-.-

      Renyi-Kahre disinformation: their information can become negative

            Copyright (C) 2002 - 2004, Jan Hajek, Netherlands

Version 1.05 of April 28, 2004, 88 lines of < 78+CrLf chars in ASCII,
written with "CMfiler 6.06f" from WWW; submitted to the webmaster of
   http://www.matheory.info  aka  www.matheory.com
This epaper has facilities for fast finding and browsing. Please save the
last copy and use a file differencer to see only how versions differ.
Your comments (preferably interlaced into this .txt file) are welcome.

   Search for the markers  !  ?  and  [

   NO part of this document nor the results here presented may be published,
   posted on any net, implemented, programmed, copied or communicated by any
   means without an explicit & full reference to this author together with
   the full title and the website WWW.MATHEORY.INFO  plus the COPYRIGHT note
   in the texts and in ALL references to this.  An implicit, incomplete,
   indirect, disconnected or unlinked reference (in the text and/or on www)
   does NOT suffice. All based on experience.  ALL rights reserved.


Renyi-Kahre information is asymmetrical and can become negative:

Renyi entropy and information has been known since 1960 and published in
[ Renyi 1961 ] and later.
Shannon's mutual information I(A;B) is symmetrical with respect to both
random variables A, B, as it holds :

     H(A) - H(A|B) = I(A;B) = I(B;A) = H(B) - H(B|A)

Due to its symmetry, Shannon's I(A;B) is provably suboptimal for the tasks
like :
pattern classification, identification, decision making, feature selection,
diagnosing and pattern recognition, where we want high hit-rates i.e. low
miss-rates i.e. low probability of (a classification) error.

Analogously re-formulated Renyi information [Kahre 2002, p.106] is not only
asymmetrical (I do not know if its asymmetry has been pointed out somewhere),
i.e.

   R(A@B) <> R(B@A),   which should be good for higher hit-rates,

but it has also been believed to be non-negative like Shannon's information.
I do consider such an asymmetry to be a highly desirable property for tasks
like :
pattern classification, identification, decision making, feature selection,
diagnosing and pattern recognition.

But negativity disqualifies an information measure as such, unless its
negative values can be meaningfully interpreted as disinformation.
During the days prior to August 12, 2002, I have disproven the ruling
conjecture of non-negativity of  R(A@B) . The disproof is easily done by
counterexamples generated by systematically trying out many values of the
probabilities necessary to specify  R(A@B)  for a pair of 2-valued i.e.
binary random variables  A, B.

The results obtained so far with my program ARenyi.pas for varying inputs
including the parameter alpha :

The lowest value of  alpha  =  2.010  for which R(A@B) was negative while
taking into account the finite precision of even an extended arithmetic.
My other numerical & graphical investigations with XPL have confirmed that

!                       R(A@B) <  0.0 is possible for alpha >  2.00
!  The lowest value of  R(A@B) = -0.048 approx.,  for alpha =  7.05
!  The lowest value of  R(A@B) = -0.076 approx.,  for alpha = 29.93


+Conclusion : Kahre's Renyi information is no information for alpha's for
              which it can become negative.
        Quiz: Is it then a disinformation ?

-.-

+References :

In the titles of books and periodicals all words start with a CAP (except
for the words like eg: a, the, and, or, in, of, to, with, der).

Renyi Alfred:  On measures of entropy and information, Proc. Fourth Berkeley
   Symposium on Probability and Statistics, 1960, published 1961, vol.1,
   pp 547-561.
Kahre Jan: The Mathematical Theory of Information, 2002, Kluwer Academic
   Publishers
-.-