>

How Uncertainty Decreases Define Information

By using a decrease in uncertainty to define information (equation (10)), we also avoid dealing with absolute quantities. Information is gained when a machine changes from an indeterminate state to a more determined state [Schneider, 1991]. There are a large number of microstates in both the before and after states, but since we are only concerned with changes of state, the large numbers are removed from consideration when the subtraction is made. Although this might appear to be a difference between large numbers, it is not: it is the logarithm of the ratio of large numbers (the sphere volumes in [Schneider, 1991]), which can be quite small. Because of this we can even legitimately speak about single bits for changes in a macroscopic object without knowing the detailed state of its molecules. Consider a coin flipping in the air. The entropy of this system is enormous, on the order of H_before = 10²³ bits in a 3 gram copper penny at 300$\mbox{K}$ using equation (6) and data from [Weast et al., 1988]. If all states were equally likely, $P_i = \frac{1}{\Omega}$, and equation (3) would reduce to $H_{equal} = \log_2 \Omega$. Since $H_{equal} \geq H_{before}$ [Shannon, 1948,Shannon & Weaver, 1949], $\Omega \geq 2^{10^{23}}$ states. Yet, after the coin has settled on one side, the uncertainty is only one bit lower because there are half as many microstates: if $H_{before} = log_2(1 \Omega)$ and $H_{after} = log_2(\Omega/2)$ then R = H_before - H_after $= log_2(1 \Omega / (\Omega/2))$ = 1 bit.

This assumes, of course, that either result of the coin flip is useful for some function. A coin-flip operation by a molecular machine can be useful if either result helps the survival of the organism that makes the machine. A striking molecular example is the mechanism used by the immune system, where the random joining of gene segments helps to insure the creation of a wide variety of antibodies [Watson et al., 1987].

However, random choices are not repeatable, so they are not useful to most molecular machines. If a coin flip mechanism were to be used, $H_{before} = log_2(2 \Omega)$ but in the ensemble of all possible after states, H_after also equals $log_2(2 \Omega)$, so R = 0. No information could be gained in the long run. For example, if the restriction enzyme EcoRI did not reliably and repeatably recognize one pattern, GAATTC, the bacterium might die by the destruction of its own genetic material [Heitman et al., 1989]. Likewise, if a DNA polymerase did not reliably insert adenosine opposite every thymidine, many mutations would occur. It is not ``simply a matter of putting in the right one'' (as we often have a tendency to think); biological systems evolve to avoid mistakes. Macroscopic communications devices must also select one particular state from several possible states. For example, a teletype selects only one character from many incorrect ones because, at any given moment, there is only one correct character to be printed. All others are errors. In both human and biological machines, there is a bias toward one particular state which is preferentially chosen from several possible states.

Even a very energetic penny can gain only one bit of information when it settles down. The following shows that there is a minimum amount of energy that a coin has to give up to specify heads or tails.