## What exactly is information, anyway?

[Note: this post is somewhat related to an ongoing discussion about “interestingness” and “complexidynamics” that was initially inspired by Sean Carroll‘s FQXi presentation and has been subsequently discussed by Scott Aaronson (see here and here) and Charlie Bennett.]

We know how to measure information (both classical and quantum), we know how to encode information, and we know how to transmit information.  But do we really know what it is?

Consider the following example.  Imagine a painter who paints an exquisite painting.  Is the painting itself information?  Certainly we can encode it in bits and qubits, but is the encoding of the painting the same thing as the original painting?  In other words, is information the same thing as that which it represents?  I can get plenty of copies of the Mona Lisa, but they aren’t the Mona Lisa.

Now suppose our painter never shows anyone else and the painting is eventually lost in a fire.  The painter forgets about it and eventually dies.  No record of this painting ever existed (after the fire) outside of the painter’s own head and he/she eventually died.  To use a turn of phrase from a novel I’m currently reading (Murakami’s 1Q84), quite clearly something – who knows what – was “irretrievably lost” once that painter died.  Now this raises some interesting questions.

If we take Wheeler’sit from bit” seriously, then information is the core component of the universe.  While the physical atoms of the painting and the painter are, of course, not lost, the specific “macrostate” (“message”), if you will, that they represented, is clearly gone.  The same can be said if we merely say that information represents reality rather than constructs it and even if we say it merely encodes reality.

Given recent results concerning information causality and our penchant for referring to entropy as a measure of information, this should raise some intriguing questions.  Now, even if you postulate an infinite universe or a multiverse to compensate for the lost information (if that is indeed what is lost), there’s still a problem: the fact remains that that painting in that particular universe (or part of an infinite universe) was destroyed (I suppose there are some subtle distinguishability issues that this raises).

The ongoing discussion about “interestingness” and complexidynamics aside, I think this raises a very thorny issue: just what exactly is information anyway?

### 16 Responses to “What exactly is information, anyway?”

1. ravithekavi Says:

Isn’t information just the specific arrangement/organization of things (like the arrangement of colours etc. on a painting), and as long as we preserve/encode the specific organizational scheme that we call ‘information’ (via encoding it or making any other isomorphic map that transforms it from one substrate to another, like from a real painting to a copy of it), we can say that information is preserved — albeit, one then has to have the additional information about the specific map that carries out this transformation if one wants to undo it and recover the original ‘information’. So the cost of transforming information in any reversible way has to be borne out by the specific map that enables the transformation — and that means keeping track of more information than you want to recover. One has to store the map in some memory if one wants to later recover the original information. And also process it. To recover a destroyed painting, the corresponding map that will undo the destruction process seems (intuitively) incredibly hard to encode in any efficient way. So in that sense, a destroyed painting is really lost. Because even the map that took the painting from the painter’s brain on to the canvas is probably corrupted after the original painting of it, even in the painter’s head. So any reproduction by the painter (assuming the painter is not dead) will not be exactly the same as the original (unless the hypothetical map that brought the painting into the world is very stable and preserved in the painter’s brain like it was when the painting was originally painted).

• quantummoxie Says:

Well, Wheeler clearly didn’t think that (the arrangement) was information. In Wheeler’s definition (or my interpretation of it), the loss of that painting amounts to the loss of information which raises some problematic issues.

• ravithekavi Says:

sorry, i didn’t notice that i started a new comment (below) instead of continuing this thread.

2. ravithekavi Says:

“While the physical atoms of the painting and the painter are, of course, not lost, the specific “macrostate” (“message”), if you will, that they represented, is clearly gone”

Do you deny that the “message” is the “information” here?. By an “arrangement” I meant a meaningful message, which in this case is the specific “macrostate”/configuration of the atoms that constituted the painting.

To me it seems that information is physical because it needs a substrate that embodies it (at any rate, one’s brain cells). Talking of information in the abstract (without reference to the specific “representation” or encoding in a particular substrate) is useful only as a theoretical convenience, if you will. It helps one simplify the process of understanding the common properties that information (as embodied in any substrate) must obey. Once one has this abstract understanding, one can apply it to do information processing in a more specific context, using a specific substrate.

As an analogy, it doesn’t make sense to talk of time if one doesn’t have a clock — time is very much an abstraction of the notion of relative rates of physical processes (circular though the use of “rates” may seem, intuitively I just mean that there’s a correlation between at least two processes that enables you to treat one of them as a clock and the other one as the observed process evolving in “time”). That the notion of time has properties independent of the particular clock-realization (essential though the clock is if one wants to measure time) is what it means to say time is a real thing. Ditto for ‘information’ vis-a-vis the ‘encoding substrate’.

• quantummoxie Says:

I don’t deny anything. In fact, that’s the point. Your argument is certainly persuasive. My point is merely that others have made different arguments.

Personally, I haven’t formulated an opinion yet.

3. Is the basis of existence binary?…

This is an interesting and long-standing open question.  John Wheeler certainly thought the answer to that was ‘yes.’  He referred to this as ‘it from bit’ in a 1990 paper (available in Complexity, Entropy, and the Physics of Information edited by …

4. Don’t conflate the physical and the symbolic. Arguably, physical information is never “really lost”. But symbolic information can be lost, and often has been, because symbols require interpretation, mortal interpreters.

Once upon a time a class of Incas communicated by tying knots in cords. It was clearly a sophisticated language or code, whatever. Then the Europeans came to western South America. They themselves testified that the Inca system was a coherent system of communication and a very few even learned it. But they also destroyed the culture that gave rise to the system and, like the Indians, died without passing the system on. No dictionary, no oral tradition remains. Just lots of cords with meaningless knots tied in them.

What if there’d been no Rosetta Stone? Who can read Linear A?

• quantummoxie Says:

While you make a good point, I’m not convinced that there is a difference between the physical and the symbolic. I take the attitude that things like thoughts and abstractions of that kind correspond to something physical, be it a configuration of brainwaves or whatever. It’s an arrangement of something physical somewhere in the universe. So I’m not so sure that there’s a difference. But even if there is, I think I actually made the point in the original post (somewhere?) that what I’m talking about is partly the symbolic anyway.

Think of it this way. That painting was a real collection of particles at some point (say) as were those knots you mentioned (there was a great article about them in Wired many years ago, by the way). But, someone had to think those things up in the first place so they began their existence in a more abstract “space.”

5. The difference could be that the symbolic is emergent from the basic physics. Which raises a whole domain of discourse in and of itself. Downward Causation; Downward Entailment. Physicists tend not to deal with this (Paul Davies is a, but possibly not the most important, current exception.)

Dealing with “Information” at this point in time may be somewhat like Sadi Carnot and Clausius (and even the far more sophisticated Boltzmann later on) feeling their way toward a viable concept of Energy. You have all this disjointed stuff which intuitively you feel should fit together — but where’s the connective tissue? I’m not up to addressing this tonight. But a thought: you can’t perceive or define Energy any more than you can Information. You “know” it only by discerning its observable manifestations and thinking hard about what they might have in common.

And both Energy and Information come coded. In the case of Information that’s obvious: If you don’t speak German you can’t read Goethe in the original; if you don’t speak Quantum you need to wait for your superposed qubit to be measured and become a Classical bit. Energy also comes coded. You can’t drop your air conditioner out of an airplane and expect it to read kinetic energy, or hold it over a bonfire and ask it to operate on thermal energy. It requires electrical energy. You need a transponder. But yet we believe we do (and we probably do) have a solid handle on Energy in the theoretical abstract. Information may be harder because it’s even more basic.

6. Should’ve said transducer, not transponder. And mentioned that epiphenomena obey strictly epiphenomenal rules as well as being generated and constrained by the substrate physics. But apparently all information is destructible (Here are Brukner and Zeilinger, discussing the Double Slit experiment along with the Heisenberg Microscope in “Quantum Physics as a Science of Information” page 48 of the Quo Vadis book):

“Indeed, the most interesting situations arise if the path information is present at some point in time, but deleted or erased in an irrevocable way later on. Then, as soon as that information is irrevocably deleted, the interference patterns can occur again. …”

Say wot.

• quantummoxie Says:

Huh. I’ll have to ask Caslav what he meant by that.

You make some very good points, by the way. I think I need to absorb your comments a bit more.

7. They explain in the paper how best to delete the path information. You detect the particle in the focal plane of the Heisenberg microscope’s lens instead of the imaging plane. Goes back to Fourier optics.

8. How about information being the ‘states’ something can be in? Assume a closed box, some particles in it. They interact. Now, how many states can they take, measurable? I say measurable because I do not like ‘weak measurements’, at least not if presented as some final proof of a ‘reality unfolded’.

So I would say, use what you expect yourself able to measure, even if you never do it, firstly. The rest belongs to what might be deducted as ‘strong possibilities’, although more close to philosophy than to what’s really measurable.

As for those ‘states’ I presume the same problem will exist. Not all of them will materialize as a outcome, although all are possible and so some ‘paintings will be lost’. But there it only, as I expect, should be a limited number of states at least, even if fast closing in on ‘infinity’ as you go up in particles.

The problem is what is ‘real’ here. The possibilities of outcomes, or only the outcomes finally existing? What is ‘real’ for us normally seems to be limited, as in a lot of possibilities never, or at least incredibly seldom, leading to a finalized outcome. If you think for example that EM is the ‘answer as existing for real’ then you’re a true pragmatic 🙂 If you allow for all possibilities, then you’re a theoretican.

And if you believe weak measurements to unfold reality?
Well, you just gotta love philosophy..
And math.

Or as some do, state that it’s ‘totally different’, ahem.

It’s tricky, and the bottom line is that..
I don’t really know..

• quantummoxie Says:

Actually, that’s exactly how I perceive information. When I say “configurations” or something like that, I really mean the states something can be in. As for reality, well … is there such a thing? 😉

9. I don’t know. I can feel pain, I may fear death, at least if painful. What I am is thoughts, cloaked in matter. Reality, possibly (?) could be seen as what restricts me as matter? Still, in my thoughts I’m free, as free as my imagination can make me 🙂

Da*n, we’re getting into poetry here.

10. But, assuming that thoughts are information too, which they by all means should be, as without thoughts, how would anyone ever reach the concept? And all information is conserved, in some form or other? Ahh, we’re closing in on the cave here methinks.