I have been involved in an interesting discussion on Google+ with a few people regarding the proper way to interpret entropy,
The Bayesian interpretation takes S to be a state of ignorance such that when we update our probabilities, the entropy subsequently updates. The question is, is the updating of the probabilities themselves subjective or objective?
Here’s a very simple example. Consider a chamber divided into two sub-chambers of equal volume. This setup can be observed from two sides, with no communication occurring between the two sides as shown in the figure.
Suppose that Observer One thinks that Gas A = Gas B. On the other hand, suppose Observer Two thinks that Gas A ≠ Gas B. In short, Observer One believes that Gases A and B are indistinguishable. Observer Two believes the opposite. Both these states of knowledge can be reflected in terms of the probabilities for the associated microstates of the gases and thus in the entropies.
Now let us suppose that the partition is removed and the two gases are allowed to mix. Since Observer One thinks that Gas A = Gas B, he will find that the entropy of mixing will be zero. Since Observer Two thinks that Gas A ≠ Gas B, she will find the entropy of mixing to be non-zero. It is important to note here, that both observers have correctly carried out their experiments with the equipment and knowledge that they have at their disposal. That is not under debate. What is under debate is how do we interpret these results?
Let us take entropy to be a measure of the state of knowledge, then. Let us assume that the number of molecules of Gas A is the same as the number of molecules of Gas B, i.e. N(A) = N(B). Since this is a case of free expansion, there is also no change in internal energy, U, of either Gas A or Gas B. The thermodynamic identities, then, reduce to
where the PdV term can be interpreted as mechanical work. Suppose both Observer One and Observer Two have an identical device attached to the chamber on their respective sides that measures mechanical work. Call these measurements M(A) and M(B). According to the above thermodynamic identity, Observer One expects to find that M(A) = 0 while Observer Two expects to find that M(B) ≠ 0.
Now let us adopt a principle of consistency for classical physics:
Experiments on classical systems must yield consistent results.
This is a slight variation on the Principle of Relativity. If this principle is correct, then it must be that M(A) = M(B). That is, either the devices both measure non-zero mechanical work (in which case there’s something wrong with Observer One’s assumptions) or they both measure zero mechanical work (in which case there’s something wrong with Observer Two’s assumptions). In other words, classically, the measurements of mechanical work represent a state of reality independent of the Observer’s knowledge of the system. As such, the thermodynamic identity can be interpreted to imply that
[knowledge] = [reality].
Now, this can be interpreted in two ways. The usual Bayesian interpretation (or what passes for it), at least when applied to classical situations, would say that reality should inform our knowledge, i.e. with new information, we can update our knowledge (in other words, the equal sign is really directional, in a way). On the other hand, it is possible to interpret this as implying that by changing our knowledge of the system we can change reality! While this may be true for certain quantum systems, it nevertheless implies that, for instance, prior to our discovery of the existence of Antarctica, it didn’t exist which is absurd!
I prefer to interpret entropy as a measure of the number of possible configurations of a system. This removes the ambiguity in the classical case and it still works in the quantum case. In quantum situations we assume there is some subjectivity to the measurements (in fact there is in some classical cases as well). But at the quantum level, we become part of the system. As such, there are degrees of freedom (configurations) that the act of measurement can introduce. These degrees of freedom are minor in classical systems, but not so in quantum systems. Nevertheless, they are always still there.
So, yes, to some extent, we do “make” reality, i.e. it is a “participatory universe”, as Wheeler has suggested, just not quite in the way that everyone assumes. Or, another way of looking at it is to say that, if we apply Ockham’s razor to all the possible interpretations, this is the simplest and most consistent. Are there still issues with it? Sure. No theory is perfect. But until someone comes up with something more consistent, I’m sticking with this one.
