## Some perils regarding interpretations of entropy

I have been involved in an interesting discussion on Google+ with a few people regarding the proper way to interpret entropy,

$S=-k\sum_{i}p_{i}\textrm{log}p_{i}$.

The Bayesian interpretation takes S to be a state of ignorance such that when we update our probabilities, the entropy subsequently updates.  The question is, is the updating of the probabilities themselves subjective or objective?

Here’s a very simple example.  Consider a chamber divided into two sub-chambers of equal volume.  This setup can be observed from two sides, with no communication occurring between the two sides as shown in the figure.

Suppose that Observer One thinks that Gas A = Gas B.  On the other hand, suppose Observer Two thinks that Gas A ≠ Gas B.  In short, Observer One believes that Gases A and B are indistinguishable.  Observer Two believes the opposite.  Both these states of knowledge can be reflected in terms of the probabilities for the associated microstates of the gases and thus in the entropies.

Now let us suppose that the partition is removed and the two gases are allowed to mix.  Since Observer One thinks that Gas A = Gas B, he will find that the entropy of mixing will be zero.  Since Observer Two thinks that Gas A ≠ Gas B, she will find the entropy of mixing to be non-zero.  It is important to note here, that both observers have correctly carried out their experiments with the equipment and knowledge that they have at their disposal.  That is not under debate.  What is under debate is how do we interpret these results?

Let us take entropy to be a measure of the state of knowledge, then.  Let us assume that the number of molecules of Gas A is the same as the number of molecules of Gas B, i.e. N(A) = N(B).  Since this is a case of free expansion, there is also no change in internal energy, U, of either Gas A or Gas B.  The thermodynamic identities, then, reduce to

$TdS=PdV$

where the PdV term can be interpreted as mechanical work.  Suppose both Observer One and Observer Two have an identical device attached to the chamber on their respective sides that measures mechanical work.  Call these measurements M(A) and M(B).  According to the above thermodynamic identity, Observer One expects to find that M(A) = 0 while Observer Two expects to find that M(B) ≠ 0.

Now let us adopt a principle of consistency for classical physics:

Experiments on classical systems must yield consistent results.

This is a slight variation on the Principle of Relativity.  If this principle is correct, then it must be that M(A) = M(B).  That is, either the devices both measure non-zero mechanical work (in which case there’s something wrong with Observer One’s assumptions) or they both measure zero mechanical work (in which case there’s something wrong with Observer Two’s assumptions).  In other words, classically, the measurements of mechanical work represent a state of reality independent of the Observer’s knowledge of the system.  As such, the thermodynamic identity can be interpreted to imply that

[knowledge] = [reality].

Now, this can be interpreted in two ways. The usual Bayesian interpretation (or what passes for it), at least when applied to classical situations, would say that reality should inform our knowledge, i.e. with new information, we can update our knowledge (in other words, the equal sign is really directional, in a way).  On the other hand, it is possible to interpret this as implying that by changing our knowledge of the system we can change reality!  While this may be true for certain quantum systems, it nevertheless implies that, for instance, prior to our discovery of the existence of Antarctica, it didn’t exist which is absurd!

I prefer to interpret entropy as a measure of the number of possible configurations of a system.  This removes the ambiguity in the classical case and it still works in the quantum case.  In quantum situations we assume there is some subjectivity to the measurements (in fact there is in some classical cases as well).  But at the quantum level, we become part of the system.  As such, there are degrees of freedom (configurations) that the act of measurement can introduce.  These degrees of freedom are minor in classical systems, but not so in quantum systems.  Nevertheless, they are always still there.

So, yes, to some extent, we do “make” reality, i.e. it is a “participatory universe”, as Wheeler has suggested, just not quite in the way that everyone assumes.  Or, another way of looking at it is to say that, if we apply Ockham’s razor to all the possible interpretations, this is the simplest and most consistent.  Are there still issues with it?  Sure.  No theory is perfect.  But until someone comes up with something more consistent, I’m sticking with this one.

### 5 Responses to “Some perils regarding interpretations of entropy”

1. “. . . the measurements of mechanical work represent a state of reality independent of the Observer’s knowledge of the system.”

This must be equally true for classical and quantum systems. Only in the later case, it’s more complicated. 🙂

2. I found ” It is important to note here, that both observers have correctly carried out their experiments with the equipment and knowledge that they have at their disposal. That is not under debate. What is under debate is how do we interpret these results?” a little hard to digest 🙂 Directly wanted to argue reading that, as it in the end should become a question if there can be two different ‘realities’ ‘simultaneously’ (don’t involve SR for this particular case please)

Anyway.

You and the experiment, assuming you are involved, are part of a same ‘system’ as I think of it. Your subsequent measurement and outcome is partly a reaction to your involvement, partly due to the equipment. If you by it, on the other tentacle, would mean that a repeatable experiment should differ, depending on whether there is a observer, or not?

Then I think I need a proof.

3. The mixing of ideal gases is the superposition of two free expansion processes that don’t interact.

A free expansion begins at a non-equilibrium position when the wall is removed). It is not in equilibrium because the gas is distributed unevenly in the larger volume V=V_final, the pressure P is not spatially constant, and the entropy S is less than the value S_e(P,V) given by an equilibrium relation such as the Sackur-Tetrode equation.

The expansion proceeds through non-equilibria, irreversibly and with entropy production, to an equilibrium. But the equation

dU = -P dV + T dS (*)

can’t be used. It can only be used for a system that satisfies the equilibrium relation S=S_e(P,V) for all times, which is incompatible with internal entropy production. (Entropy can be transferred with heat across the chamber boundary, just not get produced internally.)

I agree that during a free expansion, U remains constant and S increases. But also V remains constant (equal to the final value). So we have during the expansion,

dU/dt = 0, dS/dt > 0, dV/dt = 0,

whereas P is not defined (not a function of t alone). So we can see directly that the evolution does not satisfy (*). Indeed, S(t) < S_e(U,V) during the evolution, and the system is not in equilibrium. (Recall that V(t) = V_final for this calculation.) But it is precisely the equilibrium equation S=S_e(U,V) whose differential yields (*).

When you use (*), you have to capture the P dV work by a mechanism on the boundary. You have to support the T dS term by heat transfers (energy with entropy) across the boundary. No other sources of work or entropy are foreseen. If you follow these restrictions, your process is reversible (as long as you don't discard your advantages somewhere else in the system).

In particular, if you have an insulated chamber but capture the p dV work during the expansion, you have "reversible adiabatic expansion". The work you capture reduces U and the gas cools (since T = cU for an ideal gas).

Sorry to go on so long, it is very hard for me to keep these things straight so I have to say them aloud.

4. Just to clarify:

Above, I was just addressing the case A ≠ B and the main point is that you can’t use dU = -P dV + T dS in that case because it’s not in equilibrium during the process.

But if you capture the work with two pistons (each permeable to the other gas, if this is possible) then you can use it and you get the non-zero work you mentioned. There’s a reference or two but I can’t find it at the moment.