Entropy and Probability

How is entropy related to probability?  Well, suppose you walked out of your bedroom one morning to eat breakfast, and died because all the air in your house had moved spontaneously into your bedroom.  Not too worried about this happening?  You shouldn't be, and entropy is the reason why.  There are a lot of molecules in your house – too many to draw – so let's start with a really simple system and see if we can figure out what is going on.

Suppose we have a system of two containers, joined by a valve.  If we put just one molecule into one container, it would bounce around at random.  Open the valve, and the molecule might move into the other container – or it might stay where it is.   There are no other choices, so there would be a 50:50 chance the molecule would be in one container, or the other.

Number of possible arrangements Odds all molecules are in one of the two containers 2 1/2

Now, what if we had two molecules (the molecules aren't different, but assigning them different colours lets us see where they are).

Number of possible arrangements Odds all molecules are in one of the two containers 4 1/4

Three molecules could be arranged in eight different ways.

Number of possible arrangements Odds all molecules are in one of the two containers 8 1/8

Finally, lets look at the possibilities for four molecules.

Number of possible arrangements Odds all molecules are in one of the two containers 16 1/16

You can see that trying to do this with many more molecules will get ridiculously complex.  The number of drawings is going up very fast – exponentially, actually.   Perhaps you can see the pattern:

The number of possible ways of arranging a number of n molecules between two containers is 2 (the number of containers) to the power of n (the number of molecules): 2ⁿ.

There is only 1 possible way to get all the molecules in the one left-hand container, so the odds that all the molecules will be in one container is .  Now, if you had a real number of molecules – say a mole of them, 6.02 x 10²³ – the odds that all of them would be in one container are pretty small.  In fact, is so small that it is pretty nearly zero.  So don't hold your breath when you go to the kitchen tomorrow.  It's not very likely that all the air will move out!

You can also see that there is a fairly good chance that the molecules will be nearly evenly distributed.  For example, with just four molecules between our two containers, there are 6/16 chances to have 2 molecules on each side.  Statistically, it is just a lot more likely that things will be randomly distributed between the containers than that they will all stay in one place.