Physics449-StatisticalMechanicsI-CourseNotespeople.ucalgary.ca/~dfeder/449/notes.pdf · The ideal...

Physics 449 - Statistical Mechanics I - Course Notes

David L. Feder

September 5, 2013

Contents

1 Energy in Thermal Physics (First Law of Thermodynamics) 31.1 Thermal Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2 The Ideal Gas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.2.1 Thermodynamic Derivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2.2 Mechanical Derivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.3 Equipartition of Energy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.4 Heat and Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71.5 Compression Work: the Adiabat . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71.6 Heat Capacity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2 The Second Law of Thermodynamics (aka The Microcanonical Ensemble) 132.1 Two-State Systems (aka Flipping Coins) . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.1.1 Lots and lots of trials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 152.1.2 Digression: Statistics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.2 (This section skipped) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172.3 Interacting Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172.4 Large Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.4.1 Discrete Random Walks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182.4.2 Continuous Random Walks . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202.4.3 (maybe) Quantum Walks and Quantum Computation . . . . . . . . . . . . . 22

2.5 (This section skipped) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252.6 Entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

2.6.1 Boltzmann . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 252.6.2 Shannon Entropy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 262.6.3 (maybe) von Neumann Entropy . . . . . . . . . . . . . . . . . . . . . . . . . . 29

3 Interactions and Implications (aka Equilibrium) 323.1 Temperature . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323.2 Entropy, Heat, and Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3.2.1 Thermodynamic Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . 343.2.2 Statistical Approach . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

3.3 Paramagnetism . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 363.4 Mechanical Equilibrium and Pressure . . . . . . . . . . . . . . . . . . . . . . . . . . . 383.5 Diffusive Equilibrium and Chemical Potential . . . . . . . . . . . . . . . . . . . . . . 39

1

PHYS 449 - Course Notes 2012 2

4 Engines and Refrigerators 414.1 Heat Engines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 414.2 Refrigerators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444.3 Real Heat Engines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

4.3.1 Stirling Engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 444.3.2 Steam Engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 464.3.3 Internal Combustion Engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

4.4 Real Refrigerators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 494.4.1 Home Fridges . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 494.4.2 Liquefaction of Gases and Going to Absolute Zero . . . . . . . . . . . . . . . 50

5 Free Energy and Chemical Thermodynamics 515.1 Free Energy as Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

5.1.1 Independent variables S and V . . . . . . . . . . . . . . . . . . . . . . . . . . 515.1.2 Independent variables S and P . . . . . . . . . . . . . . . . . . . . . . . . . . 525.1.3 Independent variables T and V . . . . . . . . . . . . . . . . . . . . . . . . . . 525.1.4 Independent variables T and P . . . . . . . . . . . . . . . . . . . . . . . . . . 535.1.5 Connection to Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 535.1.6 Varying particle number . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

5.2 Free Energy as Force toward Equilibrium . . . . . . . . . . . . . . . . . . . . . . . . 54

6 Boltzmann Statistics (aka The Canonical Ensemble) 566.1 The Boltzmann Factor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 566.2 Z and the Calculation of Anything . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

6.2.1 Example: Pauli Paramagnet Again! . . . . . . . . . . . . . . . . . . . . . . . 606.2.2 Example: Particle in a Box (1D) . . . . . . . . . . . . . . . . . . . . . . . . . 616.2.3 Example: Particle in a Box (3D) . . . . . . . . . . . . . . . . . . . . . . . . . 626.2.4 Example: Harmonic Oscillator (1D) . . . . . . . . . . . . . . . . . . . . . . . 646.2.5 Example: Harmonic Oscillator (3D) . . . . . . . . . . . . . . . . . . . . . . . 656.2.6 Example: The rotor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

6.3 The Equipartition Theorem (reprise) . . . . . . . . . . . . . . . . . . . . . . . . . . . 676.3.1 Density of States . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

6.4 The Maxwell Speed Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 706.4.1 Interlude on Averages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 726.4.2 Molecular Beams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

6.5 (Already covered in Sec. 6.2) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 746.6 Gibbs’ Paradox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

7 Grand Canonical Ensemble 767.1 Chemical Potential Again . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 767.2 Grand Partition Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 777.3 Grand Potential . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79


Introduction

The purpose of these course notes is mainly to give your writing hand a break. I tend to write lotsof equations on the board, because I want to be rigorous with the material. But I write very quicklyand my handwriting isn’t pretty (this is probably a huge understatement). So these course notescontain (hopefully) all the equations that I will be writing on the board, so that when you take notesduring class you can focus on the concepts and my mistakes, rather than furiously trying to scribbledown everything I am writing, which will probably contain mistakes anyhow. Not to say that thesenotes don’t contain mistakes! These notes also have occasional non-mathematical expressions (i.e.sentences).

Chapter 1

Energy in Thermal Physics (FirstLaw of Thermodynamics)

This chapter deals with very fundamental concepts in thermodynamics, many of which you canintuit from your experience.

1.1 Thermal Equilibrium

Some questions to ponder:

• What are the ways that you measure room temperature?

• What are the ways to measure temperatures that are much hotter? Colder?

• What exactly is temperature?

• What is absolute zero?

• How do systems reach a given temperature?

• What does it means to say a system is in equilibrium?

• What is thermal equilibrium?

1.2 The Ideal Gas

1.2.1 Thermodynamic Derivation

Robert Boyle (1627-1691) was an Irish alchemist (!) who helped to establish chemistry as a legitimatefield. After much observing, he found in 1662 that gases tended to obey the follow equation:

PV = k, (1.1)

where P is the pressure, V is the volume, and k is some constant that depends on the specificgas. This equation was known as Boyle’s Law. In 1738 Daniel Bernoulli derived it using Newton’s

3


equations of motion (see more about this in the next section), under the assumption that the gas wasmade up of particles too tiny to see, but no one paid any attention because these particles were notbelieved to actually exist. Later, Joseph-Louis Gay-Lussac (1778-1850) observed that at constantpressure, one always has V ∝ T , or V1T2 = V2T1. This is known as Charles’ Law after some guynamed Charles. Benoıt Paul Emile Clapeyron (1799-1864) put the two laws together to obtain

PV = P0V0 (267 + T ) , (1.2)

in which the temperature is measured in degrees Celcius. The number 267 came from observations ofGay-Lussac. This was pretty impressive, since absolute zero is known today to be -273.15C. LorenzoRomano Amedeo Carlo Avogadro di Quaregna (Quaregga) e di Cerreto (1776-1856), otherwise knownas Avogadro, showed in 1811 that the P0V0 out front of Clapeyron’s equation was related to the‘amount of substance’ of the gas, and wrote:

PV = nR (267 + T ) , (1.3)

where n is the number of moles of the gas, and R = 8.31 J/mol/K is a universal constant (indepen-dent of the type of gas). It’s easier to think of the number of particles N (atoms or molecules) ratherthan the number of moles, so one can write N = nNA, where NA = 6.02214179(30)× 1023 is knownas Avogadro’s number and corresponds to the number of atoms in a mole of gas. Finally, if wemeasure temperature in units of Kelvin (K = 273.15 + C) and make the substitution R = NAkB,where kB = 1.381× 10−23 J/K is Boltzmann’s constant, we finally obtain the ideal gas law:

PV = NkBT. (1.4)

This is nice because we don’t have to worry about moles. What’s a mole anyhow?? In the textbook,note that they write k for kB .

That’s about all the history you’re going to get!

The ideal gas law (1.4) is called the Equation of State for the ideal gas, because it relates the threestate variables (thermodynamic coordinates) P , V , and T at equilibrium. These are macroscopicproperties of the system, and their relationhip doesn’t depend on how the system was changed toget to where it is. I could write it instead as P = P (V, T ), i.e. the pressure is given by somefunction that depends on two variables V and T . All systems at equilibrium have such constitutiveequations, relating parameters of the system over which one has external control. The van der Waals(non-ideal) gas, for example, was found to have the equation of state

NkBT =(

P +a

V 2

)

(V − b) , (1.5)

where a and b are molecule-dependent constants. Magnets have a similar equation that relatesthe temperature, magnetic field B, and the amount of magnetization M . See Section 3.3 for moredetails.

1.2.2 Mechanical Derivation

The ideal gas law can also be derived from first principles, which is what both Bernoulli (approx-imately) and Boltzmann did (thus the konstant with his initial on it) by assuming an atomistictheory. We’ll do a good job of this in Chapter 3, but a rough job of it can be done right now. Let’s


Figure 1.1: One molecule bouncing into a piston.

assume that there is a single molecule or atom, bouncing around elastically in a volume V = AL,where the length along the x-direction is L, as shown in Fig. 1.1. After many bounces against thepiston, the average pressure directly on it is

P =F x, on piston

A= −Fx, on molecule

A= −

m(

∆vx∆t

)

A. (1.6)

Let’s set ∆t = 2L/vx, the time it takes to make a full round-trip. When it undergoes one elasticcollision, its change in velocity is ∆vx = −2vx. Putting these together gives

P = −m(

∆vx∆t

)

A=

2mvx2AL/vx

=mv2xV

. (1.7)

If there were actually lots of molecules, each colliding with each other and the walls totally elastically,then we can forget the averageness of the pressure to obtain

PV = m[(

v2x)

1+(

v2x)

2+ . . .

]

= m

N∑

i=1

(

v2x)

i= mNv2x = NkBT (1.8)

using the ideal gas law at the end. So we have the mean kinetic energy per molecule in the x-directionis equivalent to temperature:

Kx

N=

1

2mv2x =

1

2kBT. (1.9)

This means that the mean kinetic energy is

K =1

2mv2x +

1

2mv2y +

1

2mv2z =

1

2mv2 = (3/2)NkBT. (1.10)

The Root-mean square (RMS) speed (which is usually close to the average value) of each atom/molecule

is defined as vRMS =√

v2 and is equal to√

3kBT/m in this case. Suppose we have oxygen at room


temperature. Diatomic oxygen has atomic mass 32, which is 5.297 ∗ 10−26 kg. Assuming roomtemperature is 300 K, one obtains vRMS = 484 m/s. This is really fast! How fast? Consider thatthe speed of sound in air at standard temperature and pressure is only 343 m/s. Is it reasonablethat the mean speed of the atoms is higher than the sound speed?

1.3 Equipartition of Energy

The equipartition theorem states that ‘every degree of freedom contributes an energy of 12NkBT to

the total energy of the system.’

To make sense of this, we need to talk about what a degree of freedom actually is. We’ll be muchmore rigorous about it in Section 1.13, but because this will be months from now it makes senseto talk about it now. A degree of freedom tells you about what the atom or molecule can actuallydo. Maybe it can move around (translate), or jiggle (vibrate) or turn around (rotate), or scream,or shine, or a host of other things. Each one of these capabilities contributes an energy of 1

2kBT toits energy. Likewise, suppose all your particles were constrained in some way to move only in onedimension; then the total translational energy would be a kBT factor smaller than molecules movingin free space. The real value of the equipartition theorem is that it allows you to calculate the totalenergy without having to do any work. So lazy people use it all the time in the real world!

It helps to remember your classical mechanics, though. For example, the kinetic energy of a rotatingbody is given by something that looks like

Krot =1

2

(

Ixxω2x + Iyyω

2y + Izzω

2z

)

, (1.11)

where Iii are the moments of inertia in the diagonalized inertial basis, and the ωi are the angularfrequencies in radians. Each of these terms is quadratic, so each one corresponds to adegree of freedom. But recall that the energy of an oscillator is made up of both a kinetic energyand potential energy:

Eosc =1

2m

(

p2x + p2y + p2z)

+m

2

(

ω2x + ω2

y + ω2z

)

, (1.12)

where I have already used that ωi =√

ki/m. Because there are two kinds of quadratic termscomprising the energy in each dimension, the three-dimensional oscillator contributes 3kBT perparticle!

So the total energy of the system (in some limit that will be explained more clearly later on) is

U =f

2NkBT, (1.13)

where f is the number of degrees of freedom.

Many examples of equipartition will be covered in class.

It is interesting to note that, even though these results were derived under the assumption of anideal gas, the equipartition theorem remains correct even when the gases are interacting. In fact, itcan be applied to a huge variety of physical systems, many of which would surprise you.


1.4 Heat and Work

So far we have seen a connection between temperature and energy through the equation of state ofthe ideal gas and the idea of molecules bouncing around in a box. Using the equipartition theorem, itlooks like the temperature is closely related to the heat of a system, if one considers the heat as beingsomehow related to the amount of energy a system has. In fact, the word ‘heat’ in thermodynamics isnot the same as what we think of as the amount of ‘hotness,’ i.e. is not equivalent to the temperature.

In fact, ‘heat’ is the amount of energy that spontaneously flows between two systems: heat movesfrom the system at higher temperature to the system at lower temperature. You might worry thatthis definition is circular, though, because temperature is itself a measure of a system’s tendencyto either absorb or release energy. But now we can define temperature slightly better through theequipartition theorem: it is a measure of the total energy of the system U i.e. T ≈ 2U/fkB, wheref is the number of degrees of freedom.

D∗: Heat: The heat Q is the energy added to the system by a spontaneous (i.e. not forced) process.

Please take careful note: the magnitude of Q is positive if heat is added to the system, and isnegative if it is removed. Of course, we could also change a system’s energy by doing somethingto it. This is called ‘work,’ and it roughly corresponds to what you naturally think of as work: ittakes work to do something to something! But it corresponds to anything where forces are present,it doesn’t have to be you doing the work.

D: Work: The work W done on a system is the energy added to the system by a forced process.

Again, the work is positive if energy is added to the system; one says that work is done on thesystem. If energy is expended (released) by the system by the work, then one says that work is doneby the system. But probably these concepts are familiar to you, because they are the same onesused in classical mechanics. Work in that case was defined as

W =

∫

F · dr, (1.14)

and it is the same now. Of course, in situations where the applied force doesn’t change withposition, then it reduces to W = F ·∆r. Anyhow, heat and work together yield the First Law ofThermodynamics:

∆U = Q+W. (1.15)

This is really just a statement of the conservation of energy, so it is weird that it is called a Law,but there you go. Processes of heat transfer include conduction, convection, and radiation.

1.5 Compression Work: the Adiabat

There are many ways to do work on a system (or for a system to do work). One of the simplest isto squeeze it, such as pushing or pulling the piston in Fig. 1.1. In the case where we push on thepiston (force to the left), the work is W = −F∆x. Now, F ≈ PA as long as the pressure is always

∗The D stands for a definition.


uniform and at equilibrium during the operation. So we have W = −PA∆x. But A∆x = ∆V isthe change in volume of the system, so

W = −P∆V. (1.16)

In our case ∆V < 0 so work W > 0 is done on the system, as expected, because we are the onesdoing the work!

If the pressure changes during the course of the process, then the above result will no longer becorrect. This is entirely possible because the volume is changing and generally this will change thepressure. In this case one needs to know the equation of state P = P (V, T ) and determine

W = −∫ Vf

Vi

P (V, T )dV. (1.17)

Let’s consider an ideal gas. Obviously, squeezing the container will pump energy into the system,because the work (1.17) is positive when dV < 0. But let’s suppose that somehow we manage tokeep the temperature completely constant. Then the work done is

W = −NkBT∫ Vf

Vi

dV

V= −NkBT ln

(

VfVi

)

> 0. (1.18)

Because the temperature is closely tied to the total internal energy of the system, this is equivalentto stating that the total energy must remain constant. In this case, we must have as much heatflowing out of the system as there is work being done on the system, i.e.

Q = NkBT ln

(

VfVi

)

< 0 (1.19)

which is negative because heat is flowing out. This process is called an isotherm.

Suppose now that we don’t let any heat escape at all. Then the internal energy and temperaturemust both go up, ∆U =W . Using the equipartition theorem (1.13), we have

∆U =f

2NkB∆T = −P∆V. (1.20)

Because things can vary during the course of the changes, it is better to write this as a differentialequation, (f/2)NkBdT = −PdV . Using the ideal gas formula again gives

f

2

dT

T= −dV

V(1.21)

which can be integrated to yield (f/2) ln(Tf/Ti) = − ln(Vf/Vi) = ln(Vi/Vf ). This yields

(

TfTi

)f/2

=

(

ViVf

)

⇒ V T f/2 = constant ⇒ V γP = constant, (1.22)

where γ = (f + 2)/f is the adiabatic exponent, and the process is called an adiabat.


A few comments here are in order. Since no heat is flowing in or out of the system in an adiabaticprocess, then in the absence of any friction we could completely recover the original state of thesystem by simply reversing the procedure, i.e. by pulling the piston back to where it was before. Thiswill allow the system to do the work, and the total energy and temperature will go back to wherethey were at the beginning. Though we haven’t talked about entropy yet, it turns out that sucha process conserves the entropy, because nothing has changed when get back to where we started.The process is then called isentropic. I mention this because often when people are talking aboutadiabatic processes, they sometimes actually mean isentropic, and vice versa.

The adiabatic heating of a gas on compression has many practical consequences. When gases arecompressed (eventually into liquids) for use in scuba gear or industrial processes, they rapidly heat.So this needs to be performed under cold conditions where the compressing gases can quickly cool,with metal (i.e. thermally conducting) tanks under cold water in the case of scuba gear.

You might be interested to know that this compression is precisely the principle behind diesel engines:when the diesel gas is hot enough it spontaneously explodes. This is why diesel engines don’t needspark plugs and why it is better to have one of those cars if you need to ford a deep river on yourway to the mountains! Here’s another application: the Chinooks. When the air descends from thetops of the mountains into the foothills, the air pressure increases because of the lower altitude,which heats it as an adiabatic process. It is largely for this reason that Chinook winds are warm.A similar thing happens in Antarctica with the katabatic winds, which can blow continuously formonths. Unfortunately for researchers down there, in that case the air loses its adiabatic heat dueto equilibration with the glaciers.

1.6 Heat Capacity

The heat capacity of an object not exactly what it sounds like, i.e. how much heat can it hold. Butit’s close: it is how much heat is needed to raise its temperature by one degree. Of course, theproperties of the object can change as it heats or cools (like it might freeze, catch fire, etc.), so tothe heat capacity needs to be carefully defined. Here’s the way the book does it: the heat capacityis C ≡ Q/∆T . In books you often hear about the ‘specific heat’ instead, which is the heat capacityper unit mass c ≡ C/m. The problem with this definition is that Q = ∆U −W , so that the heatcapacity depends on how the heat is added because of the work term. To make things simpler weassume that the applied work is zero. Assuming the work is of the form W = −P∆V then thismeans the volume is kept constant. So we finally have

CV =

(

∆U

∆T

)

V

=

(

∂U

∂T

)

V

. (1.23)

Most real objects actually prefer to expand when heated, keeping their pressure constant. Thiseffect is pretty small in solids and liquids (see more on this below) but is important in gases. So itis convenient to include the work done by the object during expansion:

CP =

(

∆U − (−P∆V )

∆T

)

P

=

(

∂U

∂T

)

P

+ P

(

∂V

∂T

)

P

. (1.24)

In solid-state physics people usually just say ‘heat capacity’ or ‘specific heat’ and you should alwaysassume they are talking about CV . You should convince yourself that for an ideal gas CP =


CV + NkB = CV + R if N = NA. Often in circumstances where the system is allowed to dovolume-changing work it is useful to define the enthalpy:

H ≡ U + PV. (1.25)

Then it’s obvious that CP = (∂H/∂T )P . The enthalpy doesn’t really mean anything particularlyphysical to me; it seems more like a mathematical devide to me. But the textbook says that it canbe thought of as the total energy that you need to create something out of nothing, i.e. it includesthe energy you need to displace the air to put something there. The enthalpy is hugely popular inchemistry, so because this is a physics class I’ll try to avoid it as much as possible!

Let’s calculate the heat capacity for an ideal gas. Using the equipartition theorem (1.13) we have

CV =∂

∂T

f

2NkBT =

f

2NkB =

f

2R, (1.26)

where the last equality follows if N = NA (see Sec. 1.2). In fact, for many people this equation isactually the equipartition theorem, i.e. that every degree of freedom contributes NkB/2 to the heatcapacity. The chemists (who have moles on the brain) state that every degree of freedom contributesR/2. With this fact in mind, let’s consider the heat capacity of a gas of hydrogen (recall thathydrogen is diatomic, as are all the HOBFINC gases: hydrogen, oxygen, bromine, fluorine, iodine,nitrogen, and chlorine): It is clear from this figure that the heat capacity is strongly dependent on

Figure 1.2: The heat capacity of hydrogen gas.

the temperature. Is there an error in the ideal gas result above? Or perhaps is there somethingwrong with what we call a degree of freedom? I will justify the shape of this curve properly inSec. 6.3.

At the risk of getting ahead of ourselves, it turns out that at a phase transition one can add heat toa system without having it increase its temperature at all (phase transitions will be considered quitea bit in Statistical Mechanics II). Think about a pot of water just at the boiling point. You keep the


burner on, but it stay at 100C. It’s heat capacity is infinite! A similar thing happens when you coolwater down to the freezing point: a huge amount of heat leaves the system without the temperaturedropping below 0C. In fact, a sudden spike in the energy as a function of temperature is used asa signature of a phase transition. In 1908 (almost exactly 100 years ago!), Kamerlingh Onnes wascooling helium and found that it turned into a liquid at 4.2 K. Cooled even further, it started goingbananas at Tλ = 2.17 K, with a huge release of energy, but below this it stayed a liquid. Only laterwas it realized that it had turned into a superfluid! These days, the fluid-superfluid transition iscalled the ‘lambda point’ because of the shape of the heat capacity curve:

Figure 1.3: Lambda point of liquid helium, showing the energy released at the superfluid transition.

Most materials have a so-called ‘cooling curve’ that shows how the cooling occurs as a functionof time. Fig. 1.4 shows a curve for a generic metal as it cools from a liquid to solid. Notice thenon-linear shape of the curve even far from the phase transition. For the hottest cup of coffee (ortea or hot chocolate as per your preference!), is it better to add the milk right away when you pourit, or just before you drink it? Do you need to use the graph to figure this out?

The heat capacity is not a very useful quantity at phase transitions because of this divergence. Butit would still be able to make quantitative statements at this point. Instead, one uses the ‘latentheat of formation,’ or the ‘latent heat’ for short:

L ≡ Q

m, (1.27)

which measures the amount of heat needed (or released) to fully transform the substance from onephase to the other. One assumes that the pressure is constant and that no other work is done either.The latent heat for water to boil is L = 2260 J/g, or 540 cal/g (1 J≈ 0.239 cal). Contrast this withthe 418.4 J (100 cal) needed to bring water from 0 to 100!

While we’re on the topic of phase transitions, you might be interested in looking at the genericphase diagram of many materials. What important biological consequences are there of water’s‘anomalous’ behaviour?


Figure 1.4: Cooling curve for a generic metal as it goes from liquid to solid.

Figure 1.5: Generic phase diagram. The dotted line corresponds to the ‘anomalous’ behaviour ofwater.

Chapter 2

The Second Law ofThermodynamics (aka TheMicrocanonical Ensemble)

2.1 Two-State Systems (aka Flipping Coins)

D Classical Probability: All events are equally likely.

D Statistical Probability: Probability that an event occurs is the measured relative frequency ofoccurrence.Q∗ Is this definition circular?

D Trials: Experiments or tests.

D Events: Results of above, designated ei.

The collection of events forms an abstract space called event space. Each point i in space is assigneda probability pi = p(ei) which is the probability of event i, so that

∑

i

pi = 1.

The classical probability of event i is pi = 1/Ω, ∀i when there are Ω points in event space. For asingle coin, Ωc = 2; for a die, Ωd = 6.

Suppose we toss our coin four times. How many events are there? Ω = Ω4c = 24 = 16. Each event is

listed in the table on the next page. The probability of obtaining the result HHHH is just as likelyas obtaining HTTH or HTHT. Does this seem right?

∗The Q stands for a question.

13


Combination (#H, #T)HHHH (4,0)HHHT (3,1)HHTH (3,1)HHTT (2,2)HTHH (3,1)HTHT (2,2)HTTH (2,2)HTTT (1,3)THHH (3,1)THHT (2,2)THTH (2,2)THTT (1,3)TTHH (2,2)TTHT (1,3)TTTH (1,3)TTTT (0,4)

Table 2.1: All the possible outcomes of tossing a coin four times.

Because the result of each coin flip is completely independent of the results of any previous coinflips, the probability of obtaining H or T is always 1

2 every time. So the probability of getting anyof the 4-coin-flip events is pi =

12 × 1

2 × 12 × 1

2 = 116 .

D Microstate: Each compound event. For this example, there are 16 microstates. All microstatesare formed from single events that are themselves equally likely.

Theorem: For a system in equilibrium, all microstates are equally possible.

Ergodic Hypothesis (Boltzmann 1871, Ehrenfest 1911): Given a sufficiently long time, all mi-crostates will be observed and all points in event space will be accessed. THIS IS THE UNPROVENFOUNDATION OF STATISTICAL MECHANICS!

Now, how many microstates have 4 heads and no tails, or (4, 0)? Or (3, 1), (2, 2), (1, 3), or (0, 4)? Ifwe don’t care what order the faces come up, then we have #(4, 0) = #(0, 4) = 1; #(3, 1) = #(1, 3) =4; #(2, 2) = 6. Recognize the pattern?

D Macrostate: Collection of microstates with some common property.

In the coin case above, the macrostates correspond to all events that share the same number ofheads and tails.

Theorem: The state of the system (the macrostate that is actually observed) is the macrostatewith the largest number of microstates.


2.1.1 Lots and lots of trials

How to assign values to the macrostate when the number of trials (N) gets really huge?

Suppose that we have a weighted coin, i.e. where the probability that it comes up heads is p(H) ≡ pand probability of seeing tails is p(T ) ≡ q = 1−p so that p+ q = 1. Then the microstate probabilityPmicro is

Pmicro = Pmicro(n1) = pn1qn2 = pn1(1− p)N−n1

where n1 and n2 are the number of p and q events, respectively, for a given trial. The total numberof coin tosses is N = n1 + n2.

Then the macrostate probability Pmacro(n1) = Ω(n1)pn1qN−n1 , where Ω(n1) is the number of ways

of arranging n1 events with p and N − n1 events with q. We also know that

∑

n1

Pmacro(n1) =∑

n1

Ω(n1)pn1qN−n1 = 1. (2.1)

So now let’s use the binomial theorem:

(p+ q)N = 1 =

N∑

n=0

(

N !

n!(N − n)!

)

pnqN−n, (2.2)

Comparison of Eqs. (2.1) and (2.2) shows that the number of ways of distributing microstates in amacrostate is given by:

Ω(N,n) =N !

n!(N − n)!≡(

Nn

)

. (2.3)

These are the binomial coefficients, which is why those numbers (1,4,6,4,1) appeared when count-ing microstates in the 4-coin macrostates above. By the way, the right hand side of the above equationis read ‘N choose n.’ Note that if p = q = 1/2 then

N∑

n=0

N !

n!(N − n)!= 2N . (2.4)

The binomial coefficients for increasing values of N are shown in Fig. 2.1.

What is the intuitive reason for this expression for Ω? Suppose we have 4 objects, labeled A, B, C,and D. How many ways can these be arranged? Let’s see:

ABCD, ABDC, ACBD, ACDB, ADBC, ADCB, BACD,BADC, BCAD, BCDA, BDAC, BDCA,CABD, CADB, CBAD, CBDA, CDAB, CDBA, DABC, DACB, DBAC, DBCA, DCAB, DCBA.

So there’s 4! = 24 ways. In general, N objects can be arranged N ! ways. Why? Well, for theexample above there’s 4 ways of arranging the first letter, 3 ways of arranging the second, 2 waysfor the third, and only one way for the last: i.e., 4× 3× 2× 1 = 4!.

But A, B, C, and D might not be completely distinguishable! Suppose all of these letters actuallyrepresented heads (H). Then obviously there would be only one way of arranging them. If n of theletters stand for heads, then one would need to divide by n!, which is just the number of ways ofarranging the previously distinguishable letters, which are now all heads. But we also need to divide


Figure 2.1: Normalized binomial coefficients for increasing N . These are the binomial coefficientsdivided by the total, 2N .

by (N − n)!, which is the number of ways of arranging the previously distinguishable letters thatnow stand for tails. And so the total number of arrangements is N !/n!/(N − n)!.

From the above discussion, it should now be obvious how to extend this analysis to ‘coins’ of morethan two sides (whatever they would look like!). A three-sided coin would have

Ω(N,n1, n2) =N !

n1!n2!(N − n1 − n2)!,

while an m-sided coin would have

Ω(N,n1, n2, . . . , nm−1) =N !

∏mi=1 ni!

=N !

∏m−1i=1 ni!

(

N −∑m−1

i=1 ni

)

!. (2.5)

This equation is going to be important later in the term.

I’ll cover some examples of calculating probabilities and Ω’s in class.

2.1.2 Digression: Statistics

Suppose that there is some variable u that have values u1, u2, . . . , um appearing with probabilitiesp(u1), p(u2), . . . , p(um) = p1, p2, . . . , pm. Then


D Mean Value (Average): of u is designated u and is defined as

u ≡ u1p(u1) + u2p(u2) + · · ·+ ump(um)

p(u1) + p(u2) + · · ·+ p(um)=

∑mi=1 uip(ui)∑m

i=1 p(ui)=

∑mi=1 uipi∑m

i=1 pi=

m∑

i=1

uipi (2.6)

because∑m

i=1 pi = 1 by the definition of probability.

In general, f(u) =∑m

i pif(ui) =∑m

i pifi.

D Variance about the mean: of u is designated (∆u)2 and is defined as

(∆u)2 ≡m∑

i=1

pi (ui − u)2 ≡ (u− u)

2

= u2 − 2uu+ u2 = u2 − u2. (2.7)

Note that u2 is just u2.

The variance is also called the fluctuations about the mean, the second moment of u aboutthe mean, and the dispersion of u. The square root of the variance is called the standard

deviation: ∆u ≡√

(∆u)2.

D Root-Mean Square (RMS) value: This is just what it sounds like. It is the square root of

the mean of the square of the function u: RMS(u) ≡√

u2.

2.2 (This section skipped)

2.3 Interacting Systems

The discussion of flipping coins in the previous section must seem a bit out of place after all thediscussion of gases in Chapter 1. In fact, it relates closely with the behaviour of a one-dimensionalgas (which might also seem a bit artificial!), as will be seen in the following section. A simpleextension relates it to a real 3D gas. But before we get to that, it turns out that the simple model ofa two-sided coin corresponds closely to something called the Pauli paramagnet. This will be studiedcompletely later on in Sec. 3.3, but it is useful to introduce it here, to illustrate how equilibriumworks.

The Pauli paramagnet is a collection of tiny atomic magnetic spins that can either point up or down.One could imagine iron atoms this way. If you associate ↑ with heads and ↓ with tails, for example,then the correspondence with flipping coins is explicit. If you apply a magnetic field, all the spinswill want to align themselves with the field, for example all pointing up. If you do this at hightemperature and then cool the metal down, the spins will be frozen in the up position, and you get abar magnet, something where all the little spins add up to a macroscopic magnet. For many metalsthe spin directions aren’t frozen even at low temperature, and quickly become disordered, which iswhy permanent magnets tend to be iron. If you heat up the metal, though, then the spins will againbecome disordered and the total magnetization will go to zero, that is on average there will be asmany ↑ as ↓ atoms.


With our statistical mechanics hats on, we should ask: why is the configuration with equal numbersof ↑ as ↓ atoms the most likely? Consider again Fig. 2.1, which shows the binomial coefficientsΩN (n1) defined in Eq. (2.3) as a function of the number n1 = n↑ of ↑ atoms for a given totalnumber N . Recall that each of these numbers corresponds to the number of microstates in a givenmacrostate. It is clear from the figure that the largest macrostate corresponds to n↑ = N/2, i.e.where there are equal numbers of up and down spins. The most populated macrostate is also theone the system is most likely to be in.

If another magnet that was at the same temperature – though I am sweeping issuesof temperature under the rug at the moment – was brought into contact with the first (totallydisordered) magnet, then what would happen? Suppose that the first magnet has NA spins in it,and the second has NB spins. The most likely macrostates for each system separately correspond to

ΩA =NA!

[(

NA

2

)

!]2 ; ΩB =

NB![(

NB

2

)

!]2 , (2.8)

i.e. the configurations with equal numbers of spin up and down atoms. Just before contact, themacrostate for the total system is the product of the two, Ωtotal = ΩAΩB. Just after contact, itbecomes

Ωtotal =(NA +NB)![(

(NA+NB)2

)

!]2 . (2.9)

This corresponds to a total system with N = NA + NB atoms, half of which are in spin up andthe other half in spin down. Let’s put in some numbers for concreteness. Suppose each system hasNA(B) = 10, so that ΩA(B) = 252. Then before contact we have ΩAΩB = 63 504, which is a prettybig number. But after contact we have instead Ωtotal = 184 756, which is almost three times bigger!Even if we had artificially managed to keep all the spins aligned in system B prior to contact, thestate with the largest occupation of the total system has a mixture of ups and downs in total.

The above example illustrates that nature will adjust itself to a new environment by spontaneously‘flowing’ to another state which has a higher ‘multiplicity’ of the macrostate. This is one perspectiveon the Second law of thermodynamics: systems will flow toward a state with the largest numberof microstates. As we will see in the next few sections, these concepts are closely related to theincrease of entropy in the universe. There is also an apparent paradox in the above example: thespins in system A need to somehow ‘know’ about the spins in system B in order to maximize thevalue of Ω, because Ωtotal > ΩAΩB. What do you think that this means?

2.4 Large Systems

2.4.1 Discrete Random Walks

Random (or drunken) walks underpin many important algorithms in physics and computer science,among other things. They also provide a nice description of how classical particles diffuse towardsome kind of equilibrium.

Suppose that there is a walker that at every step will randomly choose between moving right or left.Maybe she flips a coin and moves one step to the right if it comes up heads, and left otherwise.The question is: what is the probability of finding her m steps away after N steps are taken? Put


another way, if she is a very precise drunk, so that the length of step is always exactly ℓ, then whatis the probability of finding her a distance mℓ from where she started?

We know that N = n1+n2, where n1 and n2 are the number of steps to the right an left, respectively.Evidently, the distance after N steps is given by the difference, m = n1 − n2. These two equationscan be inverted, n1 = 1

2 (N +m) and n2 = 12 (N −m), so that the probability of being m steps away

is

PN (m) =N !

n1!n2!pn1qn2 =

N ![

12 (N +m)

]

![

12 (N −m)

]

!p

1

2(N+m)q

1

2(N−m).

This is not very pretty. But if p = q = 12 , then

p1

2(N+m)q

1

2(N−m) =

(

1

2

)N

and the result becomes a bit nicer,

PN (m) =N !

n1!n2!pn1qn2 =

N ![

12 (N +m)

]

![

12 (N −m)

]

!

(

1

2

)N

.

It’s nice to know some other things, like the mean number of steps taken to the right, and thestandard deviation. Let’s calculate these now.

n1 =N∑

n1=0

p(n1)n1 =N∑

n1=0

N !

n1!(N − n1)!pn1qN−n1n1 (2.10)

=

N∑

n1=0

N !

n1!(N − n1)!qN−n1 [n1p

n1 ]

=

N∑

n1=0

N !

n1!(N − n1)!qN−n1

[

pd

dp(pn1)

]

(2.11)

= pd

dp

N∑

n1=0

N !

n1!(N − n1)!qN−n1pn1

= pd

dp

[

(p+ q)N]

from binomial theorem

= pN(p+ q)N−1 = pN. since p+ q = 1.

So the mean number of steps to the right is just the total number of steps times the probabilityof taking a step to the right. No surprises here! Obviously we also know that n2 = qN and som = N(p− q).

Now let’s calculate the standard deviation (∆n1)2 = n2

1 − n12. Because we already know n1, we

only need to obtain n21. But now, instead of a factor of n1 appearing in the expression (2.10), there

is now a factor of n21. It would be nice to use a derivative trick like in line (2.11) above. Using

d

dp(pn1) = n1p

n1−1 andd2

dp2(pn1) = n1(n1 − 1)pn1−2


one gets

p2d2

dp2(pn1) = (n2

1 − n1)pn1 ⇒ n2

1pn1 = p2

d2

dp2(pn1) + n1p

n1 .

So right away the result can be written down:

n21 = p2

d2

dp2[

(p+ q)N]

+ pN

= p2[

N(N − 1)(p+ q)N−2]

+ pN

= p2N(N − 1) + pN.

So(∆n1)

2 = n21 − n1

2 = p2N2 − p2N + pN − p2N2 = Np(1− p) = Npq.

The square root of the standard deviation is therefore ∆n1 =√Npq. The relative width of

the distribution is therefore

∆n1

n1=

√Npq

Np=

1√N

√

q

p=

1√N

when p = q.

The upshot is that as the number of steps N increases, n1 also increases but the relative width ofthe distribution actually sharpens like N−1/2.

2.4.2 Continuous Random Walks

What is the limiting distribution as the number of steps approaches infinity (N → ∞), but thestep size goes to zero? This transforms the discrete random walk discussed above into a continuous

walk. The problem, though, is that when N gets huge, factors like N ! and (N −n)! appearing in thebinomial distribution above get insanely huge, and it becomes difficult to know what the result isof dividing them. One way to get around the problem is to take the logarithms of both sides, sincelogs of insanely huge numbers are often tractable large numbers. Here goes:

ln [PN (n1)] = ln

[

N !

n1!(N − n1)!pn1qN−n1

]

= ln(N !)− ln(n1!)− ln[(N − n1)!] + n1 ln(p) + (N − n1) ln(q).

Now use Stirling’s formula to handle the factorials,

D Stirling’s formula: ln(n!) ≈ n ln(n)− n for n≫ 1.

One then obtains

ln [PN (n1)] = N ln(N)−N−n1 ln(n1)+n1−(N−n1) ln[(N−n1)]+N−n1+n1 ln(p)+(N−n1) ln(q).

And so

d

dn1ln [PN (n1)] = − ln(n1)− 1 + ln(N − n1) + 1 + ln(p)− ln(q) = ln

[

N − n1

n1

p

q

]

. (2.12)


D The most probable value for n1 is designated n1 and is defined by

d

dn1ln [PN (n1)] = 0.

Using this givesN − n1

n1

p

q= 1 because ln(1) = 0.

Straightforward manipulation gives n1 = Np = n1, as expected.

What is the probability PN (n1) away from the most probable value n1? Taylor series around n1 = n1:

ln [PN (n1)] = ln [PN (n1)] + (n1 − n1)d

dn1ln [PN (n1)]n1=n1

+(n1 − n1)

2

2

d2

dn21

ln [PN (n1)]n1=n1+ . . .

⇒ PN (n1) ≈ PN (n1) exp

(n1 − n1)2

2

d2

dn21

ln [PN (n1)]n1=n1

.

The result (2.12) can be used to evaluate the mess in the curly brackets:

d2

dn21

ln [PN (n1)] = − 1

n1− 1

N − n1.

At n1 = n1,

d2

dn21

ln [PN (n1)] = − 1

Np− 1

N −Np

= − 1

Np− 1

Nq

= −p+ q

Npq= − 1

Npq

So the second derivative is negative, indicating that n1 = n1 is a maximum. Does this expressionlook familiar?

So finally one obtains the expression for the probability distribution of a continuous random walk:

PN (n1) = PN (n1) exp

(n1 −Np)2

2

−1

Npq

=1

√

2π(∆n1)2exp

−(n1 − n1)2

2(∆n1)2

, (2.13)

where the prefactor of the exponential is the normalization constant, since

∫

dn1PN (n1) ≡ 1.


Fig. 2.1 shows that the distribution of probabilities (assuming p = q = 12 ) gets closer to a Gaussian

as N increases.

The continuous random walk discussed above assumes that space is continuous, but the number ofsteps taken N is still discrete. We can think of the number of steps as being equivalent to time:each step takes a certain amount of time, after all! What does the random walk look like whenboth space and time are continuous? If the probability of hopping left or right is the same, then theprobability of being at the site m on the nth step, Pn(m), satisfies the equation

Pn+1(m) =1

2Pn(m− 1) +

1

2Pn(m+ 1).

So we also know that

Pn+1(m)− Pn(m) =1

2Pn(m− 1) +

1

2Pn(m+ 1)− Pn(m). (2.14)

Now, if the distance travelled is x = ma with a the stepsize, and the elapsed time is t = nτ with τthe time needed to make a step, then we can convert this discrete equation to a continuous equationwhen a → 0 and τ → 0, and the probability Pn(m) is transformed into P (x, t). To make furtherprogress we need to remember how derivatives are defined:

∂P (x, t)

∂t≡ lim

τ→0

P (x, t+ τ) − P (x, t)

τ,

and∂2P (x, t)

∂x2≡ lim

a→0

P (x+ a, t) + P (x− a, t)− 2P (x, t)

a2.

Comparison of these expressions with Eq. (2.14) gives

τ∂P (x, t)

∂t=a2

2

∂2P (x, t)

∂x2

or alternatively∂P (x, t)

∂t= D

∂2P (x, t)

∂x2

where the diffusion constant D ≡ a2/2τ . Thus, diffusion of a gas is exactly the same as the gasdoing a random walk!

2.4.3 (maybe) Quantum Walks and Quantum Computation

The discrete-time quantum walk on the line behaves much as the classical version, except thatinstead of the walker choosing to move right or left at each step, she moves in both directionssimultaneously, with some probability amplitude. A simple approach is to allow the walker to carrya ‘quantum coin,’ which can be put into a superposition of heads and tails. So the walker needs tokeep track of both her coin coordinates, and her spatial coordinates. The coin states will be encodedin a spin degree of freedom, with |0〉C and |1〉C representing heads and tails, respectively. On a 1Dlattice with M +1 points, her spatial coordinate will be encoded in the vector |j〉S , 0 ≤ j ≤M . So,the total wavefunction can be written

|ψ〉 =1∑

σ=0

M∑

j=0

ασj |σ〉C ⊗ |j〉S .


The quantum walk is effected by repeated application of the operator U , which consists of first ap-plying the Flip operator F on the coin, and then shifting the walker with S in a direction conditionalon the spin state: right if |0〉C and left if |1〉C . For N steps, the resulting operator is U = [S(F⊗I)]Nwhere I is the M -dimensional identity operator and

F = [cos(θ)|0〉C + sin(θ)|1〉C ] 〈0|C + [− sin(θ)|0〉C + cos(θ)|1〉C ] 〈1|C ;

S =∑

i

(|0〉C〈0|C ⊗ |j + 1〉S〈j|S + |1〉C〈1|C ⊗ |j − 1〉S〈j|S) ,

where θ is some angle defining the fairness of the coin, with θ = π/4 giving the balanced coin. (Notethat in 1D, a real coin operator is completely general, but this is not the case for more interestinggraphs). Because the coin operator isn’t really randomizing (the operation is perfectly unitary andthe evolution of the wavefunction is coherent), I prefer to call this a ‘quantum walk’ rather than a‘quantum random walk.’ The analysis of this discrete-time quantum walk is a bit involved, so I’llsimply show the results in Fig. 2.2 for the random and quantum walks after 20 timesteps. In bothof these cases, I chose a balanced coin and started the walks with full probability at the centralvertex, and for the quantum case the initial coin state was chosen to be |0〉C + i|1〉C which gives anice symmetric pattern.

0 10 20 30 40vertex

0

0.05

0.1

0.15

0.2

prob

abili

ty

Figure 2.2: Comparison of the discrete-time random walk and quantum walk on a line. The solidand dashed lines show the probability of finding the classical and quantum walkers at a given vertex,respectively, after 20 timesteps.

It is clear that the probability distributions for the two walks are markedly different. The classical


distribution remains peaked at the origin, and the half-width at half-max is around 5 ≈√20 lattice

spaces. In contrast, the quantum probability distribution is strongly peaked away from the center,and the spread is much greater. Fig. 2.3 compares the rms width of the resulting distributions asa function of the number of steps. While the classical rms value spreads like the square root ofthe number of steps so that ∆j ∼

√N or alternatively ∆x ∼

√t where x and t are position and

time respectively, the quantum value spreads linearly in the number of steps,√

〈x2〉 ∼ t. Thus, inprinciple the quantum walk provides a quadratic speed-up in the ability to access a given vertex overthe classical random walk; that is, starting the walker from the central vertex, a quantum walk willgive a high probability of hitting the vertex at either end quadratically faster than will a randomwalk.

This polynomial speed-up is a bit misleading, though. There is a classical strategy for reaching oneof the end vertices starting from the center that also scales linearly with the number of steps. It issimply to move in only one direction! This is a cautionary note when discussing speed-ups usingquantum walks: there can be a classical algorithm that looks nothing like a random walk that canstill perform as well.

0 2 4 6 8 10 12 14 16 18 20timestep

0

2

4

6

8

10

√⟨x2 ⟩

Figure 2.3: Comparison of the rms spread for the discrete-time random walk (solid line) and quantumwalk (dashed line).


2.5 (This section skipped)

2.6 Entropy

Entropy is the fundamental postulate of statistical mechanics (at least, after the ergodicprinciple!). Boltzmann suggested that the entropy of a system S is somehow related to the probabilityof being in a set of microstates, so that entropy and equilibrium are closely related concepts. Acompletely independent formulation by Shannon links the concept of entropy to information capacityof a channel. Yet a third concept of entropy is that of the purity of quantum states. We’ll exploreall these ideas in detail now.

2.6.1 Boltzmann

Suppose S = φ(Ω), i.e. the entropy is some function of the number of microstates in a macrostate.How to determine the function φ(Ω)? Consider two independent systems A and B so that SA =φ(ΩA) and SB = φ(ΩB). The combined system obviously has entropy SAB = φ(ΩAB). Boltzmannassumed that entropy is an additive quantity, i.e. that the entropy of the joint system is simply thesum of the entropies of each:

φ(ΩAB) = φ(ΩA) + φ(ΩB).

But we already know that ΩAB = ΩAΩB. Why? Because probabilities for compound events arealways multiplicative: the total macrostate probability P = ΩABPAB = PAPB = ΩApAΩBpB =ΩAΩBpApB. But pApB = pAB which immediately yields ΩAB = ΩAΩB. This leaves us with theimportant behaviour for φ:

φ(ΩAB) = φ(ΩAΩB) = φ(ΩA) + φ(ΩB).

The only function that has these properties is the logarithm, so φ(Ω) ∝ log(Ω), or

S = kB ln(Ω)

where kB is Boltzmann’s constant (in principle we don’t yet know what value this takes). Thisequation underlies all of statistical mechanics.

At this point you might object to the cavalier way in which I simply substituted a log with aln. There’s nothing about the previous discussion that can tell the difference, though. After all,ln(x) = loge(x) = log10(x)/ log10(e) ≈ 2.3 log10(x). So changing the base of the log just gives aconstant factor. If all of statistical mechanics were in base-10 instead of base-e, then this wouldchange the value of Boltzmann’s constant, but that’s about it. The nice thing about working withlns instead of logs is that the inverse of the former is an exponential which is nice to work with. Justa bit below you’ll encounter situations where base-2 is more natural.

Recall that n1 in the random walk example considered earlier was found by (d/dn1) ln[PN (n1)] = 0,i.e. by asking where is the maximum probability of finding n1. But ln[PN (n1)] ∝ ln[ΩN (n1)] ∝ S.So the most likely event is determined by maximizing the entropy. If you believe the theorem thatthe state of the system is the macrostate with the largest number of microstates, then this yieldsanother form of the Second law of thermodynamics, namely that the equilibrium state of a


system is found by maximizing the entropy. Maybe this gives a clue as to why entropy mustkeep increasing with time. . . .

Let’s now return to a general system where any given trial can have m outcomes, Eq. (2.5). Recallthat m = 2 for a coin, m = 6 for a die, etc. What is the explicit expression for the entropy?

ln(ΩN ) = ln

(

N !∏m

i=1 ni!

)

= N ln(N)−N − ln

(

∏

i

ni!

)

using Stirling’s formula since N ≫ 1

= N ln(N)−N −∑

i

[ni ln(ni)− ni] if ni ≫ 1 ∀ni

= N ln(N)−N −∑

i

[Npi ln(Npi)−Npi] maximizing ΩN gives ni = Npi

= N ln(N)−N −N ln(N)∑

i

pi −N∑

i

pi ln(pi) +N∑

i

pi but∑

i pi = 1

= −N∑

i

pi ln(pi).

So,

S = −NkB∑

i

pi ln(pi). (2.15)

Let’s also calculate the entropy for an ideal gas of N monatomic atoms in a box of volume V , toshow how this alone can yield the equation of state. First we need to find Ω: how many ways can wedistribute the atoms in the box? A crude method is to subdivide the box into M boxlets of volume∆V each, so that M∆V = V . So, there are M ways of distributing each atom within the volume.Assuming that an arbitrary number of atoms can be in each boxlet (this is the definition of an idealgas), then the total number of ways to distribute N atoms is

Ω =

(

V

∆V

)N

.

This immediately yields S = kB lnΩ = NkB ln(V/∆V ). If we were to change the volume, but keepthe boxlet size fixed, then ∆S = Sf − Si = NkB ln(Vf/∆V )−NkB ln(Vi/∆V ) = NkB ln(Vf/Vi) ≡NkB ln(V/Vi), which is independent of the arbitrary choice of boxlet size ∆V . This immediatelyimplies the existence of the following thermodynamic relation: P = T (∂S/∂V ), because from it wewould immediately obtain the equation of state for the ideal gas, P = NkBT/V . Justification ofthis equation, though, will have to wait for Sec. 3.1.

2.6.2 Shannon Entropy

This expression for S above was ‘derived’ using pretty hokey assumptions. It would be nice ifsomething so fundamental to statistical mechanics could be obtained in a more satisfying way. Infact, there is an information theoretic way to think about entropy that was developed completelyindependently by Claude Shannon in the mid-twentieth century.


Suppose that we have a radio antenna that broadcasts digital signals encoded in the bitstrings X1,X2, . . . , Xm, where for example X1 = 0110010111 . . .. The question Shannon posed is: what are theresources required to represent all of these bitstrings? An alternative way of asking the question is:how much memory is required to store all of the bitstrings? Shannon found that the quantity H(X),now called the Shannon entropy, gives the mean number of bits per string required. Formally, ifthe probability of the antenna to produce bitstring Xi is pi, then the Shannon entropy is

H(X) = H(p1, p2, . . . , pm) ≡ −m∑

i=1

pi log2(pi). (2.16)

D Shannon Entropy formally quantifies resources required to store information. Alternatively,it measures the uncertainty about the value of bitstring X before the measurement is made. Notethat the information content doesn’t depend on what the actual variables are (could be coins, dice,drunks, spins, etc.)

Example (handout in class)

D Shannon’s Noiseless Coding Theorem states that the Shannon entropy gives the lower boundon the ability to compress information.

Clearly, the Shannon entropy for a balanced coin (aka a true coin, where the probability of comingup heads is 1

2 ) is

Hcoin = −1

2log2

(

1

2

)

− 1

2log2

(

1

2

)

= log2(2) = 1,

so that only one bit need be used to represent the behaviour of the coin. In other words, we canencode heads as a 0 and tails as a 1.

But the Shannon entropy for a balanced die is

Hcoin = −1

6log2

(

1

6

)

× 6 ≈ 2.585 < 3

And so using three bits to represent six numbers is sub-optimal, because two words (110 = 7 and111 = 8) go unused.

Let’s return to general binary systems (unfair coins), where the probability of obtaining one result(heads) is p and the other (tails) is q = 1− p. The associated binary entropy is concave, that is:

Hbin [px1 + (1− p)x2] ≥ pHbin(x1) + (1 − p)Hbin(x2),

where 0 ≤ p, x1, x2 ≤ 1. This concavity can be most easily seen graphically by plotting Hbin as afunction of p:Example What can this concavity of entropy tell you? Suppose that Alice has both a U.S. anda Canadian quarter, but both have been rigged so that the probabilities of obtaining heads are pUand pC, respectively. Alice’s friend Bob knows that she tends to flip the U.S. coin with probability qand the Canadian coin with probability 1− q. Alice then tells Bob the result of her coin toss (headsor tails).

Q How much information has Bob gained, on average?


0 0.2 0.4 0.6 0.8 1p

0

0.2

0.4

0.6

0.8

1H

bin

Binary Entropy Hbin(p)

A Bob gets the result, and information about which coin was flipped. Suppose pU = 13 and pC = 5

6 .If heads come up, Bob can say that it was more likely to have been a Canadian coin that was flipped,that is:

Hbin [qpU + (1− q)pC ] ≥ qHbin(pU ) + (1 − q)Hbin(pC).

The concavity of binary entropy also tells us how certain we can be about the fairness of the coin.An entropy of zero implies p = 0 or p = 1 (note that 0 log(0) = 0), corresponding to a completelyunfair coin that is either two-headed or two-tailed. An entropy of one corresponds to a perfectlybalanced coin with p = 1

2 . So the entropy gives a measure of how uncertain the result of a givencoin toss will be: when Hbin = 1, we are most uncertain about the result. The same is true of Hand S in general: they provide a measure of the uncertainty in the result of testing the system (i.e.the events resulting from a series of trials on compound objects). This is why you might have heardthat entropy is a measure of uncertainty.


2.6.3 (maybe) von Neumann Entropy

So far, we have seen that entropy gives a measure of the number of ways of classifying compoundevents (like enumerating results of coin tosses, or distributing atoms in space), and a measure ofthe resources required to represent a certain quantity of information (or the information gained onmaking a series of measurements). In fact, entropy also gives a measure of the entanglement of aquantum system and of how mixed it is.

Suppose Alice has a quantum mechanical coin, where the state |0〉 represents heads and |1〉 representstails. Each of these states are called basis vectors. Unlike classical coins, quantum coins can be ina state that is simultaneously heads and tails, with different weights in each. If Alice’s coin is in apure state, then it can be written as a single wavefunction |ψA〉 = (a|0〉+ b|1〉), where it is assumedthat the wavefunction is normalized, |a|2 + |b|2 = 1.

Before discussing quantum mechanical entropy, it’s important to introduce the concept of the den-sity matrix ρ. If Alice’s coin is in the pure state above, then her density matrix is

ρA = |ψA〉〈ψA| = (a|0〉+ b|1〉) (a∗〈0|+ b∗〈1|)

= |a|2|0〉〈0|+ ab∗|0〉〈1|+ ba∗|1〉〈0|+ |b|2|1〉〈1| =(

|a|2 ab∗

ba∗ |b|2)

.

In general, though, her density matrix might not be derivable from a single pure-state wavefunctionat all, so that it would be a complete mixture of heads and tails:

ρ = α|0〉〈0|+ β|0〉〈1|+ γ|1〉〈0|+ δ|1〉〈1| =(

α βγ δ

)

.

Pure and mixed states can be distinguished as follows. First, diagonalize the density matrix; i.e.find its eigenvalues and write them on the diagonal. Then sum up all the diagonal elements. This iscalled taking the trace, and is denoted Tr(ρ). Both pure and mixed states have Tr(ρ) = 1 but onlypure states also satisfy Tr(ρ2) = 1; mixed states have Tr(ρ2) < 1.

Let’s work out some examples. If |ψA〉 = |0〉, then

ρ = |0〉〈0| =(

10

)

( 1 0 ) =

(

1 00 0

)

; ρ2 =

(

1 00 0

)

; Tr(ρ) = Tr(ρ2) = 1.

If |ψA〉 = (|0〉+ |0〉) /√2, then

ρ =1

2(|0〉+ |1〉) (〈0|+ 〈1|) = 1

2

(

11

)

( 1 1 ) =1

2

(

1 11 1

)

= ρ2.

Diagonalizing this density matrix gives eigenvalues 0 and 1, which again gives Tr(ρ) = Tr(ρ2) = 1.But the mixed state

ρ =1

2(|0〉〈0|+ |1〉〈1|) = 1

2

(

1 00 1

)

; ρ2 =1

4

(

1 00 1

)

has Tr(ρ) = 1 while Tr(ρ2) = 12 .

D The von Neumann Entropy of a binary quantum state described by density matrix ρ is

S(ρ) = −Tr (ρ log2 ρ) .


For all pure states, S(ρ) = 0 and for mixed states 0 < S(ρ) ≤ 1. For the mixed state in the exampleabove, S(ρ) = 1, so this state is maximally mixed or maximally uncertain. So the von Neumannentropy gives a measure of the purity of a quantum state.

Now suppose Bob has a quantum coin described by the wavefunction |ψB〉 = (c|0〉+ d|1〉), where|c|2 + |d|2 = 1. If Alice has her pure-state wavefunction, then the total wavefunction of both partiesis said to be separable, |ψ〉 = |ψA〉|ψB〉, because it can be written as a product of two wavefunctionsthat you can ‘take apart’ without damaging either one. The combined state of the system is

|ψ〉 = |ψA〉|ψB〉 = (a|0〉+ b|1〉) (c|0〉+ d|1〉)= (ac|00〉+ ad|01〉+ bc|10〉+ bd|11〉)

Clearly, if the combined state were instead

|ψ〉 = (α|00〉+ β|01〉+ γ|11〉) , |α|2 + |β|2 + |γ|2 = 1

there would be no way to separate the components into two independent wavefunctions belongingto Alice and Bob. This is an entangled state.

A maximally entangled pure state that Alice and Bob can share is

|ψ〉 = 1√2(|00〉+ |11〉) . (2.17)

If Alice measures her coin and finds that it is in state |0〉, then Bob is guaranteed to also measure|0〉. Note that this result doesn’t depend on who does the measuring first. Also, they would need tocommunicate classically to compare notes to detect the correlated results. So, there is no violationof causality here. This is the famous EPR paradox of early quantum mechanics.

The combined state shared by Alice and Bob also can be written in the form of a density matrix,but now it will be a 4×4 matrix instead of a 2×2. Now the question is: what information does Alicehave about the state of her coin after Bob measures the state of his coin? Measurement is carriedout by Bob tracing over his part of the two-coin density matrix, so that Alice’s density matrix isdefined as ρA ≡ TrB(ρ). The von Neumann entropy of this two-coin system is defined as

S(ρ) = −Tr (ρA log2 ρA) = −Tr (ρB log2 ρB) ,

so that the entropy measure doesn’t depend on whether Alice or Bob measures their coin statefirst. It’s easy to check that separable pure states shared by Alice and Bob give zero von Neumannentropy. Suppose |ψA〉 = |0〉 while |ψB〉 = |1〉 for simplicity. Then |ψ〉 = |01〉 and ρ = |01〉〈01|.Tracing over Bob’s state is accomplished by turning his information ‘inside out,’ that is by turninghis outer products into inner products,

TrB(ρ) = (|0〉〈0|) (〈1|1〉) = |0〉〈0| =(

1 00 0

)

,

which has zero von Neumann entropy because log(1) = 0 and 0 log(0) = 0. But the maximallyentangled state (2.17) gives an entropy of one:

ρ =1

2(|00〉〈00|+ |00〉〈11|+ |11〉〈00|+ |11〉〈11|) ;


ρA = TrB(ρ) =1

2[(|0〉〈0|) (〈0|0〉) + (|0〉〈1|) (〈0|1〉) + (|1〉〈0|) (〈1|0〉) + (|1〉〈1|) (〈1|1〉)]

=1

2[|0〉〈0|+ |1〉〈1|] = 1

2

(

1 00 1

)

because 〈0|1〉 = 〈1|0〉 = 0.

But this has the same form as the mixed-state example above, so that S(ρ) = 1. So, the maximallyentangled pure state also has an entropy of one! The von Neumann entropy also provides ameasure of the entanglement of a pure bipartite quantum state.

So, why is the entropy of the universe steadily increasing??

Chapter 3

Interactions and Implications (akaEquilibrium)

3.1 Temperature

Consider two isolated systems A and B, characterized by UA and UB, respectively. When theseare brought together, dS ≥ 0. This is because Ω is maximized at equilibrium, so ln(∆Ω) ≥ 0 ord ln(Ω) ≥ 0, and we have already justified that S = kB ln(Ω). This means that

dS =∂SA

∂U∆UA +

∂SB

∂U∆UB ≥ 0.

Because this is a closed system, ∆UA = U(f)A − U

(i)A ≡ Q and ∆UB = U

(f)B − U

(i)B ≡ −Q. Using this

we obtain(

∂SA

∂U− ∂SB

∂U

)

Q ≥ 0.

What does this mean? Suppose Q > 0, so that energy moves into system A. We know then that

∂SA

∂U≥ ∂SB

∂U.

But we also know from ‘everyday experience’ that if energy (heat) flows into system A, then system

B must have initially been hotter than system A, i.e. T(i)A ≤ T

(i)B . It therefore seems reasonable

to make the association ∂SA/∂U ∝ 1/T(i)A . In fact we can take the analogy further and define

temperature this way:1

T≡ ∂S

∂U=

∂

∂UkB ln (Ω) . (3.1)

Note that in taking the partial derivative one must ensure that the number of particles remains fixed,as does any variable that would be associated with work (like volume, magnetic field, etc.). Also,I haven’t proven that this definition is consistent with other things that we now know. We’ll seebelow that if I know the explicit form for the entropy in terms of the internal energy U , it yields anexpression for the temperature consistent with expectations. Before proceeding further, it is usefulto introduce the new parameter β ≡ 1/kBT , which has units of inverse energy, that is widely used

32


in statistical mechanics. Then Eq. (3.1) reads

β ≡ ∂ ln (Ω)

∂U. (3.2)

Unfortunately, an expression for S that is explicitly dependent on U is not always easy to comeby! For this reason, it is often simpler to obtain expressions that relate S to other thermodynamicvariables, such as volume, magnetic field, etc. So far, we know that temperature, or rather β, isrelated to the change in the entropy (or alternatively the change in the number of states in themacrostate Ω) as the energy of the system is changed. But suppose that U and Ω also depend onsome other variable x, so that Ω = Ω(U, x) and U = U(x). Then

∂ ln(Ω)

∂x=∂ ln(Ω)

∂U

∂U

∂x≡ X

∂ ln(Ω)

∂U= βX,

where I have defined a generalized force X as

X ≡ ∂U

∂x=

1

β

∂ ln(Ω)

∂x= T

∂S

∂x. (3.3)

A simple example of a generalized force would be pressure. Since the pressure changes as the volumechanges, it seems reasonable that the generalized force associated with the variable x = V will bethe pressure X = P , i.e.

P =1

β

∂ ln(Ω)

∂V= T

∂S

∂V. (3.4)

Let’s use this idea to obtain the equation of state of an ideal gas, as was done in Section 2.1. Wehad

S = kB ln(Ω) = NkB ln

(

V

∆V

)

.

Using the definition of the pressure (3.4), we have

P = T∂S

∂V= NkBT

∂

∂V

[

ln

(

V

∆V

)]

=NkBT

V,

as expected. Note that this results also does not depend on the arbitrary choice of ∆V . Also thisis the first hint that our definition of the temperature (3.1) was probably o.k., since we correctlyreproduced the ideal gas law.

You might be interested to learn that the correspondence between a generalized variable x and itsassociated generalized force X has close parallels with the idea of ‘canonically conjugate variables’in classical and quantum mechanics. For example, position and momentum, or time and energy,are conjugate variables much like volume and pressure. Together, they constitute what is knownas ‘phase space.’ In statistical mechanics, Eq. (3.3) indicates that inverse temperature and energyare the conjugate variables. In fact, there is a close relationship between inverse temperature andimaginary time that might be surprising: real gases with density ρ(r, t) diffuse through space in timethrough the diffusion equation that looks something like this:

∂

∂tρ(r, t) = D∇2ρ(r, t),


where D is the diffusion coefficient, which measures how quickly particles diffuse. As you mightexpect, D increases with temperature. Meanwhile, the Schrodinger equation for the motion ofquantum mechanical particles is

ih∂

∂tψ(r, t) = −h ∂

∂(it)ψ(r, t) = − h2

2m∇2ψ(r, t),

where now ρ(r, t) = |ψ(r, t)|2. This looks much like the diffusion equation in imaginary time tif it → t. A more rigorous correspondence between t and β−1 requires quantum field theory,unfortunately. I will come back to this point in Chapter 6, but in the meantime I hope that Ihave given you a further hint at the relationship between (the arrow of) time, temperature, andentropy. . . .

3.2 Entropy, Heat, and Work

3.2.1 Thermodynamic Approach

Suppose now that the system A comes into contact with a heat reservoir B, where there are manymore microstates in B than in A. Then the change in β of system B if it absorbs heat Q from A isgoing to be tiny,

∣

∣

∣

∣

∂βB∂UB

Q

∣

∣

∣

∣

≪ βB.

Assuming ∂βB/∂UB ∼ βB/UB, then the inequality becomes βBQ/UB ≪ βB or Q/UB ≪ 1. So nowwe can evaluate the change in the entropy due to the addition of the heat Q by expanding in aTaylor series around small Q:

∆S = kB ln[ΩB(UB +Q)]− kB ln[ΩB(UB)] ≈ QkB∂ ln(ΩB)

∂UB+Q2

2kB

∂2 ln(ΩB)

∂U2B

+ . . .

= kBQβB + kBQ2

2

∂βB∂UB

+ . . .

≈ kBQβB + kBQ2

2

βBUB

= QkBβB

(

1 +Q

2UB

)

≈ Q

T

So, Q = T∆S or

dS =Q

T(3.5)

for infinitesimal variations. This expression was in fact the thermodynamic definition for entropyintroduced by Rudolf Clausius (sounds very Christmas!) in 1865, as the thing that changes by Q/Twhen heat is added to a system at temperature T .

The textbook has a much simpler derivation. Since dS = dU/T then if you add heat Q but don’tdo any work dU = Q so that dS = Q/T . But this is really unsatisfactory, because in fact dS = Q/Teven if the volume is changing, as long as the reservoir remains a reservoir during the process. Thetextbook states that the process needs to be ‘quasistatic’ but doesn’t explain what this actuallymeans.

This is the first time I have mentioned the idea of a reservoir, and you might be curious about it.Basically, a reservoir is some exremely large system that can absorb or release energy, particles, etc.


without affecting its own macroscopic properties at all. In the present context it is used to fix thetemperature/internal energy of the smaller system A. This is the spirit of the MicrocanonicalEnsemble: the system of interest (A) has a fixed energy and number of particles, set to the valueof some well-defined reservoir.

The Third Law of Thermodynamics states that as T → 0+, S → S0, i.e. that the entropyapproaches a limiting value as the temperature goes to absolute zero.

3.2.2 Statistical Approach

Suppose that some system exhibits periodic motion, i.e. that a particle moves so that it comes backto where it started. In classical mechanics, there is something called the Action J , that is a constantof the periodic motion. It is defined as:

J =

∮

p dx, (3.6)

where p is the momentum and dx is the element of length. In quantum mechanics, J isn’t justconserved, but it is exactly equal to nh, where h is Planck’s constant and n is an arbitrary positiveinteger. This principle is called Bohr-Sommerfeld quantization, and can be used to obtain theenergy spectrum for a range of interesting quantum systems.

Example: Particle in a Box. The particle is confined in a one-dimensional box of length L,and is travelling from one side to the other with momentum p = mv. When it hits the wall, itbounces back, and so on. The energy is obviously E = 1

2mv2 = p2/2m, so p =

√2mE. The action

is therefore

J =

∫ 2L

0

√2mE dx =

√2mE 2L ≡ nh.

⇒ En(1D) =n2h2

2m(4L2)=n2π2h2

2mL2. (3.7)

Quantum mechanics tells us that the accessible states of a system are quantized: the energy levelsabove for a particle in a box are what heads and tails were for a coin. If we set En = p2n/2m then fromEq. (3.8) one also obtains the quantization of momentum, pn = (πh/L)n. In a three-dimensionalbox of length L, the energy levels are

En(3D) =π2h2

(

n2x + n2

y + n2z

)

2mL2. (3.8)

It is useful to estimate how many energy levels are occupied for air in a one-metre qubed box atroom temperature. Air is mostly made of nitrogen, mN = 14.0067 amu× 1.6605 · 10−27 kg/amu =2.33 · 10−26 kg. With h = 1.05457 · 10−34 J s, I obtain En = 2.36 · 10−42n2 J. The characteristictemperature is simply this divided by Boltzmann’s constant kB = 1.3807 · 10−23 J/K to give Tn =En/kB = 1.7 · 10−19 K. Since room temperature is around 300 K, something like 2 · 1021 states areoccupied! We’ll do a better job of this a bit later.

The above quantum interlude suggests that the accessible states of a physical system are quantizedenergy levels, which I’ll now denote ǫi. So, the mean energy U is given by

U ≡∑

i

niǫi (3.9)


and the mean energy per particle N is

U

N=

1

N

∑

i

niǫi =∑

i

ni

Nǫi =

∑

i

piǫi.

These simple expressions lead us to our first explicit connection between statistical mechanics andthermodynamics:

dU =∑

i

(ǫidni + nidǫi)

=∑

i

ǫidni +∑

i

nidǫi

= Q (heat absorbed by system) +W (work done on system).

Unfortunately, knowing the single-particle energy levels given in a box (3.8) is not enough to directlydetermine the equation of state. For this one needs to know how to construct the entropy andtherefore the population of the macrostate Ω. A rough estimate can be obtained as follows. First,we have pn = (πh/L)n =

√2mEn so that n = (L/πh)

√2mEn. Suppose that each particle accesses

many of these n levels, and that there are N particles overall. Then it seems reasonable to assumethat the number of states in the macrostate is proportional to the number of occupied energy levelsn, which corresponds to the ‘volume in n-space’ with n the radius of the hypersphere:

Ω ∝ 4π

3n3N =

4π

3

(

L

πh

)3N

(2mEn)3N/2

=4π

3

V N

(πh)3N

(2mEn)3N/2

. (3.10)

If we now make the identification En → U then Ω ∝ V NU3N/2. The temperature is now immediatelyfound:

1

kBT=

∂

∂Uln (Ω) =

3N

2

∂ ln(U)

∂U=

3N

2U. (3.11)

Notice that the proportionality factors are unimportant for the present purpose, since only theoverall U -dependence of the entropy is needed. Rearranging immediately gives U = (3/2)NkBT ,which is just the result of the equipartition theorem. Likewise, the pressure can be found from

P = kBT∂

∂Vln (Ω) = NkBT

∂ ln(V )

∂V=NkBT

V, (3.12)

which is simply the equation of state for the ideal gas again.

3.3 Paramagnetism

Suppose that we have N atoms arranged in a crystal. All but the outermost of the electrons in agiven atom are inert, hybridized with those of neighbouring atoms. The outermost electrons havetotal spin J . This means that if the atom has only one outermost electron the total spin would beJ = 1

2 ; if there were two then J = 1; three electrons would give J = 32 , etc. In quantum mechanics

the spin projection mJ can take the values mJ = −J,−J + 1, . . . , J − 1, J . So one electron can givem = 1

2 and − 12 at each site, while two electrons give m = 1, 0, and −1, etc. A lot of quantum

mechanical work shows that in a magnetic field B, the energy of these outermost electrons is

ǫJ = gJµBmJB,


where gJ is the J-dependent Lande g-factor and µB = eh/2me = 9.2741 · 10−24 Am2 is the Bohrmagneton. But you don’t really have to know all this: it is sufficient to assume that ǫJ = cmJ ,where c is some arbitrary constant.

Consider the concrete example of J = 12 , so that there are two energies, ǫ1 = −ε and ǫ2 = ε. If there

are n1 spins with energy −ε and n2 = N − n1 spins with energy ε, then according to Eq. (2.15) wehave

S = −NkB[(n1

N

)

ln(n1

N

)

+(n2

N

)

ln(n2

N

)]

.

But we also can make use of the definition of the mean energy, Eq. (3.9):

U = n1(−ε) + (N − n1)ε = ε(N − 2n1) = Nε

(

1− 2n1

N

)

.

⇒ U

Nε= 1− 2n1

N

⇒ n1

N=

1

2

(

1− U

Nε

)

and obviouslyn2

N= 1− n1

N=

1

2

(

1 +U

Nε

)

.

If we define y ≡ U/nε, then the expression for the entropy becomes

S = −NkB[(

1 + x

2

)

ln

(

1 + x

2

)

+

(

1− x

2

)

ln

(

1− x

2

)]

.

Now what? We have above that dQ = TdS, or 1/T = ∂S/∂U . Using ∂S/∂U = (∂S/∂x)(∂x/∂U) =(1/Nε)(∂S/∂x), we obtain

1

T=

1

Nε(−NkB)

∂

∂x

[(

1 + x

2

)

ln

(

1 + x

2

)

+

(

1− x

2

)

ln

(

1− x

2

)]

= −kBε

[

1

2ln

(

1 + x

2

)

− 1

2ln

(

1− x

2

)

+

(

1 + x

2

)(

1

1 + x

)

−(

1− x

2

)(

1

1− x

)]

= −kB2ε

ln

(

1 + x

1− x

)

.

We’re making progress! Inverting the last line above, we have

1 + x

1− x= exp

(

− 2ε

kBT

)

⇒ 1 + x = (1 − x) exp

(

− 2ε

kBT

)

⇒ (1 + x) exp(βε) = (1− x) exp(−βε) since β = 1/kBT .

Rearranging, one obtains

exp(βε)− exp(−βε)2

= −xexp(βε) + exp(−βε)2

⇒ −x = tanh(βε).


Finally, we are done, because this immediately gives

U = −Nε tanh(

ε

kBT

)

.

Recall that U = n1(−ε) + n2ε = ε(n2 − n1). This means that the net spin of the system is

n1 − n2 =U

ε= N tanh

(

ε

kBT

)

.

What are the limiting values of the net spin in the limit of high and low magnetic field or high andlow temperature? Why?

This isn’t quite the equation of state for the Pauli paramagnet yet, though. The magnetizationM is defined as the net spin times the Bohr magneton, so that

M = µB(n1 − n2) = NµB tanh

(

ε

kBT

)

= NµB tanh

(

g 1

2

µBB

kBT

)

. (3.13)

This is the equation of state for a magnetic system, relating the magnetization to the magneticfield and temperature, rather than relating the pressure to the volume and temperature. Where theenergy levels for a particle in a box depended on volume, giving pressure as the generalized force,now the energies depend on magnetic field, and the generalized force is the magnetization.

3.4 Mechanical Equilibrium and Pressure

Recall that we made a connection between statistical mechanics and thermodynamics by consideringthe mean energy of a system of particles, U =

∑

i niǫi. Also, for a particle in a box, remember thatthe energy levels ǫi were functions of the box volume. So

dU =∑

i

(ǫidni + nidǫi) =∑

i

(

ǫi∂ni

∂SdS + ni

∂ǫi∂V

dV

)

= Q+W

=∂

∂S

(

∑

i

niǫi

)

dS +∂

∂V

(

∑

i

niǫi

)

dV = TdS − PdV.

In obtaining the first half of the final result, I used the fact that 1/T = ∂S/∂U or T = ∂U/∂S. Irecognized that the second term corresponds to the work W = −PdV according to Eq. (1.16). Sothis implies that

P = −∂U∂V

. (3.14)

This is another handy expression for the pressure, in circumstances where Eq. (3.4) is not convenient.Note, however, that in the calculation of the total energy for a 3D gas of particles performed inSec. 3.2.2, the equipartition total energy U = (3/2)NkBT doesn’t explicitly depend on volume, onlyon the temperature! So this equation won’t be useful in this context.


3.5 Diffusive Equilibrium and Chemical Potential

Now suppose that the energy of some small system (I’ll call it a subsystem) also depends on the(fluctuating) number of particles N in it. Then

dU = TdS − PdV +∂U

∂NdN ≡ TdS − PdV + µdN.

This can also be inverted to give

dS =dU + PdV − µdN

T.

So the chemical potential for the system is defined as

µ ≡ ∂U

∂N= −T ∂S

∂N, N =

∑

i∈subsystem

ni.

What does the chemical potential mean? Consider the change in the entropy of the entire system(subsystem plus reservoir) with the number of particles in the subsystem Ns in terms of the changeof the entropies of the reservoir SR and subsystem Ss:

dSsystem =∂SR

∂NsdNs +

∂Ss

∂NsdNs = dNs

(

∂SR

∂NR

∂NR

∂Ns+∂Ss

∂Ns

)

.

But NR = N −Ns so ∂NR/∂Ns = −1 giving

dSsystem = dNs

(

∂Ss

∂Ns− ∂NR

∂NR

)

= −dNs

T(µs − µR) .

Now, equilibrium corresponds to maximizing entropy, or dS = 0. This means that the condition forequilibrium between the subsystem and the reservoir is that the chemical potentials for each shouldbe equal, µs = µR. But even more important, as equilibrium is being approached, the entropy ischanging with time like

dSsystem

dt= −dNs

dt

(

µs − µR

T

)

≥ 0

because the entropy must increase toward equilibrium (unless they are already at equilibrium). Ifinitially µR > µs, then clearly dNs/dt ≥ 0 to satisfy the above inequality. This means that in orderto reach equilibrium when the chemical potential for the reservoir is initially larger than that ofthe subsystem, particles must flow from the reservoir into the subsystem. So the chemical potentialprovides some measure of the number imbalance between two systems that are not at equilibrium.What else does it tell you?

To clarify the various meanings of the chemical potential, let’s return first to the Pauli paramagnet.Recall that in the microcanonical ensemble, the entropy for the spin- 12 case was

S = −NkB[(n1

N

)

ln(n1

N

)

+(

1− n1

N

)

ln(

1− n1

N

)]

.


Assuming that we have some subsystem with number N in contact with a reservoir at temperatureT , the chemical potential is

µ = −T ∂S∂N

= −kBT ln

(

N

N − n1

)

= kBT ln(

1− n1

N

)

= kBT ln(n2

N

)

after a bit of algebra. What does this mean? First, you can see that the chemical potential hasunits of energy. In this case, µ = 0 when the number of spin-up atoms is zero, n1 = 0. The chemicalpotential is less than zero for any other value of n1, and |µ| ≫ 0 for n1 → N . What does µ = 0mean? Suppose that the total number of particles is not zero. Then a zero chemical potential meansthat the change in the number of particles in the reservoir is not related to the change in the numberof particles in the subsystem; alternatively, the entropy is invariant under changes in the numberof particles. This implies that a zero chemical potential means that the system doesn’tconserve the number of particles. For the Pauli paramagnet, I can keep increasing the numberof atoms with spin down, and as long as I don’t creat a single spin up, then the system’s entropydoesn’t change: it remains exactly zero.

We’ll return to the chemical potential again in Chapter 7.

Chapter 4

Engines and Refrigerators

We’re surrounded by machines, devices that consume energy. How is this energy produced? Howdo engines actually work? How can we reduce our environmental impact without sacrificing ourtechnological progress? These are the kinds of questions that will be addressed in this chapter.

4.1 Heat Engines

Engines of various kinds are central to our lives in Canada. It turns out that one can understandmany of the main characteristics of engines without really knowing anything about how they actuallywork. One only really needs to invoke the first and second laws of thermodynamics.

Suppose that we have some engine as shown in Fig. 4.1. Some unspecified device is sandwiched

Figure 4.1: Theoretical heat engine.

between two reservoirs, one at a temperature Th, called the heat ‘source,’ and the other at a lowertemperature Tc ≤ Th, called the heat ‘sink.’ The device (called the engine) pulls heat out ofthe hotter reservoir and dumps it into the colder reservoir. We would like to know how muchpower (‘work’ in thermodynamics language) the engine could provide, simply using considerationsof thermodynamics. We already know from the conservation of energy (aka the First Law) that in

41


principle, we can’t get more energy out than we put in. So it seems reasonable to define the efficiencyas e = |W |/|Qh|, with e = 1 when the work is equal to the heat pulled in, and e = 0 if no workis done at all. I’m using absolute values everywhere because all I care about are the magnitudesof the energies, not about which direction they are flowing. Using conservation of energy we have|Qh| = |Qc|+ |W | so the efficiency is

e =|Qh| − |Qc|

|Qh|= 1− |Qc|

|Qh|. (4.1)

Unfortunately, this isn’t the whole story. The change in the entropy during the process has to bepositive semidefinite (fancy way of saying dS ≥ 0). The entropy ‘from’ the hot reservoir is |Qh|/Th,and that ‘to’ the cold reservoir is |Qc|/Tc. This isn’t very precise language. But mathematically wemust have

dS =|Qc|Tc

− |Qh|Th

≥ 0 ⇒ |Qc||Qh|

≥ TcTh. (4.2)

So the efficiency is

e ≤ 1− TcTh. (4.3)

This is a bit depressing. It says that the only way that one can achieve a perfect engine, even inprinciple, is if the hot reservoir had infinite temperature and the cold reservoir was at absolute zero!For example, the maximum efficiency of a steam engine, based only on the temperature differencebetween boiling and freezing water, would be a mere 27%. So at least 73% of the energy consumedwould be irretrievably lost. In fact, the steam engine doesn’t use only water but also steam, sothe maximum theoretical efficiency of a steam engine is a bit better than this at 48%; this will beexplained at greater length in Section 4.3.

In fact, even from a purely theoretical point of view, the maximum efficiency is significantly worsethan Eq. (4.3) would predict. First, it is never possible to have real reservoirs that exactly maintaintheir temperature near the location where the heat is extracted or dumped; the reservoirs will takesome finite amount of time to equilibrate to the same temperature everywhere, even if it assumed tobe so huge that the final temperature at the end will be the same as it was before. Second, the enginetemperature at the moment the heat is extracted from the hotter reservoir might not be exactly Th,which will lead to an increase in entropy; likewise when it dumps energy into the colder reservoir.And of course entropy could easily be increased by the various (as yet unspecified) processes insidethe engine, and heat could be produced by friction which would either heat the engine, or bereleased to another (uncontrolled) reservoir, changing the energy conservation formula. Given thisset of issues, it’s a wonder that engines exist at all!

You might be interested to know that entropy was discovered experimentally by experimentalists inthe early to mid-19th century who were studying the gases in the context of engine design. At thetime, steam engines were in wide use but their scientific properties were basically unknown. NicolasLeonard Sadi Carnot (1796-1832), in particular, showed that the efficiencies were significantly worsethan the simple relation (4.1) would indicate, and was the first to identify the quantity Q/T asimportant, though its significance was not appreciated for some time after his death. Speakingof death, Carnot died during the cholera epidemic that ravaged Europe, and because of fears ofcontamination most of his scientific writings were buried with him. So unfortunately his generalwork is not that well-known. Clausius picked up the ball and coined the term ‘entropy’ in 1865, but


nobody understood what it really was until Boltzmann came along and developed a full statisticaltheory of it in 1877.

To make further theoretical progress (I’ll give some practical advice for engine building later), it isimportant to at least figure out how to design the most efficient engine, subject to the constraint (4.3).A crucial concept is reversibility. We saw in Section 3.1 that if two systems are in thermal contact,then the flow of heat Q from the hotter one to the cooler one is always associated with an increase ofentropy. The only time the entropy change is zero is if the temperatures are the same. But supposethat the temperature difference was only infinitesimal. The heat could flow with only a negligibleincrease in entropy. Likewise, if the reservoir temperatures were suddenly reversed the heat wouldflow in the opposite direction, still without significantly increasing the total entropy. Here’s theofficial definition:

D: Reversible Process: Process that can be reversed by changing the conditions only infinitesi-mally.

So, if we had a reversible engine, the inequality in Eq. (4.3) would turn into an equality, and thiswould be our best-case scenario. The engine is made up of a gas, and the process is called theCarnot Cycle. It runs like this:

1. Set the temperature of the gas Tgas <∼ Th;

2. Keep the gas at Tgas so it will expand as it absorbs heat (recall that the total energy is conservedso if heat is coming in it better do some −PdV work to let some energy out);

3. When the engine dumps heat into the cold reservoir, we want Tc <∼ Tgas. So we need to allowthe gas to adiabatically expand as it cools (recall from Section 1.5 that adiabatic compression– compression without heat flow – heats the gas, so expansion must cool it);

4. Dump the heat from the gas into the cold reservoir isothermally (the reverse of step 2);

5. Adiabatically compress the gas to raise the temperature from Tc to Th (the reverse of step 3).

You might worry about steps 3 and 5. The adiabatic expansion of the gas has the gas doing thework, in principle; but to make sure it doesn’t expand too much and get too cold Tgas < Tc, youprobably have to do some work to stop it. The easiest way is to simply put the gas ‘bin’ in a largerbox that prevent the gas from expanding forever, so you don’t have to expend any real work instopping it. Except you might worry that I am transferring momentum in this process. Where doesthat energy come from? Likewise, the adiabatic compression step 5 has the bin doing work on thegas. Where is this energy coming from?

Just because the Carnot engine is the most efficient, it doesn’t mean it is the most useful. In fact,it is probably the most useless engine, because being essentially reversible the amount of work youcan extract from it is also infinitesimal, i.e. essentially zero. So I wouldn’t build one of these in afuel crisis if I were you! You’re much better off coming up with a scheme with a huge temperaturedifference between the heat source and sink, which will put the efficiency higher to begin with!


4.2 Refrigerators

A fridge is just an engine in reverse, literally. Simply take all the arrows in Fig. 4.1 and reversetheir directions, so now the work is pointing into the engine, and the heat is flowing out of thecold reservoir and into the hot reservoir. It is pretty obvious from our day-to-day experience, andfrom the mathematics we have developed so far on the statistical mechanics side of things, that thisis very unlikely to happen spontaneously. So clearly we would need to do some work to make ithappen; thus the fact that we need to plug our fridge into the wall!

The figure of merit for a fridge is no longer the efficiency to avoid confusion; it is now called the‘coefficient of performance.’ It is defined as c = |Qc|/|W |, i.e. the ratio of the heat pulled out of thecold reservoir to the amount of work needed to accomplish it. Using the first law gives

c =|Qc|

|Qh| − |Qc|=

1

|Qh|/|Qc| − 1. (4.4)

Now the second law is simply Eq. (4.2) but with the hot and cold labels reversed c↔ h, |Qh|/|Qc| ≥Th/Tc. Combining this with Eq. (4.4 gives

c ≤ 1

Th/Tc − 1=

TcTh − Tc

. (4.5)

This result is a bit more interesting than the engine efficiency, because it is somewhat counterintu-itive. Notice that if Th ≫ Tc the coefficient of performance goes to zero, so that the fridge doesn’twork at all! This is the opposite of the engine case, where this was the best-case scenario. Whatdo you think is going on? Likewise, the best situation is when the temperature of the hot reservoiris only slightly larger than that of the cold reservoir, Th → T+

c . In this case the coefficient ofperformance can be arbitrarily large!

The textbook discusses your fridge, for which the interior is usually set at 4C, or 277 K. The insideof your house is a pretty decent reservoir, so let’s assume that the kitchen is kept at 20C, or 293 K.Then c ≈ 17. This means that for every Joule of electrical energy consumed, 17 Joules of heat areremoved from the interior of the fridge, and 18 Joules of heat are dumped into the kitchen. Butconsider instead a deep freezer, which is set around −20C, or 253 K. Now one obtains c ≈ 6, whichis much less efficient. Things get much worse as you attempt to get much colder. This is part of thereason why in practice it is difficult to cool things down to absolute zero (the real reason is quantummechanics). To get to around 1 K, you need to cool very slowly, so that at any stage the differencebetween the hot and cold reservoirs remains small. More about these issues in the discussion of realfridges below.

4.3 Real Heat Engines

4.3.1 Stirling Engine

The real engine that perhaps most closely resembles the idealized engine shown in Fig. 4.1 is theStirling Engine, shown in Fig. 4.2. This was originally invented by a priest named Robert Stirling(1790-1878) in 1818, as an alternative to the steam engine that had a tendency at the time to giveyou steam burns or explode. The idea was to make sure that the engine material, in this case anideal (or nearly ideal) gas, was completely sealed, and that the pressures would be much lower than


those in steam engines. In fact, soon after his invention a better steel making process was inventedthat eliminated the explosion problem, and the Stirling engines never really took off. But it turnsout that there is a lot of current interest again for reasons that I will get to once I explain theprinciple.

Figure 4.2: Stirling engine. Thanks to wikipedia!

As shown in Fig. 4.2, the Stirling engine works with a sealed gas in a single chamber made up ofthree compartments. The upper-right compartment is in contact with a heat source while the lowercompartment is in contact with a cold reservoir; these are connected by a long ‘regenerator’ overwhich the temperature of the gas changes slowly from hot to cold. Surrounding the regenerator(not shown) is a material with a large heat capacity, so that it is always almost exactly at the sametemperature as the gas at each point. Like the Carnot engine, the Stirling engine has four distinctsteps:

1. Some external heat source (top right) dumps heat into the upper compartment. This causesrapid isothermal expansion (PdV work). This pushes the upper piston to the left, turning theflywheel (upper left) clockwise;

2. The turning flywheel also pulls the lower piston acting on the cold compartment, decreasingthe pressure locally;

3. The combined lower pressure in the cold compartment and the flywheel’s momentum pushingthe hot piston to the right pushes the gas from the hot compartment to the cold compartment,through the regenerator. Heat is absorbed in the process by the regenerator to bring the gastemperature from Th to Tc.

4. The cold piston moves down, isothermally compressing the cold gas and releasing heat intothe cold reservoir. The hot piston meanwhile is moving out and the gas is pulled back fromcold compartment toward the hot compartment through the regenerator; it absorbs heat fromthe regenerator during this process to bring the temperature from Tc to Th.

It is clear that there is no net energy transferred to or from the regenerator on each cycle, so theengine efficiency (not including friction, losses, etc.) is entirely determined only by the difference intemperature between the hot and cold compartments. So, in principle this engine has the potentialto achieve something close to maximum efficiency e = 1! It is for this reason that Stirling engines


have enjoyed something of a rennaissance in recent years: its potential efficiency is a boon in thisrelatively energy-starved age. Furthermore, Stirling engines have the potential to effectively boostthe efficiency of many traditional engines, simply by employing their necessary waste heat as a powersource. In fact, the most efficient home furnaces have a Stirling-type engine attached for this reason.They are also central to a solar-power generation scheme, where solar light is collected by reflectivedishes and concentrated on a Stirling engine. The efficiency rivals that of the best solid-state solarcells and there are no environmental costs whatsoever.

Stirling engines have some downsides, though. They need to withstand huge temperature gradientswithout melting or warping. They have moving parts which can wear over time. The amount ofgenerated torque is not a high as in other engines (like steam) because of the low gas pressures. Theytake some time to get going, so they wouldn’t be that useful for transportation. The moving partsneed to be optimized for the cycle frequency, so they aren’t well-suited to environments where therelative temperatures vary widely. And with the huge temperature differences needed for efficiencyit is difficult to find cold reservoirs with a sufficiently high heat capacity to keep things going.

4.3.2 Steam Engine

The steam engine has been the workhorse of industry for centuries, and remains so today: nuclearand coal electric power plants in fact power giant steam engines that turn turbines to generate theelectricity. The steam engine is known as an external combustion engine because the heat comesfrom a source outside. It runs on the four-step Rankine cycle, which is shown in Fig. 4.3. Thesteps are as follows:

Figure 4.3: Rankine cycle. Thanks again to wikipedia!

1. Water is pumped up (actively, this takes work) from low-pressure to high-pressure before it isfed to the boiler;

2. The water in the boiler is heated at constant pressure by an external source, turning it intovapour. The increased temperature means its volume expands so the vapour travels up thetube toward the turbine;


3. The steam expands adiabatically in the turbine, turning the turbine and thereby generatingthe power. As it does so it cools, ending up at the original low pressure;

4. The partially condensed vapour is further cooled in the condenser, a pipe network in contactwith the cold reservoir.

The condenser temperatures are around 30C, and the maximum high temperature corresponds tothe point at which the steel starts to warp, around 550C. This gives a Carnot-type efficiency upperbound of around e = 63%, though most coal-fired plants operate around 40%. In practice, the actualoperation of the steam engine is more complicated than the Rankine cycle.

4.3.3 Internal Combustion Engine

The internal combustion engine comes in two main flavours: gasoline (gas or petrol) and diesel,with the latter more efficient than the former. Their cycles are shown in Figures 4.4 and 4.5. Let’sexamine the gas engine first:

Figure 4.4: The gasoline engine. Thanks to Encyclopedia Brittanica online.

1. A mixture of vapourized gas and air are sucked into a cylinder as the piston moves outward.It is then compressed adiabatically as the piston moves inward, which raises its temperature;

2. A spark plug ignites the mixture, raising the temperature and pressure at constant volume;

3. (power stroke) The high pressure gas pushes the piston outward, expanding adiabatically andproducing work;

4. The hot gases are pushed out (exhaust) by the inward stroke of the piston and replaced by anew mixture at a lower temperature and pressure in step 1. No work is done in this process.


So by this description, the piston actually moves in and out twice over the course of a single cycle.It is called the Otto cycle, named after inventor Nikolaus August Otto. Anyhow, it seems like asuitable name!

Let’s analyze the efficiency of this cycle now. Recall the adiabatic relation (1.22) V γP = constant.Using the ideal gas law PV = NkBT gives TV γ−1 = new constant. So for the adiabatic compression(step 1) we have T1V

γ−12 = TcV

γ−11 . Likewise, for the adiabatic expansion (step 3) we have T3V

γ−11 =

ThVγ−12 . Therefore we must have

T1Tc

=

(

V1V2

)γ−1

=ThT3. (4.6)

The efficiency is then found using Eq. (4.1), but what are the values for Qh and Qc? Recallthe definition of the heat capacity (1.23) CV = ∂U/∂T . In the event that no work is done thenQ =

∫

CV dT = CV (Tf − Ti) if the heat capacity is assumed to be temperature-independent. Thisis true pretty much only for an ideal gas, cf. Eq. (1.26). Putting everything together gives

e = 1− |Qc||Qh|

= 1− CV (T3 − Tc)

CV (Th − T1)= 1− Tc(Th/T1)− Tc

Th − T1= 1− Tc

T1= 1−

(

V2V1

)γ−1

= 1− TcTh

(

ThT1

)

= 1− TcTh

(

T3Tc

)

. (4.7)

Because the ratios T3/Tc = Th/T1 ≥ 1, in general the efficiency of the Otto cycle is lower thanthat of the Carnot cycle. In automobile parlance, the ratio V1/V2 between the maximum andminimum volume of the cylinder is known as the compression ratio. For regular cars, its value issomething like 8− 10, while for sports cars it is something like 11 or 12. Together with the value ofγ = (f + 2)/f = 7/5 for air (assuming it is diatomic oxygen so that f = 5), one obtains automobileefficiencies of

e ≈ 1−(

1

8

)2/5

≈ 56%, (4.8)

which turns out to be optimistic. In reality, most gas engines get something like 30% efficiency.

Figure 4.5: The diesel engine. Thanks again to Encyclopedia Brittanica online!


Fig. 4.5 shows the corresponding cycle for the fuel-injection type of diesel engine. It is much likethe cycle for the gas engine, except that no spark plugs are needed. Rather, on the intake step onlyregular air is pumped into the cylinder, and the diesel vapour is only injected when the air is atsufficiently high pressure (and therefore temperature) following the compression step. At this stageit spontaneously ignites and explodes. It frankly sounds rather dangerous to me! The advantage isthat diesel engines are more efficient than gas ones, closer to e ≈ 40%, because of the much highercompression ratios used (closer to 20). also, you won’t have trouble starting your car after fordinga river that turned out to be deeper than you had hoped. The downside is that they are sometimesdifficult to start in cold weather. Maybe this is why they are more popular in Europe and Asia thanin Canada. Or perhaps it is because they sound like trucks and stink.

4.4 Real Refrigerators

4.4.1 Home Fridges

The kind of fridge that you probably have in your kitchen runs on a cycle that is almost exactlythe reverse of the Rankine engine shown in Fig. 4.3. The working engine again transitions from aliquid to a gas and back, but now at much lower temperature. The preferred working substance iscalled HFC-134a, which is a cousin of freon without the ozone-layer damaging chlorine. Here is theprocedure:

1. The gas, initially at around room temperature, is compressed adiabatically, raising its temper-ature and pressure to become a superheated high-pressure gas. This is where the energy fromthe wall plug is needed;

2. It passively releases heat and gradually liquefies in the condenser, a network of pipes in contactwith the ‘hot’ reservoir (actually at room temperature). These are usually under the fridge.The liquid is still hot (a bit warmer than room temperature) but stays a liquid because of thehigh pressures;

3. It passes through a throttler (a porous plug) after which it has substantially lowered its tem-perature and pressure. This process actually requires energy as well.

4. It absorbs heat from the inside of the fridge and turns back into a gas in the evaporator, anetwork of pipes in contact with the ‘cold’ reservoir. These are usually in the back of thefridge, behind the back false wall. Of course the ‘cold’ reservoir is understood to be at a highertemperature than the gas.

Let’s look a bit more closely at the throttler process. Because there is no heat during the process,the change in energy is

Uf − Ui = Q+W = 0 +Wleft +Wright = PiVi − PfVf , (4.9)

where the negative sign in the last relation follows from the fact that the fluid is doing PV work on

the piston to the right. So we have Ui + PiVi = Uf + PfVf or conservation of enthalpy, Hi = Hf .Of course we already knew that from the discussion near (Eq. 1.25). Suppose that the liquid isactually an ideal gas. Then H = (f/2)NkBT + NkBT = (f + 2)NkBT/2, which means that ifthe enthalpies are the same before and after the throttling process, the temperature must not havechanged! So clearly we can’t use an ideal gas as the working liquid. Of course, an ideal gas can’t


be also be a liquid, which it needs to be in this cycle. To become a liquid, the gas molecules mustattract each other enough, and this additional attractive potential energy is enough to make surethat the temperatures do change: denser liquids have more of the attractive (negative potential)energy contribution. After the throttling the potential energy is higher, so to conserve energy thekinetic energy (which is related to the temperature) is lower. The properties of real interacting gaseswill only be analyzed next term.

There are times when having a compressor is not convenient, either because of its noise or the lack ofan available source of electricity. In these cases, the energy comes from a heat source instead, suchas from solar or burning kerosene. Rather than developing an engine that converts heat directly toelectricity and using this, one can make an absorption refrigerator that uses the heat directly. I won’tbother going into details, suffice it to say that it uses (toxic) ammonia and (explosive) hydrogen gas.It turns out Einstein and his student Leo Szilard worked on a version that was less dangerous, butit hasn’t seen wide production even though there are indications that it could be incredibly efficient.

4.4.2 Liquefaction of Gases and Going to Absolute Zero

This part is well-done in the textbook and is mostly talk, so I’ll simply cover it in class.

Chapter 5

Free Energy and ChemicalThermodynamics

5.1 Free Energy as Work

5.1.1 Independent variables S and V

Recall that dU = Q +W = TdS − PdV . Clearly then U is a simultaneous function of the twoparameters S and V , i.e. U = U(S, V ). So we can write:

dU =

(

∂U

∂S

)

V

dS +

(

∂U

∂V

)

S

dV ≡ TdS − PdV, (5.1)

which immediately yields the following two equations

T =

(

∂U

∂S

)

V

, P = −(

∂U

∂V

)

S

. (5.2)

The first of these we used extensively in Chapter 3, or rather its inverse relationship 1/T =(∂S/∂U)V , though I didn’t make a big deal of the fact that we could only evaluate the expres-sion assuming constant volume. The second is sort of obvious using W = −PdV if we assume thatthe total energy only depends on work done on it, but the above is more rigorous.

The fact that dU is an exact differential of the quantity U allows us to derive another useful relation.If we took a second derivative of U , the result can’t depend on the order, i.e.

∂2U

∂V ∂S≡ ∂2U

∂S∂V(

∂

∂V

)

S

(

∂U

∂S

)

V

≡(

∂

∂S

)

V

(

∂U

∂V

)

S

⇒(

∂T

∂V

)

S

= −(

∂P

∂S

)

V

. (5.3)

This is a pretty handy formula, though the restriction of constant entropy for the first, or alterna-tively finding the pressure as a function of entropy, is a bit difficult in practise.

51


5.1.2 Independent variables S and P

Similar equations can be obtained by making a Legendre transformation. Consider for example thequantity PV . Evidently we have d(PV ) = PdV + V dP , So

dU = TdS − PdV = TdS − d(PV ) + V dP ⇒ d(U + PV ) = dH = TdS + V dP. (5.4)

which is the constitutive equation for the enthalpy H that we first encountered in Eq. (1.25) ratherthan the internal energy U . Following the arguments above, we have H = H(S, P ) so that

dH =

(

∂H

∂S

)

P

dS +

(

∂H

∂P

)

S

dP ≡ TdS + V dP, (5.5)


T =

(

∂H

∂S

)

P

, V =

(

∂H

∂P

)

S

. (5.6)

Likewise using the second-derivative trick above, one readily obtains:(

∂T

∂P

)

S

=

(

∂V

∂S

)

P

. (5.7)

5.1.3 Independent variables T and V

One can go on doing similar transformations for all the relevant variables, which is a bit of a dullexcercise. But in fact there are two particular choices which are the most useful in practise, namelyhaving the independent variables T and V , or the variables T and P . For the first of these, considerthe quantity TS, for which we have d(TS) = TdS + SdT . Then

dU = TdS − PdV = d(TS)− SdT − PdV ⇒ d(U − TS) ≡ dF = −SdT − PdV, (5.8)

which is the constitutive equation for the Helmholtz free energy F ≡ U − TS. Most of the timethat physicists talk about the free energy, it is this one, and often it is simply called the ‘free energy’period. Now, F = F (T, V ) so that

dF =

(

∂F

∂T

)

V

dT +

(

∂F

∂V

)

T

dV ≡ −SdT − PdV, (5.9)


S = −(

∂F

∂T

)

V

, P = −(

∂F

∂V

)

T

. (5.10)

These are very useful equations! And again, using the second-derivative trick above, one finds:(

∂S

∂V

)

T

=

(

∂P

∂T

)

V

. (5.11)

Thus, knowing how the pressure varies with temperature (at constant volume) is enough to tell youhow the entropy varies with volume (at constant temperature)! Consider an ideal gas, P = NkBT/V ;then ∂P/∂T = NkB/V , which yields S = NkB ln(Vf/Vi), just the expression that we obtained atthe beginning of Chapter 3 when considering the number of macrostates, S = NkB ln(V/∆V ).


5.1.4 Independent variables T and P

The other useful case comes from considering a combination of PV and TS:

dG ≡ d(U − TS + PV ) = TdS − PdV − TdS − SdT + PdV + V dP = −SdT + V dP, (5.12)

where I have introduced the Gibbs free energy G ≡ U − TS+PV . The Gibbs free energy is nowa function of T and P , so that

dG =

(

∂G

∂T

)

P

dT +

(

∂G

∂P

)

T

dP ≡ −SdT + V dP, (5.13)


S = −(

∂G

∂T

)

P

, V =

(

∂G

∂P

)

T

. (5.14)

These are also very useful equations! And again, using the second-derivative trick, one finds:

(

∂S

∂P

)

T

= −(

∂V

∂T

)

P

. (5.15)

Again, consider the ideal gas, V = NkBT/P ; then ∂V/∂T = NkB/P , which yields S = −NkB ln(Pf/Pi),which is not an expression we thought of obtaining previously.

Just so you know, Eqs. (5.7), (5.11), and (5.15), together with the related equation

(

∂T

∂V

)

S

= −(

∂P

∂S

)

V

(5.16)

are collectively called Maxwell’s relations, not to be confused with Maxwell’s equations in elec-tromagnetism! That Maxwell certainly got around. Of course, all of them can be obtained directlyfrom the original expression for the internal energy by various transformations. They are all basicallyrestatements of the fact that the various thermodynamic quantities (temperature, pressure, volume,entropy) are all related to one another. But you should keep in mind that none of these were derivedknowing exactly how they relate, i.e. no equation of state was used. Likewise, the quantities U , H ,F , and G are called thermodynamic potentials.

5.1.5 Connection to Work

O.k. so now that we have derived all of these relations, what can we do with them? Consider theHelmholtz free energy F = U − TS. Clearly dF = dU − TdS − SdT = Q +W − TdS − SdT =TdS − PdV − TdS − SdT = −PdV − SdT . If the change was made isothermally then dT = 0 andso the change in the free energy is exactly the same as the work, dF = −PdV = W . In derivingthis and expressions above, I used the fact that Q = TdS, as I derived in Eq. (3.5). In fact, closeinspection of this equation shows that this is only an equality if the amount of heat added is verysmall; in general one has dS ≥ Q/T when the heat is not infinitesimal. So in general TdS > Q andtherefore

dF ≤W (5.17)

at constant temperature.


If we’re working at constant pressure instead, then it is more convenient to work with the Gibbsfree energy: dG = Wother + SdT only if dP = 0. I have added the Wother to indicate that other(non-PdV ) forms of work could be included. In this case we instead have

dG ≤Wother (5.18)

at constant temperature and pressure. Values for the (changes in) Gibbs free energy for manysystems, particularly those in chemistry, can be found in tables. The textbook covers many examplesin chemistry that I find boring so I won’t discuss them.

5.1.6 Varying particle number

If the number of particles in the system is not fixed, then we also need to include the chemicalpotential µ (see Section 3.5). Then the change in internal energy, and the changes in the enthalpy,Helmholtz, and Gibbs free energies are given respectively by

dU = TdS − PdV + µdN ;

dH = TdS + V dP + µdN ;

dF = −SdT − PdV + µdN ;

dG = −SdT + V dP + µdN.

The first of these we saw back in Section 3.5. The rest of them are pretty trivially changed fromtheir cases seen above without allowing for changes to the total particle number! So we obtain thefollowing rules:

µ =

(

∂U

∂N

)

S,V

=

(

∂H

∂N

)

S,P

=

(

∂F

∂N

)

T,V

=

(

∂G

∂N

)

T,P

. (5.19)

If there is more than one kind of particle, then one would need to insert the additional conditions,i.e. the chemical potential for species 1 would be

µ1 =

(

∂F

∂N1

)

T,V,N2,N3,...

.

5.2 Free Energy as Force toward Equilibrium

We have now pretty firmly established that entropy increases as equilibrium is reached. How aboutthe free energies discussed in the previous section? Suppose that we decompose our total systeminto two pieces, corresponding to a subsystem and a reservoir. The reservoir is defined as somethingwhich can absorb or release energy without changing its temperature (recall Section 3.2.1). Thechange in the entropy of the total system can be written

dStotal = dS + dSR,

where the reservoir is labelled with an ‘R’ and subsystem has no label. We can rewrite this as

dStotal = dS +1

TR(dUR + PRdVR − µRdNR) .

We cand always assume that the temperature of the reservoir remains constant i.e. that the temper-ature is defined by that of the reservoir, T = TR. If furthermore the volume VR and the number of


particles NR in the reservoir are fixed (which also implies that V and N are fixed in the subsystem,because we assume that the total system is well-defined), then this relation becomes

dStotal = dS +dUR

T.

Now, since dU + dUR = 0 for the closed total system, this becomes

dStotal = dS − dU

T= − 1

T(dU − TdS) = −dF

T.

This says that if the total entropy of the total system increases, then the Helmholtz free energy ofthe subsystem must decrease! The same result holds for the Gibbs free energy. That is, equilibriumcorresponds to the minimization of the free energies. This is just a restatement of something thatyou have probably heard for a long time: a system will seek its lowest (free) energy. But be careful:the Helmholtz free energy is U −TS; if the entropy is increasing the free energy is going to decreaseat constant temperature even if the internal energy stays constant or even increases! So this is aresult specific to the free energies, not the internal energy.

It is convenient to classify the various thermodynamics quantities according to whether they are‘extensive’ or ‘intensive.’ These are terms that are used very often in the literature, and so I shouldlet you know what they mean. Extensive quantities are those that get larger when you increase theamount of stuff that you started with, and intensive quantities are those that remain the same underthis operation. Obvious things that will change are N and V , and obvious things that won’t are Tand P . Here is a list:

• Extensive quantities: V , N , S, U , H , F , G, mass;

• Intensive quantities: T , P , µ, density.

If you multiply an intensive and extensive quantity together, you get an extensive quantity; likewise,dividing two extensive quantities yields an intensive quantity. Multiplying two extensive quantitiestogether gives nonsense, so don’t do this! But there is nothing wrong with multiplying two intensivequantities. The mnemonic I use to remember which is which is that the ‘in’ in intensive stands foran ‘in’ternal quantity, i.e. something that doesn’t change when I change an ‘ex’ternal parameter.

Chapter 6

Boltzmann Statistics (aka TheCanonical Ensemble)

In the preceding chapter, we have seen the microcanonical ensemble, where the total numberof particles N =

∑

i ni and U =∑

i niǫi are fixed (constant). Suppose now that the total energyis not fixed, but rather than energy can be exchanged with some ‘bath,’ or energy reservoir, thecanonical ensemble. This is the situation when the system is no longer thermally isolated (i.e.your mug of steaming something, rather than your vacuum thermos full of hot something, but itfeels cool to the outside). We saw in the previous chapter that changes in the heat are associatedwith the rearrangement of particles in energy states; more heat means more particles are found inhigher quantum energy levels. So now the question is: how to calculate the ni?

6.1 The Boltzmann Factor

To find the ni in the canonical ensemble, we again use our old trick, maximizing the entropy:d ln(Ω) = 0 and d2 ln(Ω) < 0. First the first:

d ln(Ω) =∂ ln(Ω)

n1dn1 +

∂ ln(Ω)

n2dn2 + . . . = 0

=∑

i

∂ ln(Ω)

nidni = 0. (6.1)

Recall that in general we can write

Ω =N !∏

i ni!

which means

ln(Ω) = N ln(N)−N −∑

i

ni ln(ni) +∑

i

ni = N ln(N)−∑

i

ni ln(ni)

as we have seen before. Now,

∂ ln(Ω)

∂nj= − ∂

∂nj

∑

i

ni ln(ni) (Note the derivative w.r.t. the variable nj)

56


= −∑

i

δij [ln(ni) + 1] = − [ln(nj) + 1] .

Substituting this into Eq. (6.1), we have:

d ln(Ω) = −∑

i

[ln(ni) + 1] dni = 0. (6.2)

Because N =∑

i ni and U =∑

i niǫi are constants, we also know that

dN =∑

i

dni = 0 and dU =∑

i

ǫidni = 0.

You might object to the second equation here, for two reasons. First, I have assumed that thevariation of the mean energy is zero, even though I started out by stating that the total energy isn’tfixed! But while energy is allowed to flow between the system and the reservoir, the mean energy

is assumed to be a constant at equilibrium. Part of this process is to determine what the meanenergy U actually is, if the various ni are allowed to vary. Second, you might object that I haven’tincluded any variations in the energy levels themselves (i.e. I haven’t included a term that looks like∑

i nidǫi). I am explicitly assuming here that the container walls themselves are fixed, so that thequantum energy levels are always well-defined; it’s just that now the walls can conduct heat.

O.k., now I have three equations that I need to satisfy simultaneously:∑

i

dni = 0;∑

i

ǫidni = 0; and∑

i

ln(ni)dni = 0.

How can I solve for the ni? This is just like a classical mechanics problem where I have to solvefor the dynamics of some object, subject to some constraints. The first and second of the aboveequations are the holonomic equations of constraint for the third equation! If you remember, theseequations of constraint could be incorporated using he method of undetermined multipliers, otherwiseknown at Lagrange multipliers. Because I have two equations of constraint, I need two Lagrangemultipliers: call them α and β. Now the equation I need to solve is

∑

i

[ln(ni) + α+ βǫi] dni = 0.

I have effectively added zero to my original equation, twice! Now, the dni variations are completelyarbitrary, but no matter what I choose for the various dni the left hand side always sums to zero.The only way to guarantee this is if the term in the square brackets is itself zero,

ln(ni) + α+ βǫi = 0.

Inverting givesni = exp [−βǫi − α] = A exp (−βǫi) , (6.3)

where A = exp(−α) is a constant. How to find α and β? Use the equations of constraint again!Since N =

∑

iA exp (−βǫi), knowing β and given N , A immediately follows. How about β? Well,because we haven’t allowed any work to be done (the volume of the system is assumed fixed), thenthe change in mean energy is just due to the heat:

dU =∑

i

ǫidni (6.4)

= TdS = kBTd ln(Ω) = −kBT∑

i

ln(ni)dni. (6.5)


Comparing (6.4) and (6.5) immediately shows that ǫi = −kBT ln(ni). Inverting this gives

ni = exp

(

− ǫikBT

)

. (6.6)

Comparison between Eq. (6.3) and (6.6) shows that β = 1/kBT , as I derived using a hokey methodlast chapter. On the other hand, I also used the identity dU = TdS here, which was derived usingthe same hokey method, so I’m not sure if I’m any further along, really. . . .

Now that I know the explicit form for ni, I can put it back into the equation for the total numberof particles: N = A

∑

i exp(ǫi/kBT ). Because the probability of a given outcome is simply equal tothe population of a given energy state, divided by the total number of particles, I obtain:

pi =ni

N=

A exp(−ǫi/kBT )A∑

i exp(−ǫi/kBT )≡ exp(−ǫi/kBT )

Z,

where the partition function is defined as

Z ≡∑

i

exp

(

− ǫikBT

)

.

The Z stands for Zustandsumme, or sum over states. Of course, N = AZ.

6.2 Z and the Calculation of Anything

Once you know Z, you know everything. In other words, knowing the occupation of the variousenergy levels using formula (6.6) allows you to calculate any thermodynamic quantity of interest.First, let’s write the entropy in terms of Z:

S = kB ln(Ω) = kB

[

N ln(N)−∑

i

ni ln(ni)

]

= kB

N ln(N)−∑

i

ni [ln(A)− βǫi]

using Eq. (6.6).

= kB N ln(N)−N ln(A) + βU using the definition of U .

Now, N = A∑

i exp(−βǫi) so ln(N) = ln(A) + ln [∑

i exp(−βǫi)] = ln(A) + ln(Z). Inserting thisabove we obtain

S = kB [N ln(A) +N ln(Z)−N ln(A) + βU ] .

Finally, we have the result for the entropy in terms of the partition function:

S = NkB ln(Z) +U

T. (6.7)

Another important quantity that immediately follows without even knowing the explicit dependenceof U on Z is the Helmholtz free energy:

F = U − TS = −NkBT ln(Z). (6.8)

Your textbook has a different derivation of this relationship, given on pp. 247-248.


We still don’t know the explicit form for U as a function of Z, so here goes. We already know thatU =

∑

i ǫini = A∑

i ǫi exp(−βǫi). This expression can be ‘simplified’ using the fact that

ǫi exp(−βǫi) = − ∂

∂βexp(−βǫi) = −∂T

∂β

∂

∂Texp(−βǫi) =

1

kBβ2

∂

∂Texp(−βǫi).

Putting it together we have

U =A

kBβ2

∂

∂T

∑

i

exp(−βǫi) = AkBT2 ∂Z

∂T=NkBT

2

Z

∂Z

∂T.

Now we’re finally done:

U = NkBT2∂ ln(Z)

∂T. (6.9)

Some simple examples of how to think about the partition function will be covered in class.

Another very important thermodynamic quantity is the specific heat, or the heat capacity. Thisis often what is really measured in an experiment. The specific heat at constant volume CV isdefined as

CV = T

(

∂S

∂T

)

V

.

We can also express the specific heat in terms of the mean energy U :

∂S

∂T=

∂

∂T

[

NkB ln(Z) +NkBT∂ ln(Z)

∂T

]

= 2NkB∂ ln(Z)

∂T+NkBT

∂2 ln(Z)

∂T 2

=1

T

∂

∂T

[

NkBT2∂ ln(Z)

∂T

]

=1

T

∂U

∂T. (6.10)

Inserting this into the above expression for the specific heat we obtain

CV =

(

∂U

∂T

)

V

.

We can also express CV in terms of the Helmholtz free energy F , using Eqs. (6.7), (6.8), and (6.9):(

∂F

∂T

)

V

= −NkB ln(Z)−NkBT

(

∂ ln(Z)

∂T

)

= −NkB ln(Z)− U

T= −S. (6.11)

This immediately yields

CV = −T(

∂2F

∂T 2

)

V

.

Which of these three expressions you choose to use depends in large part on which one is easiest tocalculate for a given problem!

The connection between the Helmholtz free energy and the partition function means that we cancarry over all the relationships obtained Sec. 5.1.3. One of these is simply Eq. (6.11):

S = −(

∂F

∂T

)

V

= NkB

(

∂T ln(Z)

∂T

)

V

= NkB ln(Z) +NkBT

(

∂ ln(Z)

∂T

)

V

,


which we already knew from combining Eqs. (6.7) and (6.9). The other one is

P = −(

∂F

∂V

)

T

= NkBT

(

∂ ln(Z)

∂V

)

T

.

The last one follows from Eq. (5.19):

µ =

(

∂F

∂N

)

T,V

= −kBT(

∂N ln(Z)

∂N

)

T,V

= −kBT ln(Z)−NkBT

(

∂ ln(Z)

∂N

)

T,V

.

Because Z = N/A, this last relation can be solved directly: ∂ ln(N/A)/∂N = ∂ ln(N)/∂N = 1/N sothat µ = −kBT [ln(Z) + 1]. At high temperatures, the arguments of the exponentials in Z will allbe small, which implies that Z > 0. This means that the chemical potential for classical particles isalways negative. At low temperatures it might approach zero or get positive. This possibility willbe explored next term when we discuss quantum statistics.

6.2.1 Example: Pauli Paramagnet Again!

You’ll see that using the canonical ensemble is much easier! Again, we assume a spin- 12 system wherewe have two energy levels ǫ and −ǫ. The partition function is therefore

Z = exp(ǫ/kBT ) + exp(−ǫ/kBT ) = 2 cosh(ǫ/kBT ). (6.12)

Using Eq. (6.9), we obtain

U = NkBT2 ∂ ln(Z)

∂T=NkBT

2

Z

∂

∂T2 cosh(ǫ/kBT ) =

NkBT2

Z

(

− 2ǫ

kBT 2

)

sinh(ǫ/kBT )

= −Nǫ tanh(

ǫ

kBT

)

. (6.13)

Meanwhile, the entropy is

S = NkB ln(Z) +U

T= NkB ln

[

exp

(

ǫ

kBT

)

+ exp

(

− ǫ

kBT

)]

+U

T.

What about the magnetization? The energy levels now depend on the magnetic field rather thanthe volume as was the case for the gas in the box. The magnetization for the magnetic system is thegeneralized force associated with doing work on the system, in the form of changing the magneticfield (and therefore the accessible energy states), just as the pressure is the generalized force due tochanging the volume. So,

dΩ =∑

i

nidǫi =∑

i

ni∂ǫi∂B

dB ≡ −MdB,

where the negative sign is a convention. To make further progress, we use the definition of theHelmholtz free energy F = U − TS: dF = dU − SdT − TdS. But dU = dQ + dΩ = TdS −MdBas shown just now, so dF = TdS −MdB − TdS − SdT = −MdB − SdT . We are left with theimportant relation

M = −∂F∂B

= NkBT∂ ln(Z)

∂B=NkBT

Z

∂Z

∂B.


It is easy to show that using this with the expression (6.12) reproduces Eq. (3.13). It’s also importantto note that the same reasoning gives the definition of the pressure in terms of the partition function:

P = −∂F∂V

= NkBT∂ ln(Z)

∂V=NkBT

Z

∂Z

∂V. (6.14)

At very low temperatures, T → 0, kBT ≪ ǫ so the second term in the square brackets becomesvanishingly small. Also, the tanh(ǫ/kBT ) → 1 in the expression for U in Eq. (6.9). Putting thesetogether gives

S(T → 0) ≈ NkBǫ

kBT− Nǫ

T→ 0.

So, the entropy of the Pauli paramagnet goes to zero at zero temperature. This is because all of thespins align, so the disorder of the system vanishes (Boltzmann philosophy); alternatively, it becomestrivially easy to describe the state of the system (Shannon philosophy).

At the opposite limit of very high temperatures T ≫ 0, we obtain ǫ/kBT → 0 and so U →−Nǫ(ǫ/kBT ) because the tanh function has a linear slope near the origin. Meanwhile, Z → 2because each of the exponentials approaches 1. So the entropy is

S(T ≫ 0) → NkB ln(2)− Nǫ2

kBT≈ NkB ln(2).

Does this make sense?

6.2.2 Example: Particle in a Box (1D)

Recall that the energy levels for particles of mass m in a one-dimensional box of length L were foundusing Bohr-Sommerfeld quantization to be

ǫn =h2π2n2

2mL2.

The partition for this system is now

Z =

∞∑

n=0

exp(−γn2),

where

γ ≡ h2π2

2mL2kBT.

In general, we’re at high temperatures so that γ ≪ 1. To give you an idea of how small γ is, let’sagain assume that we are dealing with nitrogen in a 1 m box at room temperature, as described inSection 3.2.2. There we saw that kBT/ǫn ∼ 2× 1021, so γ ∼ 10−22. Obviously, with this incrediblysmall coefficient in the exponential, this is a series that converges very slowly. It makes sense toconvert it to an integral:

Z ≈∫ ∞

0

dn exp(−γn2).

Substituting x =√γn converts this integral to

Z =1

γ

∫ ∞

0

dxe−x2

=

√π

2√γ=

√

π

4γ=

√

L2mkBT

2πh2≡ L

λD, (6.15)


where the de Broglie wavelength λD is defined as

λD ≡

√

2πh2

mkBT.

This is an important length scale that I’ll discuss in a moment. But first, now that we have thepartition function, let’s calculate some thermodynamic properties, such as the mean energy and theheat capacity:

U =NkBT

2

Z

∂Z

∂T=NkBT

2λDL

1

2

L

TλD=

1

2NkBT,

CV =∂U

∂T=

1

2NkB .

Note that the specific heat has the same units as the entropy. Hmmmmmm.

Now let’s get to what λD means. Suppose that the atoms in the 1D box only have kinetic energy(there’s no gravity, and they all bounce elastically off the walls). Then, the mean energy perparticle is simply U/N = 1

2mv2, where v is the mean kinetic energy of an atom. Putting the

above result together gives 12mv

2 = 12kBT or v =

√

kBT/m. Alternatively, the mean momentum is

p = mv =√mkBT . But we already know that

√mkBT =

√2πh2/λD from the definition of the de

Broglie wavelength. So we obtain

p =

√2πh2

λD=

1√2π

h

λD.

But, apart from a constant factor of√2π, this is just de Broglie’s relation p = h/λ showing that

particles are waves in quantum mechanics! In other words, the ‘wavelength’ of a particle is inverselyproportional to the root of both its mass and its temperature: the lower the temperature, or thelighter the particle, the longer its wavelength. So at sufficiently low temperatures we might expectsmall particles to ‘fuzz out’ and behave very non-classically! This is indeed what happens, but thisinteresting story will have to wait until later in the term.

6.2.3 Example: Particle in a Box (3D)

In three-dimensions, the quantum energy levels are almost identical to those in 1D:

ǫnx,ny,nz=

h2π2

2mL2

(

n2x + n2

y + n2z

)

, nx, ny, nz = 0, 1, 2, . . . .

The lowest few accessible energy levels are therefore (where for clarity I define ǫ ≡ h2π2

2mL2 ):The various energy levels are all equally spaced, and the lowest energy level is unique. But the

second lowest, with energy ǫ = h2π2

2mL2 , is triply degenerate, as are the third and fifth levels 2ǫ and 4ǫ.So we should write the partition function for the 3D case, including these degeneracy factors, as

Z =∑

m

gm exp(−βǫm) = 1 + 3 exp(−βǫ) + 3 exp(−2βǫ) + exp(−3βǫ) + 3 exp(−4βǫ),

where gm is the degeneracy factor. While formally correct, it is very difficult to figure out howon earth to sum this series to get a closed-form expression for Z.


energy level nx ny nz

0 0 0 0ǫ 1 0 0ǫ 0 1 0ǫ 0 0 12ǫ 1 1 02ǫ 1 0 12ǫ 0 1 13ǫ 1 1 14ǫ 2 0 04ǫ 0 2 04ǫ 0 0 2

Our task is simplified considerably by noticing that the the energies for each dimension are additive,which means that the partition function can be written more conveniently as a product of thepartition functions in each dimension:

Z =∑

nx,ny,nz

exp[−γ(n2x + n2

y + n2z) =

∞∑

nx=0

exp(−γn2x)

∞∑

ny=0

exp(−γn2y)

∞∑

nz=0

exp(−γn2z)

= (Z1D)3=L3

λ3D=

V

λ3D.

Generalizing the calculation of the 1D mean energy to the 3D case, we obtain

U =3

2NkBT and CV =

3

2NkB.

Let’s now obtain the equation of state using the equation for the pressure in terms of the free energy(6.14). First calculate F :

F = −NkBT ln(Z) = −NkBT [ln(V )− 3 ln(λD)]

= −NkBT[

ln(V )− 3

2ln

(

2πh2

mkB

)

+3

2ln(T )

]

.

So,

P = −∂F∂V

=NkBT

V,

which is just the expected equation of state for an ideal gas in a cubic box. Putting this resulttogether with that for the mean energy gives

PV =2

3U =

2V

3VU.

So the work dW done by changing the volume can be immediately obtained:

dΩ = −PdV = −2U

3VdV.


6.2.4 Example: Harmonic Oscillator (1D)

Before we can obtain the partition for the one-dimensional harmonic oscillator, we need to find thequantum energy levels. Because the system is known to exhibit periodic motion, we can again useBohr-Sommerfeld quantization and avoid having to solve Schrodinger’s equation. The total energyis

E =p2

2m+kx2

2=

p2

2m+mω2x2

2,

where ω =√

k/m is the classical oscillation frequency. Inverting this gives p =√2mE −m2ω2x2.

Insert this into Eq. (3.6):∮

pdx =

∮

√

2mE −m2ω2x2dx = nh,

where the integral is over one full period of oscillation. Let x =√

2E/mω2 sin(θ) so that m2ω2x2 =2mE sin2(θ). Then

∮

pdx =

√

2E

mω2

∫ 2π

0

√2mE cos2(θ)dθ =

2E

ω

1

22π =

2πE

ω= nh.

So, again making the switch E → ǫn, we obtain

ǫn = nhω

2π= nhω.

The full solution to Schrodinger’s equation (a lengthy process involving Hermite polynomials) givesǫn = hω(n+ 1

2 ). Except for the constant factor, Bohr-Sommerfeld quantization has done a fine jobof determining the energy states of the harmonic oscillator.

Armed with the energy states, we can now obtain the partition function:

Z =∑

n

exp(−ǫn/kBT ) =∑

n

exp(−βǫn) = 1 + exp(−βhω) + exp(−2βhω) + . . . .

But this is just a geometric series: if I make the substitution x ≡ exp(−βhω), then Z = 1+x+x2+x3+ . . .. But I also know that xZ = x+x2 +x3+ . . .. Since both Z and xZ have an infinite numberof terms, I can subtract them and all terms cancel except the first: Z − xZ = 1, which immediatelyyields Z = 1/(1− x), or

Z =1

1− exp(−βhω) . (6.16)

Now I can calculate the mean energy:

U = NkBT2∂ ln(Z)

∂T=NkBT

2

Z

∂Z

∂T= NkBT

2 [1− exp(−βhω)][1− exp(−βhω)]2

(−1)hω

kBT 2(−1) exp(−βhω)

= Nhωexp(−βhω)

1− exp(−βhω) =Nhω

exp(βhω)− 1.

= Nhω〈n(T )〉, where 〈n(T )〉 ≡ 1

exp(hω/kBT )− 1is the occupation factor.

At very high temperatures T ≫ 1, exp(hω/kBT ) ≈ 1 + (hω/kBT ), so 〈n(T )〉 → kBT/hω and

U(T ≫ 0) → NkBT and CV (T ≫ 0) → NkB.

Notice that these high-temperature values are exactly twice those found for the one-dimensionalparticle in a box, even though the energy states themselves are completely different from each other.


6.2.5 Example: Harmonic Oscillator (3D)

By analogy to the three-dimensional box, the energy levels for the 3D harmonic oscillator are simply

ǫnx,ny,nz= hω(nx + ny + nz), nx, ny, nz = 0, 1, 2, . . . .

Again, because the energies for each dimension are simply additive, the 3D partition function canbe simply written as the product of three 1D partition functions, i.e. Z3D = (Z1D)

3. Because

almost all thermodynamic quantities are related to ln (Z3D) = ln (Z1D)3= 3 ln (Z1D), almost all

quantities will simply be mupltiplied by a factor of 3. For example, U3D = 3NkBT = 3U1D andCV (3D) = 3NkB = 3CV (1D).

One can think of atoms in a crystal as N point masses connected to each other with springs. Toa first approximation, we can think of the system as N harmonic oscillators in three dimensions.In fact, for most crystals, the specific heat is measured experimentally to be 2.76NkB at roomtemperature, accounting for 92% of this simple classical picture. It is interesting to consider theexpression for the specific heat at low temperatures. At low temperature, the mean energy goes toU → 3Nhω exp(−hω/kBT ), so that the specific heat approaches

CV → −3Nhω

kBT 2(−hω) exp

(

− hω

kBT

)

= 3NkB

(

hω

kBT

)2

exp

(

− hω

kBT

)

.

This expression was first derived by Einstein, and shows that the specific heat falls off exponentiallyat low temperature. It provided a tremendous boost to the field of statistical mechanics, becauseit was fully consistent with experimental observations of the day. Unfortunately, it turns out to bewrong: better experiments revealed that CV ∝ T 3 at low temperatures, not exponentially. This isbecause the atoms are not independent oscillators, but rather coupled oscillators, and the low-lyingexcitations are travelling lattice vibrations (now known at phonons). Actually, even CV ∝ T 3 iswrong at very low temperatures! The electrons that can travel around in crystals also contribute tothe specific heat, so in fact CV (T → 0) ∝ T .

6.2.6 Example: The rotor

Now let’s consider the energies associated with rotation. In classical mechanics, the rotational kineticenergy is

T =1

2~ω · I · ~ω,

where I is the moment of inertia tensor and ~ω is the angular velocity vector. In the inertial ellipsoid,this can be rewritten

T =L2x

2Ixx+

L2y

2Iyy+

L2z

2Izz,

where Lj is the angular momentum along direction and Ijj is the corresponding moment of inertia.Suppose that we have a spherical top, so that Ixx = Iyy = Izz = I:

T =1

2I

(

L2x + L2

y + L2z

)

=L2

2I.

In the quantum version, the kinetic energy is almost identical, except now the angular momentumis an operator, denoted by a little hat:

T =L2

2I.


The eigenvalues of this operator are ℓ(ℓ + 1)h2/2I, where ℓ = −L,−L + 1,−L + 2, . . . , L − 1, L sothat ℓ can take one of 2L+ 1 possible values.

For a linear molecule (linear top), the partition function for the rotor can then be written as

Z =

∞∑

L=0

L∑

ℓ=−L

exp

(

− ℓ(ℓ+ 1)h2

2IkBT

)

≈∞∑

L=0

(2L+ 1) exp

(

−L(L+ 1)h2

2IkBT

)

,

where the second term assumes that the contributions from the different ℓ values are more or lessequal. This assumption should be pretty good at high temperatures where the argument of theexponential is small. In this case, there are simply 2L + 1 terms for each value of L. Again,because we are at high temperatures the discrete nature of the eigenvalues is not important, we canapproximate the sum by an integral:

Z ≈∫ ∞

0

(2L+ 1) exp

(

−L(L+ 1)h2

2IkBT

)

dL.

We can make the substitution x = L(L+ 1) so that dx = (2L+ 1)dL, which is just term already inthe integrand. So the partition function becomes

Z =

∫ ∞

0

exp

(

− h2x

2IkBT

)

dx =2IkBT

h2.

Again, I can calculate the mean energy

U = NkBT2∂ ln(Z)

∂T=NkBT

2

Z

∂Z

∂T= NkBT

2 h2

2IkBT

2IkB

h2= NkBT.

This is exactly the contribution that we expected from the equipartition theorem: there are twoways the linear top can rotate, so there should be two factors of (1/2)NkBT contributing to theenergy.

For a spherical top, each of the energy levels is (2L+ 1)-fold degenerate. The partition function forthe rotor can then be written as

Z =

∞∑

L=0

L∑

ℓ=−L

(2L+ 1) exp

(

− ℓ(ℓ+ 1)h2

2IkBT

)

≈∞∑

L=0

(2L+ 1)2exp

(

−L(L+ 1)h2

2IkBT

)

,

where the second term assumes that the contributions from the different ℓ values are more or lessequal. This assumption should be pretty good at high temperatures where the argument of theexponential is small. In this case, there are simply (2L + 1)2 terms for each value of L. Again,because we are at high temperatures so that the discrete nature of the eigenvalues is not important,we can approximate the sum by an integral:

Z ≈∫ ∞

0

(2L+ 1)2exp

(

−L(L+ 1)h2

2IkBT

)

dL.

At high temperatures, one needs large values of L before the argument of the exponentials will besignificant, so it is reasonable to make the substitution L(L+ 1) → L2 and (2L+ 1)2 → 4L2. Thisyields

Z =

∫ ∞

0

4L2 exp

(

− h2L2

2IkBT

)

dL =√π

(

2IkBT

h2

)3/2

.


Again, I can calculate the mean energy

U = NkBT2∂ ln(Z)

∂T=NkBT

2

Z

∂Z

∂T= NkBT

2√π(

h2

2IkBT

)3/21√π

3

2

(

2IkB

h2

)3/2

T 1/2 =3

2NkBT.

For the speherical top, there are now three contributions to the total energy, which accounts for theadditional factor of (1/2)NkBT over the linear top.

6.3 The Equipartition Theorem (reprise)

The examples presented in the previous section show that the high-temperature limits of the meanenergy for the particle-in-the-box and harmonic oscillator problems were very similar: the meanenergies were all some multiple of NkBT/2 and the specific heat some multiples of NkB/2. Noticethat for both the particle in the box and the harmonic oscillator, both quantities were three timeslarger going from 1D to 3D, i.e. where the number of degrees of freedom increased by a factor of 3.Perhaps U and CV at high temperatures provide some measure of the number of degrees of freedomof the particles in a given system.

To make further progress with this idea, we need to revisit the hokey derivation of the equation ofstate for an ideal gas presented at the end of Section 2.6.1. Recall that, in order to enumerate allthe accessible states in a volume V , we subdivided the volume into little ‘volumelets’ of size ∆V ,each of which defined an accessible site that a particle can occupy. Then we stated that the entropywas given by S = kB ln(V/∆V ). But quantum mechanics tells us that we can’t know both the exactposition and momentum of a particle at the same time. The relationship between the uncertaintyin the position ∆x and the momentum ∆p is quantified in the Heisenberg uncertainty principle∆x∆p ≥ h, where h again is Planck’s constant. For our purposes, this means that Planck’s constantsets a fundamental limit on the size of the volumelets on can partition the system into. In a sense,the Bohr-Sommerfeld quantization condition already stated this: the integral of momentum overspace is minimally Planck’s constant.

Mathematically, this means that we can write the general partition function in one dimension as acontinuous function

Z1D → 1

h

∫ ∞

−∞

dx

∫ ∞

−∞

dpx exp

(

− ǫ(x, px)kBT

)

,

where the accessible classical states ǫ are explicitly assumed to be functions of both position x andmomentum px. In three dimensions, it would be

Z3D → 1

h3

∫

d3r

∫

d3p exp

(

− ǫ(r,p)kBT

)

.

The six dimensions (r,p) together constitute phase space, and the two three-dimensional integralsare denoted phase-space integrals.

To make these ideas more concrete, let’s again calculate the partition function for a particle in a 1Dbox of length L. The classical energy is entirely kinetic, so ǫ(p) = p2/2m, and

Zbox =1

h

∫ L/2

−L/2

dx

∫ ∞

−∞

dp exp

(

− p2

2mkBT

)

=1

hL√π√

2mkBT = L

√

mkBT

2πh2=

L

λD,


consistent with Eq. (6.15). As found previously, U(T ≫ 0) → NkBT/2 and CV (T ≫ 0) → NkB/2.Let’s also consider the 1D harmonic oscillator, for which the total energy is ǫ(x, p) = p2/2m +mω2x2/2. Now the partition function is

Zh.o. =1

h

∫ ∞

−∞

dx

∫ ∞

−∞

dp exp

(

−p2

2m + mω2x2

2

kBT

)

=1

h

∫ ∞

−∞

dx exp

(

−mω2x2

2kBT

)∫ ∞

−∞

dp exp

(

− p2

2mkBT

)

.

The second one of these we already did, giving us L/λD. The first one is equally easy to integrate,since it’s also a Gaussian integrand. So we obtain altogether

Zh.o. =

√π

2π

√

2kBT

mω2

√π√

2mkBT =kBT

hω,

which is identical to the high-temperature limit of the 1D harmonic oscillator partition function(6.16):

limT≫0

1

1− exp(−βhω) =1

1− (1− βhω)≈ kBT

hω.

The mean energy in this limit is

U(T ≫ 0) =NkBT

2

Z

∂Z

∂T≈ NkBT

2

kBT

hωkBhω

= NkBT.

But we already know that the second of the two integrals contributed NkBT/2 to the mean energyU at high temperature, because this was the result for the particle in the 1D box. This means thatthe first integral also contributed NkBT/2 to the mean energy. This is very much analogous to eachspatial integral contributing a factor of NkBT/2 to the mean energy going from 1D to 3D. If youthink of the kinetic and potential energy terms in the classical expression for the energy as eachstanding for one degree of freedom, then one can finally state the

D Equipartition Theorem: Each degree of freedom that contributes a term quadratic in positionor momentum to the classical single-particle energy contributes an average energy of kBT/2 perparticle.

Many examples showing how one can predict the value of U at high temperatures will be covered inclass.

6.3.1 Density of States

Recall that for a particle in a 3D box, the energy levels are given by

ǫn =h2π2

2mL2

(

n2x + n2

y + n2z

)

.

We can think of the energy levels as giving us the coordinates of objects on the surface of a spherein Cartesian coordinates, except only in the first octant (because nx, ny, nz ≥ 0). So, instead, we


can think of the energies as continuous, ǫ = γn2, where γ = h2π2/2mL2 and n2 is the length of thevector in ‘energy space.’ So, n =

√

ǫ/γ and therefore

dn =1

2√γ

dǫ√ǫ.

In spherical coordinates, we can write

dn = d3n = dnxdnydnz = n2 sin(θ)dndθdφ →(

1

8

)

4πn2dn (integrating over angles)

=π

2

(

ǫ

γ

)

1

2√γ

dǫ√ǫ=π√ǫdǫ

4γ3/2≡ g(ǫ)dǫ.

So, the density of states per unit energy g(ǫ) for a particle in a 3D box is given by

g(ǫ) ≡ dn

dǫ=

π

4γ3/2√ǫ = V

m√2m

2h3π2

√ǫ =

V (2m)3/2

4h3π2

√ǫ,

where in the last line I have put back in the explicit form for γ. The most important thing to noticeis that the density of states per unit energy for the 3D box goes like

√ǫ.

If you found this derivation confusing, here’s another one. One way to think about the energy levelsfor a particle in a box is that they are functions of k, which is proportional to the particle momentum(the Fourier transform of the coordinate):

ǫk =h2

2m

(πn

L

)2

≡ h2k2

2m,

which is just the usual expression for the kinetic energy if you recognize that p ≡ hk. This is knownas the free-particle dispersion relation. Now, the energy sphere is in ‘k-space,’ or ‘Fourier space.’Then the density of states is the number of single-particle states in a volume element in k-space,times the density of points:

g(ǫ)dǫ = k2 sin(θ)dθdφdk

(

L3

π3

)

.

Integrating over the angles gives

g(ǫ)dǫ =V

2π2k2dk ⇒ g3D(ǫ) =

V

2π2k2(

dk

dǫ

)

for the 3D box. Now, ǫ = h2k2/2m so k =√

2mǫ/h. Putting it together we obtain

g(ǫ)dǫ =V

2π2k2dk =

V

2π2

2mǫ

h2

√

2m

h21

2√ǫdǫ =

V

2π2

(2m)3/2√ǫ

2h3dǫ,

as we obtained above.

It is straightforward to generalize the 3D box to the 2D and 1D cases. Going through the sameprocedure gives

g2D(ǫ) =Ak

2π

(

dk

dǫ

)

; g1D(ǫ) =L

π

(

dk

dǫ

)

.


Explicitly using the free-particle dispersion relation as above shows that the density of states perunit volume is a constant (independent of energy) for 2D, and goes like 1/

√ǫ in 1D.

What is all this helping us to do? Let’s consider how many states are occupied at room temperature.The total number of states G(ǫ) is the integral of the density of states up to energy state ǫ:

G(ǫ) =

∫ ǫ

0

g(ǫ′)dǫ′ =V

2π2

(2m)3/2

2h3√ǫ′dǫ′ =

V

6π2

(2m)3/2

h3ǫ3/2 =

π

6

(

2mL2

h2π2ǫ

)3/2

.

At room temperature, ǫ = 32kBT where T = 300 K. So this means that the total number of states

accessible for nitrogen in a 1 m box at room temperatures is approximately G(ǫ) ≈ (π/6)(1020)3/2 ∼1030. But the density of air at sea level is about 1.25 kg/m3, or about 1025 molecules per m3. Soeven with these rough estimates it is clear that there are far more states accessible to the moleculesin the air at room temperature than there are particles. Alternatively, the probability that a givenenergy state has a particle in it is something like 10−5.

6.4 The Maxwell Speed Distribution

Let’s make the assumption that the energy levels are so closely spaced that we are left with amore-or-less continuous distribution of them, i.e.

N =∑

i

ni ⇒∫

dǫn(ǫ) =

∫

dǫg(ǫ) exp(−βǫ)

=A(2m)3/2V

4h3π2

∫ ∞

0

dǫ√ǫ exp(−βǫ), in three dimensions.

Because Z = N/A, this expression also quickly yields Z = V/λ3D in the usual way. Alternatively, wecan set A = N/Z and obtain

n(ǫ) = Ag(ǫ) exp(−βǫ) = N

V

(

2πh2

mkBT

)3/2(2m)3/2V

4h3π2

√ǫ exp(−βǫ) = 2π

√ǫN

(πkBT )3/2exp(−βǫ).

Now suppose that the energy levels were simply classical kinetic energy states: ǫ = mv2/2, so that√ǫ =

√

m/2v and dǫ = mvdv. Then,

n(v)dv =

√

2

πN

(

m

kBT

)3/2

v2dv exp

(

−βmv2

2

)

.

This is the Maxwell-Boltzmann distribution of velocities. The important thing to notice is that forsmall velocities, the distribution increases quadratically, n(v)dv ∝ v2, while for large velocities itdecreases exponentially, n(v)dv ∝ exp(−βmv2/2). This means that the distribution is not even, i.e.is not symmetric around any given velocity. As shown below, this will have interesting consequencesfor the statistics.

Let’s obtain the mean velocity v first:

v =

∫

vn(v)dv∫

n(v)dv=

∫∞

0v3 exp

(

−βmv2

2

)

dv

∫∞

0 v2 exp(

−βmv2

2

)

dv,


where I’ve cancelled all the common constant terms. To turn these into integrals we can evaluate,let’s set x2 ≡ βmv2/2, so that v = x

√

2kBT/m. Then

v =

√

2kBT

m

∫∞

0x3 exp

(

−x2)

dx∫∞

0x2 exp (−x2) dx .

Now it is very useful to know the following integrals:

∫ ∞

0

x2n exp(−a2x2)dx =(2n)!

√π

n!2(2a)2naand

∫ ∞

0

x2n+1 exp(−a2x2)dx =n!

2a2n+2.

In the current case, we have a = 1 and

v =

√

2kBT

m

1

2

8

2√π

=

√

8kBT

πm≈ 1.596

√

kBT

m.

Now let’s calculate the RMS velocity, ∆v ≡√

v2:

v2 =

∫

v2n(v)dv∫

n(v)dv=

∫∞

0v4 exp

(

−βmv2

2

)

dv

∫∞

0 v2 exp(

−βmv2

2

)

dv

=2kBT

m

∫∞

0x4 exp

(

−x2)

dx∫∞

0 x2 exp (−x2) dx=

2kBT

m

24√π

64

8

2√π=

3kBT

πm

⇒ ∆v =

√

3kBT

πm≈ 1.73

√

kBT

m.

The most probable speed v corresponds to the point at which the distribution is maximum:

∂

∂v

[

√

2

πN

(

m

kBT

)3/2

v2dv exp

(

−βmv2

2

)

]

= 0

2v exp

(

−βmv2

2

)

− v2(

m

2kBT

)

2v exp

(

−βmv2

2

)

= 0

⇒ v =

√

2kBT

m≈ 1.414

√

kBT

m.

The amazing thing about the Maxwell-Boltzmann distribution of velocities is that all the quantitiesv, ∆v, and v are different. This is in contrast to the Gaussian distribution seen early on in the term.The various values are in the ratio

∆v : v : v ≈ 1.224 : 1.128 : 1.

Why do you think that ∆v > v > v?

One factoid that you might find interesting is that the numbers you get for air are surprisingly large.Assuming that air is mostly nitrogen molecules, with m = 4.65×10−26 kg at a temperature of 273 K,


one obtains v = 454 m/s, and ∆v = 493 m/s. But the speed of sound in air is 331 m/s at 273 K. Sothe molecules are moving considerably faster than the sound speed. Does this make sense?

There is an interesting application of this for the composition of the Earth’s atmosphere. First, let’sestimate the velocity a particle at the Earth’s surface would need to fully escape the gravitationalfield. The balance of kinetic and potential energies implies (1/2)mv2 = mgR, where R is the radiusof the Earth. (Actually I think that this implies that the particle is at the Earth’s center, where allof it’s mass would be concentrated?). In any case, we obtain vescape =

√2gR ≈ 11 000 m/s. The

mean velocity for hydrogen molecules (mass of 1.66×10−27 kg) from the formula v =√

8kBT/πm =1600 m/s. But the Maxwell-Boltzmann velocity distribution has a very long tail at high velocities,which means that there are approximately 2 × 109 hydrogen molecules that travel at more than 6times the average velocity. So there are lots of hydrogen molecules escaping forever all the time.Thankfully, our supply is continually replenished by protons bombarding us from the sun. Muchmore serious is helium, which is being lost irretrievably, with no replenishing. In fact, the U.S.Government has been stockpiling huge reserves of liquid helium for years in preparation of a world-wide shortage. But over the past few years the current administration has softened its policy onhelium conservation and these stockpiles are slowly dwindling. In any case, for oxygen and nitrogenmolecules, whose mean velocities are about a factor of 4 lower than that of hydrogen, very few ofthese can actually escape. Whew!

6.4.1 Interlude on Averages

When the various averages were calculated above, we made explicit use of the particular form of theenergy, ǫ = mv2/2. Of course, if the energy levels are different, or the dimension of the problem isnot 3D like it was above, then the way we take averages is going to be different. So how does onetake averages in general using the canonical ensemble?

Recall that the general form for the average of something we can measure, call it B, is

B =∑

i

piBi,

where the sum is over all the accessible states of the system, and pi are the probabilities of occupyingthose states. In the canonical ensemble, those probabilities are

pi =gi exp(−βǫi)

Z,

where Z is the usual partition function, and I have explicitly inserted the degeneracy factor. So theaverage of B is defined as

B =

∑

i giBi exp(−βǫi)∑

i gi exp(−βǫi)≈∫

dk g(k)B(k) exp[−βǫ(k)]∫

dk g(k) exp[−βǫ(k)] =

∫∞

0 dǫg(ǫ)B(ǫ) exp(−βǫ)∫∞

0dǫg(ǫ) exp(−βǫ) .

Thus, the dimensionality, the dependence of the energy ǫ on the wavefector k or velocity v, and theinherent degeneracy of a given energy level are all buried in the density of states per unit energy,g(ǫ). In order to calculate any average, this must be done first.

For example, suppose that our fundamental excitations were ripples on the surface of water, where


ǫ(k) = αk3/2. This is an effective 2D system, so we use the expression for the 2D density of states,

g(ǫ) =Ak

2π

dk

dǫ=Ak

2π

d

dǫ

( ǫ

α

)2/3

=A

2π

( ǫ

α

)2/3(

2

3α2/3ǫ1/3

)

=A

3πα4/3ǫ1/3.

So this is what we would use to evaluate averages. Of course, the constant terms would disappear,but the energy-dependence of the density of states would not. For example, the mean energy perparticle for this problem would be

U =

∫∞

0 dǫ ǫ4/3 exp(−βǫ)∫∞

0dǫ ǫ1/3 exp(−βǫ) =

(kBT )7/3Γ

(

73

)

(kBT )4/3Γ(

43

) =4kBT

3.

6.4.2 Molecular Beams

One of these is operational in Nasser Moazzen-Ahmadi’s lab, so it’s good that you’re learning aboutit! Suppose that we have an oven containing lots of hot molecules. There’s a small hole in oneend, out of which shoot the molecules. On the same table in front of the hole is a series of wallswith small holes lined up horizontally with the exit hole of the oven. The idea here is that mostof the molecules moving off the horizontal axis will hit the various walls, and only those moleculesmoving in a narrow cone around the horizontal axis will make it to the screen. The question is:what distribution of velocities do the molecules have that hit the screen?

Evidently, once the molecules leave the last pinhole, they spread out and form a cone whose base isarea A on the screen. The number of molecules in the cone with velocities between v and v + dv,and in angles between θ and θ + dθ and between φ and φ+ dφ is

number of particles = Avt cos(θ)n(v) dv dΩ = Avt cos(θ)n(v) dvdθ sin(θ)dφ

4π.

The flux of molecules f(v)dv is the number of molecules per unit area per unit time,

f(v)dv =vn(v)dv

4π

∫ π/2

0

dθ cos(θ) sin(θ)

∫ 2π

0

dφ =vn(v)dv

4.

For the first integral, set x = sin(θ) so that dx = cos(θ). The integral is therefore x2/2 =

sin2(θ)/2|π/20 = 1/2. And the second integral is 2π. Using the Maxwell-Boltzmann distributionof velocities, we obtain the flux density as

f(v) =Nπ

8

(

2m

πkBT

)3/2

v3 exp

(

− mv2

2kBT

)

=Nλ3D8π2

(m

h

)3

v3 exp

(

− mv2

2kBT

)

.

What is the point? I’m not really sure, actually. Sometimes it’s good to know the distribution ofvelocities hitting a screen to interpret results of an experiment. One thing we can do right now is toderive the equation of state for an ideal gas using it! A crude way to do this is to assume that everytime the molecule strikes the surface with velocity v, and bounces off elastically, the screen picksup a momentum 2mv cos(θ), accounting for the angle off the horizontal axis. The mean pressure istherefore the integral over the pressure flux:

P =

∫

dv

∫ π/2

0

dθ

∫ 2π

0

dφ2p cos(θ)v cos(θ)n(v)dvsin(θ)dθdφ

4πV

=1

3V

∫ ∞

0

mvn(v)v dv =m

3V

∫ ∞

0

v2n(v)dv =Nm

3Vv2 =

Nm

3V

3kBT

m=NkBT

V.


A slightly less hokey derivation (maybe) is to say that the pressure per unit volume is the meanforce along the horizontal per unit area:

P =F

A=

∫

dvf(v)mv2x =N

Vmv2x.

But v2 = v2x + v2y + v2z = 3v2x. So

P =Nm

3Vv2 =

NkBT

V.

6.5 (Already covered in Sec. 6.2)

6.6 Gibbs’ Paradox

It turns out that much of what we have done so far is fundamentally wrong. One of the first peopleto realize this was Gibbs (of Free Energy fame!), so it is called Gibbs’ Paradox. Basically, heshowed that there was a problem with the definition of entropy that we have been using so far.The only way to resolve the paradox is using quantum mechanics, and we’ll cover this in the nextchapter. Of course, we have been using quantum mechanics already in order to define the accessibleenergy levels that particles can occupy. But so far, we haven’t been concerned with the particlesthemselves. In this section we’ll see the paradox, and next chapter we’ll resolve it using quantummechanics.

Recall that the entropy in the canonical ensemble is defined as

S = NkB ln(Z) +U

T= NkB ln(Z) +NkBT

∂ ln(Z)

∂T.

As shown in Section 6.2.3, the partition function for a monatomic ideal gas in a 3D box is simply

Z = V/λ3D, where V is the volume and λD =√

2πh2/mkBT is the de Broglie wavelength. Plugging

this in, we obtain

ln(Z) = ln(V )− 3

2ln

(

2πh2

mkB

)

+3

2ln(T ).

Evidently, ∂ ln(Z)/∂T = (1/Z)∂Z/∂T = 3/2T so

S = NkB

[

ln(V )− 3

2ln

(

2πh2

mkB

)

+3

2ln(T ) +

3

2

]

≡ NkB

[

ln(V ) +3

2ln(T ) + σ

]

,

where

σ =3

2

[

1− ln

(

2πh2

mkB

)]

is some constant we don’t really care about. Now to the paradox.

Consider a box of volume V containing N atoms. Now, suppose that a barrier is inserted thatdivides the box into two regions of equal volume V/2, each containing N/2 atoms. Now the total


entropy is the sum of the entropies for each region,

S =N

2kB

[

ln

(

V

2

)

+3

2ln(T ) + σ

]

+N

2kB

[

ln

(

V

2

)

+3

2ln(T ) + σ

]

= NkB

[

ln

(

V

2

)

+3

2ln(T ) + σ

]

6= NkB

[

ln(V ) +3

2ln(T ) + σ

]

.

In other words, simply putting in a barrier seems to have reduced the entropy of the particles inthe box by a factor of NkB ln(2). Recall that this is the same as the high-temperature entropy forthe two-level Pauli paramagnet (a.k.a. the coin). So it seems to suggest that the two sides of thepartition once the barrier is up take the place of ‘heads’ or ‘tails’ in that there are two kinds of statesatoms can occupy: the left or right partitions. But here’s the paradox: putting in the barrier hasn’tdone any work on the system, or added any heat, so the entropy should be invariant! And simplyremoving the barrier brings the entropy back where it was before. Simply reducing the capabilityof the particles in the left partition from accessing locations in the right partition (and vice versa)shouldn’t change the entropy, because in reality you can’t tell the difference between atomson the left or on the right.

Clearly, we are treating particles on the left and right as distinguishable objects, which has given riseto this paradox. How can we fix things? The cheap fix is to realize that because all of the particles arefundamentally indistinguishable, the N ! permutations of the particles among themselves shouldn’tlead to physically distinct realizations of the system. So, our calculation of Z must be too large bya factor of N !:

ZN ≡ ZNcorrect =

ZNold

N !,

where I explicitly make use of the fact that the N -particle partition function is N -fold product ofthe single-particle partition function Zold. With this ‘ad hoc’ solution,

ln(ZN ) = N ln(Zold)− ln(N !) = N ln(Zold)−N ln(N) +N.

Now the entropy of the total box is

S = NkB

[

ln(V ) +3

2ln(T ) + σ − ln(N) + 1

]

= NkB

[

ln

(

V

N

)

+3

2ln(T ) + σ + 1

]

.

Now, if we partition the box into two regions of volume V/2, each withN/2 particles, then ln(V/N) →ln(V/N) and the combined entropy of the two regions is identical to the original entropy. So Gibbs’paradox is resolved.

The equations for the mean energy, entropy, and free energy in terms of the proper partition functionZN are now

U = kBT2∂ ln(ZN )

∂T; S + kB ln(ZN ) + kBT

∂ ln(ZN )

∂T; F = −kBT ln(ZN ).

That is, the expression are identical to the ones we’ve seen before, except the N factors in fronthave disappeared, and the Z is replaced by a ZN .

The justification for the replacement of ZN by ZN/N ! was pretty hokey, however. Most of the nextchapter will be devoted to a formal quantum mechanical justification for it.

Chapter 7

Grand Canonical Ensemble

In the canonical ensemble, we considered a small subsystem with heat-conducting walls, in contactwith a gigantic system at equilibrium with itself. This gigantic system could be considered as a heatbath, or reservoir, because if the small system’s temperature was lower (or higher), heat could flowinto (or out of) the subsystem out of (or into) the reservoir, and it wouldn’t affect the temperatureof the reservoir at all. Now suppose that we draw only imaginary walls between our subsystem andthe reservoir, so that not only heat but also particles can flow between them. How do we properlydescribe the subsystem, i.e. how do we enumerate the number of ways the particles can occupy theaccessible (energy) quantum states, if both the energy and the particle number is fluctuating? Theanswer is that we must now use the Grand Canonical Ensemble.

7.1 Chemical Potential Again

Now let’s return to the particle in the 3D box. Including the Gibbs’ correction, the entropy is

S = NkB

[

ln

(

V

N

)

+3

2ln

(

mkBT

2πh2

)

+5

2

]

.

Now, U = 32NkBT , so kBT = 2U/3N . Inserting this into the above expression for the entropy gives

S = NkB

[

ln

(

V

N

)

+3

2ln

(

mU

3πh2N

)

+5

2

]

.

Now we can take the derivative with respect to N to obtain the chemical potential:

µ = −kBT[

ln

(

V

N

)

+3

2ln

(

mU

3πh2N

)

+5

2

]

+ kBT

(

1 +3

2

)

.

µ = −kBT[

ln

(

V

N

)

+3

2ln

(

mkBT

2πh2

)]

= kBT ln(

nλ3D)

,

where n = N/V is the density. Note that the subsystem stops conserving the number of particleswhen nλ3D = 1, or when the mean interparticle separation ℓ becomes comparable to the de Brogliewavelength, ℓ ≡ n−1/3 ∼ λD. But we already know that something quantum happens on the lengthscale of the de Broglie wavelength. We’ll come back to this a bit later. For very classical systems,

76


ℓ≫ λD which means that the chemical potential is usually large and negative for the particle in the3D box.

How can you calculate the chemical potential from the free energy? I’m sure that you were burningto find this out. So let’s do it. The free energy is F = U − TS so dF = dU − TdS − SdT =TdS − PdV + µdN − TdS − SdT , so

µ =∂F

∂N= −kBT

∂ ln (ZN)

∂N.

Let’s check that this gives the right answer for the particle in the 3D box. In that case, ZN =(V/λ3D)N/N ! so ln(ZN ) = N ln(V/λ3)−N ln(N)+N . So ∂ ln(ZN )/∂N = ln(V/λ3D)− ln(N)− 1+1,or µ = kBT ln(nλ3D). Suppose that the energies of the particles were larger by a constant factor of∆, so that ǫk = (h2k2/2m) + ∆. Then

Z =∑

k

exp

[

−β(

h2k2

2m+∆

)]

= exp(−β∆)∑

k

exp

(

−βh2k2

2m

)

=V

λ3Dexp(−β∆).

So, the chemical potential is nowµ = kBT ln(nλ3D) + ∆.

Clearly, the shift in the energies has led to exactly the same shift in the chemical potential. Thisadditional piece can be thought of as an ‘external chemical potential’ that increases the total value.If the shift had depended on position or momentum, though, then we would have needed to use theequipartition theorem to evaluate the contribution of the additional piece to the chemical potential.Also, if we had included vibration and rotation for molecules, say, then the chemical potential wouldhave increased as well. These would be ‘internal chemical potential’ contributions. Would thesetend to increase or decrease the chemical potential?

7.2 Grand Partition Function

Recall that in the derivation of the Boltzmann distribution in the canonical ensemble, we maximizedthe entropy (or the number of microstates) subject to the two constraints N =

∑

i ni and U =∑

i niǫi. So we had

d ln(Ω) + α

(

dN −∑

i

dni

)

+ β

(

dU −∑

i

ǫidni

)

= 0,

where α and β were the Lagrange multipliers fixing the two constraints (alternatively, they areunknown multiplicative constants in front of terms that are zero). So we immediately have thefollowing relations:

α =∂ ln(Ω)

∂N; β =

∂ ln(Ω)

∂U.

But above we have ∂S/∂N = −µ/T = kB(∂ ln(Ω)/∂N) because S = kB ln(Ω). So we immediatelyobtain α = −µ/kBT = −βµ. So the chemical potential is in fact the Lagrange multiplier that fixesthe number of particles in the total system. Putting these together, we have

µ = −kBT∂ ln(Ω)

∂N;

∂ ln(Ω)

∂U=

1

kBT.


⇒ ln(Ω) = βU − βµN + const.

⇒ Ω = A exp [β (U − µN)] ,

where A = exp(const.).

So far, this enumeration has been for the reservoir, which contains the fixed temperature. To findΩ for the subsystem, we note that UR = U − Es and NR = N −Ns. So

ΩR = A exp [β (UR − µNR)] = A exp [β (U − Es)− βµ (N −Ns)]

= A exp [β (U − µN)] exp [−β (Es − µNs)] = ΩsystemΩsubsystem.

Finally we obtainΩs = A exp [−β (Es − µNs)] .

The physical system is actually comprised of a very large number of these subsystems, all of whichhave the same temperature and chemical potential at equilibrium, but all of which have differentenergies Ei and number of particles Ni. The total number of ways of distibuting the particles Ω istherefore the sum of all of the subsystems’ contributions, Ω =

∑

i Ωs. So the probability of occupyinga given subsystem is the fraction of the distribution of a given subsystem over all of them,

pi =Ωi

W=

exp [−β (Ei − µNi)]∑

i exp [−β (Ei − µNi)]≡ exp [−β (Ei − µNi)]

Ξ,

where Ξ is the grand partition function,

Ξ ≡∑

i

exp [−β (Ei − µNi)] .

It is important to note that the way one obtains the grand partition function is different from theway it was done in the canonical ensemble. Suppose that the subsystem contains three accessibleenergy levels 0, ǫ, and 2ǫ. Now suppose that there is only one atom in the total system. How manyways can I distribute atoms in my energy states? I might have no atoms in any energy state (Es = 0,Ns = 0). I might have one atom in state 0 (Es = 0, Ns = 1). I might have one atom in state ǫ(Es = ǫ, Ns = 1), or I might have one atom in state 2ǫ (Es = 2ǫ, Ns = 1). So my grand partitionfunction is:

Ξ = exp[−β(0− µ0)] + exp[−β(0− µ1)] + exp[−β(ǫ− µ1)] + exp[−β(2ǫ− µ1)]

= 1 + exp(βµ) [1 + exp(−βǫ) + exp(−2βǫ)] .

Suppose instead that there are an unknown number of particles in the total system, but that eachof these three energy levels in my subsystem can only accommodate up to two particles. Then thegrand partition function is

Ξ = 1 + exp(βµ) [1 + exp(−βǫ) + exp[−2βǫ)]

+ exp(2βµ) [1 + exp(−βǫ) + 2 exp(−2βǫ) + exp(−3βǫ) + exp(−4βǫ)] ,

where in the second line I have recognized that with two particles we can have Es = 0 (both in state0), Es = ǫ (one in state 0, the other in state ǫ), Es = 2ǫ (either both in state ǫ, or one in state 0while the other is in state 2ǫ, thus the factor of two out front of this term), Es = 3ǫ (one in state ǫ,the other in state 2ǫ), and Es = 4ǫ (both in state 2ǫ). That’s how one constructs the grand partitionfunction in practice!


7.3 Grand Potential

As we did for the canonical ensemble, we can obtain thermodynamic quantities using the grandpartition function instead of the regular partition function. The entropy is defined as

S = −kB∑

i

pi ln(pi) = −kB∑

i

pi [−β(Ei − µNi)− ln(Ξ)] =U

T− µN

T+ kB ln(Ξ),

where U =∑

i piEi and N =∑

i piNi are the mean energy and mean particle number for thesubsystem, respectively. This expression can be inverted to yield

U − µN − TS = −kBT ln(Ξ) ≡ ΦG,

where ΦG is the grand potential. Sometimes this is also written as ΩG in statistical mechanicsbooks. Recall that in the canonical ensemble we had U − TS = F = −kBT ln(ZN ), where F is theHelmholtz free energy. So the grand potential is simply related to the free energy by ΦG = F −µN .Anyhow, the following thermodynamic relations immediately follow:

S = −∂ΦG

∂T; N = −∂ΦG

∂µ= kBT

∂ ln(Ξ)

∂µ.

Two other relations that follow in analogy with the results for the canonical ensemble are:

P = −∂ΦG

∂V; U = −∂ ln(Ξ)

∂β= kBT

2∂ ln(Ξ)

∂T.

Date post:	03-Apr-2020
Category:	Documents
Upload:	others
View:	0 times
Download:	0 times

Physics449-StatisticalMechanicsI-CourseNotespeople.ucalgary.ca/~dfeder/449/notes.pdf · The ideal...

Documents