Exercises — Boltzmann's entropy S = k_B ln(Ω)

• (a) $S$ is entropy, units J/K (joule per kelvin). It is a macroscopic, thermodynamic quantity.
• (b) $k_B$ is Boltzmann's constant, $1.380649\times 10^{-23}$ J/K. Its whole job is to carry the units.
• (c) $\Omega$ is the multiplicity — a pure number, the count of microstates matching the macrostate. A count has no units.
• $\ln$ of a pure number is a pure number, so $\ln \Omega$ is dimensionless. All the J/K in $S$ comes from $k_B$. ✔

$\Delta S=S_Y-S_X=k_B(\ln 32-\ln 8)=k_B\ln \frac{32}{8}=k_B\ln 4$. Why the subtraction becomes a ratio: $\ln a-\ln b=\ln (a/b)$ — logarithms turn division into subtraction. Since $4=2^2$, $\ln 4=2\ln 2\approx 1.386$. So $\Delta S\approx 1.386k_B$. ✔

The number of ways to choose which $r$ of $4$ coins are heads is the binomial coefficient $\left(\genfrac{}{}{0ex}{}{4}{r}\right)=\frac{4!}{r!(4-r)!}$ (this is just "how many distinct arrangements have $r$ heads"):

• 0 heads: $\left(\genfrac{}{}{0ex}{}{4}{0}\right)=1$
• 1 head: $\left(\genfrac{}{}{0ex}{}{4}{1}\right)=4$
• 2 heads: $\left(\genfrac{}{}{0ex}{}{4}{2}\right)=6$
• 3 heads: $\left(\genfrac{}{}{0ex}{}{4}{3}\right)=4$
• 4 heads: $\left(\genfrac{}{}{0ex}{}{4}{4}\right)=1$

Total $=1+4+6+4+1=16=2^4$ ✔ (every coin independently H or T). The peak is "2 heads" with $\Omega =6$, so $S/k_B=\ln 6\approx 1.792$. The bell shape in the figure shows why the middle macrostate dominates. ✔

The multiplicity of a gas factorizes into a position part and a momentum part: $\Omega \propto (\text{position volume})\times (\text{momentum-shell volume})$. Only the position part changes here.

• Momentum part is unchanged. In free expansion into vacuum the gas does no work ($p_{\text{ext}}=0$) and exchanges no heat, so its total kinetic energy $E$ is unchanged. The momentum-shell volume depends only on $E$ (it is the surface $\sum p_i^2=2mE$), so with $E$ fixed the momentum count is identical before and after. This is why the process is also isothermal for an ideal gas ($E$ depends only on $T$).
• Position part doubles per particle. Doubling the accessible volume doubles each particle's position choices, so $\Omega \to 2^N\Omega$ (each of $N$ particles independently gains a factor 2).

$\Delta S=k_B\ln (2^N\Omega )-k_B\ln \Omega =k_{B}N\ln 2=R\ln 2.$ Numerically $\Delta S=8.314\times 0.693147=5.76\text{J/K}$. ✔ This equals the textbook $nR\ln (V_f/V_i)$ with $n=1,V_f/V_i=2$ — the counting picture reproduces classical thermodynamics.

(a) Each spin is independently up/down, so choosing which $n$ of the $N$ are up gives $\Omega (n)=\left(\genfrac{}{}{0ex}{}{N}{n}\right)=\frac{N!}{n!(N-n)!}$. (b) $\left(\genfrac{}{}{0ex}{}{N}{n}\right)$ peaks at $n=N/2=50$. There $\Omega =\left(\genfrac{}{}{0ex}{}{100}{50}\right)\approx 1.0089\times 10^{29}$, so $\frac{S}{k_B}=\ln \left(\genfrac{}{}{0ex}{}{100}{50}\right)=\mathrm{66.78.}$ (c) Stirling's approximation comes in two grades. The crude form $\ln m!\approx m\ln m-m$ handles huge factorials but loses the sub-leading piece. The refined form keeps one more term: $\ln m!\approx m\ln m-m+\frac{1}{2}\ln (2\pi m).$ Substitute this refined form into $\ln \left(\genfrac{}{}{0ex}{}{N}{N/2}\right)=\ln N!-2\ln (N/2)!$. The leading $m\ln m-m$ pieces collapse to $N\ln 2$ (the same crude answer), and the three $\frac{1}{2}\ln (2\pi m)$ terms combine: $\frac{1}{2}\ln (2\pi N)-2\cdot \frac{1}{2}\ln (2\pi \frac{N}{2})=\frac{1}{2}\ln \frac{2\pi N}{(2\pi N/2{)}^2}=\frac{1}{2}\ln \frac{2}{\pi N}=-\frac{1}{2}\ln \frac{\pi N}{2}.$ That is exactly the correction term — now derived, not quoted. So $\ln \left(\genfrac{}{}{0ex}{}{N}{N/2}\right)\approx N\ln 2-\frac{1}{2}\ln (\pi N/2)$. With $N=100$: $100\ln 2-\frac{1}{2}\ln (50\pi )=69.315-2.531=\mathrm{66.78.}$ The two agree to two decimals — Stirling is superb even at $N=100$. ✔ (For real gases $N\sim 10^{23}$, where exact factorials are impossible and Stirling is essential.)

Take the log, keeping every factor explicit (nothing hidden in a "const"): $\frac{S}{k_B}=N\ln V-N\ln h^3-\ln N!+\frac{3N}{2}\ln (2\pi mE)-\ln \Gamma (\frac{3N}{2}).$ Now apply Stirling to both $\ln N!\approx N\ln N-N$ and $\ln \Gamma (\frac{3N}{2})\approx \frac{3N}{2}\ln \frac{3N}{2}-\frac{3N}{2}$. Group terms: $\frac{S}{k_B}=N\ln \frac{V}{N}+\frac{3N}{2}\ln \left(\frac{2\pi mE}{(3N/2)}\right)+\frac{5N}{2}-3N\ln h.$ Recognising $E=\frac{3}{2}N{k}_{B}T$ and tidying gives the famous Sackur–Tetrode form $S=N{k}_B\left[\ln \left(\frac{V}{N}{\left(\frac{4\pi mE}{3N{h}^2}\right)}^{3/2}\right)+\frac{5}{2}\right],$ in which Planck's constant $h$ sits inside the log — that is where quantum mechanics enters the entropy of a classical gas, and it is why $S$ has an absolute zero-point (the Third Law of Thermodynamics). (a) Extensivity test: scale $V\to \lambda V,N\to \lambda N,E\to \lambda E$. Inside the bracket $\frac{V}{N}\to \frac{V}{N}$ (intensive) and $\frac{E}{N}\to \frac{E}{N}$ (intensive), so the whole bracket is unchanged; only the prefactor $N{k}_B\to \lambda N{k}_B$. Hence $S\to \lambda S$. Extensive ✔ — the $-\ln N!$ is what converted $\ln V^N$ into $N\ln (V/N)$, replacing $V$ by the intensive density ratio $V/N$. (b) Without $1/N!$: the $-N\ln N$ from Stirling disappears, leaving $\frac{S}{k_B}\supset N\ln V$ instead of $N\ln (V/N)$. Scaling now gives $\lambda N\ln (\lambda V)=\lambda N\ln V+\lambda N\ln \lambda$, an extra $\lambda N\ln \lambda$ term that has no business being there. Mixing two identical gas boxes ($\lambda =2$) then predicts a spurious entropy of mixing $\Delta S=2N{k}_B\ln 2\ne 0$ even though nothing physical happened — Gibbs' paradox. The $1/N!$ removes it. ✔ See Gibbs Paradox & Indistinguishability.

(a) $S/k_B=\ln \left(\genfrac{}{}{0ex}{}{N}{n}\right)=\ln \frac{N!}{n!(N-n)!}$. (b) Why we may treat $n$ as continuous: $\Omega (n)$ jumps by discrete steps, but for $N$ large the fractional change from $n$ to $n+1$ is tiny ($\sim 1/N$), so the staircase is smooth on the scale that matters and we may differentiate. Using the refined Stirling form (as derived in L3.1) on each factorial, $\frac{d}{dn}\ln \left(\genfrac{}{}{0ex}{}{N}{n}\right)=\ln \frac{N-n}{n}(\text{valid for }1\ll n\ll N-1).$ Edge caution: at $n=0$ or $n=N$ this derivative diverges ($\ln$ of $0$ or $\infty$) and the continuous approximation fails — there $\Omega =1$, $S=0$ exactly, and one must treat the ends discretely. Away from the edges the formula is excellent. Since $E=n\epsilon$, $\frac{\partial }{\partial E}=\frac{1}{\epsilon }\frac{d}{dn}$, so $\frac{1}{T}=\frac{k_B}{\epsilon }\ln \frac{N-n}{n}.$

• If $n<N/2$: $\frac{N-n}{n}>1$, $\ln (\cdot )>0$, so $T>0$ (normal). Adding energy adds microstates.
• At $n=N/2$: $\ln 1=0$, $1/T=0$, so $T\to +\infty$ — multiplicity is maximal, entropy peaks.
• If $n>N/2$: $\frac{N-n}{n}<1$, $\ln (\cdot )<0$, so $\boxed{T<0}$. Adding energy now removes microstates ($\partial S/\partial E<0$). Negative temperature — hotter than any positive $T$. See Negative Temperature & Spin Systems. ✔

(c) $N=1000,n=750$ (well away from the edges, so the formula applies): $\frac{N-n}{n}=\frac{250}{750}=\frac{1}{3}$, so $\ln \frac{1}{3}=-1.0986$. $\frac{1}{T}=\frac{k_B}{\epsilon }(-1.0986)\Rightarrow T=\frac{\epsilon }{k_B}\cdot \frac{1}{-1.0986}=-0.9102\frac{\epsilon }{k_B}.$ So $T\approx -0.910\epsilon /k_B$ — negative, as predicted. ✔ The figure shows $S(E)$ rising, peaking at $E=N\epsilon /2$, then falling; $n=750$ lives on the falling ($T<0$) side.

If each of $N_A$ molecules has $W=3/2$ independent orientational choices, the total multiplicity is $\Omega =W^{N_A}$ (independent choices multiply). Then $S_0=k_B\ln \left(W^{N_A}\right)=N_A{k}_B\ln W=R\ln \frac{3}{2}.$ Numerically $S_0=8.314\times \ln 1.5=8.314\times 0.405465=3.37\text{J/(mol⋅K)}$. ✔ This matches the experimental $\approx 3.4$ J/(mol·K) beautifully — a striking confirmation that $S=k_B\ln \Omega$ literally counts orientational disorder, and that the Third Law of Thermodynamics ($S\to 0$) requires a perfectly ordered crystal, which ice is not.