Information & Bits

The unifying principle

Information is the reduction of uncertainty. If a system can be in one of $N$ equally likely states, identifying the actual state carries

$H = \log_2 N \quad \text{bits.}$

More generally, for a distribution $\{p_i\}$, Shannon's entropy is

$H = -\sum_i p_i \log_2 p_i \quad \text{(bits).}$

The bridge appears when we notice this is the same functional as physical entropy. Boltzmann/Gibbs entropy is

$S = -k_B \sum_i p_i \ln p_i = (k_B \ln 2)\, H,$

so one bit of information corresponds to a definite chunk of thermodynamic entropy, $S_{\text{bit}} = k_B \ln 2$. Landauer's principle turns this into a hard physical cost: irreversibly erasing one bit must dissipate at least

$E_{\min} = k_B T \ln 2 \approx 2.9 \times 10^{-21}\ \text{J at } 300\,\text{K}.$

The unifying claim: the number of distinguishable physical states sets both the storage capacity and the energy budget. Everything below is a realization of $H = \log_2(\text{number of states})$.

How it shows up in each field

Coding/CS

Here the bit is an idealized symbol in a channel. Shannon's noisy-channel theorem gives the maximum reliable rate through a channel of bandwidth $B$ and signal-to-noise ratio $S/N$:

$C = B \log_2\!\left(1 + \frac{S}{N}\right) \quad \text{bits/s.}$

The "number of distinguishable states" reappears as the number of noise-separated signal levels. Error-correcting codes (Hamming, Reed–Solomon, LDPC) trade extra bits for reliability.

Physics

The bit is a microstate count and a thermodynamic resource. Landauer and Bennett showed logically reversible computation can in principle cost zero energy, but erasure cannot. Maxwell's demon is exorcised precisely by charging the demon $k_B T \ln 2$ per bit erased from its memory. In quantum mechanics the unit is the qubit, a state $|\psi\rangle = \alpha|0\rangle + \beta|1\rangle$; a measured qubit yields at most one classical bit (Holevo bound), yet $n$ qubits span a $2^n$-dimensional space.

Biology

DNA is a literal quaternary code: each base $\in \{A,C,G,T\}$ carries up to $\log_2 4 = 2$ bits.

$H_{\text{max}} = 2\ \text{bits/base}.$

The human genome ($\sim 3.2\times 10^9$ bases) is therefore $\sim 6.4\times 10^9$ bits $\approx 800\ \text{MB}$ of raw capacity. Codons ($3$ bases $= 6$ bits) redundantly encode $20$ amino acids ($\log_2 20 \approx 4.3$ bits)—built-in error tolerance, exactly the coding-theory logic of redundancy against copying noise.

Hardware

A bit is a charged node. In DRAM it is charge $Q = CV$ on a capacitor (present = 1, empty = 0); in flash it is trapped charge on a floating gate; in SRAM it is the stable state of cross-coupled transistors. Distinguishability requires the stored energy to exceed thermal noise:

$\tfrac{1}{2}CV^2 \gg k_B T,$

otherwise the bit flips spontaneously. Every logic gate that overwrites its inputs pays (in practice far above) the Landauer floor, which is why data-center energy scales with bit operations.

Why this bridge matters

• Coding → Physics: Shannon entropy is thermodynamic entropy up to $k_B \ln 2$; this lets physicists treat measurement and computation as physical processes with energy budgets (resolving Maxwell's demon).
• Physics → Hardware: The Landauer and thermal-noise bounds set the ultimate floor for how small and low-power transistors can get—a north star for chip design as we approach few-electron devices.
• Coding → Biology: Error-correction and channel-capacity thinking explain codon redundancy, DNA repair enzymes, and mutation rates as an evolved coding scheme operating over a noisy replication channel.
• Biology → Hardware: DNA's density ($\sim 10^{18}$ bytes/gram) inspires molecular data storage, transferring the $2$-bit/base insight to archival memory.

The single transferable intuition: count the distinguishable states, and you simultaneously know the information capacity and the minimum energy to manipulate it.

Connections

• 01 Shannon Entropy & Channel Capacity
• 02 Error-Correcting Codes
• 03 Landauer's Principle & Reversible Computing
• 04 Boltzmann–Gibbs Entropy
• 05 Qubits & Quantum Information
• 06 Maxwell's Demon
• 07 DNA as a Genetic Code
• 08 DRAM, Flash & Charge Storage
• 09 Thermal Noise in Devices

#bridge