Stochastic processes — Markov chains, steady-state, random walks: Visual walkthrough

We use the 2-state weather chain from the parent as our running example, because two states can be drawn on a flat picture and watched frame by frame.

Step 1 — Draw the world: states and arrows

WHAT. Our system has exactly two possible situations, called states. We name them:

S = "today is Sunny"
R = "today is Rainy"

Each morning the weather is in exactly one of these two boxes. Tomorrow it may jump to the other box or stay. The chance of each jump is a fixed number that never changes — that is the whole "rule".

WHY draw arrows. A number like "80%" is invisible. An arrow from S back to S labelled $0.8$ makes "if sunny, 80% chance still sunny tomorrow" something you can point at. Four arrows carry the entire law of the system.

PICTURE. Look at Step figure 1. Two circles (states), four arrows (possible one-day moves), each arrow tagged with its probability.

Reading the arrows off the picture:

P_{SS} = 0.8, P_{SR} = 0.2, P_{RS} = 0.4, P_{RR} = 0.6.

Step 2 — Pack the arrows into a grid (the matrix $P$ )

WHAT. Instead of four loose labels, arrange them in a $2 \times 2$ grid so the machine is one object.

WHY a grid, and why this layout. We want a bookkeeping table where row = today, column = tomorrow. Then "the arrows leaving today's row" sit together and must sum to 1 — the picture's rule becomes "every row adds to 1".

PICTURE. Step figure 2: the same four arrows, now dropped into their grid cells; the row-sum $= 1$ highlighted.

Nothing new has happened — we only reorganised the four arrows. But now the whole rule is a single symbol $P$ .

Step 3 — Describe belief, not fact: the probability vector

WHAT. We rarely know today's weather for certain; we hold a belief: "70% chance sunny, 30% chance rainy". Write that as a pair of numbers.

WHY a vector, and why a row. Two beliefs need two slots — a length-2 list. We write it horizontally (a row) on purpose: in Step 4 the row will slide across the grid's rows, and that motion is exactly what mixes belief with the arrows. Writing it as a column would multiply the wrong way (that is the classic $P π$ mistake from the parent note).

PICTURE. Step figure 3: a horizontal bar split into a blue chunk (weight on S) and an orange chunk (weight on R), the two chunks filling the bar completely.

Step 4 — Push belief forward one day (where $π P$ is born)

WHAT. Given today's belief $π^{(t)} = (s, r)$ , compute tomorrow's chance of Sun.

WHY we sum two channels. Tomorrow can be sunny via two separate roads:

it was sunny today (weight $s$ ) and stayed sunny (arrow $0.8$ ),
it was rainy today (weight $r$ ) and flipped to sunny (arrow $0.4$ ).

These roads don't overlap (today is either S or R, never both), so we add their chances — again the Law of Total Probability, now summing over where you were.

PICTURE. Step figure 4: two coloured streams pouring into the S circle — a blue stream $s \cdot 0.8$ and an orange stream $r \cdot 0.4$ — merging into tomorrow's sunny probability.

Notice the pattern: to get tomorrow's S entry we ran the row $(s, r)$ down the S-column $(0.8, 0.4)$ , multiplying pairs and adding. That "slide-and-add across each column" is the operation we call multiplying a row vector by a matrix.

Step 5 — Watch the numbers march (and freeze)

WHAT. Start from a lopsided belief, say $π^{(0)} = (1, 0)$ (certain it is sunny today), and apply Step 4 over and over.

WHY iterate at all. The stationary claim is about the long run. We cannot see "long run" in one equation — we see it by watching the sequence and noticing it stops moving.

π^{(1)} π^{(2)} π^{(3)} π^{(\infty)} = (1, 0) P = (0.8, 0.2) = (0.8, 0.2) P = (0.72, 0.28) = (0.688, 0.312) ⋮ \to (0.6 \overline{6}, 0.3 \overline{3}) = (\frac{2}{3}, \frac{1}{3}) .

Check the $π^{(3)}$ step against Step 4: sunny inflow $= 0.72 \cdot 0.8 + 0.28 \cdot 0.4 = 0.576 + 0.112 = 0.688$ , and rainy $= 0.72 \cdot 0.2 + 0.28 \cdot 0.6 = 0.144 + 0.168 = 0.312$ . Every day the gap to $2/3$ shrinks.

PICTURE. Step figure 5: the S-probability plotted against day number, starting at $1$ and curving down to a flat dashed line at $2/3$ . Two different starting beliefs both drain toward the same line.

Step 6 — The frozen equation: $π = π P$

WHAT. "Frozen" means: applying one more day changes nothing. Call the frozen belief $π = (s, r)$ .

WHY this becomes an eigen-equation. "One more day does nothing" is literally $π P = π$ . That is the definition of $π$ being a left eigenvector of $P$ with eigenvalue $1$ (see Linear Algebra — Eigenvalues and Eigenvectors): a direction the machine leaves unchanged.

Set $π^{(t + 1)} = π^{(t)} = π$ in Step 4's S-equation:

frozen S s = stay-sunny inflow s \cdot 0.8 + rain-to-sun inflow r \cdot 0.4 .

PICTURE. Step figure 6: the S circle with inflow arrows balanced by an equal outflow arrow — a see-saw exactly level. Flow into S equals flow out of S.

Step 7 — Edge & degenerate cases (so no scenario surprises you)

A picture that only shows the nice case is a trap. Here are the ways the "settle to a unique $π$ " story can break, each with its own frame in Step figure 7.

Case A — a trap door (reducible). If some arrows are missing so a state, once entered, can never be left (an absorbing state), belief drains into it and stays. There can then be several stationary distributions — one for each trap — so $π$ is not unique.

Case B — a forced pendulum (periodic). Suppose from S you always go to R and from R you always go back to S (arrows $0.8, 0.4$ replaced by $1$ and $1$ ). Then belief flips $(1, 0) \to (0, 1) \to (1, 0) \to \dots$ forever. A stationary $π = (\frac{1}{2}, \frac{1}{2})$ still exists, but a lopsided start never converges to it — it just oscillates.

Case C — the healthy case (irreducible + aperiodic). Our weather chain has some chance of staying put ( $0.8, 0.6 > 0$ ) and every state reaches every other. This kills both diseases: no traps, no forced cycle. Then $π$ is unique and every start converges to it — the guarantee comes from the Perron–Frobenius Theorem.

PICTURE. Step figure 7: three mini-diagrams side by side — a trap door, a rigid two-cycle, and our healthy self-looping chain — with a ✓ only on the third.

The one-picture summary

Everything above compressed into one frame: a lopsided starting bar, the arrow-grid $P$ acting once per day, the belief-bar re-splitting and drifting, and the frozen $\frac{2}{3} : \frac{1}{3}$ line where inflow balances outflow.

π^{(0)} \times P π^{(1)} \times P π^{(2)} \dots ⟶ π where π = π P, \sum π_{i} = 1.

Recall Feynman retelling (explain the whole walkthrough to a 12-year-old)

Picture a frog on two lily pads, Sunny-pad and Rainy-pad. Each morning it flips a loaded coin whose odds depend only on the pad it's sitting on right now — never on yesterday. I drew those odds as arrows (Step 1) and stacked them into a little 2×2 table (Step 2). My "guess" about where the frog is I wrote as a bar split into a blue part and an orange part (Step 3). To guess tomorrow, I pour today's blue and orange into each pad through the arrows and see how much lands on each (Step 4) — that pouring is multiplying my row-guess by the table, written $π P$ . When I keep pouring day after day, the split stops sliding and locks at two-thirds Sunny, one-third Rainy (Step 5), no matter where I started. "Locked" means pouring one more day changes nothing, $π = π P$ , which is just "as much probability flows into a pad as flows out" — a level see-saw (Step 6). The one catch: this only locks nicely if the frog can reach every pad and isn't forced to alternate like a metronome; a trap-pad or a rigid flip-flop breaks it (Step 7).

Connections

Parent topic — the full theory this page draws.
Linear Algebra — Eigenvalues and Eigenvectors — $π = π P$ is a left eigenvector for eigenvalue 1.
Perron–Frobenius Theorem — guarantees uniqueness/convergence in Case C.
Law of Total Probability — the summing-over-yesterday that builds Step 4.
Google PageRank — the same $π = π P$ solved on the whole web.