TensorBoard - Weights & Biases logging: Visual walkthrough

You already met the parent note: you sprinkle writer.add_scalar('Loss/train', loss, step) into your training loop, and later a smooth loss curve appears in your browser. But how? What actually happens between "a single floating-point number in Python" and "a pixel curve you can zoom into"?

This page derives that whole pipeline from zero, one picture per step. No prior knowledge of files, servers, or plotting is assumed. By the end you will be able to draw the entire path a number takes, from RAM to retina.

Step 1 — A metric is just a `(step, value)` dot

WHAT. The smallest unit of logging is a pair of numbers: when something was measured and what it was. We write this pair as $(s, v)$ .

$s$ = the step (an integer: which batch or epoch we are on). It plays the role of "time".
$v$ = the value (a real number: the loss, the accuracy, whatever we measured).

WHY a pair and not just $v$ ? If we only kept the value, all our loss numbers would pile up on top of each other with no order — like a diary with no dates. The step $s$ is the horizontal coordinate that spreads the values out so a shape can emerge. This is exactly why the parent's call is add_scalar(name, value, step) — the step is not optional decoration, it is the x-axis itself.

PICTURE. One measurement is one dot in a 2D plane: step across the bottom, value up the side.

Figure — TensorBoard - Weights & Biases logging

Step 2 — Many dots, in the order they were born

WHAT. Training runs a loop. Each time we call add_scalar, one new dot $(s_{i}, v_{i})$ is created. After $n$ calls we have a sequence:

$(s_{1}, v_{1}), (s_{2}, v_{2}), \dots, (s_{n}, v_{n})$

Reading term by term: the subscript $i$ just numbers the events $1, 2, 3, \dots$ in the order they happened; $s_{i}$ is the step recorded at event $i$ ; $v_{i}$ is the value recorded at event $i$ .

WHY keep them in order and never edit old ones? Because the event stream is append-only: we only ever add to the end, never rewrite the middle. This is the single most important design choice, and it buys us crash-safety — if training dies at event 57, events 1–57 are already safely on disk. The parent note calls these "append-only binary logs → crash-safe, can resume." Here is why that property matters, drawn out.

PICTURE. The dots accumulate left-to-right as the loop turns. A crash just stops the growth; it never corrupts what came before.

Step 3 — Serialization: turning a dot into bytes on disk

WHAT. RAM is temporary. To survive, each event must be written to a file as a fixed pattern of bytes (a byte = 8 bits = a number from 0 to 255). Turning the pair $(s, v)$ into such a byte pattern is called serialization.

Conceptually, the writer lays down, for each event:

$how many bytes follow [length] the string [name bytes] 4 bytes [s] 4 bytes [v] integrity guard [checksum]$

Each bracket is a fixed field. The length at the front says how far this record stretches, so the reader knows where one event ends and the next begins. The checksum at the end is a small number computed from the bytes; if a byte gets corrupted, recomputing it won't match, and the reader can skip that record instead of showing garbage.

WHY bytes and not human-readable text (like CSV)? Two reasons. Speed: a 4-byte float is written in one shot, no formatting. And self-describing framing: the length prefix lets the reader recover its place even after a partial/corrupt record — which is what makes append-only logs robust. This is the "event files"/binary log the parent mentions.

PICTURE. One dot becomes one framed record; records stack head-to-tail into the growing file.

Step 4 — The reader replays the file back into dots

WHAT. A separate program — the TensorBoard server — opens the event file and walks it from the start. For each record it reads the length, grabs that many bytes, checks the checksum, and deserializes the bytes back into a pair $(s, v)$ . Deserialization is just serialization run backwards.

WHY a separate program? Because the reader is decoupled from the trainer. Your training process can be finished, or even crashed, and the server still reads whatever is on disk. This is why in the parent note you launch tensorboard --logdir=runs/ as its own command — it is a different process entirely, coupled to your training only through the file.

PICTURE. The byte stream flows into the reader and reconstitutes the exact same sequence of dots from Step 2 — an inverse of Step 3.

Step 5 — Connecting the dots into a line

WHAT. The reader now holds the dots in step-order. To draw a curve, the plotter connects consecutive dots with straight line segments. Between event $i$ and event $i + 1$ it draws the segment from $(s_{i}, v_{i})$ to $(s_{i + 1}, v_{i + 1})$ .

WHY straight segments (piecewise-linear)? Because we have no information about what happened between two logged steps — the loss between step 100 and step 200 was never recorded. The honest choice is the simplest interpolation: a straight line. It never invents wiggles that weren't measured. This is also why logging more often gives a smoother, more truthful curve — you supply more real dots, so the straight segments get shorter.

PICTURE. Dots from Step 2, now joined by segments — the recognizable loss curve is born.

Step 6 — Smoothing: why the raw curve is often jagged

WHAT. Mini-batch loss is noisy: batch 41 happened to be easy, batch 42 hard, so $v$ jitters. TensorBoard's SCALARS view offers a smoothing slider. The smoothed value $\tilde{v}_{i}$ is an exponential moving average:

$\tilde{v}_{i} = α \tilde{v}_{i - 1} + (1 - α) v_{i}$

Term by term: $\tilde{v}_{i}$ is the new smoothed point; $v_{i}$ is the raw value just measured; $\tilde{v}_{i - 1}$ is the previous smoothed point (carrying the memory of the past); $α$ (between 0 and 1) is the smoothing weight — the slider. When $α$ is near $1$ , the past dominates and the curve is very smooth but lags; when $α = 0$ , $\tilde{v}_{i} = v_{i}$ and you see the raw jitter.

WHY exponential and not a plain average of the last $k$ points? Because the exponential form needs only one stored number ( $\tilde{v}_{i - 1}$ ) to summarize all history — it is cheap and streams naturally as new dots arrive. A window average would need to remember the last $k$ raw values.

WHY does this matter? Smoothing lets you see the trend (is loss really going down?) without being fooled by batch-to-batch noise. But keep the raw curve visible too — extreme spikes there can signal exploding gradients, a diagnostic tied to gradient flow.

PICTURE. Raw jagged curve in one accent, the smoothed trend line overlaid in another.

Step 7 — Edge and degenerate cases (the ones that trip people up)

Real logs are messy. Each of these is a scenario you will hit.

Case A — a single dot ( $n = 1$ ). With only one event there is nothing to connect. TensorBoard shows a lone point, no line. Why: a segment needs two endpoints. If your curve is "empty", you probably logged only once.

Case B — NaN or Inf value. If $v$ becomes NaN (not-a-number, e.g. loss overflowed), it cannot be placed on the axis. TensorBoard drops it, leaving a gap. Why it happens: often exploding gradients — a real bug, not a plotting glitch. See vanishing/exploding gradients.

Case C — repeated / decreasing step. As warned in Step 2, duplicate steps put two $v$ at the same $x$ ; the line back-tracks. Why: the plotter trusts your $s$ blindly.

Case D — multiple runs, same metric name. Two directories under runs/ both log Loss/train. TensorBoard overlays them as separate colored curves — this is exactly the "compare runs" superpower, and the seed of hyperparameter tuning where each run is one hyperparameter combo.

PICTURE. A 2×2 panel, one degenerate case per quadrant.

The one-picture summary

The whole journey of a number, on a single canvas: it is born as a dot in the loop, gets framed into bytes and appended to a crash-safe file, is read back by an independent server, joined into a piecewise-linear curve, and optionally smoothed — with degenerate inputs handled gracefully at every stage.

Recall Feynman retelling — say the whole walkthrough in plain words

Imagine every loss number is a tiny messenger. In the training loop it's born knowing two things: when it exists (the step) and how big it is (the value) — that's a dot on graph paper. Because the loop runs fast and forgets, the messenger immediately gets sealed into an envelope of bytes, with a length written on the front and a little integrity stamp on the back, and dropped onto a growing stack of envelopes (the event file) — always at the top, never rewriting the middle, so a crash can't ruin the earlier ones. Later, a totally separate helper (the TensorBoard server) opens each envelope, checks the stamp, and unpacks the messenger back into the exact same dot. It lines up all the dots by their "when" and draws straight roads between neighbors — that's your loss curve. If the roads look too bumpy, a smoothing knob blends each dot with the memory of the ones before it, showing the trend without the jitter. And when things go weird — one lonely dot, a NaN gap, a run drawn on top of another — none of it is a bug in the drawing; each is the picture honestly telling you what the numbers did.

The one line to remember ::: A metric survives as an append-only, self-framed byte record, and a decoupled reader turns that record back into ordered dots joined into a piecewise-linear (optionally smoothed) curve. Why the step $s$ is mandatory ::: It is the x-axis; without it values pile up with no order and no shape can form. Why append-only bytes ::: Crash-safety and fast, self-describing framing so a corrupt record can be skipped. What smoothing's $α$ trades off ::: Higher $α$ = smoother trend but more lag; $α = 0$ shows raw jitter.

Where this leads

Overlaying many runs (Step 7, Case D) is the mechanism behind hyperparameter tuning.
The append-only + checksum idea (Steps 3–4) generalizes to model versioning and artifact tracking in W&B.
Curve spikes and NaN gaps (Step 6, Step 7B) are your early-warning system for exploding/vanishing gradients.
Prefer Hindi? Yeh walkthrough Hinglish mein.