TensorBoard - Weights & Biases logging: Question bank

This page is a misconception hunt. Each line below is a trap: a statement that sounds right, a piece of code that looks fine, or a "why" that people fumble. Read the left side, commit to an answer out loud, then reveal the right side. The reveals are short but they always explain the reasoning, never just "yes/no".

If any word here is unfamiliar, first read the parent note TensorBoard / W&B logging — this page assumes you already met SummaryWriter, wandb.init, scalars, histograms, event files, and the step argument.

True or false — justify

Every item: decide true or false, and say why — a bare T/F earns nothing.

TensorBoard needs your training script to be running in order to show plots.

False. TensorBoard reads event files off disk independently; the writer and the server are decoupled, so you can view completed or crashed runs later.

writer.add_scalar('Loss/train', loss, step) with step left out will still plot correctly.

False-ish / dangerous. If step is omitted it auto-increments from an internal counter, so two writers or a resumed run will collide and the curve zig-zags or overwrites.

W&B stores your metrics only in the cloud, so if the network drops you lose the run.

False. wandb.log writes to a local wandb/ directory first and syncs; on reconnect it uploads. Offline mode (WANDB_MODE=offline) makes this explicit.

TensorBoard and W&B are mutually exclusive — you pick one.

False. They log different things well (TensorBoard = local, fast, histograms/graphs; W&B = cloud, sweeps, artifacts) and W&B can even ingest TensorBoard event files.

Logging every batch instead of every N batches makes the plots more accurate.

False in spirit. It makes them noisier and slower to load; the per-batch loss is high-variance, so you usually smooth or subsample precisely to see the trend.

Weight histograms and loss scalars answer the same question.

False. Scalars answer "is the model getting better?"; histograms answer "is each layer alive and learning?" — a flat, unchanging weight histogram signals a dead or non-learning layer even if loss drops.

wandb.config values can be edited freely mid-training to reflect changes.

False (mostly). config is meant to record the inputs of the run for reproducibility; treating it as a mutable scratchpad breaks the promise that config = "how this run was launched".

If two runs share the same logdir subfolder, TensorBoard merges them into one clean curve.

False. It overlays both event streams in that folder; steps interleave and you get a tangled, misleading line — one run = one subdirectory.

Spot the error

Each snippet has one conceptual bug. Name it and say what it causes.

writer = SummaryWriter('runs/exp') … then the loop logs, but there is no writer.close().

Bug: unflushed buffer. Without close() (or flush()) the last events may never hit disk, so the tail of your curve silently disappears.

Inside the batch loop: writer.add_scalar('Loss', loss.item(), epoch).

Bug: wrong step variable. Using epoch for a per-batch log means every batch in an epoch writes to the same x-value, overwriting itself — use global_step = epoch * len(loader) + batch_idx.

writer.add_scalar('Loss/train', loss, step) — loss is the raw tensor, not loss.item().

Bug: logging a tensor holds a reference to the whole computation graph, leaking GPU memory (and may error). Always log a Python float via .item().

wandb.init(project="x") is called inside the epoch loop.

Bug: a new run is created every epoch, so you get 10 fragmented runs instead of 1. init belongs once, before the loop.

wandb.log({"loss": train_loss}); wandb.log({"acc": val_acc}) (two separate calls, same epoch).

Bug: each log call advances W&B's internal step, so loss and acc land on different x-values and never line up. Log related metrics in one dict per step.

writer.add_image('pred', img_grid) where img_grid is shape (H, W, 3).

Bug: channel order. add_image expects CHW (3, H, W) by default; passing HWC scrambles the image unless you set dataformats='HWC'.

total_norm += p.grad.norm(2) then logging total_norm ** 0.5 as the gradient norm.

Bug: you must square each per-parameter norm before summing (norm(2)**2), then take the root. Summing raw norms and rooting gives the wrong global norm.

Logging wandb.Image(tensor) where tensor is un-normalized (values in [-2, 2] after standardization).

Bug: display range. W&B/PIL expect [0,1] or [0,255]; the un-de-normalized tensor clips to garbage. De-normalize before visualizing.

Why questions

The reveal is the reasoning, not a label.

Why are event files append-only binary logs instead of, say, a CSV you overwrite?

Append-only means a crash mid-write can't corrupt earlier data, and TensorBoard can tail the file live; binary keeps histograms/images compact where CSV can't hold them at all.

Why does W&B log the git commit and library versions automatically?

This is provenance — six months later "which code produced this curve?" is answerable, giving true reproducibility instead of a lonely number. (See model versioning.)

Why log gradient norms as a diagnostic scalar rather than trusting the loss curve?

Loss can look fine while gradients quietly vanish or explode inside deep layers; the norm is a direct early-warning signal. (See vanishing / exploding gradients.)

Why does W&B exist at all if TensorBoard already plots everything?

TensorBoard is local and single-user; W&B adds cloud sharing, sweep tracking, and artifact versioning — features that matter when a team runs hundreds of experiments.

Why is a hyperparameter sweep better tracked in W&B than by naming folders run_lr0.01_bs64?

Folder names don't scale, aren't queryable, and can't be sorted by a metric; W&B's sweep table links each config to its result so you can rank and filter automatically. (See hyperparameter tuning.)

Why should you log validation loss on epoch but training loss on global_step?

You compute validation once per epoch, so epoch is its natural x-axis; training loss updates every batch, so it needs the finer global_step to show intra-epoch dynamics.

Why does smoothing in the TensorBoard SCALARS view not change your saved data?

Smoothing is a display transform (an exponential moving average) applied in the browser; the underlying event file still holds the raw values, so you can toggle it freely.

Edge cases

The boundary and degenerate scenarios people never test until they break.

What does TensorBoard show if you log zero scalars but many histograms?

The SCALARS tab is empty (not an error) and only HISTOGRAMS populates — each tab is independent, so missing one data type just hides that tab's content.

You resume training from a checkpoint at epoch 50 but restart global_step at 0. What happens?

The new curve overwrites/interleaves the old x-range, so the plot appears to "go back in time." Continue the step counter from where the checkpoint left off.

You call wandb.log({"loss": float('nan')}). What does W&B do?

It records the NaN and the point drops out / shows a gap on the plot — a useful signal that training diverged rather than a crash. Treat gaps as "check for NaN".

Two teammates run the same script; both call wandb.init(project="x") with no name. Do runs collide?

No — each init mints a unique run ID, so they appear as two distinct runs even with identical (auto-generated) display names.

You log an image every step for a 100k-step run. What's the failure mode?

Disk / upload bloat — images are large; either the event dir balloons or the W&B upload lags far behind. Log images every N steps or only on validation.

An empty val_loader (dataset of size 0) reaches val_loss /= len(val_loader).

Division by zero — the metric becomes inf/nan and pollutes the plot. Guard degenerate loaders before logging any averaged metric.

You point tensorboard --logdir=runs/ at a folder with no event files.

TensorBoard starts fine but shows "No dashboards are active" — it's a valid empty state, usually meaning the writer wrote to a different path than you're pointing at.

Recall Fast self-test (peek only after answering all four sections)

One-tool-vs-the-other ::: TensorBoard = local, decoupled event files, great histograms/graphs; W&B = cloud, sweeps, artifact versioning, collaboration. The single most common bug on this page ::: using the wrong step variable (epoch where global_step belongs, or separate wandb.log calls) so points misalign. The safety habit for artifacts ::: always close()/flush() the writer and log related metrics in one dict per step.