5.3.17 · D2MLOps & Deployment

Visual walkthrough — Edge deployment and ONNX

2,045 words9 min readBack to topic

This is the central result behind Model Quantization and Pruning and the reason edge deployment fits models onto small chips.


Step 1 — Two rulers that must agree

PICTURE. Two horizontal rulers stacked. Top: the real weights, crowded near zero. Bottom: the 256 integer slots, evenly spaced. Our whole job is to glue the top ruler onto the bottom one.


Step 2 — Only the used range matters

PICTURE. The top ruler now has a green bracket around . Only that bracket gets stretched to fit the 256 integer slots below. The far tails are greyed out — wasted if included.


Step 3 — The simplest honest map: a straight line

PICTURE. A single straight red line on a graph: horizontal axis (integers), vertical axis (reals). Its slope is ; the point where it crosses sits at .


Step 4 — Solve for the slope by matching the ends

PICTURE. The green real-bracket of width sitting above 255 integer gaps; a callout shows = one gap's real width.


Step 5 — Solve for the zero-point , and why zero must be exact

PICTURE. The line again, with a bright yellow dot at — the exact spot where the ruler passes through real zero — and read off the horizontal axis.


Step 6 — Going both ways: quantize and dequantize

PICTURE. A real value shown as an arrow up to the line, snapping right onto integer , then back down to . The tiny gap between and is the rounding error, capped at .


Step 7 — Run the numbers (the worked example, drawn)

PICTURE. The specific line for this example, with the real point , the snapped integer , and the recovered marked, plus the shaded error band.

Reveal-to-check:

for range , int8
Integer code for
Recovered value

Step 8 — The degenerate cases (never let the reader fall off the edge)

PICTURE. Three mini number lines side by side: (A) collapsed range, (B) one outlier dragging the ruler, (C) an off-centre range with snapped to the nearest tick.


The one-picture summary

Recall Feynman retelling — the whole walkthrough in plain words

Imagine you own a shelf with exactly 256 labelled boxes (that's int8). Your model's weights are a pile of numbers that mostly huddle near zero. First you find the smallest and largest number you actually own — that's your green bracket. You stretch that bracket evenly across all 256 boxes; the real-width of one box is the scale . You make sure one special box sits exactly on the number zero — that box's label is the zero-point — because your model makes lots of zeros and you don't want to smudge them. To pack a number away, you round it to its nearest box; to use it later, you read the box's real value back. You lose at most half a box's worth of accuracy, so if you make the boxes narrow (a tight bracket) you barely lose anything. Special warnings: if all your numbers are the same, don't stretch anything (); if one crazy number is far away, ignore it when choosing the bracket, or it wastes all your boxes; and if your numbers aren't centred on zero, just snap the zero-box to the nearest label. That's quantization — the trick that shrinks a model 4× so it runs on a phone.


Connections