5.4.3 · D2Scientific Computing (Python)

Visual walkthrough — Indexing and slicing — basic, boolean masking, fancy indexing

1,937 words9 min readBack to topic

We build everything from one object: an array a = [10, 20, 30, 40, 50].


Step 1 — What an array actually is: a line of boxes + a recipe

WHAT. Before any indexing, we draw the truth: the five numbers do not float in the air. They sit in one continuous strip of computer memory — box after box, no gaps. Call the address of the first box the offset. Call the number of boxes you jump to reach the next element the stride.

WHY. Every single indexing rule in this chapter is a consequence of how you are allowed to walk this strip. If you can describe your desired elements as "start here, jump this far each time," memory needs no copying. If you can't, it must copy. So we must see the strip first.

PICTURE. Look at the strip below. The number inside each box is its value; the number below each box is its position index (0,1,2,…). The blue label offset points at where index 0 lives; the pink arrow labelled stride = 1 shows the jump to the next element.

Figure — Indexing and slicing — basic, boolean masking, fancy indexing

See NumPy arrays — shape, strides, dtype for the full recipe object.


Step 2 — A slice just rewrites the recipe (this is the whole secret)

WHAT. We take the slice a[1:5:2]. It should give values at positions — that is [20, 40]. We now show what NumPy does not do: it does not copy 20 and 40 into a new strip. Instead it writes a new recipe pointing at the same strip.

WHY this tool — arithmetic on the recipe, not on the data. The question a slice answers is "can I reach every element I want with a single fixed jump?" For 1:5:2 the answer is yes: start at box 1, jump 2 boxes each time. So the only thing that changes is:

  • new offset → box 1 (where the slice starts),
  • new stride → 2 boxes (the slice's step),
  • new length → how many steps fit.

No value is touched, so it is instant even for a billion-element array. That is exactly why a slice returns a viewViews vs Copies — memory model.

PICTURE. Same strip as Step 1. The yellow recipe box on the left is the original . The blue recipe box is the slice . Follow the blue arrows: they land only on boxes 1 and 3 — inside the original strip, never on a new one.

Figure — Indexing and slicing — basic, boolean masking, fancy indexing

Step 3 — Deriving the slice length from the boxes

WHAT. The parent gave the length formula . Let's see it, not trust it. For a[1:5:2]: , , .

WHY a ceiling and not plain division? Because you land on boxes and then box would be too far ( is stop, excluded). You always get a whole number of jumps — and a partial jump still counts as one landing. Rounding a partial jump up is exactly what (ceiling = "round up to the next whole number") does.

PICTURE. The strip with green landing-dots on boxes 1 and 3, and a red on box 5 marking the excluded stop. The bracket underneath measures the distance ; the pink ticks chop it into jumps of length , giving landings.

Figure — Indexing and slicing — basic, boolean masking, fancy indexing

Step 4 — Why a boolean mask is forced to copy

WHAT. Now the mask a[a > 30]. First a > 30 builds a stencil [F, F, F, T, T] — one True/False per box. Then a[mask] collects the True boxes: values [40, 50].

WHY a copy and not a view — the stride test fails. Ask the Step-2 question again: "can I reach every True box with one fixed jump?" Here the True boxes are 3 and 4 — jump of 1. Fine so far. But a mask can pick any scattered pattern, e.g. [T, F, F, T, F] → boxes 0 and 3, jump of 3, versus 0 and 1, jump of 1. There is no single stride that works for all masks. So NumPy cannot fake it with a recipe; it must gather the chosen values into a fresh strip. That gather = a copy.

PICTURE. The stencil (a card with holes over the True boxes) laid over the strip. Only boxes 3 and 4 "show through." A blue arrow carries those two values down into a brand-new short strip labelled copy (fresh memory) — physically separate from the original.

Figure — Indexing and slicing — basic, boolean masking, fancy indexing

Step 5 — Fancy indexing: you name the boxes, order and repeats allowed

WHAT. Fancy indexing a[[3, 0, 0, 2]]. You hand a list of positions. Output = the values at those boxes, in your order, with your repeats: [40, 10, 10, 30].

WHY a copy too. Position 0 appears twice; positions jump backward . No fixed stride can produce "3, then 0, then 0, then 2." Again the stride test fails, so NumPy gathers into fresh memory — a copy. Bonus rule: the result's shape copies the index array's shape, not the source array's. Give it a 2×2 index grid, you get a 2×2 result.

PICTURE. The index list [3,0,0,2] sits on top as chalk-blue tickets. Arrows fan out from each ticket to its box in the original strip, then down into the output strip — showing 10 pulled twice and the reversed order.

Figure — Indexing and slicing — basic, boolean masking, fancy indexing

Step 6 — The trap step: A[[0,2],[1,3]] pairs, it does not block

WHAT. For 2-D A, when you pass two integer lists, NumPy zips them pointwise: the -th element of the row-list pairs with the -th element of the col-list. So A[[0,2],[1,3]] = [A[0,1], A[2,3]] — two elements, not a 2×2 block of rows {0,2} × cols {1,3}.

WHY it feels like a block (and why it isn't). Basic slicing A[0:2, 1:3] does give a rectangle, so we over-generalize. But fancy index lists behave like paired coordinates: list them side by side and read down the columns. To actually get the block, ask for the outer product of the two lists with np.ix_.

PICTURE. A 3×4 grid A. Left panel: pink zip-lines connect and , highlighting only cells and — a diagonal-style pick. Right panel: np.ix_([0,2],[1,3]) shades the full 2×2 rectangle .

Figure — Indexing and slicing — basic, boolean masking, fancy indexing

The one-picture summary

The single diagram below compresses the whole walkthrough: one strip of memory feeds three doors. The slice door only rewrites the recipe (arrow loops back onto the same strip → view). The mask door and fancy door both gather scattered values into a new strip (→ copy). The stride test at the top is the judge that decides which door you're allowed through for free.

Figure — Indexing and slicing — basic, boolean masking, fancy indexing
Recall Feynman retelling — the whole walk in plain words

Picture a shelf of numbered boxes glued in a line. A NumPy array is that shelf plus a sticky note: "start at box offset, jump stride boxes each time."

  • Slicing just writes a new sticky note — new start, new jump — pointing at the same shelf. Nothing is moved, so it's instant and it's a view: rearrange a toy through the note and the real box changes.
  • Boolean masking hands over a punch-card with holes over the boxes you want. But holes can be anywhere, so no single "jump every N" rule can reach them all — NumPy has to carry the chosen values into a new little shelf. That's a copy, and it comes out as a flat line.
  • Fancy indexing hands over a written list of box numbers, in any order, repeats allowed — "give me 3, 0, 0, 2." Same problem: no fixed jump makes that pattern, so again NumPy carries them into a fresh shelf (copy), and the answer takes the shape of your list.
  • The one gotcha: with two lists for a 2-D shelf, NumPy pairs them down the columns[0,2] with [1,3] means boxes (0,1) and (2,3), not the whole rectangle. Want the rectangle? Use np.ix_. The single deciding question for "free view or must-copy?" is always: can I reach all my elements with one fixed jump? Yes → view. No → copy.

Connections

  • Parent: Indexing & slicing (topic note)
  • NumPy arrays — shape, strides, dtype — the offset/stride recipe drawn in Step 1
  • Views vs Copies — memory model — the view/copy verdict of every step
  • Broadcasting — how paired index arrays line up in Step 6
  • np.where and conditional selection — masks as conditions
  • Vectorization — replacing Python loops — why we index instead of looping
  • Pandas .loc / .iloc indexing — the same three doors at DataFrame level