Noesis
1.3.11 · D3Python Intermediate

Worked examples — Regular expressions — re module, patterns, groups, findall, sub

3,703 words17 min read↑ Back to topic

Before anything else, two boxes of vocabulary we will lean on constantly:

The scenario matrix

Every worked example below is tagged with the cell it covers. The two dimensions that change a regex call's behaviour the most are: how many matches exist, and how many capturing groups the pattern has — because those two together decide the exact shape of what findall hands back. We also sweep every verb in the re API, not just the headline five.

Cell Situation What changes Example
A 0 matches search → None; findall → [] Ex 1
B many matches, 0 groups findall → list of whole-match strings Ex 2
C many matches, 1 group findall → list of the group's string Ex 3
D many matches, ≥2 groups findall → list of tuples Ex 4
E greedy vs lazy on same text * grabs max, *? grabs min Ex 5
F empty / degenerate input empty string, pattern that matches nothing-width Ex 6
G sub with backreferences reformat captured pieces Ex 7
H real-world word problem parse a log line, pull fields Ex 8
I exam-style twist match vs search trap + anchors Ex 9
J rest-of-API verbs finditer positions, fullmatch whole-string, split Ex 10

Notice the matrix has three axes. Cases A–D sweep the count of groups; F sweeps degenerate inputs; E sweeps greediness; G–I sweep the action verbs and traps; J sweeps the remaining API verbs so "every regex scenario" really is every one. Together they touch every corner.

Ex 1 — Cell A · zero matches

Forecast: guess before reading. Will search throw an error? Will findall return [] or None?

  1. Read the pattern. \d+ means "one or more digits in a row." Why this step? Every regex question starts by translating the pattern into plain words, so you know what shape you are hunting.
  2. Scan the string. The text "no numbers here" contains only letters and spaces — no digit anywhere. Why this step? A match only exists if the shape actually occurs; here it never does.
  3. Apply each verb's "empty" rule. re.search returns the special value ==None== when it finds nothing. re.findall returns an ==empty list []== — never None. Why this step? The two verbs report "nothing found" differently, and mixing them up is the #1 crash cause (None.group() blows up).

Result:

None
[]

Verify: search result is None (a real object comparison), and findall returns a list of length 0. Both checked in VERIFY.

Ex 2 — Cell B · many matches, zero groups

Forecast: how many strings? Are 8080 split or kept whole?

  1. Translate. \d+ = a run of one or more digits. Why this step? The + is what glues consecutive digits together — this is the whole point.
  2. Walk the cursor left→right. It finds 7, then 8080, then 42. Each maximal run is one match. Why this step? + is greedy, so it extends each run as far as it can before stopping.
  3. Zero capturing groups → whole matches. Because the pattern has no ( ), findall returns each whole match as a string. Why this step? The number of groups controls findall's output shape (this is exactly the axis our matrix sweeps).

Result: ['7', '8080', '42']

Verify: list equals ['7', '8080', '42']. Checked below.

Ex 3 — Cell C · many matches, exactly one group

Forecast: does each element include the word port, or just the number?

  1. Translate. port (literal word + space) followed by (\d+) — a captured run of digits. Why this step? The literal port restricts where we match; the group ( ) decides what we keep.
  2. Find matches. Full matches are "port 80" and "port 443". But group 1 captured "80" and "443". Why this step? We must separate what the pattern matched from what the group captured.
  3. One group → findall returns the group's text only. With exactly one capturing group, findall drops the whole match and hands back only group 1's string. Why this step? This is the special-case rule: 0 groups → whole matches; 1 group → that group; ≥2 groups → tuples.

Result: ['80', '443'] — the word port is gone.

Verify: list equals ['80', '443'].

Ex 4 — Cell D · many matches, two groups (tuples)

Forecast: list of strings, or list of tuples? What's inside each element?

  1. Translate. (\w+) = a captured word-chunk (name), = literal, (\d+) = a captured number. Why this step? Two ( ) means two groups per match — the trigger for tuple output.
  2. Match. Three matches: name/number pairs (x,10), (y,25), (z,3). Why this step? We list matches so we can predict the tuple contents exactly.
  3. ≥2 groups → list of tuples. findall returns each match as a tuple of its group strings, in group order. Why this step? This is the third branch of the same rule — the axis our matrix is built around.

Result: [('x', '10'), ('y', '25'), ('z', '3')]

Verify: list equals [('x','10'),('y','25'),('z','3')].

Ex 5 — Cell E · greedy vs lazy (with figure)

Forecast: does greedy give three tags or one big blob?

The picture below shows why they differ. Read it like this: the yellow boxes are the nine characters of "<a><b><c>". The coral arrow is greedy .* — it sweeps from the first < all the way to the last > in a single leap, so it captures the whole string as one match. The three mint arrows are lazy .*? — each takes the shortest hop from a < to the very next >, so you get three separate tag matches. In short: coral = one long sweep, mint = three short hops.

Figure — Regular expressions — re module, patterns, groups, findall, sub

Figure: the coral arrow (greedy <.*>) spans the entire string <a><b><c> as one match; the three mint arrows (lazy <.*?>) each stop at the nearest >, giving <a>, <b>, <c>.

  1. Greedy .*. After the first <, .* swallows everything up to the end, then backs up to the final >. One giant match: "<a><b><c>". Why this step? Greedy quantifiers try the largest count first (see the coral sweep in the figure).
  2. Lazy .*?. After each <, .*? swallows as little as possible, stopping at the nearest >. Three small matches. Why this step? The ? after * flips it to try the smallest count first (the mint minimal hops).
  3. Zero groups → whole matches for both. Why this step? No ( ), so findall returns whole matched strings (ties back to Cell B).

Result:

['<a><b><c>']            # greedy
['<a>', '<b>', '<c>']    # lazy

Verify: both lists checked.

Ex 6 — Cell F · empty and degenerate inputs

Forecast: for a* on "baa", how many empty matches sneak in?

  1. Empty string. There is no text to scan, so no digit run exists → []. Why this step? Establishes the trivial degenerate base case: nothing in, nothing out.

  2. a* allows zero as. a* matches "zero or more as" — crucially it can match the empty string at any position where no a starts. Why this step? Zero-width matches are the classic degenerate case people forget.

  3. Walk "baa" position by position, watching where the cursor lands. Label the positions 0:'b', 1:'a', 2:'a', 3:end.

    • Position 0 (before b): no a here, so a* matches zero-width ''. A zero-length match does not advance the cursor by itself, so the engine bumps the cursor forward by one to avoid an infinite loop — landing on position 1.
    • Position 1 (the first a): now a* matches the real run 'aa' greedily, consuming both as and moving the cursor to position 3. This is why no empty match appears at the first a: the greedy run has already eaten it, so there is no leftover empty slot there.
    • Position 3 (end of string): no a left, so a* matches zero-width '' one final time.

    Why this step? The cursor-advance rule after a zero-length match is exactly what decides how many empty strings you see — one before b, none between the as (they were consumed by the real run), one at the end.

Result:

[]                       # empty input
['', 'aa', '']           # zero-width matches at b and at end

Verify: both lists checked exactly.

Ex 7 — Cell G · sub with backreferences

Forecast: which number ends up first?

  1. Translate the pattern. (\d{4}) = exactly four digits (the {4} means "exactly this many") = year, -, (\d{2}) = exactly two digits = month, -, (\d{2}) = exactly two digits = day. Groups are: 1=year, 2=month, 3=day. Why this step? We must number the groups correctly, because the replacement refers to them by number. Note {4} and {2} are the fixed-count quantifiers from the recap box.
  2. Read the replacement. \3/\2/\1 means "day, slash, month, slash, year" — a backreference \n inserts the text captured by group n. Why this step? sub rebuilds the output from captured pieces; getting the numbers right is the whole task.
  3. Substitute. Year 2024 → \1, month 05 → \2, day 12 → \3, giving 12/05/2024. Why this step? Plug each captured string into its slot.

Result: '12/05/2024'

Verify: output equals '12/05/2024'. (Uses Escape characters and raw strings — raw r"..." keeps \3 intact instead of turning it into an escape code.)

Ex 8 — Cell H · real-world word problem (log parsing)

Forecast: how many groups will you need? (Count the fields you want.)

  1. Decide the fields = decide the groups. We want 4 things → we need 4 capturing groups. Why this step? One group per field you plan to pull out (ties directly to Cell D's tuple rule).

  2. Build atom by atom.

    • IP: (\d+\.\d+\.\d+\.\d+) — four digit-runs joined by escaped dots \.. A bare . means "any char" (see recap box), so we escape it to demand a literal dot.
    • Method: "(\w+) — inside the quote, a word-chunk.
    • Path: (\S+) — one or more non-whitespace characters (\S from the recap box), i.e. a token with no spaces.
    • Status: " (\d+) — after the closing quote and a space, a digit-run.

    Why this step? \. is escaped because an unescaped . would match anything, letting 1x2 masquerade as part of an IP.

  3. Why the greedy .* between fields is safe here. The gaps we skip (.*) sit between literal landmarks: .* runs from just after the IP up to the " that begins "GET ..., and a second .* runs from just after the path's HTTP up to the " before the status. Even though .* is greedy and initially over-reaches, it is forced to backtrack (Ex 5's mechanic) until the required literal after it — the " and then HTTP — can match. Because those literals appear only where we intend, greedy .* cannot leak into the IP, method, path or status: those fields are pinned by the surrounding literal ", space, and HTTP anchors.

    pat = r'(\d+\.\d+\.\d+\.\d+).*"(\w+) (\S+) HTTP.*" (\d+)'
m = re.search(pat, line)
m.groups()

    Why this step? Understanding why greedy skipping doesn't corrupt the fields is the difference between a pattern that works by luck and one you can trust — the literals " and HTTP act as fences that greedy .* must respect.

Result: ('192.168.0.7', 'GET', '/home', '200')

Verify: the 4-tuple matches exactly. See Python Intermediate — File I/O and parsing logs for reading such lines from a file, and String methods — split, join, format for the non-regex alternative.

Ex 9 — Cell I · exam-style twist (match vs search + anchors)

Forecast: which of these is None?

  1. (a) match is anchored at position 0. re.match only checks whether the pattern fits starting at index 0. "hello world" starts with h, not w → ==None==. Why this step? This is the classic trap from the parent note: match ≠ "contains".
  2. (b) anchors ^ and $. ^world$ means "the string is exactly world, nothing before or after." "world" fits perfectly → a Match object. Why this step? ^ pins the start, $ pins the end. A regex engine is really a little state machine (see Finite State Machines): anchors are zero-width edges that only pass when the cursor sits at the very start or very end — they consume no characters, they just test position.
  3. (c) a leading space breaks the anchor. " world" has a space before w, so ^world can't match at the start → ==None==. Why this step? ^ is strict: even one stray whitespace character defeats it.

Result:

None                              # (a)  match not at start
<re.Match object; ... 'world'>    # (b)  exact anchored match
None                              # (c)  leading space

Verify: (a) and (c) are None; (b) is not None and its .group(0) is 'world'. Checked below.

Ex 10 — Cell J · the rest of the API (finditer, fullmatch, split)

Forecast: does fullmatch accept "123x"? What does split do with messy spaces?

  1. (a) finditer gives Match objects with positions. Unlike findall (strings only), ==re.finditer== yields a Match per hit, and .start() is the index where it began. "a1bb22" has 1 at index 1 and 22 at index 4. Why this step? When you need where a match is (not just its text), finditer is the tool — findall throws positions away.
  2. (b)/(c) fullmatch demands the WHOLE string. ==re.fullmatch== is like putting ^...$ around your pattern automatically. "123" is all digits → a Match; "123x" has a trailing x → None. Why this step? It answers "is the entire input exactly this shape?" — perfect for validating a whole field.
  3. (d) split cuts the string ON the pattern. ==re.split== breaks text wherever the pattern matches, returning the pieces. \s*,\s* = "a comma with optional spaces around it", so it cleanly separates a, b, c, d regardless of stray spaces. (Compare the plain str.split, which can't absorb the ragged spaces.) Why this step? split is the inverse of matching — you keep the gaps between matches, not the matches.

Result:

[1, 4]                    # (a) finditer positions
<re.Match ... '123'>      # (b) fullmatch ok
None                      # (c) trailing x rejected
['a', 'b', 'c', 'd']      # (d) split on comma-with-spaces

Verify: all four checked below.

Recall One-line summary of every cell

findall shape depends only on group count ::: 0 groups → whole matches, 1 group → that group, 2+ groups → tuples. How does "no match" differ between verbs? ::: search→None, findall→[]. When do empty-string matches appear? ::: With zero-allowing quantifiers (*, ?, {0,n}) at positions where the atom is absent. Greedy vs lazy on <.*> vs <.*?>? ::: greedy → one big match, lazy → each tag separately. Why does re.match("world","hello world") give None? ::: match only tries position 0. What does \. mean vs a bare .? ::: \. = a literal dot; bare . = any character. finditer vs findall? ::: finditer yields Match objects (with positions); findall yields only strings/tuples. What does re.fullmatch require? ::: The pattern must match the entire string (like auto ^...$). What does re.split return? ::: The pieces of the string between matches of the pattern.

Connections

  • Parent topic
  • Greedy vs Backtracking algorithms — why greedy .* backtracks (Ex 5, Ex 8): gulp everything, then rewind char-by-char until the trailing atom fits.
  • Escape characters and raw strings — why r"\3" and \. behave (Ex 7, 8): \ is special in both Python strings and regex, so raw strings pass it through untouched.
  • Python Intermediate — File I/O and parsing logs — log parsing at scale (Ex 8, Ex 10).
  • String methods — split, join, format — the non-regex alternative; re.split beats str.split on ragged separators (Ex 10).
  • Finite State Machines — anchors as zero-width states (Ex 9): the engine is a state machine and ^/$ are position-only edges that consume no characters.