Exercises — Regular expressions — re module, patterns, groups, findall, sub

• r"\d" — one single digit, i.e. one character from 0–9.
• r"\w" — one "word" character. In plain ASCII terms it is a letter, a digit, or the underscore _. Careful: by default Python treats regex strings as Unicode, so \w also matches accented letters and non-Latin scripts (e.g. é, ü, 字). To restrict it to the pure ASCII set [A-Za-z0-9_], pass the re.ASCII flag or write the class out explicitly.
• r"." — any single character except a newline. The dot is a wildcard for exactly one char.
• r"[^0-9]" — the ^ inside a class means "not". So: one character that is not a digit.

What it looks like: think of each atom as a single stamp that presses down on exactly one letter of the text (see the figure above).

A quantifier repeats the previous atom (here a).

• r"a*" — zero or more as (so it can match the empty string too!).
• r"a+" — one or more as (needs at least one).
• r"a?" — zero or one a (optional single a).
• r"a{2,4}" — between 2 and 4 as in a row.

Mnemonic (from parent): Star = Sometimes (0+), Plus = Present (1+), Question = Quitable (0/1).

re.findall(r"\d+", "room 7, floor 12, unit 305")
# -> ['7', '12', '305']

Why \d+ and not \d? \d alone matches one digit, so you'd get ['3','0','5', ...] — every digit split apart. The + says "grab a whole run of consecutive digits as one match", clumping 3,0,5 into '305'.

Basic version (shape only):

m = re.search(r"(\d{2}):(\d{2})", "alarm at 14:30 sharp")
m.group(0)   # '14:30'   whole match
m.group(1)   # '14'      group 1 = hour
m.group(2)   # '30'      group 2 = minute
m.groups()   # ('14', '30')

Why parentheses? Each ( ) is a capturing group — a little labelled box on your rule card. The colon : between them is a literal character (it matches an actual colon). group(0) is always the entire match; group(1), group(2) are the boxes, numbered left to right by their opening (.

Limitation: (\d{2}):(\d{2}) accepts nonsense like 99:99 — it only checks the shape (two digits each), not that the hour is 00–23 or the minute 00–59.

Range-guarded version: we enumerate the valid digit ranges with alternation |. Look at the figure: the hour splits into "0–9", "10–19", "20–23", the minute into "00–59".

pat = r"([01]\d|2[0-3]):([0-5]\d)"
re.search(pat, "alarm at 14:30 sharp").groups()  # ('14', '30')
re.search(pat, "bad 99:99 time")                 # -> None (rejected!)

• [01]\d — hours 00–19 (a 0 or 1, then any digit).
• 2[0-3] — hours 20–23.
• [0-5]\d — minutes 00–59 (first digit 0–5, second any digit). Regex checks shape, so ranges must be spelled out as digit classes; it cannot do arithmetic like "≤ 23".

[('x', '1'), ('y', '2'), ('z', '30')]

Why tuples, not strings? The rule: when the pattern has 2+ capturing groups, findall returns each match as a tuple of its groups — not the whole match. Here each match x=1 yields the tuple ('x','1'). (With 0 groups you'd get whole matches; with exactly 1 group you'd get just that group's string.)

re.findall(r"\".*\"",  'say "hi" then "bye"')   # -> ['"hi" then "bye"']
re.findall(r"\".*?\"", 'say "hi" then "bye"')   # -> ['"hi"', '"bye"']

Why the greedy one swallows the middle: .* is greedy — it first grabs everything to the far right, then backtracks left just enough to let the final " match. So it matches from the first quote all the way to the last quote, eating hi" then "bye. Look at the figure: the greedy red arrow shoots to the end, then walks back.

Why lazy splits into two: .*? (the ? after *) tries the smallest number of characters first. From the first " it takes nothing, checks for a closing ", finds it right after hi, and stops — giving "hi". Then it continues from bye and does the same. See Greedy vs Backtracking algorithms for the backtracking machinery.

re.sub(r"(\d{2})/(\d{2})/(\d{4})", r"\3-\2-\1", "due 05/12/2024 ok")
# -> 'due 2024-12-05 ok'

Step by step:

1. WHAT: capture three boxes — (\d{2}) day, (\d{2}) month, (\d{4}) year. The / are literal slashes between them.
2. WHY groups: we need to reorder the pieces, so each piece must be remembered separately.
3. WHAT the replacement means: in r"\3-\2-\1", \3 = year, \2 = month, \1 = day — these are backreferences to the captured boxes. We glue them with literal dashes.
4. re.sub walks the string, and for every match it substitutes the rebuilt text — non-matching parts ("due ", " ok") are left untouched.

re.sub(r"\s+", " ", "a\t b\n\nc")
# -> 'a b c'

Why \s+? \s matches one whitespace char (space, tab \t, newline \n). The + grabs a whole run as one match, so "\n\n" (two newlines) becomes one replacement, not two. Replacing each run with a single " " normalises the spacing. Contrast with String methods — split, join, format: " ".join(s.split()) does the same job without regex.

Named groups use the syntax (?P<name>...) — the ?P<name> right after the opening ( labels the box with a name instead of just a number. You then read it with m.group("name") or m.groupdict().

line = "2024-05-12 14:30:07 ERROR user=42 msg=disk full"
pat = (r"^(?P<date>\d{4}-\d{2}-\d{2}) \S+ "
       r"(?P<level>\w+) user=(?P<uid>\d+) msg=(?P<msg>.+)$")
m = re.search(pat, line)
m.group("date")    # '2024-05-12'
m.group("level")   # 'ERROR'
m.group("uid")     # '42'
m.group("msg")     # 'disk full'
m.groupdict()      # {'date': '2024-05-12', 'level': 'ERROR', 'uid': '42', 'msg': 'disk full'}

Reading the pattern left to right:

• ^ — anchor to start of string, so we parse from the very beginning.
• (?P<date>\d{4}-\d{2}-\d{2}) — named box date: 4 digits, dash, 2, dash, 2.
• \S+ — a space, then \S+ = a run of non-whitespace characters (defined in the callout just above this exercise; this eats the 14:30:07 time we don't want to capture), then a space. Using \S+ instead of a group means we skip it cleanly.
• (?P<level>\w+) — named box level: the word ERROR.
• user=(?P<uid>\d+) — literal user=, then named box uid captures the digits.
• msg=(?P<msg>.+)$ — literal msg=, then named box msg = .+ grabs the rest of the line, and $ pins us to end of string.

Why .+ at the end (greedy is fine here): the message may contain spaces (disk full), so we want greedy "grab to the line's end" behaviour, with $ stopping exactly at line end.

The newline caveat: by default . does not match a newline character. So if a message itself spans multiple lines (e.g. a stack trace after msg=), .+$ silently stops at the first line break and the parse loses the rest. To let . swallow newlines too, pass the ==re.DOTALL== flag: re.search(pat, line, re.DOTALL). (And remember from the L5 trap below: ^/$ line behaviour is controlled by a different flag, re.MULTILINE.)

num = re.compile(r"\d+")          # parse the pattern ONCE
total = 0
for line in lines:
    total += len(num.findall(line))

Why re.compile? Every call to re.findall(r"\d+", line) re-parses the pattern string into an internal machine (a state machine — see Finite State Machines). Doing that a million times is wasteful. re.compile parses it once into a reusable pattern object; then num.findall(line) reuses the compiled machine. Same result, far less repeated work — and it reads cleaner.

For example, on ["a1b2", "33", "no digits"] the counts are 2 + 1 + 0 = 3.

When does findall return tuples? ::: When the pattern has 2+ capturing groups — one tuple of group strings per match. How do you make .* lazy, and what changes? ::: Add ? → .*?; the engine tries the smallest count first instead of the largest. What does \3 mean inside a re.sub replacement? ::: A backreference to the text captured by group 3. How do you name a capturing group and read it back? ::: Write (?P<name>...) in the pattern, read with m.group("name") or m.groupdict(). Which flag lets . match a newline? ::: re.DOTALL. Which flag makes ^/$ match at each line boundary? ::: re.MULTILINE. What does \S match? ::: One character that is NOT whitespace (the opposite of \s). How do you restrict \w/\d to ASCII only? ::: Pass the re.ASCII flag (or spell the class out, e.g. [A-Za-z0-9_]). Why compile a pattern used in a loop? ::: It parses the pattern into its state machine once, so each reuse skips re-parsing. Why does re.match return None for a pattern sitting mid-string? ::: match only tries position 0; use search to scan anywhere.