Regular expressions — re module, patterns, groups, findall, sub: Question bank

Before we start, one shared vocabulary reminder so every trap below reads cleanly:

An atom is one pattern piece — a single character like a, a class like \d, or a group like (...).
A quantifier (*, +, ?, {m,n}) always attaches to the atom immediately before it.
A match object is what search/match return when they succeed; it is "truthy", and None is "falsy".
Capturing means a (...) group remembers the exact text it swallowed, numbered left-to-right by opening bracket.

Two visuals below carry the hardest dynamic ideas — how the engine scans to make zero-width matches, and how greedy matching backtracks. Refer to them as you read the traps.

Two quick pictures for the dynamic traps

The regex engine is literally a little machine that walks a cursor left→right, trying to match at each position. The figure below shows that cursor sweeping across "baa" with the pattern a*, and why it produces the empties.

Figure — Regular expressions — re module, patterns, groups, findall, sub

Greedy matching is the other big surprise. The next figure shows <.*> swallowing everything, then backtracking — giving characters back one at a time until the final > fits. This is the same "overshoot then retreat" idea behind greedy-with-backtracking search, where an algorithm grabs the largest option first and unwinds when it fails.

True or false — justify

re.match(r"dog", "the dog barks") finds the word "dog".

False. match anchors at position 0 only; the string starts with "the", not "dog", so it returns None. Use search to look anywhere.

re.search(r"^dog$", "dog" + chr(10)) matches even though the string has a trailing newline.

True in default mode. $ matches at end-of-string or just before a single trailing newline, so a string ending in a newline still matches. This surprises people who think $ means "the very last character".

a* will fail to match the string "bbb".

False. a* means "zero or more a's", and zero a's is a valid match — it matches the empty string at position 0. It never fails; it can match nothing.

The pattern . matches every possible character.

False. By default . matches any character except newline. You need the re.DOTALL flag (or [\s\S]) to include newlines.

re.findall(r"\d", "12") returns ['12'].

False. Without +, \d matches one digit at a time, so you get ['1', '2']. The + is what clumps consecutive digits into one match.

re.findall(r"(a)(b)", "ab ab") returns ['ab', 'ab'].

False. With 2+ capturing groups, findall returns a list of tuples of the groups: [('a','b'), ('a','b')]. The whole match is discarded.

Adding a ? after + turns + into "optional".

False. +? is not "maybe one or more" — it's the lazy version of +: still one-or-more, but matching as few as possible. ? after a quantifier means "be lazy", not "be optional".

(?:...) and (...) behave identically for matching text.

True for what text gets matched. They differ only in bookkeeping: (?:...) is non-capturing, so it doesn't create a numbered group or appear in .groups().

In re.sub(r"(\w+)", r"\1\1", "hi") the result is "hihi".

True. \1 in the replacement is a backreference to group 1's captured text "hi", written twice.

The string "\d" and the raw string r"\d" are identical to the regex engine.

False in general. For \d they happen to coincide today (Python leaves the unknown escape alone, though this is deprecated), but for escapes like \b a normal string turns "\b" into a backspace character before the regex engine sees it — a different pattern entirely. Treat them as not interchangeable and always use r"...".

Spot the error

Why does re.match("the dog", "the dog") succeed but re.match("dog", "the dog") fail?

The first pattern starts where the string starts (position 0); the second requires "dog" at position 0, but the string begins with "the ". match never scans forward.

A student writes re.findall(r"*", "abc") and gets an error. What's wrong?

* is a quantifier — it needs an atom before it to repeat. With nothing before it there's "nothing to repeat", so the engine raises an error. They probably wanted .*.

re.sub(r"cat", r"dog", "concatenate") returns "condogenate" — surprised? Why?

Regex matches substrings, not whole words. "cat" appears inside "concatenate", so it gets replaced. Use \bcat\b to require word boundaries.

Someone expects re.search(r"a.b", "a" + chr(10) + "b") to match but it returns None. Why?

. does not match the newline by default, and their string has a newline between a and b. They need re.DOTALL or the class [\s\S].

p = re.compile(r"\d+"); p.match("abc123") returns None. They think compile is broken. Why isn't it?

compile only pre-parses the pattern; the method still matters. .match anchors at the start, and the string starts with "a", not a digit. p.search(...) would find "123".

re.findall(r"<.*>", "<a> text <b>") grabs the whole "<a> text <b>" instead of two tags. Why?

* is greedy: it matches as far right as it can, so .* swallows everything up to the last > (then backtracks just enough to place that final >). Use lazy <.*?> to stop at the first >. See figure s02.

re.findall(r"aba", "ababa") returns only one match, not two. Why?

findall scans non-overlapping: after matching "aba" at index 0, the cursor resumes at index 3, so the second "aba" (which overlaps at index 2) is never seen. Overlapping matches need a lookahead trick like (?=(aba)).

m = re.search(r"(\d+)?", "abc"); m.group(1) returns None, not "". Why?

The optional group (...)? matched zero times, so it never captured anything — its value is None, distinct from an empty string that a group would capture if it had participated.

Why questions

Why must patterns often be written as raw strings?

Both Python string literals and regex use backslash. Without r"...", Python eats the backslash first ("\b" → backspace char) before the regex engine ever sees it, silently changing the pattern. See Escape characters and raw strings.

Why does findall return strings in some cases and tuples in others?

It depends on group count: 0 groups → list of whole matches (strings); exactly 1 group → list of that group's text; 2+ groups → list of tuples. The rule is "return the groups if there are any".

Why does making .* lazy sometimes change which text matches, not just how much?

Greedy grabs maximally then backtracks; lazy grabs minimally then expands. When another > or delimiter follows, the two strategies stop at different places, so you literally get different matched substrings — critical for tags/quotes.

Why is group(0) special?

It always means the entire matched substring, regardless of groups. Numbered groups 1, 2, ... start at the first opening parenthesis; 0 is the reference to "the whole match" that always exists.

Why prefer re.compile in a loop even though behaviour is identical?

Parsing a pattern into an internal state machine costs work; compiling once reuses that machine every call, avoiding repeated parsing. Conceptually a regex compiles to a finite state machine — a fixed set of "states" with arrows saying "on this character, go here" — and you build that machine once. Behaviour is the same, speed improves.

Why does the engine "backtrack"?

A greedy quantifier over-commits, then the rest of the pattern fails to match; the engine steps back, giving up characters one at a time until the whole pattern fits (see figure s02). This is the same "grab big, then unwind" idea as Greedy vs Backtracking algorithms.

Why is ^abc$ different from just abc?

abc matches "abc" anywhere inside a longer string; ^abc$ demands the entire string be exactly "abc" (start-anchor + end-anchor with nothing else between).

Edge cases

What does re.findall(r"a*", "baa") return, and why the empties?

['', 'aa', '']. Walk the cursor (figure s01): at index 0 the char is "b", so a* matches zero a's → ""; the cursor advances past "b" to index 1; there it matches "aa" greedily; landing at index 3 (end of string) it matches zero a's again → a final "". That's three matches: empty, "aa", empty.

Why doesn't "baa" also produce an empty match between the two a's?

After matching "aa" the cursor sits at end-of-string, not between them — the greedy a* already consumed both a's in one match, so no scanning position is left inside "aa" to produce another empty.

re.search(r"", "hello") — what happens?

It succeeds with an empty match at position 0. An empty pattern matches the empty string that exists at every position, so the first one (offset 0) is returned.

re.findall(r"aa", "aaaa") returns two matches, not three — why not the overlapping middle one?

Matching is non-overlapping: "aa" at index 0, then resume at index 2 for "aa" at index 2. The overlapping "aa" at index 1 is skipped because the cursor already moved past it.

What does re.split(r"(\d)", "a1b") include that a groupless split wouldn't?

['a', '1', 'b'] — when the split pattern has a capturing group, the captured delimiters are kept in the result list, not thrown away. Relevant when combined with String methods — split, join, format.

If a Match object is None, what breaks when you call .group()?

You get AttributeError: 'NoneType' object has no attribute 'group'. Always test if m: first — a non-match returns None, which is falsy, not an empty match object.

Does [a-z] match an uppercase A?

No — character-class ranges are case-sensitive; [a-z] is lowercase only. Use [a-zA-Z] or the re.IGNORECASE flag.

Inside a character class, is . a wildcard?

No. Inside [...], most metacharacters lose their power: [.] matches a literal dot only. That's actually the safest way to match a real period.

What does \b mean inside a normal (non-raw) Python string, and why is that a bug?

In a normal string "\b" is the backspace control character (ASCII 8), not a word boundary. The regex engine then receives a backspace, silently matching the wrong thing — the classic reason to always use r"\b".

Recall One-line summary of the traps

Anchors (match vs ^$), greediness, zero-width matches, non-overlapping scanning, group-count changing return types, and raw-string backslash handling are the six families of regex surprises. The unifying fix ::: read every pattern literally, remember quantifiers bind to the previous atom only, and remember . skips newlines while * allows zero.