Firewalls — stateless vs stateful packet filtering: Worked examples

Before we begin, three plain-word reminders (we will not use a term before this):

We reuse the parent's tiny ruleset throughout:

#	Dir	Proto	Src	Dst	Dst Port	Flag	Action
1	out	TCP	10.0.0.0/24	any	80,443	—	ALLOW
2	in	TCP	any	10.0.0.0/24	>1023	ACK=1	ALLOW
3	`*`	`*`	any	any	any	—	DENY

Our internal network is 10.0.0.0/24 (addresses 10.0.0.0–10.0.0.255).

The scenario matrix

Every packet a filter sees falls into one of these cells. The two axes that matter are direction (out / in) and what the flags + state say. We also fold in the degenerate inputs (empty table, port on the boundary) and limiting behaviour (table full).

Cell	Direction	Flag pattern	Legit?	Stateless verdict	Stateful verdict	Covered by
A	out	SYN (ACK=0) new conn	✅	ALLOW (rule 1)	ALLOW + create entry	Ex 1
B	in	SYN-ACK, is a real reply	✅	ALLOW (rule 2, ACK=1)	ALLOW via table	Ex 2
C	in	ACK=1, no real conn (spoof)	❌	ALLOW (leak!)	DROP (not in table)	Ex 3
D	in	SYN (ACK=0), new inbound conn	❌	DROP (rule 2 needs ACK=1)	DROP	Ex 4
E	in	dst port = 1023 vs 1024 (boundary)	—	rule 2 edge test	same	Ex 5
F	in	data on ESTABLISHED conn	✅	ALLOW (ACK=1)	ALLOW (fast path)	Ex 6
G	close	FIN vs RST tears down	—	ALLOW (if ACK=1)	ALLOW + delete entry	Ex 7
H	in	flood of SYNs, table full (limiting)	❌	shrugs (no table)	table exhaustion DoS	Ex 8
I	in	active-FTP 2nd channel (word problem)	✅	needs huge port hole	ALG opens 1 port	Ex 9
J	in	UDP DNS reply (no flags!) (exam twist)	✅	rule 2 fails (no ACK)	ALLOW via pseudo-state	Ex 10

We now work one example per cell. Directions and states are drawn in the figures — look at the arrows, not just the words.

Figure — Firewalls — stateless vs stateful packet filtering

Worked examples

Worked example Example 1 — Cell A: outbound SYN starts a browse

Laptop 10.0.0.5:50000 sends a SYN (ACK=0) to 93.184.216.34:443.

Forecast: does each firewall allow it, and does anything get remembered? Guess first.

Read the header. Direction = out, proto = TCP, dst port = 443, flag = SYN. Why this step? Filtering is header-first; we always classify before deciding.
Stateless: walk the rules top-down. Rule 1 matches (out, TCP, dst 443) → ALLOW. Why this step? Rules are ordered; the first match wins, so we stop at rule 1.
Stateful: same match → ALLOW, and it writes a row $(10.0.0.5, 93.184.216.34, 50000, 443, TCP, SYN_SENT)$ . Why this step? The whole point of "state": remember the conversation so the reply is free.

Verify: dst port $443 \leq 443$ and it's in the allowed set ${80, 443}$ → rule 1 is a true match. Both firewalls agree: legit outbound is allowed. ✅

Worked example Example 2 — Cell B: the honest reply comes back

Server answers 93.184.216.34:443 → 10.0.0.5:50000, a SYN-ACK (SYN=1, ACK=1).

Forecast: which rule / table entry catches it? Same verdict on both boxes?

Classify. Direction = in, dst = 10.0.0.5 (inside 10.0.0.0/24), dst port = 50000, ACK=1. Why this step? We must know it's inbound and check its destination port for rule 2.
Stateless: rule 1 is outbound only → skip. Rule 2: in, dst inside net, port $50000 > 1023$ , ACK=1 → ALLOW. Why this step? The ACK bit is the only evidence a stateless box has that this "belongs" to something — see the parent's ACK trick.
Stateful: compute the reverse 5-tuple of Example 1 $(93.184.216.34, 10.0.0.5, 443, 50000, TCP)$ → found → ALLOW, update to ESTABLISHED. Why this step? Return traffic is the mirror of the outgoing 5-tuple; matching that mirror is what "stateful" means.

Verify: $50000 > 1023$ true, so rule 2's port test passes; the reversed tuple's src port 443 equals the original dst port 443, so the mirror lines up. Both allow. ✅

Worked example Example 3 — Cell C: attacker's forged ACK (the leak)

Attacker 66.66.66.66:80 → 10.0.0.5:50000 with ACK=1, but no handshake ever happened.

Forecast: This is the headline difference. Predict both verdicts before reading.

Classify. Direction = in, dst inside net, port $50000 > 1023$ , ACK=1. Looks identical to Example 2 from the header alone. Why this step? The danger is precisely that headers can't tell truth from forgery.
Stateless: rule 2 matches (in, port >1023, ACK=1) → ALLOW — BAD. Why this step? A stateless box trusts the flag; it cannot ask "did I start this?"
Stateful: look up $(66.66.66.66, 10.0.0.5, 80, 50000, TCP)$ → absent → DROP — GOOD. Why this step? No remembered 5-tuple ⇒ the packet is unsolicited, whatever flags it waves. This is the ACK-scan probe the parent warned about.

Verify: The forged tuple has src 66.66.66.66:80; no row in the table has that as its destination. Absence of a match ⇒ drop. The stateless box's rule-2 conditions are all true, so it wrongly allows. Verdicts differ — exactly the topic's point. ✅

Worked example Example 4 — Cell D: attacker tries a fresh inbound SYN

Attacker 66.66.66.66:40000 → 10.0.0.5:22 SYN (ACK=0) — a new inbound connection to SSH.

Forecast: Is the ACK trick enough to stop this one on its own?

Classify. Direction = in, ACK=0 (this is the first handshake packet). Why this step? The SYN-with-ACK=0 is the fingerprint of a brand-new connection.
Stateless: rule 1 is outbound → skip. Rule 2 needs ACK=1, but here ACK=0 → no match → falls to rule 3 → DENY. Why this step? The ACK trick's whole job is to block exactly this: a new inbound conn.
Stateful: no matching entry, and no explicit inbound-allow rule for port 22 → DROP. Why this step? Deny-by-default plus empty table = drop.

Verify: ACK=0 fails rule 2's ACK=1 condition, so rule 2 cannot match; rule 3 catches all → DENY. Both agree. So for this attack the humble ACK trick already suffices — Cell C is where it failed. ✅

Worked example Example 6 — Cell F: data on an established connection

Continuing Example 2, the server sends an HTTPS data segment 93.184.216.34:443 → 10.0.0.5:50000, ACK=1, carrying payload.

Forecast: Does the stateful box re-run the whole rulebook, or take a shortcut?

Classify. In, dst inside net, port 50000, ACK=1, state of conn = ESTABLISHED. Why this step? Established-state packets are the common case; speed matters here.
Stateless: rule 2 (ACK=1, port>1023) → ALLOW. It re-checks rules every packet. Why this step? No memory means no shortcut — constant work per packet.
Stateful: table lookup hits the ESTABLISHED row → ALLOW on the fast path, no full rule scan. Why this step? Once trusted, matching a table row is one hash lookup — cheaper than walking the whole ACL.

Verify: ACK=1 and $50000 > 1023$ ⇒ rule 2 true (stateless allows); reverse tuple already in table ⇒ stateful allows. Both allow, but the cost differs. ✅

Worked example Example 7 — Cell G: tearing the connection down (FIN vs RST)

Two teardown scenarios on the Example-2 flow, then a late straggler packet. (a) Laptop sends FIN (a graceful "I'm done", ACK=1). (b) Instead, the server sends a RST (an abrupt "reset — this connection is dead now").

Forecast: Do FIN and RST close the state entry the same way? What about the late packet?

FIN path (a): FIN is a polite close — TCP normally does a FIN handshake on both sides, so the stateful box moves the row toward CLOSED and removes it after the final ACKs (or a short timeout). Why this step? A graceful close still exchanges packets, so the firewall waits for the handshake to finish before forgetting.
RST path (b): RST is an immediate kill — there is no RST handshake, and a RST may carry ACK=0. The stateful box deletes the row at once on seeing a valid RST. Why this step? RST means "the connection no longer exists," so keeping the row would be wrong; RST-specific semantics = instant teardown, unlike FIN's negotiated one.
The late straggler. After either close, a stray inbound packet arrives for that tuple. Stateful: lookup → absent → DROP. Stateless: if it has ACK=1 and port>1023 it still matches rule 2 → ALLOW — but a lone RST with ACK=0 would fail rule 2 (needs ACK=1) and be DENIED. Why this step? The straggler shows two things at once: stateful forgets closed flows, and a flagless/ACK=0 RST reveals the ACK trick's blind spots.

Verify: RST-with-ACK=0 fails rule 2's ACK=1 requirement (so stateless denies it), while a data/FIN packet with ACK=1 passes rule 2. Stateful drops any post-close straggler because the row is gone. FIN = negotiated close, RST = immediate close — distinct semantics. ✅

Worked example Example 8 — Cell H (limiting): SYN flood fills the table

An attacker sends $N = 1, 000, 000$ spoofed SYN packets from random source IPs. The state table holds at most $C = 500, 000$ half-open entries.

Forecast: Which firewall breaks under load — the "smart" one or the "forgetful" one?

Each new SYN creates one half-open entry (state SYN_SENT) on the stateful box. Why this step? Remembering has a cost: memory. Every connection start consumes a row.
Rows needed vs capacity: $N = 1 0^{6} > C = 5 \times 1 0^{5}$ . Once the $500, 00 1^{st}$ SYN arrives, the table is full → legitimate new connections can't be recorded → DoS. Why this step? Finite memory is a hard wall; the "limiting behaviour" is exhaustion.
Stateless box: stores nothing, so it just applies rules to each packet and shrugs. Why this step? No table ⇒ nothing to exhaust; the weakness of Cell C becomes a strength here.

Verify: Overflow amount $= N - C = 1, 000, 000 - 500, 000 = 500, 000$ rows that don't fit ⇒ table full ⇒ stateful is vulnerable, stateless is not. Mitigation: SYN cookies / rate limits. ✅ (See Denial of Service Attacks.)

Worked example Example 9 — Cell I (word problem): active-mode FTP's second channel

A user runs active FTP to 203.0.113.9. The control channel is on port 21; the server then opens a new data connection back to the client on a dynamically negotiated port, say 52000.

Forecast: How does each firewall let the data channel in without opening the barn door?

The problem: the data port isn't known in advance — it's chosen mid-conversation. Why this step? Static rules must be written before traffic; a dynamic port defeats them.
Stateless: to be safe it would have to permanently ALLOW inbound to all ports $> 1023$ — a huge hole (that's basically rule 2, and Cell C shows how it leaks). Why this step? Without memory, "guess a port range in advance" is the only option.
Stateful with an ALG: an Application-Layer Gateway reads the FTP control channel, sees the PORT 52000 command, and opens exactly one temporary inbound pinhole for port 52000, closing it when the transfer ends. Why this step? State lets the firewall react to what the protocol does, opening 1 port instead of ~64000.

Verify: Ports opened — stateless: $65535 - 1023 = 64512$ ; stateful-ALG: 1. The ratio $64512 : 1$ is the security win. ✅

Worked example Example 10 — Cell J (exam twist): a UDP DNS reply has no flags

Laptop 10.0.0.5:51000 sends a DNS query (UDP) to 8.8.8.8:53. The reply comes back 8.8.8.8:53 → 10.0.0.5:51000. UDP has no SYN/ACK flags at all.

Forecast: Rule 2's trick relies on ACK=1. What happens when the protocol has no flags?

UDP is connectionless — there is no handshake, so "ACK=1" is undefined. Why this step? The entire ACK trick assumes TCP; the exam twist removes that assumption.
Stateless: rule 2 requires TCP and ACK=1. A UDP reply is not TCP and has no ACK bit, so rule 2 cannot match → it falls to rule 3 → DENY. Legit DNS breaks! To make DNS work on a stateless box you'd have to add a permanent rule allowing inbound UDP src-port 53 to high ports — a standing hole an attacker can abuse (spoofed src-port-53 packets). Why this step? Flag-based tricks simply don't exist for flagless protocols, so the stateless box must choose between "break DNS" or "leave a permanent hole."
Stateful: on the outbound query it stores a pseudo-connection row $(10.0.0.5, 8.8.8.8, 51000, 53, UDP)$ with a short timeout; the reply's reverse tuple $(8.8.8.8, 10.0.0.5, 53, 51000, UDP)$ matches → ALLOW, then the row expires. No permanent hole, no forgery loophole. Why this step? State works even without flags: it remembers the tuple, not a bit — so UDP is handled as cleanly as TCP.

Verify: UDP reply has ACK undefined and proto ≠ TCP ⇒ rule 2 (needs TCP + ACK=1) cannot match ⇒ stateless denies (or needs a permanent hole). Reverse UDP tuple present in the pseudo-state table ⇒ stateful allows. This is exactly why real stateless rulesets carry ugly permanent UDP holes while stateful firewalls handle UDP cleanly. ✅

Recall Cover check — one line per cell

Cell C: stateless verdict on a forged ACK=1 packet? ::: ALLOW (the leak) — stateful DROPs it. Cell E: does dst port 1023 pass rule 2's >1023 test? ::: No — strictly greater, so DENY. Cell G: how does RST teardown differ from FIN? ::: RST closes immediately (no handshake, may have ACK=0); FIN is a negotiated close. Cell H: which firewall does a SYN flood exhaust? ::: The stateful one (its table fills); stateless shrugs. Cell J: why does rule 2 fail for a UDP DNS reply? ::: UDP has no ACK flag (and isn't TCP), so the ACK trick can't match.