Visual walkthrough — Agent architectures and reasoning loops

The parent note shows the loop as a finished machine: Observe → Reason → Act → Update. This page builds that machine one piece at a time, from a plain language model that can only produce text, until we reach the full ReAct update rule the parent states. Every symbol is drawn before it is used.

Step 1 — Start with a machine that only writes

WHAT. We define one object and nothing else:

Read $\text{LLM}(\cdot )$ as a function: a rule "give me text, I hand back the next text." The parentheses hold what we feed it.

WHY. We start here because it is the only thing we truly have. Everything else — memory, tools, loops — must be built on top of this one ability. If we assumed more, we'd be cheating. See Prompt Engineering for how the input strip is shaped.

PICTURE. In the figure, the lavender box is the $\text{LLM}$. A coral arrow carries text in; a mint arrow carries text out. Notice: the out-arrow does not come back. The box has no loop yet, no memory, no hands.

Step 2 — Give the box a name for what it is looking at

WHAT. We name the input strip:

The square brackets $[\dots ]$ mean "a list, kept in order." Each $m_i$ is one chunk of text (a user question, a note the agent wrote, a tool's answer). The subscript $t$ counts how many chunks are on the strip right now.

WHY. We need a symbol for "everything seen so far" because the loop's whole trick is: the strip grows each tick, and the box re-reads the entire grown strip every time. Without a name for the strip, we can't describe how it grows. The strip's maximum length is set by the context window.

PICTURE. The strip is drawn as stacked pastel cards. At $t=1$ there is one card. The clock in the corner reads $t=1$. Keep your eye on this clock — every later step advances it.

Step 3 — Make the box think out loud before it acts

WHAT. We split the box's output into two named pieces. First a Thought, then an Action:

• $T_t$ — the plain-English sentence the box writes, e.g. "I need Paris's population."
• $\text{parse}(\cdot )$ — a tiny reader that scans $T_t$ for a line like Action: search("...") and pulls out the command. "Parse" just means "find the structured bit inside the text."
• $A_t$ — the extracted command, e.g. search("population of Paris").

WHY $\text{parse}$ and not something cleverer? The box only emits text. A real tool needs a clean command, not a paragraph. $\text{parse}$ is the smallest possible bridge from "prose" to "callable command." This is exactly function calling. Writing the thought first is the idea behind Chain-of-Thought Prompting.

PICTURE. The mint out-arrow now forks: the top fork is the Thought (a speech bubble), the bottom fork is the Action (a small gear labelled with the command). A dashed line shows $\text{parse}$ snipping the gear out of the bubble.

Step 4 — Let the action touch the real world

WHAT. We add two named things:

• $\text{execute}(\cdot )$ — the module that runs $A_t$ for real and catches errors.
• $o_{t+1}$ — what came back, e.g. "Paris metro: ~12.3 million." The subscript is $t+1$, one ahead, because this fresh fact belongs to the next strip, not the current one.

WHY $t+1$ and not $t$? Cause before effect. The action $A_t$ happens at tick $t$; its result only exists after, so it is the first thing the world hands us going into tick $t+1$. Getting this index right is what makes the loop advance instead of standing still.

PICTURE. The gear (Action $A_t$) drops into a coral "world" cloud (search engine / code runner). A butter-coloured arrow carries the reply $o_{t+1}$ back out. The clock is still $t$, but the reply is stamped $t+1$.

Step 5 — Grow the strip: close the loop

WHAT. We build the next strip from the old one:

The plus sign $+$ here means append to the list (stick on the end), not numeric addition. The three appended items are one bundle: the thought, the action taken, and the fresh observation.

WHY. This is the entire reason agents work. Because $c_{t+1}$ contains $c_t$ plus new facts, the next call $\text{LLM}(c_{t+1})$ reasons with more knowledge than the last call had. Ignorance shrinks every tick. This growing strip is the agent's short-term memory.

PICTURE. The strip from Step 2 gains three new stacked cards (thought = lavender, action = slate, observation = butter). The clock ticks from $t$ to $t+1$. A big curved arrow sweeps the grown strip back into the $\text{LLM}$ box — the loop is now visibly closed.

Putting Steps 3–5 together gives exactly the parent note's four lines:

Step 6 — When do we stop? (the termination case)

WHAT. A decision, not a formula:

• $t_{\max }$ — a safety cap on ticks (e.g. 10). Without it a confused agent loops forever.
• If neither is true, we go back to Step 3 with the grown strip $c_{t+1}$.

WHY. Two degenerate cases force this:

1. Success case — the box's Action is respond(...) instead of a tool call. $\text{parse}$ sees "final answer," so we halt and hand it to the user.
2. Runaway case — a tool keeps failing, the box keeps retrying, the strip grows past the window. The cap $t_{\max }$ is the circuit breaker.

PICTURE. A diamond decision node after the observation: one branch (mint, "goal met") exits to the user; the other (coral, "keep going") arcs back to the $\text{LLM}$. A small counter shows $t$ climbing toward $t_{\max }$.

Step 7 — The edge case: the world says "error"

WHAT. When execution fails:

The next thought $T_{t+1}=\text{LLM}(c_{t+1})$ now sees the error and can write "I forgot to cast the input to int." This self-correcting variant is called Reflexion — a loop that learns from its own mistakes, echoing trial-and-error learning.

WHY. Because we already made observations anything the world hands back, errors need no new machinery. This is the payoff of Step 5's design: the strip absorbs good news and bad news identically.

PICTURE. The world cloud turns coral and emits a jagged error card. It slots onto the strip exactly like a normal observation; the next thought bubble points at it and says "fix it."

The one-picture summary

Everything above compresses into a single wheel: read the strip → think → parse → execute → observe → grow the strip → check stop → spin again. Ignorance falls each turn; the strip fattens; the world keeps the box honest.

Quick self-check