ReAct (reasoning + acting) framework: Visual walkthrough

Before we start, three plain words we will keep reusing:

A thought is the model talking to itself — "what should I do next, and why?"
An action is the model talking to the outside world — pressing a button on a tool (search, calculator, database).
An observation is the world talking back — the result the tool hands over.

Everything below is just these three, arranged in a ring.

Step 1 — One question, a box of tools

WHAT. We start with the raw ingredients. We have a question we call $Q$ (just a name for the text the user typed), and a toolbox we call $T$ .

WHY. Before any reasoning, you must know two things: what is being asked and what am I allowed to use. If the answer lives outside the model's memory (today's news, exact arithmetic), the model cannot invent it — it must reach into $T$ . This is exactly the gap Tool Use in LLMs fills.

PICTURE. On the left, the question $Q$ . On the right, the toolbox — a set written with curly braces:

$T = {t_{1}, t_{2}, \dots, t_{n}}$

$T$ — the curly "script T", meaning the collection of all tools. Curly braces ${}$ are math shorthand for "a set of things".
$t_{1}, t_{2}, \dots, t_{n}$ — the individual tools (tool one, tool two, up to tool number $n$ ). $n$ is just however many tools we have.

At this instant nothing has happened yet. No thought, no action. The stage is set.

Step 2 — First the model thinks (Thought₁)

WHAT. The model reads $Q$ and produces its first thought, written $Thought_{1}$ . The little $1$ (a subscript, meaning "number 1") says this is the first one in the sequence.

WHY. Why think before acting? Because an action fired blindly ("Search[...]" for what?) usually searches for the wrong thing. The thought decides the intent of the next move. This is the lesson from Mistake 2 in the parent: acting without reasoning leaves you stuck.

PICTURE. We write where the thought comes from with a squiggle:

$Thought_{1} \sim P_{LM} (thought ∣ Q, examples)$

Read this left to right, out loud:

$Thought_{1}$ — the thing we get out.
$\sim$ — the squiggle means "is drawn from" (sampled), because a language model doesn't give one fixed sentence, it rolls dice over possible sentences.
$P_{LM} (\dots)$ — $P$ is a probability: how likely each possible thought is. The subscript $LM$ says the language model is the one assigning these likelihoods.
$∣$ — the vertical bar reads "given". Everything after it is what the model already knows.
$Q, examples$ — it knows the question, plus a few worked demonstrations (the few-shot prompt) that show it the Thought→Action→Observation style.

So $Thought_{1}$ is a sentence like "I need the country the Eiffel Tower is in." Notice it doesn't answer the question — it plans the next single step.

Step 3 — Then the model acts (Action₁)

WHAT. Guided by $Thought_{1}$ , the model picks one action $a_{1}$ out of the toolbox.

WHY. A thought is a wish; an action makes it real. We must turn intent into a button-press so the outside world can respond. Which button? The one the model judges most fitting given its own thought.

PICTURE. The "most fitting" is captured by an $ar g max$ :

$a_{1} = ar g max_{a \in T} P_{LM} (a ∣ Q, Thought_{1})$

Term by term:

$a_{1}$ — the chosen first action (e.g. Search["Eiffel Tower location"]).
$a \in T$ — read " $a$ in $T$ ": we only consider actions that are actually in the toolbox. You can't press a button you don't have.
$P_{LM} (a ∣ Q, Thought_{1})$ — the model's score for each candidate action, given the question and the thought it just had.
$ar g max_{a}$ — "the argument that maximises". Not the biggest score, but which action earns the biggest score. It picks the winning tool.

The arrow in the figure goes from the thought's intent straight to one highlighted tool. Every other tool stays dim — they were considered and rejected.

Step 4 — The world answers (Observation₁)

WHAT. The tool actually runs. We call the result the observation $o_{1}$ .

WHY. This is the whole point of ReAct — grounding. The observation is real data from outside the model, so it can correct the model's guesses. This is where hallucination gets caught.

PICTURE. Unlike thinking and acting, this step has no dice:

$o_{1} = Execute (a_{1})$

$o_{1}$ — the first observation, e.g. "The Eiffel Tower is in Paris, France."
$Execute (a_{1})$ — "run the tool named by $a_{1}$ ". A calculator given 23 ** 2 always returns 529; a search given a query returns its hits.
There is a plain $=$ here, not a squiggle $\sim$ . The environment is deterministic from the model's side: press the button, get the result. No sampling.

Now the model knows something it did not know in Step 2. That new knowledge is what makes the next thought smarter.

Step 5 — Think again, now wiser (Thought₂)

WHAT. The model produces $Thought_{2}$ , but this time it may condition on everything that has happened: the question, the first thought, the first action, and the first observation.

WHY. This is the feedback loop and the source of error recovery. If $o_{1}$ was surprising ("population is only 2.1M — but maybe they mean the metro area?"), the second thought can pivot. A one-shot model can't do this; it never sees its own results.

PICTURE. The "given" list just grew:

$Thought_{2} \sim P_{LM} (thought ∣ Q, Thought_{1}, a_{1}, o_{1})$

Same $P_{LM}$ , same squiggle $\sim$ — still sampled reasoning.
After the bar $∣$ we now list four things instead of two. The model literally re-reads its whole scroll so far before speaking.
$o_{1}$ appearing here is the magic: the observation reshapes the next thought. That is grounding turning into planning.

The loop has closed: Thought → Action → Observation → Thought again. Steps 3–5 now simply repeat.

Step 6 — The loop as a growing scroll (the trajectory τ)

WHAT. Stacking every triple $(Thought, action, observation)$ gives us the trajectory, named $τ$ (the Greek letter "tau").

WHY. We need one symbol for "the whole run so far", so we can talk about it and score it. A trajectory is just the transcript.

PICTURE.

$τ = [(Thought_{1}, a_{1}, o_{1}), (Thought_{2}, a_{2}, o_{2}), \dots, (Thought_{T}, a_{T}, o_{T})]$

$τ$ — the list of all rounds.
Each $(\cdot, \cdot, \cdot)$ — one full round: think, act, observe.
$T$ — the total number of rounds (capital, because it's the last index). Some questions need $T = 1$ , multi-hop ones need $T = 3$ or more.

The scroll keeps growing until a special action ends it:

$a_{T} = Finish [answer]$

Finish is the only action that doesn't call a tool — it just declares the answer and stops the ring. Without it, the loop would spin forever.

Step 7 — Multiplying the loop into the parent's formula

WHAT. We now assemble the full probability of the trajectory — the formula the parent note stated.

WHY. Each round is a chain of dependencies: the thought depends on the past, the action depends on the thought, the observation depends on the action. When independent-per-step chances stack in sequence, their probabilities multiply. That's why we get a product $\prod$ (a capital Greek "pi", meaning "multiply all these together").

PICTURE. Build one round, then repeat:

$P (τ ∣ Q) = \prod_{t = 1}^{T} reasoning P (Thought_{t} ∣ Q, τ_{< t}) \cdot acting P (a_{t} ∣ Q, τ_{< t}, Thought_{t}) \cdot world (fixed) P (o_{t} ∣ a_{t})$

$\prod_{t = 1}^{T}$ — multiply the round-terms for $t = 1, 2, \dots, T$ .
$τ_{< t}$ — "everything in the scroll before round $t$ ". The subscript $< t$ literally means indices less than $t$ .
First factor = Step 2/5 (thinking, sampled).
Second factor = Step 3 (acting, sampled given the thought).
Third factor = Step 4 (observing) — and because the world is deterministic, this factor is $1$ for the real result and $0$ for anything else, so it never changes the shape, it only pins us to reality.

The parent's objective — maximise $P (τ ∣ Q) \cdot 1 [answer = A^{*}]$ — is now readable: the $1 [\dots]$ is an indicator (1 if the trajectory's answer equals the true answer $A^{*}$ , else 0). We want the most probable trajectory that also lands on the truth.

Step 8 — Degenerate cases (what happens at the edges)

Every honest derivation must survive its corners. Here are the three that break naive agents.

Case A — $T = 1$ (no tool needed). The question is answerable from memory ("What is $2 + 2$ ?"). Then the very first action is Finish[4], the product has a single term, and ReAct collapses to plain reasoning — indistinguishable from a chain-of-thought answer. No harm, no waste.

Case B — a bad observation. Suppose $o_{1}$ is garbage ("No results found"). The world factor is still deterministic; the recovery lives in $Thought_{2}$ , which reads the garbage and re-plans (rephrase the query, try another tool). This is precisely what an act-only agent cannot do — it has no thought slot to notice the failure. Reflexion extends this idea by letting the agent write a longer self-critique between attempts.

Case C — the loop never finishes. If Finish is never chosen, $T \to \infty$ and the product runs forever. In practice we cap $T$ at a maximum (say 8 rounds) and force a Finish. The cap is a safety rail, not part of the ideal math.

The one-picture summary

The whole framework is a ring with one exit. Thought points to Action, Action points to Observation, Observation points back to Thought — and the only way out is the Finish door. Stack $T$ turns of that ring, multiply the little probabilities on each arrow, and you have $P (τ ∣ Q)$ .

Recall Feynman retelling — say it back in plain words

Imagine solving a puzzle with a notebook and a phone. First you write a note to yourself about what you need (that's a thought). Then you use the phone to look it up (an action). The phone shows you an answer (an observation). You read it and write a new note, maybe changing your plan (thought again). You keep circling — note, look, read, note, look, read — until you write "Done, the answer is ___". The math formula is nothing but bookkeeping for this circle: each "note" and each "look" is a little chance the model rolls, each "phone result" is fixed once you dialed, and multiplying all those chances together tells you how likely this whole notebook-run was. If the puzzle needs nothing looked up, you skip straight to "Done". If the phone shows junk, your next note is where you notice and try again. And you always keep a page limit so you never circle forever.

Recall Quick self-test

Why is the observation factor an $=$ and not a $\sim$ ? ::: The environment is deterministic from the model's view — running a tool on a given input always returns the same output, so there is no sampling. What single term makes ReAct able to recover from a bad tool result? ::: The next thought, $Thought_{t + 1}$ , which conditions on the failed observation and can re-plan. What does $ar g max_{a \in T}$ pick — a number or a tool? ::: A tool (the argument achieving the highest score), restricted to tools actually in the box $T$ . When does ReAct collapse to plain chain-of-thought? ::: When $T = 1$ : the first action is Finish, so no tool is ever called.

See also: Tool Use in LLMs · Agent Architectures · MRKL · Retrieval-Augmented Generation (RAG) · Reflexion