Foundations — ReAct (reasoning + acting) framework

Before you can read the parent note, you need to be fluent in every squiggle it uses. This page builds each one from nothing, in the order that each depends on the last. If you have never seen $\sim$, $\arg \max$, or $\prod$ before — perfect, that is who this page is written for.

0. What a "language model" even is

The key thing to hold onto: a language model does not return one fixed answer. It returns a spread of possibilities, each with a chance attached. That spread is what the symbols in the next sections describe.

Look at the figure. The orange bars are the model's guesses for what word comes next after "The Eiffel Tower is in ___". Each bar has a height = how likely the model thinks that word is. Tall bar = confident. That height is a probability, our next symbol.

You will use this idea everywhere in the parent note and in Tool Use in LLMs.

1. Probability — the symbol $P$

Why does ReAct need this? Because the model is a guesser, not an oracle. When the parent writes

$P_{\text{LM}}(\text{thought}\mid Q,\text{examples})$

it is saying: "the model's chance of producing each possible thought, given the question and some examples." We are about to unpack that vertical bar $\mid$.

2. The conditioning bar — the symbol $\mid$

So $P_{\text{LM}}(\text{thought}\mid Q,\text{examples})$ reads left-to-right as:

chance of a thought — given the question $Q$ and some example demonstrations.

The comma just means "and": "given $Q$ and examples."

3. The question, the answer, and the tools — $Q$, $A$, $\mathcal{T}$

The agent's job: pick tools out of this box, in a smart order, to turn $Q$ into $A$. This is exactly the theme of Agent Architectures.

4. The three moving parts — Thought, Action, Observation

The figure shows one full turn of the loop as three coloured boxes. Blue = the model thinks, orange = it acts on a tool, green = the world observes back. The green arrow curls back up to a new blue box — that curl is ReAct. Everything else on this page is machinery for writing that curl down in symbols.

Why three parts and not two? The parent's whole argument is that pure thinking (blue only) can't touch the real world, and pure acting (orange only) can't correct itself. You need the green observation to close the loop.

5. Sampling — the symbol $\sim$

${\text{Thought}}_1\sim P_{\text{LM}}(\text{thought}\mid Q,\text{examples})$

means: the first thought is pulled at random from the model's spread of possible thoughts, weighted by how likely each is, given the question. It is not "equals" — the model could produce different thoughts on different runs, and that randomness is why Self-Consistency (running many times) even helps.

6. Picking the best action — the symbol $\arg \max$

The parent chooses an action with:

$a_1=\arg {\max }_{a\in \mathcal{T}}{P}_{\text{LLM}}(a\mid Q,{\text{Thought}}_1)$

Let's earn every piece.

Read the figure left-to-right. Each tool in $\mathcal{T}$ gets a bar = how likely the model thinks that tool is the right next move. $\max$ points at the tallest bar's height. $\arg \max$ points at the tool underneath it (here, Search) — and that tool is what we actually run. Putting it together:

$a_1$ = the tool (from the box) that the model rates as most likely to be right, given the question and its first thought.

Why $\arg \max$ and not sampling ($\sim$) here? Because for actions we usually want the single best move, not a random one — a wrong tool call wastes a step. (Some systems sample actions too; the parent picks the confident-best version.)

7. Running the tool — the symbol $\text{Execute}$

8. The trajectory — the symbol $\tau$

9. Stopping — the $\text{Finish}$ action

10. Multiplying chances — the symbols $\prod$ and $1$

The parent's big formula is

$P(\tau \mid Q)={\prod }_{t=1}^{T}P({\text{Thought}}_t\mid Q,{\tau }_{<t})\cdot P(a_t\mid Q,{\tau }_{<t},{\text{Thought}}_t)\cdot P(o_t\mid a_t)$

Two new symbols.

The figure shows two trajectories. The top one ends on the correct $A^{\ast }$, so its indicator is $1$ and its whole probability survives. The bottom one ends wrong, so the indicator zeroes it out (grey). "Maximise over $\tau$" then means: find the run that is both likely and correct.

Prerequisite map

Related deeper topics once you're fluent here: Reflexion (adding self-critique to the loop), MRKL (routing questions to tools), and Self-Consistency (sampling many trajectories).

Equipment checklist

Cover the right side and see if you can answer each before revealing.