ReAct (reasoning + acting) framework

What Is ReAct?

Why This Architecture?

Pure chain-of-thought (CoT) can reason deeply but cannot interact with external knowledge or tools—it's trapped in its training data. Pure action agents can use tools but often hallucinate or make britle decisions without explicit reasoning. ReAct gets both:

1. Grounding: Observations from real tools prevent hallucination
2. Transparency: Reasoning traces make decisions interpretable
3. Error recovery: The agent can recognize mistakes in observations and change course
4. Dynamic planning: Each thought can revise the plan based on what was just learned

The ReAct Loop: From First Principles

Let's derive how ReAct operates step-by-step.

Starting point: You have a question $Q$ and a set of available tools $\mathcal{T} = \{t_1, t_2, \ldots, t_n\}$ (e.g., search, calculator, database).

Goal: Generate answer $A$ by alternating reasoning and acting.

Step 1: Initial Reasoning

The LM receives the question and generates a thought $\text{Thought}_1$:

$\text{Thought}_1 sim P_{\text{LM}}(\text{thought} \mid Q, \text{examples})$

This thought analyzes what information is needed. Example: "To find the population of Paris in 2023, I need to search for current demographic data."

Why this step? Without reasoning first, the agent might search for the wrong thing or not know why it's searching.

Step 2: Action Selection

Based on $\text{Thought}_1$, the LM choses an action $a_1$ from the tool set:

$a_1 = \arg\max_{a \in \mathcal{T}} P_{\text{LLM}}(a \mid Q, \text{Thought}_1)$

Example: Action: Search["Paris population 2023"]

Why this step? Reasoning without action is useless for questions requiring external knowledge. The action must match the thought's intent.

Step 3: Environment Observation

The environment (tool executor) runs $a_1$ and returns observation $o_1$:

$o_1 = \text{Execute}(a_1)$

Example: Observation: Paris had a population of approximately 2.1 million in 2023 within city limits.

Why this step? The observation provides grounding—real data that the LLM's parametric knowledge might lack or get wrong.

Step 4: Reasoning with New Information

The LLM now generates $\text{Thought}_2$ conditioned on everything so far:

$\text{Thought}_2 \sim P_{\text{LLM}}(\text{thought} \mid Q, \text{Thought}_1, a_1, o_1)$

Example: "The city limits population is 2.1M, but the question might be asking about the metropolitan area. I should search for that."

Why this step? The agent uses the observation to refine its understanding. This is where error recovery happens—if $o_1$ was unexpected, $\text{Thought}_2$ can pivot.

Step 5: Iteration Until Done

Repeat steps 2-4, building a trajectory:

$\tau = [(\text{Thought}_1, a_1, o_1), (\text{Thought}_2, a_2, o_2), \ldots, (\text{Thought}_T, a_T, o_T)]$

The loop terminates when the LLM generates:

$a_T = \text{Finish}[\text{answer}]$

Example: Action: Finish[The population of Paris is 2.1 million in the city and 12.4 million in the metro area as of 2023]

Mathematical Formulation

The probability of the entire trajectory given the question is:

$P(\tau \mid Q) = \prod_{t=1}^{T} P(\text{Thought}_t \mid Q, \tau_{<t}) \cdot P(a_t \mid Q, \tau_{<t}, \text{Thought}_t) \cdot P(o_t \mid a_t)$

Where:

• $P(\text{Thought}_t \mid \cdot)$ is the LM's reasoning distribution
• $P(a_t \mid \cdot)$ is the action selection distribution
• $P(o_t \mid a_t)$ is deterministic (the environment's response)

Why this matters: This factorization shows that ReAct is a sequential decision process where each step depends on all prior steps. The LLM is doing multi-step inference, not just one-shot generation.

Concrete Example: Multi-Hop Question

Question: "What is the capital of the country where the Eiffel Tower is located?"

Trajectory:

Thought 1: I need to find which country the Eiffel Tower is in. Action 1: Search["Eiffel Tower location"] Observation 1: "The Eiffel Tower is in Paris, France."

Why this step? The agent identifies it lacks country information and uses search to ground its knowledge.

Thought 2: Now I know it's in France. I need to find France's capital. Action 2: Search["capital of France"] Observation 2: "Paris is the capital of France."

Why this step? The observation from Action 1 enables a dependent second query. This is multi-hop reasoning—each step builds on the last.

Thought 3: I have all information needed. Paris is both the city with the Eiffel Tower and France's capital. Action 3: Finish["Paris"]

Why this step? The agent recognizes the task is complete and synthesizes the answer from accumulated observations.

Contrast with Pure CoT

A pure CoT approach would try to answer from memory:

Let me think step by step:
1. The Eiffel Tower is in Paris
2. Paris is in France (I think?)
3. The capital of France is Paris
Answer: Paris

Risk: If the LLM halucinates step 2 (e.g., says "Paris is in Belgium"), there's no correction mechanism. ReAct's observations enforce correctness.

Common Mistakes and How to Fix Them

When ReAct Fails

ReAct is not magic. It struggles with:

1. Poor tool design: If tools return noisy or irrelevant observations, the agent can't recover. Garbage in, garbage out.
2. Ambiguous questions: If the question is underspecified, the agent might not know what to search for. Reasoning traces expose this, but don't solve it.
3. Long trajectories: Each thought-action-observation cycle uses tokens. Very long tasks (20+ steps) can exceed context limits or become expensive.
4. Irreducible hallucination: If the LM misinterprets a correct observation (e.g., reads "2.1 million" as "21 million"), ReAct can't fix that—it's a model capability issue.

Connections to Other Concepts

• Chain-of-Thought (CoT): ReAct extends CoT by adding actions. CoT is "think then answer"; ReAct is "think, act, observe, think, act, ..."
• Tool Use in LLMs: ReAct is a structured framework for tool use, ensuring tools are called for good reasons (the thoughts).
• Agent Architectures: ReAct is a specific agent loop. Compare with Reflexion (self-reflection) and MRKL (modular reasoning).
• Self-Consistency: ReAct can be combined with self-consistency by sampling multiple trajectories and picking the most common answer.
• Retrieval-Augmented Generation (RAG): ReAct often uses search as a tool, making it a form of interactive RAG where the agent decides when to retrieve.

#flashcards/ai-ml

What are the three components of a ReAct loop? :: Thought (reasoning trace), Action (tool call), and Observation (environment feedback).

Concept Map

Hinglish (regional understanding)