Exercises — ReAct (reasoning + acting) framework

1. Action — it is an executable tool call written in Tool["..."] form.
2. Thought — internal self-talk deciding what to do next; no tool is invoked.
3. Observation — it is the result handed back by the tool (note it is a returned fact, not a decision).

Rule of thumb: Actions are commands, Observations are results, Thoughts are reasons.

(b) grounding via real observations. Pure chain-of-thought reasons only from what is already inside the model's weights (its training memory). It never receives an $o_t$ from the outside world, so it cannot correct a wrong belief. ReAct's loop inserts $o_t$ after every $a_t$, which anchors the reasoning to real data. See Retrieval-Augmented Generation (RAG) for a related "fetch real facts" idea.

Thought 1: I need $12^3$ first, because "cube of 12" is the inner quantity. Action 1: Calculate[12 ** 3] Observation 1: 1728 Thought 2: Now take 8% of 1728, i.e. $0.08\times 1728$. Action 2: Calculate[0.08 * 1728] Observation 2: 138.24 Thought 3: I have the value; finish. Action 3: Finish[138.24]

Answer: 138.24. The order matters: you must cube before taking the percentage, so the first tool call is the inner operation.

Thought 1: I must first learn who directed Inception; I can't search a birth year without a name. Action 1: Search["director of the film Inception"] Observation 1: "Inception was directed by Christopher Nolan." Thought 2: Now that I have the name, I can look up that person's birth year. Action 2: Search["Christopher Nolan birth year"] Observation 2: "Christopher Nolan was born in 1970." Thought 3: I have the year; finish. Action 3: Finish["1970"]

Key point: Action 2 depends on Observation 1. You literally cannot write query 2 until query 1 returns. This is what "multi-hop" means.

(a) $P(o_t\mid a_t)$ is deterministic — once the action $a_t$ is fixed, the environment (the tool) returns exactly one observation, so this probability is $1$ (for the real observation) or $0$ (for any other). The other two factors are stochastic: the language model could have produced many different thoughts or picked many different actions, each with some probability. Only the LM factors carry real uncertainty.

(b) Multiply every entry: $P(\tau \mid Q)=(0.5\cdot 0.8\cdot 1)(0.4\cdot 0.5\cdot 1)(0.9\cdot 1.0\cdot 1)$ $=0.40\times 0.20\times 0.90=\mathrm{0.072.}$ So $P(\tau \mid Q)=0.072$. Notice the $P(o_t\mid a_t)=1$ terms drop out — they never change the product.

An action-only agent has no place to interpret the observation. Its next step is another action, chosen with no explicit reasoning about the failure — so it typically repeats a near-identical query or blindly moves on.

A Thought step gives the agent a slot to reason: "The query returned nothing; the phrasing may be too obscure. Let me broaden it." This produces a revised action, e.g. Search["quorum flux definition"]. This is the error-recovery property: observations flag the problem, thoughts choose the fix. Compare with Reflexion, which formalises this "reflect on failure" idea across whole episodes.

Thought 1: I need Japan's capital before I can look up its population. Action 1: Search["capital of Japan"] Observation 1: "The capital of Japan is Tokyo." Thought 2: Now I need Tokyo's city-limits population. Action 2: Search["Tokyo city limits population millions"] Observation 2: "Tokyo has about 14.0 million people within city limits." Thought 3: Compute 20% of 14.0, i.e. $0.20\times 14.0$. Action 3: Calculate[0.20 * 14.0] Observation 3: 2.8 Thought 4: I have the value; finish. Action 4: Finish["2.8 million"]

Answer: 2.8 million. This trajectory shows tool switching: two Search hops feed a Calculate hop. Each thought names exactly the next action — see Tool Use in LLMs and MRKL for how multiple tools are routed.

The loop repeats Thought → Action → Observation, and exits only when the chosen action is Finish[...]:

flowchart TD
  Q["Question Q"] --> T["Thought t"]
  T --> A["Action a_t"]
  A --> Check{"is a_t Finish"}
  Check -->|no| O["Observation o_t"]
  O --> T
  Check -->|yes| End["Return answer"]

The decision diamond is the only exit: if $a_t=\text{Finish[...]}$ we stop; otherwise we get an observation and loop back to a new Thought.

(a) This is ReAct combined with Self-Consistency — sample multiple trajectories, then vote on the final answers. (b) Voting helps when the errors are independent and the correct answer is the most common single outcome: if each trajectory is more likely right than any specific wrong answer, majority vote amplifies the correct one. Concretely, suppose each trajectory is correct with probability $p$ and, when wrong, wrong in scattered non-agreeing ways. Then the majority of 3 is correct if at least 2 are correct: $P(\text{maj correct})=p^3+3p^2(1-p).$ For $p=0.6$: $0.6^3+3(0.6^2)(0.4)=0.216+0.432=0.648>0.6$ — voting helped. Voting does not help when trajectories share a systematic bias (all hallucinate the same wrong fact) — then they agree on the wrong answer and voting cements the mistake. Independence of errors is the whole engine.

Why CoT fails in principle: the answer depends on data created after training (today's price). No amount of internal reasoning can recover a number that was never in the weights. Only an observation from a live tool can supply it — this is the grounding argument at its sharpest.

ReAct trajectory: Thought 1: I need the current price of AAPL. Action 1: GetPrice["AAPL"] Observation 1: "AAPL current price = 190.00" Thought 2: I also need its 50-day average. Action 2: GetAverage["AAPL", 50] Observation 2: "AAPL 50-day average = 182.50" Thought 3: Compare 190.00 with 182.50. Action 3: Compare[190.00, 182.50] Observation 3: "190.00 > 182.50 → True" Thought 4: The current price exceeds the average; finish. Action 4: Finish["Yes, above its 50-day average"]

Every fact ($190.00$, $182.50$) entered through an observation, never invented. This is exactly the Agent Architectures principle: reasoning orchestrates, tools supply truth.

• Can I label any line as Thought / Action / Observation? (L1)
• Can I write a dependent multi-hop trajectory without batching? (L2)
• Do I set $P(o_t\mid a_t)=1$ and multiply the LM factors correctly? (L3)
• Do I always end with Finish[...]? (L4)
• Do I know when self-consistency helps vs. cements error? (L5)