ReAct (reasoning + acting) framework

6.2.2 · AI-ML › AI Agents & Tool Use

ReAct Kya Hai?

Yeh Architecture Kyun?

Pure chain-of-thought (CoT) deeply reason kar sakta hai lekin external knowledge ya tools ke saath interact nahi kar sakta—yeh apne training data mein trapped hai. Pure action agents tools use kar sakte hain lekin explicit reasoning ke bina aksar hallucinate karte hain ya brittle decisions lete hain. ReAct dono milata hai:

1. Grounding: Real tools se observations hallucination rokti hain
2. Transparency: Reasoning traces decisions ko interpretable banati hain
3. Error recovery: Agent observations mein mistakes pehchaan sakta hai aur course change kar sakta hai
4. Dynamic planning: Har thought abhi jo sikha uske basis pe plan revise kar sakta hai

ReAct Loop: First Principles Se

Chalte hain step-by-step derive karte hain ki ReAct kaise operate karta hai.

Starting point: Aapke paas ek question $Q$ hai aur available tools ka ek set $\mathcal{T}=\{t_1,t_2,\dots ,t_n\}$ hai (jaise search, calculator, database).

Goal: Reasoning aur acting ko alternate karke answer $A$ generate karo.

Step 1: Initial Reasoning

LM question receive karta hai aur ek thought ${\text{Thought}}_1$ generate karta hai:

${\text{Thought}}_1\sim P_{\text{LM}}(\text{thought}\mid Q,\text{examples})$

Yeh thought analyze karta hai ki kaunsi information chahiye. Example: "2023 mein Paris ki population find karne ke liye, mujhe current demographic data search karna hoga."

Yeh step kyun? Pehle reasoning ke bina, agent galat cheez search kar sakta hai ya yeh nahi jaanta ki kyun search kar raha hai.

Step 2: Action Selection

${\text{Thought}}_1$ ke basis pe, LM tool set se ek action $a_1$ choose karta hai:

$a_1=\arg {\max }_{a\in \mathcal{T}}{P}_{\text{LLM}}(a\mid Q,{\text{Thought}}_1)$

Example: Action: Search["Paris population 2023"]

Yeh step kyun? External knowledge chahiye wale questions ke liye reasoning without action bekar hai. Action ko thought ki intent se match karna chahiye.

Step 3: Environment Observation

Environment (tool executor) $a_1$ run karta hai aur observation $o_1$ return karta hai:

$o_1=\text{Execute}(a_1)$

Example: Observation: Paris had a population of approximately 2.1 million in 2023 within city limits.

Yeh step kyun? Observation grounding provide karta hai—real data jo LLM ki parametric knowledge mein miss ho sakti hai ya galat ho sakti hai.

Step 4: Nayi Information ke Saath Reasoning

LLM ab ${\text{Thought}}_2$ generate karta hai jo ab tak sab kuch pe conditioned hai:

${\text{Thought}}_2\sim P_{\text{LLM}}(\text{thought}\mid Q,{\text{Thought}}_1,a_1,o_1)$

Example: "City limits ki population 2.1M hai, lekin question metropolitan area ke baare mein pooch raha ho sakta hai. Mujhe uske liye search karna chahiye."

Yeh step kyun? Agent observation use karta hai apni understanding refine karne ke liye. Error recovery yahan hoti hai—agar $o_1$ unexpected tha, toh ${\text{Thought}}_2$ pivot kar sakta hai.

Step 5: Done Hone Tak Iteration

Steps 2-4 repeat karo, ek trajectory build karte hue:

$\tau =[({\text{Thought}}_1,a_1,o_1),({\text{Thought}}_2,a_2,o_2),\dots ,({\text{Thought}}_T,a_T,o_T)]$

Loop tab terminate hoti hai jab LLM generate karta hai:

$a_T=\text{Finish}[\text{answer}]$

Example: Action: Finish[The population of Paris is 2.1 million in the city and 12.4 million in the metro area as of 2023]

Mathematical Formulation

Question diye hue poori trajectory ki probability hai:

$P(\tau \mid Q)={\prod }_{t=1}^{T}P({\text{Thought}}_t\mid Q,{\tau }_{<t})\cdot P(a_t\mid Q,{\tau }_{<t},{\text{Thought}}_t)\cdot P(o_t\mid a_t)$

Jahan:

• $P({\text{Thought}}_t\mid \cdot )$ LM ki reasoning distribution hai
• $P(a_t\mid \cdot )$ action selection distribution hai
• $P(o_t\mid a_t)$ deterministic hai (environment ka response)

Yeh kyun matter karta hai: Yeh factorization dikhata hai ki ReAct ek sequential decision process hai jahan har step saare prior steps pe depend karta hai. LLM multi-step inference kar raha hai, sirf one-shot generation nahi.

Concrete Example: Multi-Hop Question

Question: "Jis desh mein Eiffel Tower hai, uski capital kya hai?"

Trajectory:

Thought 1: Mujhe find karna hai ki Eiffel Tower kis desh mein hai. Action 1: Search["Eiffel Tower location"] Observation 1: "The Eiffel Tower is in Paris, France."

Yeh step kyun? Agent identify karta hai ki uske paas country information nahi hai aur apni knowledge ground karne ke liye search use karta hai.

Thought 2: Ab mujhe pata hai yeh France mein hai. Mujhe France ki capital find karni hai. Action 2: Search["capital of France"] Observation 2: "Paris is the capital of France."

Yeh step kyun? Action 1 ka observation ek dependent second query enable karta hai. Yeh multi-hop reasoning hai—har step pichle pe build karta hai.

Thought 3: Mere paas saari zaruri information hai. Paris hi woh city hai jahan Eiffel Tower hai aur France ki capital bhi hai. Action 3: Finish["Paris"]

Yeh step kyun? Agent recognize karta hai ki task complete ho gayi hai aur accumulated observations se answer synthesize karta hai.

Pure CoT se Contrast

Ek pure CoT approach memory se answer karne ki koshish karta:

Let me think step by step:
1. The Eiffel Tower is in Paris
2. Paris is in France (I think?)
3. The capital of France is Paris
Answer: Paris

Risk: Agar LLM step 2 hallucinate kar de (jaise kahe "Paris Belgium mein hai"), toh koi correction mechanism nahi hai. ReAct ki observations correctness enforce karti hain.

Common Mistakes Aur Unhe Fix Kaise Karein

ReAct Kab Fail Hota Hai

ReAct koi magic nahi hai. Yeh inke saath struggle karta hai:

1. Poor tool design: Agar tools noisy ya irrelevant observations return karein, toh agent recover nahi kar sakta. Garbage in, garbage out.
2. Ambiguous questions: Agar question underspecified hai, toh agent ko pata nahi hoga kya search karein. Reasoning traces yeh expose karti hain, lekin solve nahi karti.
3. Long trajectories: Har thought-action-observation cycle tokens use karta hai. Bahut lambe tasks (20+ steps) context limits exceed kar sakte hain ya expensive ho sakte hain.
4. Irreducible hallucination: Agar LM ek correct observation ko misinterpret kare (jaise "2.1 million" ko "21 million" padhe), toh ReAct use fix nahi kar sakta—yeh ek model capability issue hai.

Doosre Concepts Se Connections

• Chain-of-Thought (CoT): ReAct, CoT ko actions add karke extend karta hai. CoT hai "sochho phir answer do"; ReAct hai "sochho, act karo, observe karo, sochho, act karo, ..."
• Tool Use in LLMs: ReAct tool use ke liye ek structured framework hai, ensure karta hai ki tools achhe reasons ke liye call hon (thoughts).
• Agent Architectures: ReAct ek specific agent loop hai. Reflexion (self-reflection) aur MRKL (modular reasoning) se compare karo.
• Self-Consistency: ReAct ko self-consistency ke saath combine kiya ja sakta hai multiple trajectories sample karke aur sabse common answer pick karke.
• Retrieval-Augmented Generation (RAG): ReAct aksar search ko tool ke roop mein use karta hai, isse interactive RAG ka ek form banata hai jahan agent decide karta hai ki kab retrieve karna hai.

#flashcards/ai-ml

ReAct loop ke teen components kya hain? :: Thought (reasoning trace), Action (tool call), aur Observation (environment feedback).

Concept Map