Agent architectures and reasoning loops — Hinglish notes

6.2.1 · AI-ML › AI Agents & Tool Use

Overview

Ek AI agent architecture define karta hai ki ek autonomous system apne environment ko kaise perceive karta hai, decisions kaise leta hai, aur goals achieve karne ke liye actions kaise leta hai. Reasoning loop woh core cycle hai jo agent behavior ko drive karta hai: Observe → Think → Act → Repeat.

Core Intuition

Key Architectural Components

1. Perception Module

Yeh kya karta hai:

Natural language instructions parse karta hai
Memory se relevant context retrieve karta hai
Tool outputs ko structured data mein format karta hai

Yeh kyun important hai: Raw inputs messy hote hain. Perception unhe standardize karta hai taaki reasoning engine ko JSON parsing, text cleaning, aur retrieval ek saath handle na karna pade.

2. Reasoning Engine

Common reasoning strategies:

Strategy	Kab Use Karein	Example
ReAct (Reason + Act)	General-purpose, tool-heavy tasks	"Mujhe current weather chahiye → weather API call karo → response format karo"
Chain-of-Thought	Multi-step logical problems	Math word problems, logical puzzles
Tree-of-Thoughts	Jab multiple solution paths explore karne hon	Strategic games, creative writing
Reflexion	Failures se seekhna	Errors ke baad self-critique ke saath retry karna

Derivation: ReAct Kyun Kaam Karta Hai

First principles se shuru karo: ek LM akele real-time data access ya code execute nahi kar sakta. Humein chahiye:

Agent Output = f (LM reasoning, Tool results)

ReAct inhe interleave karta hai:

Thought_{t} Action_{t} Observation_{t + 1} context_{t + 1} = LLM (context_{t}) = parse (Thought_{t}) = execute (Action_{t}) = context_{t} + (Thought_{t}, Action_{t}, Observation_{t + 1})

Har cycle context mein add hoti hai, isliye LLM agla step decide karte waqt apni khud ki reasoning + tool results "dekhta" hai.

3. Action Execution Module

Yeh kya karta hai:

Action syntax validate karta hai (kya search("query") tool schema se match karta hai?)
Error handling ke saath execute karta hai (timeout par retry, exceptions catch karo)
Reasoning loop ko observations return karta hai

4. Memory System

Types:

Memory = {Short-term, Long-term, Working}

Short-term (episodic): Current conversation context, tool call history
Long-term (semantic): Persistent facts, user preferences, past solutions
Working: Multi-step reasoning ke liye temporary scratch space

Yeh kyun important hai: Memory ke bina, har loop zero se shuru hota hai. Memory continuity, learning, aur aisi complex tasks enable karti hai jo kaafi steps mein span karti hain.

The Standard Reasoning Loop

Step-by-step:

Observe: Inputs aggregate karo (user message, memory, tool outputs)
Reason: LM thought + action generate karta hai
Act: Action execute karo (tool call karo ya respond karo)
Update: Observation ko context mein append karo
Terminate check: Goal met hua? Max steps? Agar nahi, goto 1.

Worked Example: Multi-Step Research Agent

Task: "France ki capital ki population kya hai, aur kya yeh London se badi hai?"

Loop ke Through Trace

Cycle 1:

Observe: User query: "France ki capital ki population kya hai..."
Reason (LLM output):
```
Action: search("capital of France")
```
Yeh step kyun? Agent compound question ko atomic sub-goals mein todta hai.
Act: search("capital of France") execute karo
Observation: "Paris is the capital of France."

Cycle 2:

Observe: Context mein ab pichla thought + "Paris is the capital" shamil hai

Reason:

Thought: Ab mujhe Paris ki population chahiye.
Action: search("population of Paris 2024")

Kyun? Agent information chain karta hai: capital → population.

Act: Search execute karo
Observation: "Paris metro area: ~12.3 million"

Cycle 3:

Reason:

Thought: Compare karne ke liye mujhe London ki population bhi chahiye.
Action: search("population of London 2024")

Observation: "Greater London: ~9.6 million"

Cycle 4:

Reason:

Thought: Mere paas dono numbers hain. Paris (12.3M) > London (9.6M).
Action: respond("Paris mein ~12.3 million log hain, jo London ke ~9.6 million se zyada hai.")

Terminate: Goal met.

Common Reasoning Patterns

ReAct (Reason + Act)

Yeh kyun kaam karta hai: Explicit "Thought:" prefix model ko action lene se pehle apni reasoning explain karne par majboor karta hai, jisse accuracy improve hoti hai.

Chain-of-Thought (CoT)

Ke liye: Aise problems jo external tools ke bina stepwise logic require karte hain.

Answer = f (problem, step_{1}, step_{2}, \dots, step_{n})

Example: "Agar 3 seb $5 ke hain, toh 7 seb kitne ke honge?"

Thought: Pehle per-apple cost nikalo: $5/3 = $1.67
Thought: Phir 7 se multiply karo: $1.67 × 7 = $11.67
Answer: $11.67

Tree-of-Thoughts (ToT)

Ke liye: Multiple solution branches explore karna, failure par backtrack karna.

Har step $k$ candidate thoughts generate karta hai, unhe evaluate karta hai, aur best ko expand karta hai:

Best path = ar g path \in Tree max node \in path \sum score (node)

Use case: Creative writing (multiple plot directions), strategic games (chess moves).

Memory Integration

Short-term Memory

Implementation: Conversation context khud hi.

context_{t} = [msg_{1}, msg_{2}, \dots, msg_{t}]

Limit: Context windows (e.g., 128k tokens). Iske baad, purane messages summarize karo ya discard karo.

Long-term Memory

Implementation: Vector database (e.g., embeddings + similarity search).

Jab agent koi fact yaad rakhne layak observe kare:

embed (fact) \to vector DB

Baad mein, jab relevant ho:

retrieved facts = top-k (similarity (query, DB))

Worked Example: Reflexion Loop

Task: Ek failing Python function debug karo.

Attempt 1:

Action: execute_code(original_function)
Observation: TypeError: unsupported operand type(s) for +: 'int' and 'str'

Reflexion:

Thought: Error dikhata hai ki main int + str add kar raha hoon. Shayad main user input cast karna bhool gaya.
Action: edit_code(type conversion add karo: int(user_input))

Attempt 2:

Action: execute_code(modified_function)
Observation: Success output is 42

Reflexion kyun kaam karta hai: Agent failures ko self-critique karta hai, apna mental model update karta hai, aur corrections ke saath dobara koshish karta hai.

Architecture Variants

Simple Reflex Agent

action = f (current percept)

Koi memory nahi, koi reasoning nahi. Percepts ko directly actions par map karta hai (e.g., thermostat).

Limitation: Aisi tasks handle nahi kar sakta jo history ya multi-step plans require karte hain.

Goal-Based Agent

action = ar g a max goal_achieved (a, state)

Woh actions choose karta hai jo defined goal ki taraf best advance karte hain.

Example: Navigation agent woh moves select karta hai jo target se distance kam karein.

Utility-Based Agent

action = ar g a max E [utility (outcome (a)]

Expected utility optimize karta hai (speed, cost, accuracy ke beech trade-offs).

Example: Delivery drone battery life vs. delivery time balance karta hai.

Learning Agent

Ek learning module add karta hai jo feedback ke basis par reasoning engine update karta hai:

θ_{t + 1} = θ_{t} + α \nabla_{θ} L (experience_{t})

Example: RL-based agent reward signals ke through time ke saath tool selection improve karta hai.

Common Mistakes

Common mistake Missing Termination se Infinite Loops

Galat idea: "Loop automatically ruk jaayega jab task ho jaayega." Yeh sahi kyun lagta hai: Agar LM smart hai, toh use pata hona chahiye kab rukna hai. Reality: LMs progress hallucinate kar sakte hain ya circular reasoning mein phans sakte hain. Explicit termination (max steps, goal check, no-progress detection) ke bina, agent forever loop karta rahta hai.

Fix: Enforce karo:

max_iterations = 10
if iterations >= max_iterations or goal_achieved():
   break

Common mistake Action Parsing Failures

Galat idea: "LM hamesha valid tool calls output karega."

Reality: LMs malformed tool calls output kar sakte hain (typos, wrong syntax, extra text) instructions ke bawajood.

Fix: Structured output use karo (JSON mode), regex validation, ya error feedback ke saath retry karo:

Connections

Prompt Engineering – Reasoning loops carefully structured prompts par rely karte hain
Tool Use and Function Calling – Action execution ke liye tool schemas chahiye
Retrieval-Augmented Generation – Long-term memory fact retrieval ke liye RAG use karta hai
Reinforcement Learning Basics – Learning agents policies update karne ke liye RL use karte hain
LM Context Windows – Memory constraints architecture choices ko shape karti hain
Chain-of-Thought Prompting – Loop ke andar ek reasoning strategy
Multi-Agent Systems – Multiple reasoning loops coordinating

Summary

Agent architectures structure karte hain ki autonomous AI systems kaise operate karte hain: perceive → reason → act → update. Reasoning loop in phases ke through cycle karta hai, state maintain karne ke liye memory aur duniya ke saath interact karne ke liye tools use karta hai. Alag-alag patterns (ReAct, CoT, ToT, Reflexion) alag-alag task types ke liye suitable hain. Key failure modes: infinite loops, context overflow, aur action parsing errors—sab explicit termination checks, memory summarization, aur validation se preventable hain.

Flashcards

#flashcards/ai-ml

Standard agent architecture mein chaar core modules kya hain? :: Perception (inputs process karta hai), Reasoning Engine (actions decide karta hai), Action Execution (tools/operations run karta hai), Memory (history aur knowledge store karta hai)

ReAct pattern kya hai?

Reasoning + Acting interleaved—agent alternating Thoughts aur Actions generate karta hai, executed actions ke observations context mein dekhte hue agla step inform karne ke liye

ReAct pure tool-use se behtar kyun kaam karta hai?

Explicit "Thought:" steps LLM ko action lene se pehle reasoning explain karne par majboor karte hain, decision quality aur debugability improve hoti hai

Agent reasoning loop formula kya hai?

state → observe → reason → action → observation → update state → loop (termination check ke saath)

Agent memory ke teen types kya hain?

Short-term (episodic/conversation context), Long-term (vector DB mein persistent facts), Working (temporary scratch space)

Chain-of-Thought reasoning kya hai?

Multi-step problems ko explicit intermediate steps mein todna, external tools ke bina, har step pichle par build karta hai

Tree-of-Thoughts kya hai?

Multiple reasoning branches parallel mein explore karna, har ko evaluate karna, aur highest-scoring paths expand karna (reasoning ke liye beam search jaisa)

Agent architectures mein Reflexion kya hai?

Self-critique loop jahan agent failures analyze karta hai, apna approach update karta hai, aur corrections ke saath retry karta hai

Reasoning loops mein ek common failure mode kya hai?

Missing termination conditions se infinite loops (LM progress hallucinate karta hai ya goal checks ke bina cycle karta hai)

Long agent sessions mein context overflow kaise prevent karte hain?

Purane steps summarize karo, context mein sirf recent k cycles rakho, ya detailed history external memory mein move karo

Action parsing failures kyun hoti hain?

LLMs malformed tool calls output kar sakte hain (typos, wrong syntax, extra text) instructions ke bawajood, jiske liye validation aur retry logic chahiye

Goal-based agent ko reflex agent se kya alag karta hai?

Goal-based agents woh actions select karte hain jo defined goal ki taraf advance karte hain (search/planning use karke), reflex agents percepts ko reasoning ke bina directly actions par map karte hain

Utility-based agent kya hota hai?

Woh agent jo expected utility maximize karne wale actions choose karta hai, trade-offs (cost vs speed vs accuracy) balance karta hai binary goal achievement ki jagah

Recall Ek 12-saal ke bachche ko explain karo

Socho tum ek video game khel rahe ho jahan tum ek character control karte ho jo puzzles solve karta hai. Tumhara character poora game ek saath nahi dekh sakta—usse:

Idhar-udhar dekhna hoga (perception): "Is room mein kya hai? Koi locked door hai?"
Sochna hoga (reasoning): "Mujhe ek key chahiye. Shayad woh us chest mein hai?"
Kuch karna hoga (action): Chest tak jaao, use kholna.
Dekhna hoga kya hua (observation): "Mujhe ek red key mili!"
Yaad rakhna hoga (memory): "Theek hai, mere paas red key hai. Door ko red key chahiye."
Repeat karo: Door par jaao, key use karo, agले room mein jaao.

Ek AI agent is game character jaisa hai, lekin real tasks ke liye. Tum usse poochte ho "Tokyo mein mausam kaisa hai?" Woh sirf guess nahi kar sakta—usse:

Sochna hoga: "Mujhe Tokyo weather search karna hoga"
Karna hoga: Actually ek search tool run karna
Result dekhna hoga: "Wahan 25°C aur sunny hai"
Respond karna hoga: "Tokyo mein mausam 25°C aur sunny hai!"

Loop important hai kyunki ek step agले ki taraf lead karta hai. Agar chest khaali tha, toh character phir se sochta: "Shayad key doosre room mein hai?" AI ke saath bhi waisa hi—agar koi tool fail karta hai, toh woh kuch aur try karta hai. Architecture woh rules hain jo batate hain ki character (agent) decisions kaise karta hai, cheezein kaise yaad rakhta hai, aur kab pata chalta hai ki kaam ho gaya. Iske bina, AI randomly baat karta instead of actually problems solve karne ke!