6.2.1AI Agents & Tool Use

Agent architectures and reasoning loops

2,595 words12 min readdifficulty · medium1 backlinks

Overview

An AI agent architecture defines how an autonomous system perceives its environment, makes decisions, and takes actions to achieve goals. The reasoning loop is the core cycle that drives agent behavior: Observe → Think → Act → Repeat.


Core Intuition


Key Architectural Components

1. Perception Module

What it does:

  • Parses natural language instructions
  • Retrieves relevant context from memory
  • Formats tool outputs into structured data

Why it matters: Raw inputs are mesy. Perception standardizes them so the reasoning engine doesn't have to handle JSON parsing, text cleaning, and retrieval simultaneously.


2. Reasoning Engine

Common reasoning strategies:

Strategy When to Use Example
ReAct (Reason + Act) General-purpose, tool-heavy tasks "I need current weather → call weather API → format response"
Chain-of-Thought Multi-step logical problems Math word problems, logical puzzles
Tree-of-Thoughts Multiple solution paths need exploration Strategic games, creative writing
Reflexion Learning from failures Retry after errors with self-critique

Derivation: Why ReAct Works

Start from first principles: an LM alone can't access real-time data or execute code. We need:

Agent Output=f(LM reasoning,Tool results)\text{Agent Output} = f(\text{LM reasoning}, \text{Tool results})

ReAct interleaves these:

Thoughtt=LLM(contextt)Actiont=parse(Thoughtt)Observationt+1=execute(Actiont)contextt+1=contextt+(Thoughtt,Actiont,Observationt+1)\begin{align} \text{Thought}_t &= \text{LLM}(\text{context}_t) \\ \text{Action}_t &= \text{parse}(\text{Thought}_t) \\ \text{Observation}_{t+1} &= \text{execute}(\text{Action}_t) \\ \text{context}_{t+1} &= \text{context}_t + (\text{Thought}_t, \text{Action}_t, \text{Observation}_{t+1}) \end{align}

Each cycle adds to context, so the LLM "sees" its own reasoning + tool results when deciding the next step.


3. Action Execution Module

What it does:

  • Validates action syntax (does search("query") match the tool schema?)
  • Executes with error handling (retry on timeout, catch exceptions)
  • Returns observations to the reasoning loop

4. Memory System

Types:

Memory={Short-term,Long-term,Working}\text{Memory} = \{\text{Short-term}, \text{Long-term}, \text{Working}\}
  • Short-term (episodic): Current conversation context, tool call history
  • Long-term (semantic): Persistent facts, user preferences, past solutions
  • Working: Temporary scratch space for multi-step reasoning

Why it matters: Without memory, every loop starts from zero. Memory enables continuity, learning, and complex tasks that span many steps.


The Standard Reasoning Loop

Step-by-step:

  1. Observe: Aggregate inputs (user message, memory, tool outputs)
  2. Reason: LM generates thought + action
  3. Act: Execute action (call tool or respond)
  4. Update: Append observation to context
  5. Terminate check: Goal met? Max steps? If not, goto 1.

Worked Example: Multi-Step Research Agent

Task: "What's the population of the capital of France, and is it larger than London's?"

Trace Through the Loop

Cycle 1:

  • Observe: User query: "What's the population of the capital of France..."

  • Reason (LLM output):

    Action: search("capital of France")

    Why this step? The agent breaks the compound question into atomic sub-goals.

  • Act: Execute search("capital of France")

  • Observation: "Paris is the capital of France."

Cycle 2:

  • Observe: Context now includes previous thought + "Paris is the capital"

  • Reason:

    Thought: Now I need Paris's population.
    Action: search("population of Paris 2024")
    

    Why? The agent chains information: capital → population.

  • Act: Execute search

  • Observation: "Paris metro area: ~12.3 million"

Cycle 3:

  • Reason:
    Thought: I also need London's population to compare.
    Action: search("population of London 2024")
    
  • Observation: "Greater London: ~9.6 million"

Cycle 4:

  • Reason:
    Thought: I have both numbers. Paris (12.3M) > London (9.6M).
    Action: respond("Paris has ~12.3 million people, which is larger than London's ~9.6 million.")
    
  • Terminate: Goal met.

Common Reasoning Patterns

ReAct (Reason + Act)

Why it works: The explicit "Thought:" prefix forces the model to explain its reasoning before acting, improving accuracy.

Chain-of-Thought (CoT)

For: Problems requiring stepwise logic without external tools.

Answer=f(problem,step1,step2,,stepn)\text{Answer} = f(\text{problem}, \text{step}_1, \text{step}_2, \ldots, \text{step}_n)

Example: "If 3 aples cost $5, how much do 7 apples cost?"

Thought: First find per-apple cost: $5/3 = $1.67
Thought: Then multiply by 7: $1.67 × 7 = $11.67
Answer: $11.67

Tree-of-Thoughts (ToT)

For: Exploring multiple solution branches, backtracking on failure.

Each step generates kk candidate thoughts, evaluates them, and expands the best:

Best path=argmaxpathTreenodepathscore(node)\text{Best path} = \arg\max_{\text{path} \in \text{Tree}} \sum_{\text{node} \in \text{path}} \text{score}(\text{node})

Use case: Creative writing (multiple plot directions), strategic games (chess moves).


Memory Integration

Short-term Memory

Implementation: The conversation context itself.

contextt=[msg1,msg2,,msgt]\text{context}_t = [\text{msg}_1, \text{msg}_2, \ldots, \text{msg}_t]

Limit: Context windows (e.g., 128k tokens). Beyond this, summarize or discard old messages.

Long-term Memory

Implementation: Vector database (e.g., embedings + similarity search).

When the agent observes a fact worth remembering:

embed(fact)vector DB\text{embed}(\text{fact}) \to \text{vector DB}

Later, when relevant:

retrieved facts=top-k(similarity(query,DB))\text{retrieved facts} = \text{top-k}(\text{similarity}(\text{query}, \text{DB}))

Worked Example: Reflexion Loop

Task: Debug a failing Python function.

Attempt 1:

  • Action: execute_code(original_function)
  • Observation: TypeError: unsupported operand type(s) for +: 'int' and 'str'

Reflexion:

Thought: The error shows I'm adding int + str. I likely forgot to cast user input.
Action: edit_code(add type conversion: int(user_input))

Attempt 2:

  • Action: execute_code(modified_function)
  • Observation: Success output is42

Why reflexion works: The agent self-critiques failures, updates its mental model, and tries again with corrections.


Architecture Variants

Simple Reflex Agent

action=f(current percept)\text{action} = f(\text{current percept})

No memory, no reasoning. Maps percepts directly to actions (e.g., thermostat).

Limitation: Can't handle tasks requiring history or multi-step plans.

Goal-Based Agent

action=argmaxagoal_achieved(a,state)\text{action} = \arg\max_{a} \text{goal\_achieved}(a, \text{state})

Choses actions that best advance toward a defined goal.

Example: Navigation agent selects moves that reduce distance to target.

Utility-Based Agent

action=argmaxaE[utility(outcome(a)]\text{action} = \arg\max_{a} \mathbb{E}[\text{utility}(\text{outcome}(a)]

Optimizes expected utility (trade-offs between speed, cost, accuracy).

Example: Delivery drone balances battery life vs. delivery time.

Learning Agent

Adds a learning module that updates the reasoning engine based on feedback:

θt+1=θt+αθL(experiencet)\theta_{t+1} = \theta_t + \alpha \nabla_\theta \mathcal{L}(\text{experience}_t)

Example: RL-based agent improves tool selection over time by reward signals.


Common Mistakes


Connections


Summary

Agent architectures structure how autonomous AI systems operate: perceive → reason → act → update. The reasoning loop cycles through these phases, using memory to maintain state and tools to interact with the world. Different patterns(ReAct, CoT, ToT, Reflexion) suit different task types. Key failure modes: infinite loops, context overflow, and action parsing errors—all preventable with explicit termination checks, memory summarization, and validation.


Flashcards

#flashcards/ai-ml

What are the four core modules in a standard agent architecture? :: Perception (processes inputs), Reasoning Engine (decides actions), Action Execution (runs tools/operations), Memory (stores history and knowledge)

What is the ReAct pattern?
Reasoning + Acting interleaved—agent alternates generating Thoughts and Actions, seeing observations from executed actions in context to inform next steps
Why does ReAct work better than pure tool-use?
Explicit "Thought:" steps force the LLM to explain reasoning before acting, improving decision quality and debugability
What is the agent reasoning loop formula?
state → observe → reason → action → observation → update state → loop (with termination check)
What are the three types of agent memory?
Short-term (episodic/conversation context), Long-term (persistent facts in vector DB), Working (temporary scratch space)
What is Chain-of-Thought reasoning?
Breaking multi-step problems into explicit intermediate steps without external tools, each step building on the prior
What is Tree-of-Thoughts?
Exploring multiple reasoning branches in parallel, evaluating each, and expanding the highest-scoring paths (like beam search for reasoning)
What is Reflexion in agent architectures?
Self-critique loop where agent analyzes failures, updates its approach, and retries with corrections
What is a common failure mode with reasoning loops?
Infinite loops from missing termination conditions (LM halucinates progress or cycles without goal checks)
How do you prevent context overflow in long agent sessions?
Summarize old steps, keep only recent k cycles in context, or move detailed history to external memory
Why do action parsing failures occur?
LLMs can output malformed tool calls (typos, wrong syntax, extra text) despite instructions, requiring validation and retry logic
What distinguishes a goal-based agent from a reflex agent?
Goal-based agents select actions that advance toward a defined goal (using search/planning), reflex agents map percepts directly to actions without reasoning
What is a utility-based agent?
Agent that choses actions maximizing expected utility, balancing trade-offs (cost vs speed vs accuracy) rather than binary goal achievement

Recall Explain to a 12-year-old

Imagine you're playing a video game where you control a character trying to solve puzzles. Your character can't see the whole game at once—it has to:

  1. Look around (perception): "What's in this room? Is there a locked door?"
  2. Think (reasoning): "I need a key. Maybe it's in that chest?"
  3. Do something (action): Walk to the chest, open it.
  4. See what happened (observation): "I found a red key!"
  5. Remember (memory): "Okay, I have a red key now. The door needs a red key."
  6. Repeat: Go to the door, use the key, enter the next room.

An AI agent is like this game character, but for real tasks. You ask it "What's the weather in Tokyo?" It can't just guess—it has to:

  • Think: "I need to search for Tokyo weather"
  • Act: Actually run a search tool
  • See the result: "It's 25°C and sunny"
  • Respond: "The weather in Tokyo is 25°C and sunny!"

The loop is important because one step leads to the next. If the chest was empty, the character would think again: "Maybe the key is in the other room?" Same with AI—if a tool fails, it tries something else. The architecture is the rules for how the character (agent) makes decisions, remembers things, and knows when it's done. Without it, the AI would just talk randomly instead of actually solving problems!

Concept Map

drives

Observe

standardizes inputs

Act

Observation feeds back

uses strategy

uses strategy

uses strategy

uses strategy

interleaves

appends to

informs next step

calls

AI Agent Architecture

Reasoning Loop

Perception Module

Reasoning Engine

Action Execution Module

ReAct

Chain-of-Thought

Tree-of-Thoughts

Reflexion

Thought + Action + Observation

Context

APIs and Tools

Hinglish (regional understanding)

Intuition Hinglish mein samjho

Agent architecture matlabek autonomous AI system ka blueprint hai—kaise wo perceive karega, sochega, aur actions lega. Core concept hai reasoning loop: Observe → Reason → Act → Update, phir repeat. Jaise tum cooking karte waqt ingredients check karte ho, next step decide karte ho (pyaz katne hain?), execute karte ho, aur result dekhte ho—agent bhi yehi karta hai par software tasks ke liye.

Architecture mein char main parts hain: Perception (inputs ko samajhna), Reasoning Engine (decide karna kya karna hai, jaise ReAct ya Chain-of-Thought use karke), Action Execution (tools ya APIs ko actually run karna), aur Memory (purani information store karna). Memory teen tarah ki hoti hai: short-term (abhi ka conversation), long-term (persistent facts vector DB mein), aur working (temporary scratch space). Bina memory ke agent har baar zero se shuru karega, multi-step tasks handle nahi kar payega.

ReAct pattern sabse useful hai tool-heavy tasks ke liye: agent alternate karta hai "Thought" (reasoning) aur "Action" (tool call) ke bech, aur har observation next decision mein use hota hai. Example: "France ki capital ki population batao"—pehle capital dhundho (Paris), phir population search karo, phir user ko jawab do. Har step explicitly previous step ko build karta hai. Common mistakes: infinite loops (termination check nahi lagaya), context overflow (sab kuch append karte raho without summarization), aur action parsing failures (LLM typo kar sakta hai tool calls mein). Fix: max iterations set karo, old steps summarize karo, aur tool outputs validate karo. Yeh architecture complex real-world tasks ko structured tarike se solve karne ke liye zaruri hai—without it, LM sirf text generate karega, goal-directed actions nahi le payega.

Go deeper — visual, from zero

Test yourself — AI Agents & Tool Use

Connections