Agent architectures and reasoning loops
Overview
An AI agent architecture defines how an autonomous system perceives its environment, makes decisions, and takes actions to achieve goals. The reasoning loop is the core cycle that drives agent behavior: Observe → Think → Act → Repeat.
Core Intuition
Key Architectural Components
1. Perception Module
What it does:
- Parses natural language instructions
- Retrieves relevant context from memory
- Formats tool outputs into structured data
Why it matters: Raw inputs are mesy. Perception standardizes them so the reasoning engine doesn't have to handle JSON parsing, text cleaning, and retrieval simultaneously.
2. Reasoning Engine
Common reasoning strategies:
| Strategy | When to Use | Example |
|---|---|---|
| ReAct (Reason + Act) | General-purpose, tool-heavy tasks | "I need current weather → call weather API → format response" |
| Chain-of-Thought | Multi-step logical problems | Math word problems, logical puzzles |
| Tree-of-Thoughts | Multiple solution paths need exploration | Strategic games, creative writing |
| Reflexion | Learning from failures | Retry after errors with self-critique |
Derivation: Why ReAct Works
Start from first principles: an LM alone can't access real-time data or execute code. We need:
ReAct interleaves these:
Each cycle adds to context, so the LLM "sees" its own reasoning + tool results when deciding the next step.
3. Action Execution Module
What it does:
- Validates action syntax (does
search("query")match the tool schema?) - Executes with error handling (retry on timeout, catch exceptions)
- Returns observations to the reasoning loop
4. Memory System
Types:
- Short-term (episodic): Current conversation context, tool call history
- Long-term (semantic): Persistent facts, user preferences, past solutions
- Working: Temporary scratch space for multi-step reasoning
Why it matters: Without memory, every loop starts from zero. Memory enables continuity, learning, and complex tasks that span many steps.
The Standard Reasoning Loop
Step-by-step:
- Observe: Aggregate inputs (user message, memory, tool outputs)
- Reason: LM generates thought + action
- Act: Execute action (call tool or respond)
- Update: Append observation to context
- Terminate check: Goal met? Max steps? If not, goto 1.
Worked Example: Multi-Step Research Agent
Task: "What's the population of the capital of France, and is it larger than London's?"
Trace Through the Loop
Cycle 1:
-
Observe: User query: "What's the population of the capital of France..."
-
Reason (LLM output):
Action: search("capital of France")Why this step? The agent breaks the compound question into atomic sub-goals.
-
Act: Execute
search("capital of France") -
Observation: "Paris is the capital of France."
Cycle 2:
-
Observe: Context now includes previous thought + "Paris is the capital"
-
Reason:
Thought: Now I need Paris's population. Action: search("population of Paris 2024")Why? The agent chains information: capital → population.
-
Act: Execute search
-
Observation: "Paris metro area: ~12.3 million"
Cycle 3:
- Reason:
Thought: I also need London's population to compare. Action: search("population of London 2024") - Observation: "Greater London: ~9.6 million"
Cycle 4:
- Reason:
Thought: I have both numbers. Paris (12.3M) > London (9.6M). Action: respond("Paris has ~12.3 million people, which is larger than London's ~9.6 million.") - Terminate: Goal met.
Common Reasoning Patterns
ReAct (Reason + Act)
Why it works: The explicit "Thought:" prefix forces the model to explain its reasoning before acting, improving accuracy.
Chain-of-Thought (CoT)
For: Problems requiring stepwise logic without external tools.
Example: "If 3 aples cost $5, how much do 7 apples cost?"
Thought: First find per-apple cost: $5/3 = $1.67
Thought: Then multiply by 7: $1.67 × 7 = $11.67
Answer: $11.67
Tree-of-Thoughts (ToT)
For: Exploring multiple solution branches, backtracking on failure.
Each step generates candidate thoughts, evaluates them, and expands the best:
Use case: Creative writing (multiple plot directions), strategic games (chess moves).
Memory Integration
Short-term Memory
Implementation: The conversation context itself.
Limit: Context windows (e.g., 128k tokens). Beyond this, summarize or discard old messages.
Long-term Memory
Implementation: Vector database (e.g., embedings + similarity search).
When the agent observes a fact worth remembering:
Later, when relevant:
Worked Example: Reflexion Loop
Task: Debug a failing Python function.
Attempt 1:
- Action:
execute_code(original_function) - Observation:
TypeError: unsupported operand type(s) for +: 'int' and 'str'
Reflexion:
Thought: The error shows I'm adding int + str. I likely forgot to cast user input.
Action: edit_code(add type conversion: int(user_input))
Attempt 2:
- Action:
execute_code(modified_function) - Observation:
Success output is42
Why reflexion works: The agent self-critiques failures, updates its mental model, and tries again with corrections.
Architecture Variants
Simple Reflex Agent
No memory, no reasoning. Maps percepts directly to actions (e.g., thermostat).
Limitation: Can't handle tasks requiring history or multi-step plans.
Goal-Based Agent
Choses actions that best advance toward a defined goal.
Example: Navigation agent selects moves that reduce distance to target.
Utility-Based Agent
Optimizes expected utility (trade-offs between speed, cost, accuracy).
Example: Delivery drone balances battery life vs. delivery time.
Learning Agent
Adds a learning module that updates the reasoning engine based on feedback:
Example: RL-based agent improves tool selection over time by reward signals.
Common Mistakes
Connections
- Prompt Engineering – Reasoning loops rely on carefully structured prompts
- Tool Use and Function Calling – Action execution requires tool schemas
- Retrieval-Augmented Generation – Long-term memory uses RAG for fact retrieval
- Reinforcement Learning Basics – Learning agents useRL to update policies
- LM Context Windows – Memory constraints shape architecture choices
- Chain-of-Thought Prompting – A reasoning strategy within the loop
- Multi-Agent Systems – Multiple reasoning loops coordinating
Summary
Agent architectures structure how autonomous AI systems operate: perceive → reason → act → update. The reasoning loop cycles through these phases, using memory to maintain state and tools to interact with the world. Different patterns(ReAct, CoT, ToT, Reflexion) suit different task types. Key failure modes: infinite loops, context overflow, and action parsing errors—all preventable with explicit termination checks, memory summarization, and validation.
Flashcards
#flashcards/ai-ml
What are the four core modules in a standard agent architecture? :: Perception (processes inputs), Reasoning Engine (decides actions), Action Execution (runs tools/operations), Memory (stores history and knowledge)
What is the ReAct pattern?
Why does ReAct work better than pure tool-use?
What is the agent reasoning loop formula?
What are the three types of agent memory?
What is Chain-of-Thought reasoning?
What is Tree-of-Thoughts?
What is Reflexion in agent architectures?
What is a common failure mode with reasoning loops?
How do you prevent context overflow in long agent sessions?
Why do action parsing failures occur?
What distinguishes a goal-based agent from a reflex agent?
What is a utility-based agent?
Recall Explain to a 12-year-old
Imagine you're playing a video game where you control a character trying to solve puzzles. Your character can't see the whole game at once—it has to:
- Look around (perception): "What's in this room? Is there a locked door?"
- Think (reasoning): "I need a key. Maybe it's in that chest?"
- Do something (action): Walk to the chest, open it.
- See what happened (observation): "I found a red key!"
- Remember (memory): "Okay, I have a red key now. The door needs a red key."
- Repeat: Go to the door, use the key, enter the next room.
An AI agent is like this game character, but for real tasks. You ask it "What's the weather in Tokyo?" It can't just guess—it has to:
- Think: "I need to search for Tokyo weather"
- Act: Actually run a search tool
- See the result: "It's 25°C and sunny"
- Respond: "The weather in Tokyo is 25°C and sunny!"
The loop is important because one step leads to the next. If the chest was empty, the character would think again: "Maybe the key is in the other room?" Same with AI—if a tool fails, it tries something else. The architecture is the rules for how the character (agent) makes decisions, remembers things, and knows when it's done. Without it, the AI would just talk randomly instead of actually solving problems!
Concept Map
Hinglish (regional understanding)
Intuition Hinglish mein samjho
Agent architecture matlabek autonomous AI system ka blueprint hai—kaise wo perceive karega, sochega, aur actions lega. Core concept hai reasoning loop: Observe → Reason → Act → Update, phir repeat. Jaise tum cooking karte waqt ingredients check karte ho, next step decide karte ho (pyaz katne hain?), execute karte ho, aur result dekhte ho—agent bhi yehi karta hai par software tasks ke liye.
Architecture mein char main parts hain: Perception (inputs ko samajhna), Reasoning Engine (decide karna kya karna hai, jaise ReAct ya Chain-of-Thought use karke), Action Execution (tools ya APIs ko actually run karna), aur Memory (purani information store karna). Memory teen tarah ki hoti hai: short-term (abhi ka conversation), long-term (persistent facts vector DB mein), aur working (temporary scratch space). Bina memory ke agent har baar zero se shuru karega, multi-step tasks handle nahi kar payega.
ReAct pattern sabse useful hai tool-heavy tasks ke liye: agent alternate karta hai "Thought" (reasoning) aur "Action" (tool call) ke bech, aur har observation next decision mein use hota hai. Example: "France ki capital ki population batao"—pehle capital dhundho (Paris), phir population search karo, phir user ko jawab do. Har step explicitly previous step ko build karta hai. Common mistakes: infinite loops (termination check nahi lagaya), context overflow (sab kuch append karte raho without summarization), aur action parsing failures (LLM typo kar sakta hai tool calls mein). Fix: max iterations set karo, old steps summarize karo, aur tool outputs validate karo. Yeh architecture complex real-world tasks ko structured tarike se solve karne ke liye zaruri hai—without it, LM sirf text generate karega, goal-directed actions nahi le payega.