Overview

Agentic frameworks are software libraries that provide abstractions and tools for building AI agents – systems where language models can reason, plan, use tools, and interact with external data sources to accomplish complex tasks autonomously.

The two dominant frameworks are:

• LangChain: Focus on chaining LM calls with tools, memory, and external APIs
• LlamaIndex: Focus on indexing and querying over large document collections

Agentic frameworks solve this by providing pre-built components (prompt templates, tool interfaces, memory stores retrieval systems) that snap together. Instead of writing 500 lines of parsing logic, you write 20 lines configuring the framework.

Think of it like web frameworks (Flask, Django) vs raw HTTP sockets. You could build everything from scratch, but frameworks eliminate boilerplate and encode best practices.

LangChain: Chains, Agents, and Tools

Core Architecture: Deriving the Agent Loop

Let's derive why LangChain's agent design looks the way it does.

Starting constraint: An LM can only output text. It cannot directly call functions or use tools.

Problem: How do we give it tool-use capability?

Solution derivation:

1. Step 1: Structured output format

   • We need the LM to express "I want to use tool X with input Y" in a parseable way
   • Define a text format: Action: tool_name\nAction Input: {"arg": "value"}
   • The LLM outputs this format when it decides to use a tool

2. Step 2: Parsing layer

   • We need code to parse this format and extract tool name + arguments
   • LangChain's AgentOutputParser does this: parse(llm_output) → AgentAction

3. Step 3: Tool execution

   • We need a registry mapping tool names to actual Python functions
   • LangChain's ToolRegistry does: execute(tool_name, args) → tool_result

4. Step 4: Observation injection

   • The tool result needs to go back to the LLM so it can continue reasoning
   • We append: Observation: <tool_result> to the prompt and call the LM again

5. Step 5: Loop until done

   • Repeat steps 1-4 until the LLM outputs Final Answer: <response>

This is ReAct (Reasoning + Acting) formalized into code.

In code:

prompt = initial_prompt
conversationhistory = []
 
while True:
    # LM decides next action
    llm_output = llm(prompt + format_history(conversation_history))
    # Parse what the LLM wants
    parsed = output_parser.parse(llm_output)
    
    if isinstance(parsed, AgentFinish):
        return parsed.output  # Done!
    
    # Execute the tool
    tool_result = tool_registry.run(parsed.tool, parsed.tool_input)
    
    # Add observation to history
    conversation_history.append({
        "action": parsed,
        "observation": tool_result
    })
    
    # Loop continues...

Why this design?

• Minimal LM modifications: We don't need to fine-tune the LLM or change its architecture
• Extensibility: Adding a new tool = writing a Python function + registering it
• Debugability: We can inspect each step of the loop

Key Components

Why chains? Some tasks have a known recipe. No need for agent reasoning when you know the steps.

calculator = Tool( name="Calculator", func=lambda x: eval(x), # Unsafe! Real code uses safer parsing description="Useful for math. Input: a math expression like '2+'" )

**Why the description?** The LM uses it to decide *when* to call this tool. A bad description = agent never uses it or it wrong.

> [!definition] Memory
> A ==state store== that persists across agent turns. Types:
> - **ConversationBufferMemory**: Keep last N messages
> - **ConversationSummaryMemory**: LM summarizes old messages to save tokens
> - **VectorStoreMemory**: Embed past messages, retrieve relevant ones

**Why memory?** Without it, the agent forgets earlier parts of the conversation. With limited context windows, we need smart retrieval.

### Example: Building a Research Agent

> [!example] LangChain Agent with Tools
> **Task**: Agent that can search Wikipedia and do math.

```python
from langchain.agents import initialize_agent, Tool, AgentType
from langchain.llms import OpenAI
from langchain.utilities import WikipediaAPIWrapper

# Define tools
wikipedia = WikipediaAPIWrapper()
tools = [
    Tool(
        name="Wikipedia",
        func=wikipedia.run,
        description="Search Wikipedia. Input: search query"
    ),
    Tool(
        name="Calculator",
        func=lambda x: str(eval(x)),
        description="Do math. Input: expression like '123 * 456'"
    )
]

# Initialize agent
llm = OpenAI(temperature=0)
agent = initialize_agent(
    tools=tools,
    llm=llm,
    agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION,
    verbose=True
)

# Run
result = agent.run("What is the population of Tokyo? Multiply it by 2.")

Why each step?

1. Tool definitions: We tell the agent what it can do. The description is crucial – the LLM reads this to decide when to use each tool.

2. ZERO_SHOT_REACT_DESCRIPTION: This agent type means:

   • Zero-shot: No example demonstrations in the prompt
   • ReAct: Uses Reasoning + Acting pattern
   • Description: Decides tools based on descriptions

3. Agent execution (what happens internally):

   LM: "I need to know Tokyo's population. I'll use Wikipedia."
   → Action: Wikipedia, Input: "Tokyo population"
   → Tool executes, returns: "~14 million"
   → Observation: "14 million"
   LM: "Now I'll multiply by 2."
   → Action: Calculator, Input: "14000 * 2"
   → Tool returns: "28000000"
   → Final Answer: "28 million"
   

Why verbose=True? So you can debug the agent's reasoning. You see each Action/Observation pair.

LlamaIndex: Data Connectors and Query Engines

Key difference from LangChain: LlamaIndex is data-centric (how to get the right context), LangChain is orchestration-centric (how to chain LM calls).

Core Concept: The Index

Problem: You have 10,000 pages of documentation. An LLM can only handle ~8k tokens. How do you answer questions?

Naive solution: Chunk docs, embed chunks, search for top-k similar chunks, stuff into prompt.

Why naive?

• Chunks might lack context
• Top-k might miss relevant info spread across multiple chunks
• No reasoning about which chunks to combine

LlamaIndex solution: Multiple index types with different tradeoffs.

Constraint 1: Context window limit $C$ tokens.
Constraint 2: Document corpus $D = \{d_1, .., d_n\}$ with total size $|D| >> C$.
Goal: Given query $q$, select subset $S \subset D$ such that $|S| \leq C$ and $S$ maximizes answer quality.

Different indices solve this differently:

1. Vector Store Index (most common): $\text{similarity}(q, d_i) = \cos(\text{embed}(q), \text{embed}(d_i))$ Select top-k by similarity. Why? Semantic search is fast and works across paraphrases.

2. List Index (sequential): Iterate through all chunks, ask LLM "Is this relevant?" Why? Comprehensive but slow (N LM calls).

3. Tree Index (hierarchical): Build a tree where leaves are chunks, nodes are summaries. Start at root, traverse to relevant leaves. Why? Logarithmic search time $O(\log n)$.

4. Keyword Table Index: Extract keywords from each chunk, build mapping keyword → chunk_ids. Why? Fast exact-match retrieval.

Tradeoff table:

Example: RAG with LlamaIndex

from llama_index import VectorStoreIndex, SimpleDirectoryReader
from llama_index.llms import OpenAI
 
# Step 1: Load documents
documents = SimpleDirectoryReader("./data").load_data()
# Why? Abstracts away PDF parsing, text extraction
 
# Step 2: Build index
index = VectorStoreIndex.from_documents(documents)
# What happens here?
# 1. Chunk documents (default: 1024 tokens with 20 token overlap)
# 2. Embed each chunk (default: OpenAI text-embedding-ada-002)
# 3. Store in vector DB (default: in-memory)
 
# Step 3: Create query engine
query_engine = index.as_query_engine()
# Why? Wraps retrieval + synthesis logic
 
# Step 4: Query
response = query_engine.query("What are the main arguments?")
print(response)

What happens during query()? (Deriving the flow):

1. Embed query: $\mathbf{q} = \text{embed}(\text{"What are the main arguments?"})$

2. Retrieve top-k chunks by cosine similarity: $S = \text{top-k}(\{d_i\}, \lambda d_i: \cos(\mathbf{q}, \text{embed}(d_i)))$ Default k=2.

3. Build prompt:

   Context:
   [Chunk 1 text]
   [Chunk 2 text]
   Question: What are the main arguments?
   Answer:
   

4. LM generates answer from context.

Why this works? The chunks are semantically close to the query, so they likely contain the answer.

Agentic Retrieval: Query Transformations

Problem: User asks "Compare X and Y". Single retrieval might not fetch info about both X and Y.

Solution: LlamaIndex's query engines can transform queries:

Create sub-question engine

query_engine = SubQuestionQueryEngine.from_defaults( query_engine_tools=[ index1.as_query_engine(), # Technical docs index2.as_query_engine(), # User reviews ] )

Query: "How does product A compare to B in performance and price?"

What happens?

1. LM decomposes into sub-questions:

- "What is the performance of product A?"

- "What is the performance of product B?"

- "What is the price of product A?"

- "What is the price of product B?"

2. Routes each sub-question to appropriate index

3. Synthesizes final answer from all sub-answers

**Why decomposition?** Complex queries need multiple retrieval steps. The LLM reasons about *what information is needed*, then retrieves it.

---

## LangChain vs LlamaIndex: When to Use Which?

> [!intuition] Mental Model: Orchestration vs Data

**LangChain** = "I have an LLM and need to connect it to tools/APIs/logic"
- Chatbots with tool use (search, calculator API calls)
- Multi-step workflows (summarize → translate → email)
- Agents that reason about external actions

**LlamaIndex** = "I have lots of documents and need to query them"
- Q&A over internal docs, research papers, codebases
- Semantic search with contextstitching
- Multi-document comparison and synthesis

**In practice**: They're often used together!
```python
# LlamaIndex query engine as a LangChain tool
from langchain.tools import Tool

doc_tool = Tool(
    name="DocumentSearch",
    func=lambda q: llamaindex_query_engine.query(q),
    description="Search company documentation"
)

# Now the LangChain agent can search docs!
agent = initialize_agent([doc_tool, calculator, ...], llm)

Common Mistakes

Why it feels right: Defaults usually work, and tuning is extra work.

Why it's wrong:

• Too large chunks: Dilute relevance (irrelevant text in retrieved chunks)
• Too small chunks: Break context (miss connections across chunks)

Example: Legal documents have long clauses. 1024 tokens might split a clause mid-sentence, losing meaning.

Fix:

• Experiment with chunk sizes (512, 1024, 2048)
• Use overlap (20% of chunk size) to preserve context at boundaries
• Measure retrieval quality on your specific data

Code:

from llama_index import ServiceContext
from llama_index.text_splitter import SentenceSplitter
 
# Custom chunk size
text_splitter = SentenceSplitter(chunk_size=512, chunk_overlap=50)
service_context = ServiceContext.from_defaults(text_splitter=text_splitter)
index = VectorStoreIndex.from_documents(docs, service_context=service_context)

Why it feels right: It's literally a search tool, what more is there to say?

Why it's wrong: The LLM uses descriptions to decide when to use the tool. A vague description means:

• The agent might never use it (doesn't understand when it's useful)
• The agent might misuse it (wrong input format)

Example:

# Bad
Tool(name="Search", func=search, description tool")
 
# Good  
Tool(
    name="Search",
    func=search,
    description="Search Wikipedia for factual information about people, places, events. Input should be a specific search query like 'Barack Obama presidency' not a question."
)

Fix: Include:

1. What it does: "Search Wikipedia for factual information"
2. When to use it: "for people, places, events"
3. Input format: "specific search query like 'X' not a question"

Why it feels right: LMs are smart, they can recover from errors.

Why it's wrong:

• Tool failures (API timeouts, parsing errors) crash the agent
• The agent might loop forever retrying the same broken action
• Costs accumulate with each retry

Example: Agent calls Calculator with invalid input "what is 2+2" (words instead of expression). Without handling:

Action: Calculator
Action Input: "what is 2+2"
→ Error: invalid syntax
→ Agent retries same input
→ Error: invalid syntax
→ Loop forever (or hit max iterations)

Fix:

from langchain.agents import initialize_agent
 
agent = initialize_agent(
    tools=tools,
    llm=llm,
    max_iterations=5,  # Prevent infinite loops
    max_execution_time=60,  # Timeout after 60s
    handle_parsing_errors=True,  # Catch tool parsing errors
    early_stopping_method="generate"  # Return partial answer on timeout
)

Even better: Wrap tools with try-except:

def safe_calculator(expr):
    try:
        return str(eval(expr))
    except Exception as e:
        return f"Error: {e}. Please provide a valid math expression."

Now the agent gets feedback and can correct its input.

Advanced: Custom Agents from Scratch

Why? Frameworks are opinionated. Sometimes you need custom logic (specialized prompting, novel tool-use patterns, integration with proprietary systems).

import openai
import re
 
# Define tools
TOOLS = {
    "calculator": lambda x: eval(x),
    "search": lambda x: f"Search result for {x}"  # Stub
}
 
# ReAct prompt template
REACT_PROMPT = """
You can use these tools: {tools}
Format: 
Thought: <your reasoning>
Action: <tool_name>
Action Input: <input>
Observation: <result>
... (repeat Thought/Action/Observation as needed)
Thought: I now know the answer
Final Answer: <answer>
 
Question: {question}
"""
 
def parse_action(text):
    """Extract Action and Action Input from LM output."""
    action_match = re.search(r"Action: (\w+)", text)
    input_match = re.search(r"Action Input: (.+)", text)
    
    if action_match and input_match:
        return action_match.group(1), input_match.group(1).strip()
    return None, None
 
def agent_loop(question, max_steps=5):
    """Run the ReAct loop."""
    prompt = REACT_PROMPT.format(
        tools=", ".join(TOOLS.keys()),
        question=question
    )
    
    for step in range(max_steps):
        # Call LM
        response = openai.ChatCompletion.create(
            model="gpt-4",
            messages=[{"role": "user", "content": prompt}]
        )
        output = response.choices[0].message.content
        print(f"Step {step}:\n{output}\n")
        
        # Check if done
        if "Final Answer:" in output:
            answer = output.split("Final Answer:")[1].strip()
            return answer
        
        # Parse action
        tool_name, tool_input = parse_action(output)
        if not tool_name:
            return "Error: Could not parse action"
        
        # Execute tool
        if tool_name in TOOLS:
            result = TOOLS[tool_name](tool_input)
            prompt += f"\n{output}\nObservation: {result}\n"
        else:
            prompt += f"\n{output}\nObservation: Tool '{tool_name}' not found\n"
    
    return "Max steps reached"
 
# Test
answer = agent_loop("What is 123 * 456?")
print(f"Final: {answer}")

What we learned by building it:

1. The prompt template is the scaffolding – it teaches the LM the format
2. Parsing is britle – regex can fail if LM doesn't follow format exactly (frameworks handle this better with retry logic)
3. Tool execution is just a dict lookup + function call
4. Appending observations to the prompt creates the feedback loop

Why use frameworks then? They add:

• Robust parsing (handle variations in LLM output)
• Streaming (show progress as agent runs)
• Memory management (summarize/truncate long conversations)
• Logging and debugging
• Pre-built integrations (vector DBs, APIs)

Key Formulas and Concepts

where $a_t = \text{LM}(\text{prompt} + \text{history}_t)$.

Translation: The agent's state (conversation history) updates based on tool results until it produces a final answer.

where $\text{retrieve}(D, q) = \text{top-k}(D, \text{similarity}(q, \cdot))$.

Interpretation: Instead of generating from the LLM's parametric knowledge alone, we condition on retrieved documents.

Connections

• 6.2.01-Definition-and-characteristics-of-AI-agents – Frameworks implement the perceive-reason-act cycle
• 6.2.04-ReAct-(Reasoning-and-Acting) – LangChain's agent types implement ReAct pattern
• 6.2.05-Tool-calling-and-function-calling-APIs – Frameworks wrap function calling APIs for easy integration
• 5.3.02-Vector-databases – LlamaIndex uses vector DBs for semantic search
• 5.3.03-Retrieval-Augmented-Generation-(RAG) – LlamaIndex is essentially a RAG framework

LangChain and LlamaIndex are like giving your friend superpowers:

LangChain = Giving your friend a toolbox. Now they can:

• Use Google (search tool)
• Use a calculator (math tool)
• Remember earlier parts of your conversation (memory)

They figure out which tool to use when by reading descriptions. If you ask "What's 25 times 30?", they think "I need the calculator tool" and use it.

LlamaIndex = Giving your friend a magical library. You have100 textbooks but they can only read 5 pages at a time (attention span!). LlamaIndex:

• Makes a map of all the books (index)
• When you ask a question, finds the 5 most useful pages (retrieval)
• Reads just those pages to answer (synthesis)

Together, they turn a smart-but-limited friend into a superhero assistant who can search, calculate, remember, and read huge libraries!

LLAMAIndex = Load Lots And Map All (data-focused)

Flashcards

#flashcards/ai-ml

What is an agentic framework? :: A library that provides abstractions (chains, agents, tools, memory, retrievers) for building AI agents without reimplementing common patterns like prompt templating, tool parsing, and conversation management.

What is query decomposition in LlamaIndex? :: Breaking a complex query into sub-questions, retrieving answers for each, then synthesizing a final answer. Useful for comparison queries ("Compare X and Y") or multi-part questions.

Concept Map

Hinglish (regional understanding)