Code-generation agents

6.2.8 · AI-ML › AI Agents & Tool Use

What Makes a Code-Generation Agent

The Agent Loop Architecture

Operational cycle kuch is tarah follow hoti hai:

State_{t} Perception Context_{t} Planning Action_{t} Execution State_{t + 1}

First principles se derivation:

State representation: Step $t$ par, agent ke paas hota hai:
- User requirements $R$
- Partial code $C_{t}$
- Execution history $H_{t} = {(c_{i}, o_{i}, e_{i})}_{i < t}$ (code, output, error)
Context construction: Prompt $P_{t}$ banao by concatenating: $P_{t} = [R, C_{t}, H_{t}, system instructions]$ Yeh encode karta hai "kya try kiya gaya aur kya hua."
Action generation: LM next action $A_{t} \sim LLM (P_{t})$ produce karta hai, jo ho sakta hai:
- Naya code snippet generate karna
- Existing code modify karna
- Clarification request karna
- Task complete declare karna
Environment interaction: $A_{t}$ ko sandbox mein execute karo: $o_{t}, e_{t} = Execute (A_{t})$ jahan $o_{t}$ stdout/output hai, $e_{t}$ error trace hai
State update: History mein append karo: $H_{t + 1} = H_{t} \cup {(A_{t}, o_{t}, e_{t})}$ Code state update karo: $C_{t + 1} = Apply (C_{t}, A_{t})$

Iterative loops kyun? Code generation mein high failure rates hoti hain (~30-70% first-try success). Iteration essential hai kyunki:

Syntax errors ko immediate fixes chahiye
Logic bugs ke liye test-driven debugging zaruri hai
Edge cases sirf execution ke dauran samne aate hain

Key Mechanisms in Code Agents

1. Tool Use & Execution Sandboxing

Agents in chezon ke saath interact karte hain:

Python/JavaScript interpreters: Generated code run karte hain
Terminal/shell: Commands execute karte hain, packages install karte hain
File system: Files read/write karte hain, directories navigate karte hain
API calls: External services, databases query karte hain

Worked example Execution Flow

Task: "Calculate mean of [1,2,3,4,5]" Iteration 1:

Agent generate karta hai:

numbers = [1,2,3,4,5]
mean = sum(numbers) / length(numbers)
print(mean)

Execution error: NameError: name 'length' is not defined
Yeh step kyun? Sandbox Python-specific error pakadta hai (hona chahiye tha len) Iteration 2:

Agent error dekhta hai, generate karta hai:

numbers = [1,2,3,4,5]
mean = sum(numbers) / len(numbers)
print(mean)

Output: 3.0 ✓
Yeh step kyun? Error feedback ne correction guide kiya—yahi agent loop action mein hai

2. Test-Driven Development (TDD) Integration

Agents pehle tests generate karte hain, phir unhe pass karne ke liye code:

Spec \to Tests \to Implementation \to Validation

Agents ke liye TDD kyun?

Ambiguity reduce karta hai: Tests requirements ko formalize karte hain
Self-verification enable karta hai: Agent apna kaam khud check kar sakta hai
Regression prevent karta hai: Nayi iterations purani functionality nahi todti

Worked example TDD Agent Workflow

Task: "Write function to check if number is prime" Step 1 - Tests generate karo (Kyun? Edge cases clarify karne ke liye):

assert is_prime(2) == True   # smallest prime
assert is_prime(1) == False  # edge case
assert is_prime(17) == True  # mid-range
assert is_prime(18) == False # composite

Step 2 - Implementation generate karo (Kyun? Specs satisfy karne ke liye):

def is_prime(n):
    if n < 2:
        return False
    for i in range(2, int(n**0.5) + 1):
        if n % i == 0:
            return False
    return True

n**0.5 kyun? Sirf $n$ tak ke divisors check karne ki zarurat hai kyunki agar $n = a \times b$ aur $a > n$ , to $b < n$ (already check ho gaya).

Step 3 - Tests run karo: Sab pass ✓

Real projects kai files mein phele hote hain. Agents ko chahiye:

Search capabilities: grep/AST search se relevant code dhundhna
Dependency tracking: Imports, function calls samajhna
Selective editing: Specific functions modify karna bina doosron ko tode

Codebases mein information retrieval:

Query $q$ (e.g., "find login function") di ho, files ko relevance se rank karo: $score (f, q) = BM25 (f, q) + α \cdot embedding_sim (e_{f}, e_{q})$

jahan:

BM25 = keyword-based retrieval score
$e_{f}, e_{q}$ = code/query embeddings
$α$ = balance parameter

Hybrid kyun? Keywords exact matches pakadti hain (e.g., function names), embeddings semantic similarity pakadti hain (e.g., "authenticate" ≈ "login").

4. Error Recovery Strategies

Jab code fail hota hai, agents yeh employ karte hain:

Error message parsing: Stack traces, line numbers extract karna
Root cause analysis: Identify karna ki kis change ne bug introduce kiya
Hypothesis generation: "Kya galat gaya hoga?"
Targeted fixes: Issue resolve karne ke liye minimal edits

Performance Metrics

Code agents ka evaluation:

Benchmark datasets:

HumanEval: 164 hand-written Python programming problems
MBPP: 974 crowd-sourced Python tasks
APPS: Competition-level coding problems

State-of-the-art (2026):

GPT-4 with iteration: ~85% pass@5 on HumanEval
Claude Sonnet 3.5: ~92% pass@5
Specialized code models (DeepSeek Coder): ~88% pass@5

Advanced: Multi-agent Code Systems

Complex projects agent teams use karte hain:

Architect agent: System structure design karna, libraries choose karna
Implementation agents: Specific modules ke liye code likhna
Testing agent: Test cases generate karna, QA run karna
Review agent: Code quality check karna, improvements suggest karna

Communication shared workspace (filesystem) aur message passing (inter-agent prompts) ke zariye hoti hai.

Multi-agent kyun? Specialization quality improve karta hai (har agent apne role ke liye optimized hai) aur parallelism enable karta hai (multiple files ek saath edit hoti hain).

Limitations & Future Directions

Current challenges:

Context limits: Bade codebases token windows exceed karte hain (RAG/summarization ki zarurat)
State management: Agents lambe sessions mein track kho dete hain
Correctness guarantees: Koi formal verification nahi, sirf empirical testing
Cost: Task per multiple LM calls expensive ho sakti hain

Emerging solutions:

Hierarchical planning: Tasks ko subtasks mein todna taaki per-step complexity kam ho
Formal verification integration: Correctness prove karne ke liye theorem provers (Coq, Lean) use karna
Learned world models: Execution ke bina code behavior predict karna (faster iteration)

Recall 12 Saal ke Bacche ko Explain Karo

Imagine karo tum ek bahut smart robot se Lego castle banane ko kehte ho. Ek normal robot bas bricks ek pile mein dump kar deta hai (yahi code completion hai). Lekin ek code-generation agent alag hai:

Yeh tumhari instructions padhta hai ("ise tall banao drawbridge ke saath")
Yeh banana shuru karta hai, ek section at a time
Jab ek tower gir jaata hai (bug!), yeh dekhta hai ki kyun gira
Yeh ek baar phir try karta hai stronger base ke saath (debugging)
Jab tak castle perfectly nahi khada ho jaata tab tak karta rehta hai

Magic yeh hai ki robot apna kaam khud test kar sakta hai aur mistakes fix kar sakta hai—bilkul waise jaise tum Legos banate waqt karte ho! Yeh ek try ke baad give up nahi karta.

Connections

Large Language Models: Wo foundation model jo code agents ko power karta hai
Tool use in AI systems: Code execution ek specialized tool hai
Prompt engineering: Agent behavior ke liye system prompts craft karna
Retrieval-Augmented Generation: Agents bade codebases kaise search karte hain
AI safety and alignment: Agents ko harmful code execute karne se rokna
Automated testing: Agents apne outputs kaise verify karte hain
Program synthesis: Code generation ka formal methods wala view

#flashcards/ai-ml

Code-generation agents ko simple code completion models se alag karne wali chaar core capabilities kya hain?

1) Task understanding (requirements parse karna), 2) Code synthesis (correct code generate karna), 3) Execution & verification (sandbox mein run karna), 4) Iterative refinement (feedback ke basis par debug karna aur adapt karna)

Code-generation agents one-shot generation ki jagah iterative loop kyun use karte hain?

Code generation mein high first-try failure rates hoti hain (30-70%). Iteration agent ko errors dekhne, issues debug karne, aur working solutions tak converge karne deta hai. Agar single-attempt success p=0.4 hai, to 5 iterations mein 1-(1-p)^5 se 92% success milti hai.

k independent attempts ke liye success probability ka formula derive karo jab per-attempt success rate p ho.

Ek attempt mein failure = (1-p). Saare k attempts mein failure = (1-p)^k (independence). Kam se kam ek mein success = complement = 1 - (1-p)^k.

State transitions ke liye agent loop equation kya hai?

State_t → Perception → Context_t → Planning → Action_t → Execution → State_{t+1}, jahan har state mein requirements R, code C_t, aur execution history H_t hoti hai.

Code agent execution mein history H_t mein kya hota hai?

Tuples {(c_i, o_i, e_i)} ka ek sequence jo pichle code snippets, unke outputs, aur error traces represent karta hai. Yeh agli iteration ke liye feedback provide karta hai.

Code agents test-driven development (TDD) kyun use karte hain?

1) Tests ambiguous requirements formalize karte hain, 2) Self-verification enable karte hain (agent apna kaam khud check karta hai), 3) Regression prevent karte hain (naya code purani functionality nahi todta).

Code generation ke liye pass@k metric kya hai?

Pass@k = E[1 - C(n-c,k)/C(n,k)], yeh probability hai ki n total samples mein se c correct wale k generated samples mein se kam se kam ek correct ho. Multiple attempts ke saath success rate measure karta hai.

Codebase search mein hybrid BM25 + embedding similarity kyun use karte hain?

BM25 exact keyword matches pakadti hai (e.g., function names). Embeddings semantic similarity pakadti hain (e.g., "authenticate" ≈ "login"). Hybrid score: BM25(f,q) + α·sim(e_f, e_q).

"IndexError: list index out of range" dekhne par agents kya common error recovery mistake karte hain?

Error message par overfitting karte hue blindly har jagah bounds checks add kar dena, bajay yeh trace karne ke ki index invalid kyun hai (aksar incorrect loop bounds ya pehle ki logic mein off-by-one errors hoti hain).

Complex projects ke liye multi-agent code systems single agents se zyada effective kyun hain?

Specialization quality improve karti hai (har agent architecture/implementation/testing/review ke liye optimized hai) aur parallelism enable karta hai (multiple files ek saath edit hoti hain). Agents shared workspace aur message passing ke zariye communicate karte hain.