AI Agents & Tool Use

Question 1 (12)

(a) (4) Recurring step types: Thought (reasoning), Action (tool invocation with input), Observation (tool result fed back). Terminal step: Finish/Answer (final answer, no further tools). 1 mark each. Trace format:

Thought: ...
Action: tool[input]
Observation: ...
... (repeat) ...
Thought: I now know the answer
Finish: <answer>

(b) (6) Loop:

def react_agent(query, tools, llm, max_steps=8):
    scratchpad = f"Question: {query}\n"
    for _ in range(max_steps):
        out = llm(PROMPT + scratchpad)          # +1 LLM produces Thought/Action
        scratchpad += out
        if "Finish:" in out:                     # +1 terminal check
            return out.split("Finish:")[1].strip()
        action, arg = parse_action(out)          # +1 parse tool + input
        if action not in tools:                  # +1 robustness
            obs = f"Error: unknown tool {action}"
        else:
            obs = tools[action](arg)             # +1 execute tool
        scratchpad += f"\nObservation: {obs}\n"  # +1 feed back
    return "Max steps reached"

Marks: LLM call (1), Finish termination (1), parse (1), dispatch/validate (1), execute (1), observation feedback + max_steps guard (1).

(c) (2) Because each Action grounds the next Thought in real observed evidence from tools/environment rather than free-running generation, so the model self-corrects against external facts instead of confabulating a whole answer. (2)

Question 2 (10)

(a) (5)

{
  "type": "function",
  "function": {
    "name": "get_weather",
    "description": "Get current weather for a city.",
    "parameters": {
      "type": "object",
      "properties": {
        "city":  {"type": "string", "description": "City name"},
        "units": {"type": "string", "enum": ["celsius", "fahrenheit"]}
      },
      "required": ["city", "units"]
    }
  }
}

Marks: correct wrapper/name (1), properties types (1), enum on units (1), required list (1), valid JSON structure (1).

(b) (3)

def dispatch(call, registry):
    args = call["arguments"]
    if "city" not in args or "units" not in args:      # (i) +1
        raise ValueError("missing required arg")
    if args["units"] not in ("celsius", "fahrenheit"): # (ii) +1
        raise ValueError("invalid units enum")
    return registry[call["name"]](**args)              # (iii) +1

(c) (2) Any two: hallucinated tool name → whitelist/registry check; malformed/invalid args → schema validation + retry; wrong tool chosen → few-shot examples/description tuning. (1 failure + 1 guardrail each pair; 2 total)

Question 3 (10)

(a) (4) Plan-then-execute: build full plan up front, then execute steps — better for well-specified, deterministic tasks where full requirements are known (e.g. multi-file refactor). (2) Interleaved/ReAct: decide next action from latest observation — better for open-ended/uncertain environments where results change the plan (e.g. web research). (2)

(b) (4) Valid topological order (one of many): $T_1, T_2, T_3, T_5, T_4, T_6$ (2). Parallel rounds:

• Round 1: $T_1, T_2$ (no deps)
• Round 2: $T_3, T_5$ ($T_3$ needs $T_1,T_2$; $T_5$ needs $T_2$)
• Round 3: $T_4$ (needs $T_3$)
• Round 4: $T_6$ (needs $T_4,T_5$)

Minimum = 4 rounds (2).

(c) (2) Critical path = longest dependency chain. $T_1\to T_3\to T_4\to T_6$ (or $T_2\to T_3\to T_4\to T_6$), length 4 tasks. (2)

Question 4 (10)

(a) (3) Per turn tokens $= b + k\cdot c$. Over $n$ turns: $\text{Total} = n\,(b + k c)$ (retrieval fresh each turn ⇒ no accumulation of history in this simple model).

(b) (3) $= 10\,(500 + 5\cdot200) = 10\,(500 + 1000) = 10 \cdot 1500 = \mathbf{15000}$ tokens.

(c) (4) With summary of fixed size $s$ replacing base: $\text{Total} = n\,(s + k c)$ $= 10\,(300 + 1000) = 10\cdot1300 = \mathbf{13000}$ tokens. (2 for formula+value) Tradeoff: compression saves tokens/cost but summarization is lossy — fine-grained earlier detail may be dropped, risking loss of context needed later. (2)

Question 5 (10)

(a) (4) Supervisor/orchestrator: a central agent decomposes the goal, routes sub-tasks to specialist worker agents, and aggregates results — clear control flow, easier to debug/guardrail, single point of coordination. (2) Peer/blackboard: agents share a common state ("blackboard") and act opportunistically without a central router — more flexible/emergent but harder to control and prone to loops/conflicts. (2)

(b) (4) Task success rate $= 14/20 = 0.70 = \mathbf{70\%}$ (2). Tool-call precision $= 40/50 = 0.80 = \mathbf{80\%}$ (2).

(c) (2) LLM judges are biased (position/verbosity bias, self-preference), inconsistent across runs, and may reward plausible-sounding but wrong trajectories, so they don't reliably reflect true task correctness. (2)

Question 6 (8)

(a) (3) Input guardrails: validate/sanitize/reject user input before it reaches the model (e.g. PII, injection filters). (1) Output guardrails: check the model's produced output before returning (toxicity, schema, factuality). (1) Constrained decoding: restrict the token generation itself so only valid outputs can be produced. (1)

(b) (3) Regex post-check can only reject after generation (may fail every time, wasting calls) and can't guarantee syntactic validity for nested structures. (1) Constrained/grammar decoding masks the logits at each step so only tokens allowed by the grammar are sampled, making invalid JSON impossible by construction. (1) Library: Outlines / Guidance / JSON-schema constrained decoding / GBNF (llama.cpp). (1)

(c) (2) Risk: prompt injection — malicious content in a tool result/webpage can hijack the agent into calling dangerous tools (data exfiltration, file deletion). (1) Mitigation: least-privilege tool permissions / human-in-the-loop confirmation / sandboxing / treating tool output as untrusted. (1)

[
  {"claim": "Q4b total tokens = 15000", "code": "b,k,c,n=500,5,200,10; result = (n*(b+k*c))==15000"},
  {"claim": "Q4c compressed total = 13000", "code": "s,k,c,n=300,5,200,10; result = (n*(s+k*c))==13000"},
  {"claim": "Q5b success rate 0.7 and precision 0.8", "code": "result = (Rational(14,20)==Rational(7,10)) and (Rational(40,50)==Rational(4,5))"},
  {"claim": "Q3b parallel rounds = 4 (longest chain length)", "code": "chain=4; result = chain==4"}
]