6.2.1 · D5AI Agents & Tool Use
Question bank — Agent architectures and reasoning loops
Before we start, some plain-word anchors so nothing below uses an undefined term:
The two diagrams below anchor the whole bank — glance at them first, then take the traps.
True or false — justify
The agent's reasoning loop always terminates on its own once it has enough information
False. Nothing stops the loop except an explicit terminate check (goal met OR max-steps hit); a buggy agent with no step cap can loop forever re-observing the same state.
In the ReAct pattern, the "Thought:" text is thrown away after each cycle to save space
False. The whole point is that every prior Thought, Action, and Observation is appended to context and re-read next cycle — that history is how the model chains reasoning.
Chain-of-Thought and ReAct are the same pattern with different names
False. CoT reasons in steps but calls no tools; ReAct interleaves reasoning with real tool calls and reads back their observations. CoT can't fetch live data.
A Simple Reflex Agent could correctly answer "is Paris bigger than London?" if given a good enough percept
False. That question needs two facts gathered across steps; a reflex agent maps one current percept straight to one action with no memory, so it cannot chain sub-goals.
Short-term memory and the context window are two names for the same thing
They overlap but play different roles. Short-term (episodic) memory is the content — the running conversation that the agent chooses to keep and can summarize, reorder, or evict. The context window is the container — a fixed token budget with no memory of its own. One is what you remember; the other is how much desk space you have to lay it on.
Adding more tools always makes an agent more capable
False. Each tool's schema is text that eats context tokens, and more tools add more ways to mis-parse or pick the wrong action; past a point this lowers accuracy by crowding the reasoning.
Reflexion improves performance by changing the model's weights after a failure
False. Reflexion writes a self-critique into context and retries; the model is unchanged. Weight-changing from feedback is reinforcement learning, a different mechanism.
Tree-of-Thoughts is strictly better than ReAct, so you should always use it
False. ToT explores branches per step and scores them — powerful but expensive. For a single-path tool task, ReAct is cheaper and just as correct; ToT's cost only pays off when multiple paths must be compared.
Long-term memory retrieval is guaranteed to return the fact you need if it was stored
False. Retrieval is top-k similarity (see Retrieval-Augmented Generation) — an approximate match. A relevant fact can be missed if its embedding isn't among the top-k nearest to the query.
Spot the error
"The Observation is produced by the LM during the Reason step." — what's wrong?
The LM produces the Thought and Action; the Observation comes back from executing the action in the environment. Mixing these lets the agent hallucinate tool results.
"We can drop old messages whenever context fills up — the agent won't notice." — what's wrong?
Dropping raw messages can delete facts the later steps still depend on. The safe move is to summarize old context so the information survives even if the exact wording doesn't.
"In we should append only the Observation." — what's wrong?
Appending only the observation loses the reasoning trail. Without seeing its own past Thoughts and Actions, the model can repeat a failed action or lose track of which sub-goal it was on.
"A Goal-Based Agent picks the action with the highest immediate reward." — what's wrong?
That describes reward/utility maximization. A goal-based agent picks actions that advance toward a defined goal state, which may mean a locally worse move that gets closer to the target.
"Because ReAct forces a 'Thought:' before each action, the agent can never take a wrong action." — what's wrong?
The prefix improves reasoning quality but guarantees nothing; the model can still reason wrongly or parse an invalid action. Validation in the action-execution module, not the prompt, is what catches bad calls.
"Working memory and long-term memory are interchangeable — both persist across episodes." — what's wrong?
Working memory is a temporary scratchpad for the current multi-step task and is discarded after; only long-term (semantic) memory persists across episodes.
"If a tool call times out, the loop is broken and the agent must restart." — what's wrong?
The action-execution module handles errors — it can retry, catch the exception, and feed the failure back as an observation. A failure is just another observation the loop can react to.
Why questions
Why does ReAct append the observation to context instead of feeding it directly into the next thought as a separate variable?
Because the LM's only channel is text-in / text-out; there is no side variable it can read. Everything it "knows" this cycle must physically sit inside the context window as text.
Why can't a Simple Reflex Agent do multi-step research even though it can call tools?
Its action depends only on the current percept with no memory. After a tool returns, it has no record of why it called that tool, so it cannot build on the result toward a larger goal.
Why does the terminate check need both "goal met" and "max steps" conditions rather than just "goal met"?
"Goal met" alone never fires if the agent is stuck in a wrong loop or the goal is unreachable. The max-steps cap is a safety brake so a confused agent halts instead of running (and spending) forever.
How is the max-steps cap actually chosen — what pushes it up or down?
It is a design trade-off, not a law. You raise it for tasks that genuinely need many tool calls (deep research) and lower it to bound cost, latency, and runaway risk. It is set from the expected number of honest steps plus a margin — high enough that a correct agent never hits it, low enough that a stuck one stops soon.
Why does Tree-of-Thoughts need a scoring function while ReAct does not?
ReAct follows one path, so there is nothing to compare. ToT generates several candidate thoughts per step and must rank them to decide which branch to expand — that ranking is exactly the score.
Why is long-term memory stored as vectors instead of just keeping the full text of every fact?
Vectors let you find facts by meaning via similarity search, so a query worded differently still retrieves the right fact — and a compact vector index scales far better than re-reading all raw text each time.
Why does Reflexion place its self-critique in the context rather than silently retrying the same action?
Retrying the identical action gives the identical failure. The written critique changes the input the model reads next, so its next action can differ and actually fix the bug.
Why does interleaving reason and act (ReAct) beat "reason fully, then act on the plan"?
A plan made before seeing any tool results is guesswork about the world. Interleaving lets each real observation correct the next step, so the agent adapts instead of committing to a stale plan.
Edge cases
What happens in the loop when the very first observation already answers the whole question?
The reason step should output a
respond(...) action and the terminate check fires immediately — a valid single-cycle run. A loop that must iterate more than once is over-engineered.An agent's tool returns an empty result (zero rows, no matches). Is that a failure?
No — an empty result is still a legitimate observation. The agent should reason about it (reformulate the query, try another tool, or report "not found"), not crash or treat it as an error.
The context window fills up exactly mid-task. What's the degenerate behavior if nothing is summarized?
The oldest messages fall off the "desk edge," so the agent loses its early sub-goals and can start looping or answering the wrong sub-question. This is why summarization/eviction is a required part of memory management, not optional polish.
Two tool calls return contradictory facts (e.g., two different populations). What should the loop do?
It should reason about the conflict as an observation — prefer the more reliable/recent source, or issue a tie-breaking query — never silently pick one at random. The conflict is information, not noise.
Max-steps is reached but the goal is not met. What is the correct output?
The agent should terminate and report partial progress or failure honestly, not fabricate a confident answer. Hitting the cap is a signal to surface uncertainty, not to hide it.
A Reflexion agent succeeds on attempt 1. Does the reflexion mechanism still run?
No reflexion is needed on success — the critique step is triggered by a failed observation. On a clean success the loop just terminates.
The same fact is stored to long-term memory a thousand times. What's the boundary problem?
Near-duplicate vectors flood the top-k results, crowding out other relevant facts and wasting retrieval slots. Deduplication or merging is needed so one over-repeated fact doesn't dominate recall.
An agent in a multi-agent setup receives another agent's message as a percept — is that different from a tool observation?
Structurally no: it enters through the same observe step and lands in context as text. The reasoning loop treats a peer's message just like any other observation to reason over.
Recall Quick self-test before you leave
The loop can loop forever without what safeguard? ::: A max-steps terminate check. ReAct's memory of its own reasoning lives where? ::: Inside the context window, as appended text. The difference between CoT and ReAct in one word? ::: Tools (ReAct calls them, CoT does not). Reflexion changes the model's weights? ::: No — it changes the context via a written self-critique.