6.2.7 · D5AI Agents & Tool Use
Question bank — Agentic frameworks (LangChain, LlamaIndex)
This bank targets the misconceptions that agentic frameworks invite. Before you read an answer, cover it and argue your own case. Every answer gives a reason, not a verdict. See the parent Agentic frameworks (LangChain, LlamaIndex) for the mechanics being tested here.
The two pictures this bank keeps testing
Before the questions, pin down the two diagrams that most traps distort. Keep them in view as you work.
Picture 1 — the agent loop. Text goes into the model, comes out as text, gets parsed into either a tool request or a final answer; a tool request runs real code, whose result is fed back as an observation, and the loop repeats. Notice the model never touches your functions directly — the framework does.
Picture 2 — why an index exists. The whole document pile is huge; the model's window is a small box. Retrieval is the act of choosing which few chunks fit inside that box. When the pile already fits, there is nothing to choose.
True or false — justify
A framework gives the language model new abilities the raw model does not have.
False. The agent loop only routes text: the LM still just predicts tokens. The framework parses those tokens and runs real code, so the "new ability" lives in the Python around the model, not inside it.
LangChain agents require a fine-tuned or specially-trained model to use tools.
False. The whole point of
ZERO_SHOT_REACT_DESCRIPTION is that tool use is prompted, not trained — the model reads tool descriptions at inference time and emits Action: lines. No weights change.A Chain and an Agent are the same thing with different names.
False. A chain is a fixed sequence you wrote (deterministic steps); an agent chooses its next step at runtime by reasoning. Chain = known recipe, agent = decides the recipe as it goes.
LlamaIndex and LangChain are competitors that do the same job.
Mostly false. LlamaIndex is data-centric (get the right context via indexing/retrieval); LangChain is orchestration-centric (chain LM calls and tools). They overlap on RAG but are routinely used together.
Setting temperature=0 guarantees the agent always picks the same tool for the same task.
Not guaranteed.
temperature=0 makes decoding near-deterministic for identical prompts, but the prompt itself changes as observations accumulate, and tool results (a live Wikipedia call) can differ, steering the agent down a different branch.A Tool's description is documentation for humans; the agent ignores it.
False. The description is fed into the prompt and is the primary signal the LM uses to decide when to call the tool. A vague description is the most common reason an agent never uses a tool it should.
Memory in an agent is the same as the model's context window.
False. The context window is a hard token limit () inside the model; memory is a framework-side store that decides what to place into that window each turn (buffer, summary, or vector-retrieved).
If the corpus fits in the context window, you still need a vector index.
False. If total size , you can stuff everything and skip retrieval entirely. Indexing only earns its keep when .
Spot the error
"I registered a tool but wrote no description, so the agent will figure out its purpose from the function name."
The error: the function body and name are not in the prompt — only the declared
name and description are. With an empty description the LM has almost no basis to select it, so it typically never fires."My agent loops forever, so the framework must be buggy."
More likely the LM never emits the
Final Answer: / AgentFinish token, so the loop never breaks. This is a prompt/parsing problem, not a bug — production loops add a max_iterations cap precisely for this."The parser failed on the LM output, so I'll just retry the exact same call."
A blind retry with an unchanged prompt at low temperature reproduces the same malformed output. The fix is to feed the parse error back as an observation so the model can correct its format.
"Top-k retrieval with k=3 always gives the agent enough context."
Relevant facts may be spread across more than 3 chunks, or a single chunk may lack surrounding context. A fixed small k silently drops information the answer depends on.
"func=lambda x: eval(x) is a fine calculator tool for production."
eval executes arbitrary Python from LM-generated (and possibly user-influenced) text — a code-execution hole. The parent flags this as unsafe; real tools use a sandboxed math parser."ConversationSummaryMemory stores everything, so nothing is ever lost."
Summarization is lossy by design — it compresses old turns to save tokens. Exact wording and minor details can vanish, which matters if a later question depends on them.
"Each Observation is the model's own reasoning about the tool."
No.
Observation: is the raw tool result injected by the framework. The model's reasoning is the Thought: text; conflating them makes you misread ReAct traces. See ReAct.Why questions
Why does the agent output a text format like Action: Calculator instead of directly calling a Python function?
Because the LM can only emit text — it has no runtime handle on your functions. The text is a serialized request that the framework's parser turns into an actual call. See tool/function-calling APIs.
Why append the tool result as an Observation and call the LM again instead of just returning it?
Because the raw result usually isn't the final answer — the model must interpret it, possibly chain another tool, or phrase a response. Re-prompting lets reasoning continue with the new fact in hand.
Why do vector-store indices need an embedding model at all?
To turn text into vectors so "similar meaning" becomes "nearby in space", enabling nearest-neighbour search in a vector database. Without embeddings you can only match exact keywords, missing paraphrases.
Why prefer a fixed Chain over an Agent when the steps are known?
Agent reasoning spends extra LM calls (cost + latency) deciding steps you already know. A chain hard-codes the recipe, making it cheaper, faster, and far more predictable.
Why is verbose=True a debugging lifeline rather than clutter?
It prints every
Thought / Action / Observation triple, so when the agent misbehaves you can see which step chose the wrong tool or misread a result — the loop is otherwise opaque.Edge cases
What happens when an agent is given zero tools?
With an empty registry it can never emit a valid
Action, so it collapses to a plain single-shot LM answer — effectively a chain of length one with no retrieval or computation.What if two tools have nearly identical descriptions?
The LM cannot reliably distinguish them and may pick either at random or alternate, wasting steps. Tool descriptions must be disjoint in the situations they claim to cover.
What if the tool returns an empty string or an error message?
That text still becomes the
Observation. A well-prompted agent can react ("the search returned nothing, I'll rephrase"), but a naive one may treat the error string as a valid fact and hallucinate on top of it.What if the corpus is a single tiny document ()?
Indexing and retrieval are pure overhead — top-k just returns that one document. Retrieval frameworks only pay off past the point where content exceeds the context window ().
What is the limiting behaviour as the number of agent iterations grows without a cap?
Cost and latency grow unboundedly and the accumulated observation history eventually overflows the context window , degrading or breaking the run. This is why
max_iterations and memory summarization exist as guardrails.Recall Fastest self-test
Which framework is "data-centric" and which is "orchestration-centric"? ::: LlamaIndex = data-centric (right context); LangChain = orchestration-centric (chain the calls).
Where does a tool's new capability actually live? ::: In the Python code the framework runs, not inside the language model.
What single token ends the agent loop? ::: The Final Answer: line, parsed as AgentFinish.
What must be true for retrieval to be worth it? ::: The corpus must exceed the window, .