Foundations — Agentic frameworks (LangChain, LlamaIndex)

Before you can understand why the parent topic designs its agent loop the way it does, every word in that design has to mean something concrete. This page names each symbol and picture from zero.

1. The atom: a Language Model (LM)

The picture: a box with one arrow going in (a string of characters) and one arrow coming out (another string).

Look at the box in the figure. The only thing crossing its boundary is text. This single fact is the seed the whole topic grows from: if the model can only speak text, then every clever thing — using a calculator, searching Wikipedia — must be smuggled through text.

The LL in LLM just means Large — the model has billions of internal numbers (parameters) tuned on huge text. For this page, LM and LLM mean the same box.

2. Tokens and the context window $C$

The picture: a fixed-width bucket. You can pour text in, but once it hits the brim ($C$ tokens), no more fits.

Why the topic needs this symbol. The parent's LlamaIndex section opens with "10,000 pages of docs, model handles ~8k tokens." That $8k$ is $C$. The entire idea of indexing and retrieval exists only because $|D|\gg C$ — the documents are far bigger than the bucket. Hold onto this inequality; §7 defines $D$.

3. The prompt

The picture: the prompt is the stuff inside the bucket of §2. It must be $\le C$ tokens.

In the parent's pseudocode, prompt = initial_prompt is the first thing poured in, and each loop appends more. The prompt is the only memory the model has during one call — it remembers nothing else.

4. Structured output: Action, Action Input, Observation

Since the model can only write text (§1), how does it say "run the calculator on 2+2"? By writing text in an agreed format that some code outside knows how to read.

The picture: a conversation transcript that grows line by line — the model writes Action lines, the framework writes Observation lines back.

Follow the arrows in the figure: model writes Action (orange) → framework runs the real tool → framework writes Observation (teal) → back into the prompt → model reads and writes the next Action. That circle is ReAct — Reasoning (the model thinking in text) plus Acting (tools running).

5. Parsing — turning the model's text into a decision

The picture: a sieve. Raw text goes in; out comes a small labelled slip: {tool: "Calculator", input: "2+2"} or {finish: "28 million"}.

Why the topic needs it: the model writes free-form text; Python needs structured data to act on. LangChain's AgentOutputParser does exactly parse(llm_output) → AgentAction or AgentFinish. Without parsing, the "Action: Calculator" line is just prose no computer can act on.

6. Tool and Tool Registry

The picture: a rack of labelled drawers. The Action line names a drawer; the registry opens that drawer and runs what's inside; the return value becomes the Observation.

Why the description matters (this is the subtle part): the model chooses a tool by reading its description in the prompt. A vague description = the model never picks the tool, or picks it wrongly. See tool calling for how modern APIs formalise this.

7. The corpus $D$ and the retrieval problem

Now the data side, which drives LlamaIndex.

The picture: a huge library ($D$), a question $q$, and a librarian who fetches only the handful of pages ($S$) that fit on your desk ($C$).

Why "relevant"? How does the librarian know which chunks match $q$? By turning text into numbers (embeddings) and measuring closeness — the job of vector databases. That machinery is a prerequisite, not built here; just know the arrow "$q\to$ closest chunks" exists.

8. Chain vs Agent — the last distinction

Prerequisite map

Read top-to-bottom: the text-only model plus a bounded prompt force the structured-output-and-parse trick, which builds the tool loop; separately, a corpus bigger than the window forces retrieval, which fills the prompt. Both streams pour into the frameworks.

Equipment checklist

Cover the right side; can you answer each before revealing?