Causal language modeling objective — Hinglish notes

4.2.6 · AI-ML › Tokenization & Language Modeling

Yeh Kaunsi Problem Solve Karta Hai?

Language mein structure hota hai: "love" word "I" ke baad zyada likely hai, "quantum" ke baad nahi. Causal LM sequences par yeh conditional probability distribution seekhta hai:

$P (sentence) = P (w_{1}) \cdot P (w_{2} ∣ w_{1}) \cdot P (w_{3} ∣ w_{1}, w_{2}) \dots P (w_{n} ∣ w_{1} \dots w_{n - 1})$

Is tarah factorize kyun karte hain? Chain rule of probability ke according, koi bhi joint distribution conditionals ke product ke roop mein likhi ja sakti hai. Causal constraint ka matlab hai ki har $P (w_{t} ∣ w_{< t})$ ek forward-looking prediction hai, jo bilkul waise match karta hai jaise hum naturally language produce karte hain.

Figure — Causal language modeling objective

First Principles Se Derivation

Step 1: Maximum Likelihood Se Shuru Karo

Hum chahte hain ek model jo real text ko high probability assign kare. Sequences ke ek dataset $D$ ke liye, maximize karo:

$\prod_{x \in D} P_{θ} (x)$

Product kyun? Training examples ke beech independence assume karte hain.

Step 2: Har Sequence Par Chain Rule Apply Karo

Ek sequence $x = (x_{1}, \dots, x_{T})$ ke liye:

$P_{θ} (x) = P_{θ} (x_{1}, x_{2}, \dots, x_{T}) = \prod_{t = 1}^{T} P_{θ} (x_{t} ∣ x_{< t})$

Yeh factorization kyun? Probability ka chain rule: $P (A, B) = P (A) P (B ∣ A)$ , $T$ variables tak extend kiya. Causal ordering ( $x_{1}$ pehle, phir $x_{2}$ given $x_{1}$ , etc.) arbitrary hai lekin text ke left-to-right flow se match karta hai.

Step 3: Optimization Ke Liye Negative Log Lo

Product maximize karna = negative log-likelihood minimize karna:

$L_{CLM} = - lo g P_{θ} (x) = - \sum_{t = 1}^{T} lo g P_{θ} (x_{t} ∣ x_{< t})$

Negative log kyun?

Numerical stability: Chhoti probabilities ke products underflow ho jaate hain; logs ke sums nahi hote.
Additivity: Ek batch par loss sirf per-example losses ka sum hota hai.
Convexity: Log-loss exponential families ke liye convex hoti hai (jaise softmax output).

Step 4: Dataset Par Average Karo

$L_{CLM} (D) = - \frac{1}{∣ D ∣} \sum_{x \in D} \sum_{t = 1}^{T} lo g P_{θ} (x_{t} ∣ x_{< t})$

Yeh true next-token distribution (one-hot) aur model ki predicted distribution ke beech cross-entropy loss hai.

Worked Examples

Example 1: Tiny Sequence

Sequence: "I love ML" Tokens: $x_{1} = I, x_{2} = love, x_{3} = ML$

Step-by-step loss:

Position 1: $L_{1} = - lo g P_{θ} (x_{1} = I ∣ \emptyset)$ Context nahi kyun? $x_{1}$ pehla token hai; model ek start-of-sequence token se predict karta hai ya unconditional distribution seekhta hai.
Position 2: $L_{2} = - lo g P_{θ} (x_{2} = love ∣ x_{1} = I)$ Sirf $x_{1}$ kyun? Causal constraint: model "I" dekhta hai, "love" predict karta hai.
Position 3: $L_{3} = - lo g P_{θ} (x_{3} = ML ∣ x_{1} = I, x_{2} = love)$ $x_{1}, x_{2}$ dono kyun? Saara previous context available hota hai.

Total loss: $L_{CLM} = L_{1} + L_{2} + L_{3}$

Concrete numbers: Maano model yeh output karta hai:

$P_{θ} (I ∣ \emptyset) = 0.1$ → $L_{1} = - lo g (0.1) \approx 2.30$
$P_{θ} (love ∣ I) = 0.4$ → $L_{2} = - lo g (0.4) \approx 0.92$
$P_{θ} (ML ∣ I love) = 0.05$ → $L_{3} = - lo g (0.05) \approx 3.00$

Total: $2.30 + 0.92 + 3.00 = 6.22$

Yeh number kyun? Kam loss = model true sequence ko zyada probability assign karta hai. Perfect prediction ( $P = 1$ ) se loss $0$ milta hai.

Example 2: Generation vs. Training

Training:

Input: "The cat sat on the"
Targets: "cat sat on the mat" (1 se shift hua)
Loss: $- lo g P (cat ∣ The) - lo g P (sat ∣ The cat) - \dots$

Generation (autoregressive sampling):

Start: "The"
$x_{2} \sim P_{θ} (\cdot ∣ The)$ sample karo → "cat" milta hai
$x_{3} \sim P_{θ} (\cdot ∣ The cat)$ sample karo → "sat" milta hai
End-of-sequence token tak continue karo

Dono ke liye same objective kyun kaam karta hai? Training har position par $P_{θ} (x_{t} ∣ x_{< t})$ seekhta hai. Generation repeatedly is seekhe hue conditional ko sequence extend karne ke liye apply karta hai.

Worked example Real Model (GPT-2 Small)

Architecture: 12-layer transformer, 768-dim embeddings, 50,257 vocab Loss computation per batch:

# Pseudo-code (PyTorch-like)
logits = model(input_ids)  # shape: (batch, seq_len, vocab_size)
shift_logits = logits[:, :-1, :]  # last position exclude karo
shift_labels = input_ids[:, 1:]   # pehla position exclude karo (shift targets)
loss = cross_entropy(
    shift_logits.view(-1, vocab_size),
    shift_labels.view(-1),
    ignore_index=pad_token_id  # padding ko penalize mat karo
)

Shift kyun? Position $t$ ko positions $< t$ se predict karo. Position $t - 1$ par input, position $t$ par label predict karta hai.

Typical loss values:

Random init: $lo g (50257) \approx 10.8$ (vocab par uniform distribution)
Trained GPT-2: validation set par $\approx 3.5$
Overfitted: training set par $< 0.1$ (memorization)

Common mistake Common Mistakes

Mistake 1: "Loss mein future tokens shamil hain"

Galat soch: "Model poori sentence dekhta hai, saare tokens ek saath predict karta hai." Kyun sahi lagta hai: Parallel training (saari positions simultaneously compute hoti hain) aisa lagta hai jaise model ko full sequence access hai.

Sachchi baat: Attention masking causality enforce karta hai. Position $t$ par, positions $> t$ ke liye self-attention scores softmax se pehle $- \infty$ set ho jaate hain, unka contribution zero kar dete hain. Computationally parallel ≠ informationally non-causal.

Fix: Attention implementation check karo:

mask = torch.triu(torch.ones(seq_len, seq_len), diagonal=1).bool()
scores.masked_fill_(mask, float('-inf'))

Mistake 2: "Pehle token ka koi loss nahi"

Galat soch: "Position 1 ka koi context nahi, isliye $L_{1} = 0$ ." Kyun sahi lagta hai: "Bina input ke predict kaise karoge?"

Sachchi baat: $P_{θ} (x_{1})$ pehle tokens par unconditional distribution hai, jo data se seekhi jaati hai. Model seekhta hai kaun se tokens commonly sentences shuru karte hain (jaise "The", "I", capital letters). Loss hai $- lo g P_{θ} (x_{1})$ .

Fix: Position 1 ko loss summation mein include karo. Aksar, ek special `<BOS>` (beginning-of-sequence) token prepend kiya jaata hai, taaki $x_{1}$ ko `<BOS>` se predict kiya jaaye.

Mistake 3: "Loss minimize karo = better text generate karo"

Galat soch: "Validation set par kam loss hamesha better generation quality matlab hai."

Kyun sahi lagta hai: Loss directly prediction accuracy measure karta hai.

Sachchi baat: Training data par Overfitting se loss minimize hota hai lekin memorized, non-creative text produce hota hai. Exposure bias (model training ke dौran apni galtiyan kabhi nahi dekhta) generation time par compounding mistakes karata hai.

Fix: Perplexity (exponentiated loss) aur human evaluation use karo. Likelihood aur diversity balance karne ke liye temperature/top- $p$ se sample karo: $P^{'} (x_{t}) \propto P_{θ} (x_{t} ∣ x_{< t})^{1/ T}, T > 1 for more randomness$

Causal vs. Other Objectives Kyun?

Objective	Prediction	Mask	Use Case
Causal LM	Next token	Future positions	Generation (GPT, GPT-2, GPT-3)
Masked LM	Masked tokens	Random positions	Understanding (BERT)
Prefix LM	Next token	Future in suffix	Hybrid (encoder-decoder)

Generation ke liye causal kyun? Inference par, tumhare paas sirf past tokens hote hain. Training objective ko test-time reality se match karna chahiye (future available nahi hai).

Masked LM generate kyun nahi kar sakta? BERT bidirectional context dekhta hai, lekin generation left-to-right hoti hai. Training aur inference ke beech mismatch hai.

Recall Ek 12-Saal-Ke Bacche Ko Samjhao

Socho tum ek story likh rahe ho, ek word at a time. Tum jo likh chuke ho woh dekh sakte ho, lekin aage kya aayega yeh nahi dekh sakte (kyunki tumne abhi likha hi nahi!).

Causal language modeling ek robot ko yahi game sikhana hai. Robot "Once upon a" padhta hai, phir guess karta hai agla word "time" ho sakta hai. Agar real agla word sach mein "time" hai, to robot ko achcha score milta hai. Agar usne "banana" guess kiya, to bura score milta hai aur woh better karna seekhta hai.

Robot laakhon stories par practice karta hai, aaise patterns seekhta hai jaise:

"Once upon a ___" → usually "time"
"I feel ___" → often "happy", "sad", "excited"
"The dog ___" → probably ek verb jaise "ran", "barked"

Trick yeh hai ki robot kabhi cheat nahi karta aage dekh ke. Woh sirf wahi dekhta hai jo already likh chuka hai, bilkul tumhari tarah. Isliye ise "causal" kehte hain — jaise cause aur effect, past future ko cause karta hai, na ki ulta!

Kaafi practice ke baad, robot next words guess karne mein itna achcha ho jaata hai ki woh khud poori stories likh sakta hai, ek word at a time.

Connections

Autoregressive Models — Causal LM text par apply kiya gaya autoregressive paradigm hai
Cross-Entropy Loss — CLM objective ka mathematical form
Transformer Decoder — Causal self-attention implement karne wala architecture
Teacher Forcing — Training technique jahan true tokens (predictions nahi) context provide karte hain
Perplexity — Evaluation metric: $exp (L_{CLM})$ , jitna kam utna better
Masked Language Modeling — Bidirectional understanding ke liye contrasting objective (BERT)
Exposure Bias — Autoregressive models mein training-inference mismatch
Sampling Strategies — $P_{θ} (x_{t} ∣ x_{< t})$ se kaise generate karein (greedy, beam search, nucleus)
GPT Family — Causal LM objective se train kiye gaye models

#flashcards/ai-ml

Causal language modeling objective kya hai? :: Negative log-likelihood $- \sum_{t = 1}^{T} lo g P_{θ} (x_{t} ∣ x_{< t})$ minimize karo, har token ko sirf previous tokens se predict karo (future access nahi).

Ise "causal" kyun kehte hain? :: Kyunki position $t$ par prediction sirf $t$ se pehle ki positions par depend kar sakti hai (cause → effect), time ki forward direction preserve hoti hai.

CLM sequence probability ko kaise factorize karta hai?

Chain rule se:

P (x) = \prod_{t = 1}^{T} P (x_{t} ∣ x_{< t})

, joint probability ko conditional next-token predictions mein decompose karta hai.

Transformers mein causality kaise enforce hoti hai?

Attention masking: positions

> t

ko softmax se pehle

- \infty

scores milte hain, position

t

ki representation mein unka contribution zero ho jaata hai.

Ek single position

t

ka loss kya hai?

L_{t} = - lo g P_{θ} (x_{t} ∣ x_{< t})

, true token

x_{t}

ko past context diye assign ki gayi negative log-probability.

Probability product ki jagah negative log-likelihood kyun use karte hain?

Numerical stability (logs underflow avoid karte hain), additivity (sum karna aasaan hai), aur convexity (better optimization properties).

CLM aur cross-entropy mein kya relation hai?

CLM, true next-token distribution (correct token par one-hot) aur model ki predicted softmax distribution ke beech cross-entropy hai.

Generation ke liye causal LM kaise use hota hai?

Autoregressively

x_{t} \sim P_{θ} (\cdot ∣ x_{< t})

sample karo, phir sampled

x_{t}

par condition karke

x_{t + 1}

predict karo, end-of-sequence tak repeat karo.

CLM ke context mein perplexity kya hai?

PPL = exp (L_{CLM})

, exponentiated average negative log-likelihood; kam perplexity = better predictions.

CLM shifted targets par kyun train karta hai?

Position

t - 1

par input, position

t

par target predict karta hai (input: "The cat sat", targets: "cat sat on"),

P (x_{t} ∣ x_{< t})

implement karta hai.

Causal LM mein exposure bias kya hai?

Training mein context ke liye true tokens use hote hain (teacher forcing), lekin generation mein model ke apne predictions use hote hain; errors compound ho jaate hain kyunki model ne kabhi apni galtiyon se recover karna practice nahi kiya.

BERT GPT ki tarah text generate kyun nahi kar sakta?

BERT bidirectional context ke saath masked LM use karta hai, lekin generation left-to-right hai (sirf past available); training-inference mismatch autoregressive sampling prevent karta hai.