Self-supervised pretraining objectives — Hinglish notes

4.3.5 · AI-ML › Pretraining & Fine-Tuning LLMs

KYUN zaroori hain self-supervised objectives?

KYA problem solve kar rahe hain? Hum chahte hain ek model jo language "samjhe", lekin labeled data (sentiment, translation, etc.) scarce aur expensive hoti hai. Raw text (web, books) almost infinite hai.

KYUN self-supervision kaam karta hai: kisi missing word ko sahi predict karne ke liye model ko implicitly grammar, facts, coreference, arithmetic, world knowledge seekhni padti hai. Toh ek trivially simple lagta task (agla word guess karo) model ko bahut saara structure apne weights mein compress karne par majboor karta hai.

Main objectives (the 80/20 core)

1. Causal / Autoregressive Language Modeling (CLM) — GPT-style

KYA: Sabhi previous tokens ko dekh kar agla token predict karo (left-to-right).

HOW loss derive karte hain (first principles se): Poore sequence $x_{1}, \dots, x_{T}$ ki probability ko hamesha probability ke chain rule se factorize kiya ja sakta hai: $p_{θ} (x_{1}, \dots, x_{T}) = \prod_{t = 1}^{T} p_{θ} (x_{t} ∣ x_{1}, \dots, x_{t - 1})$

Yeh step kyun? Koi assumption nahi ki gayi — chain rule kisi bhi joint distribution ke liye exact hai.

Train karne ke liye hum log-likelihood maximize karte hain. "Maximize log-prob" ko "minimize loss" mein badalna ( $- 1$ se multiply karke) aur positions par average lena deta hai:

Cross-entropy kyun? Kyunki $- lo g p_{θ} (x_{t} ∣ \cdot)$ sahi token ka "surprise" hai; average surprise minimize karna = likelihood maximize karna. Perplexity $PPL = e^{L_{CLM}}$ bas exponentiated loss hai.

Key trick — causal masking: ek token sirf positions $\leq t$ par attend kar sakta hai, warna woh answer "peek" karke cheat kar sakta hai.

2. Masked Language Modeling (MLM) — BERT-style

KYA: Randomly ~15% tokens ko [MASK] symbol se chupa do; left aur right dono context (bidirectional) use karke originals predict karo.

HOW: Maano $M$ masked positions ka set hai. $x \to \tilde{x}$ ko corrupt karo $x_{i}$ ( $i \in M$ ) ko [MASK] se replace karke. Sirf masked wale predict karo:

Sirf $M$ par sum kyun? Non-masked tokens visible hain, toh unhe predict karna trivial hai aur koi learning signal nahi milta.

BERT ka real recipe 80/10/10 kyun hai? Chosen 15% mein se: 80% → [MASK], 10% → ek random token, 10% → unchanged rakhte hain.

Kyun? Fine-tuning ke time koi [MASK] token hota nahi, toh sirf [MASK] par trained model ko train/test mismatch dikhega. Random/unchanged cases model ko majboor karte hain ki woh har token ki useful representation banaye, sirf mask symbol par react na kare.

3. Span Corruption / Denoising (T5, BART)

KYA: Contiguous spans mask karo aur decoder ko missing spans ek short output sequence ke roop mein generate karne do. Bidirectional encoding ko generation ke saath combine karta hai.

KYUN single tokens nahi, spans? Ek masked token predict karna aksar local n-gram statistics se solve ho jata hai; poora span mask karna genuine multi-token reasoning force karta hai.

Figure — Self-supervised pretraining objectives

Worked examples

Forecast-then-Verify

Common mistakes (Steel-manned)

Mnemonic

Feynman

Recall Ek 12-saal ke bachche ko explain karo

Socho ek story padh rahe ho jisme kuch words stickers se chhupaaye gaye hain. Unhe fill karne ke liye tumhe story samajhni padegi — kaun kya kar raha hai, kya sense banta hai. Computer yeh fill-in game lakho baar poore internet par khelta hai. Kissi ko answers likhne nahi padte, kyunki answers wahi words hain jo stickers ke neeche the! Hidden words guess karne mein bahut accha ho jaane se, computer accidentally grammar, facts, aur thodi reasoning bhi seekh leta hai.

Flashcards

Koi objective "self-supervised" kab hota hai?

Jab labels automatically input se hi derive hoti hain (jaise hidden words), toh kisi human annotation ki zaroorat nahi.

Causal LM loss batao.

L = - \frac{1}{T} \sum_{t} lo g p_{θ} (x_{t} ∣ x_{< t})

— har next token ka average negative log-prob.

CLM factorization ko konsa probability rule justify karta hai?

Probability ka chain rule:

p (x_{1} .. x_{T}) = \prod_{t} p (x_{t} ∣ x_{< t})

, kisi bhi joint distribution ke liye exact.

Cross-entropy loss aur perplexity ka kya relation hai?

PPL = e^{L_{C E}}

CLM causal attention mask kyun use karta hai?

Taaki koi token future tokens par attend na kar sake aur apna answer 'peek' na kar sake.

MLM mein konsi positions loss mein contribute karti hain?

Sirf masked positions

M

BERT ka 80/10/10 masking split aur uska purpose explain karo.

Chosen 15% tokens mein se: 80% [MASK] bante hain, 10% random token, 10% unchanged — train/test mismatch avoid karne ke liye kyunki [MASK] fine-tuning par kabhi appear nahi karta.

CLM generation ke liye MLM se zyada suited kyun hai?

Uska factorization left-to-right sampling se match karta hai; MLM bidirectional hai aur autoregressively sample nahi kiya ja sakta.

100% tokens mask kyun nahi karne chahiye?

Condition karne ke liye koi context nahi bachta, prediction unlearnable ho jaata hai.

Span corruption (T5/BART) kya hai?

Contiguous spans mask karna aur missing spans generate karna, multi-token reasoning force karte hue bidirectional encoding rakhna.

Connections

Transformer Architecture — causal vs bidirectional attention masks in objectives implement karte hain.
Cross-entropy Loss — shared underlying loss function.
Perplexity — evaluation metric = exp(loss).
Fine-Tuning LLMs — pretrained weights starting point hote hain.
Tokenization (BPE) — woh vocabulary define karta hai jis par softmax compute hota hai.
GPT vs BERT — CLM vs MLM design tradeoffs.