Evaluation of LLMs (benchmarks, LLM-as-judge)

4.4.16 · AI-ML › Alignment, Prompting & RAG

Core Problem: The Evaluation Gap

Language model evaluation teen challenges face karta hai:

1. Capability diversity: LLMs math se lekar creative writing aur code tak sab kuch karte hain
2. Output subjectivity: "Achha" har task ke hisaab se alag hota hai (factual vs. creative vs. helpful)
3. Human evaluation cost: Expert review lakhon outputs ke liye scale nahi hoti

Solution approach: Objective benchmarks (measurable skills ke liye) aur model-based judges (subjective quality ke liye) ko combine karo.

Part 1: Benchmark Evaluation

Benchmarks Kaise Kaam Karte Hain

Pipeline:

Model → Prompt with task → Generate output → Compare to answer → Compute metric

Key benchmark types:

| Type | Measures | Example Metric | |------|----------|------|-----| | Multiple choice | Knowledge recall | MMLU (57 subjects) | Accuracy | | Code generation | Programming ability | HumanEval, MBPP | Pass@k (% passing tests) | | Math word problems | Reasoning | GSM8K, MATH Exact match | | Reading comprehension | Understanding | SQuAD, DROP | F1 score | | Instruction following | Alignment | MT-Bench, AlpacaEval | Win rate vs baseline |

Pass@k First Principles Se Derive Karna

Problem: Model har problem ke liye $k$ code samples generate karta hai. Probability kya hai ki ≥1 correct hai?

Setup:

• $n$ = total problems
• $c$ = woh problems jahan $k$ samples mein se ≥1 ne sare tests pass kiye
• $p$ = true probability ki ek single sample correct hai (unknown)

Derivation: $\text{Pass@}k=\mathbb{P}(\text{≥1 correct in }k\text{ samples})$

Subtract kyun karte hain? Complement compute karna aasaan hai: $=1-\mathbb{P}(\text{all }k\text{ wrong})$

Agar samples independent hain: $=1-(1-p{)}^k$

Lekin $p$ pata nahi! Data se estimate karo: $\hat{p}=\frac{c}{n}$

Ye estimator kyun? Maximum likelihood: observed success rate = true rate ka best guess.

Final formula: $\boxed{\text{Pass@}k=1-{\left(1-\frac{c}{n}\right)}^k}$

Part 2: LLM-as-Judge

Isko Kyun Chahiye

Scenarios jahan benchmarks fail karte hain:

• Creative tasks: "Akele pan par ek haiku likho" (koi ground truth nahi)
• Open-ended QA: "Aasman neela kyun hota hai explain karo" (bahut saare valid explanations hain)
• Conversational ability: Multi-turn dialogues (context-dependent quality)
• Alignment: Kya response helpful aur safe hai? (subjective judgment)

Human evaluation ki problems:

• Expensive: Rating ke liye $0.50–5, large-scale tuning ke liye lakhs chahiye
• Slow: Crowdworker consensus ke liye days se weeks lagte hain
• Inconsistent: Inter-annotator agreement aksar 60–80% hoti hai

Solution: Ek strong LM ko human preferences mimic karne ke liye train/prompt karo.

LM-as-Judge Kaise Kaam Karta Hai

Pairwise comparison setup (sabse common):

Prompt to judge:
---
Instruction: {user_question}

Response A: {model_1_output}
Response B: {model_2_output}

Kaun sa response behtar hai? Accuracy, helpfulness, safety consider karo.
Choose: [[A]] or [[B]] or [[Tie]]
---

Judge output deta hai: "A"
Bahut examples pe aggregate karo: Model 1, Model 2 ke against 65% jeet raha hai → 1 higher ranked hai.

Absolute scoring setup:

Is response ko 1-10 scale par rate karo:
- Accuracy (factually correct?)
- Helpfulness (question ka answer deta hai?)
- Safety (harmful content se bachta hai?)

Response: {output}

Judge output deta hai: "Accuracy: 8, Helpfulness: 9, Safety: 10"

Agreement Metrics First Principles Se Derive Karna

Problem: LM judge humans se kitna agree karta hai?

Setup:

• $n$ = total comparisons
• $a$ = agreements (judge aur human dono same winner choose karte hain)
• $d$ = disagreements

Cohen's Kappa (chance agreement ke liye account karta hai):

$\kappa =\frac{p_o-p_e}{1-p_e}$

Jahaan:

• $p_o$ = observed agreement = $\frac{a}{n}$
• $p_e$ = chance se expected agreement

Chance subtract kyun karte hain? Random guessing binary choice mein ~50% agreement deta hai. Humein agreement luck se beyond chahiye.

$p_e$ derive karna: Agar human A ko probability $p_A$ se choose karta hai aur judge A ko probability $q_A$ se choose karta hai: $p_e=p_A\cdot q_A+p_B\cdot q_B$

Product kyun? Probability ki dono chance se A choose karein = (prob human A choose kare) × (prob judge A choose kare), independence assume karte hue.

Example calculation:

• 100 comparisons, 80 agreements → $p_o=0.80$
• Human 60% time A choose karta hai, judge 55% time A choose karta hai
• $p_e=0.60\times 0.55+0.40\times 0.45=0.33+0.18=0.51$
• $\kappa =\frac{0.80-0.51}{1-0.51}=\frac{0.29}{0.49}=0.59$ (moderate agreement)

Part 3: Practice Mein Key Benchmarks

MMLU (Massive Multitask Language Understanding)

• Kya hai: 57 subjects (elementary math se professional law tak), 14k multiple choice questions
• Kyun matter karta hai: Broad knowledge test, users ko "smart lagta hai" wali feeling se correlate karta hai
• Limitation: Memorization measure karta hai, reasoning depth nahi

GSM8K (Grade School Math 8K)

• Kya hai: Multi-step reasoning ki zaroorat wale 8,500 grade-school word problems
• Kyun matter karta hai: Chain-of-thought test karta hai, sirf formula recall nahi
• Example: "Agar ek train 2.5 ghante 60 mph se chale, toh kitni door jaayegi?" ($d=rt$ ki zaroorat hai)

HumanEval & MBPP (code generation)

• Kya hai: Unit tests ke saath 164 (HumanEval) / 974 (MBPP) Python programming problems
• Kyun matter karta hai: Executable = objective correctness check
• Limitation: Algorithmic coding test karta hai, software engineering nahi (design, debugging, collaboration)

MT-Bench (Multi-Turn Benchmark)

• Kya hai: 8 categories mein 80 multi-turn conversations (writing, reasoning, math, coding, roleplay, extraction, STEM, humanities)
• Kyun matter karta hai: Conversational ability + context tracking test karta hai (real-world chat use)
• Evaluation: GPT-4 as judge, har turn ko 1-10 rate karta hai

AlpacaEval

• Kya hai: 805 diverse instructions, model ko reference (GPT-4-Turbo) se compare karta hai
• Kyun matter karta hai: Instruction-following + user preference measure karta hai
• Metric: Win rate (% time model baseline se preferred hota hai)

Part 4: Pitfalls & Limitations

Comprehensive Example: Ek Naye Model Ko Evaluate Karna

Scenario: Tumne LaMA-3-70B ko medical data par fine-tune kiya hai. Ise kaise evaluate karoge?

Step 1: Benchmark suite

• MedQA (medical licensing exam questions): 78% (baseline: 72%, GPT-4: 86%)
  • Interpretation: Baseline se upar, lekin medical reasoning mein SOTA nahi
• PubMedQA (research abstracts par yes/no/maybe): 81% (baseline: 76%, GPT-4: 84%)
  • Interpretation: Solid biomedical understanding
• HumanEval (general coding): 45% (baseline: 42%)
  • Interpretation: Coding harm nahi hua, lekin improvement bhi nahi

Step 2: 500 custom medical queries par LM-as-judge (GPT-4)

• Prompt: "Tum ek expert physician ho. Response ko accuracy, clinical relevance, aur safety ke liye rate karo (har ek 1-10)."
• Results: Accuracy 7.8, Relevance 8.3, Safety 9.1 (baseline: 7.2, 7.9, 8.8)
  • Interpretation: Fine-tuning ne relevance + safety improve kiya, accuracy mein thoda gain

Step 3: Human expert review (50 samples)

• GPT-4 judge ke saath agreement: Cohen's $\kappa =0.68$ (substantial)
• Woh cases pakdo jahan judge galat tha: 4 responses mein factual errors the jo judge miss kar gaya
  • Interpretation: LLM judge screening ke liye reliable hai, lekin critical decisions ke liye humans chahiye

Step 4: Simulated clinic mein A/B test

• 20 physicians ne 100 patient questions ke liye model vs baseline use kiya
• Preference: 65% physicians ne fine-tuned model prefer kiya (AlpacaEval-style win rate)
• Qualitative: "Zyada specific drug recommendations, kam generic responses"

Conclusion: Model physician assistant role ke liye production-ready hai (autonomous diagnosis ke liye nahi). Benchmarks ne capability dikhaayi, judge ne quality improvement dikhaayi, humans ne safety validate ki.

Active Recall Practice

Connections

• Chain-of-Thought Prompting: GSM8K/MATH benchmarks CoT reasoning ability test karte hain
• RLHF (Reinforcement Learning from Human Feedback): Human preference data reward models train karta hai, LM-judge setup se milta-julta hai
• Prompt Engineering Best Practices: Benchmark prompts carefully design hone chahiye (zero-shot vs few-shot scores affect karta hai)
• RAG (Retrieval-Augmented Generation): Evaluation ko retrieval quality (recall, precision) + generation quality (faithfulness) ke liye benchmarks chahiye
• Alignment Tax: High benchmark scores safety guarantee nahi karte—alignment-specific evals chahiye (TruthfulQA, BBQ bias benchmark)
• Few-Shot Learning: MT-Bench conversation turns mein in-context learning test karta hai
• Constitutional AI: Training ke dauran harmful outputs filter karne ke liye internally model-as-judge use karta hai

#flashcards/ai-ml

LM evaluation ko traditional ML evaluation se zyada mushkil banane wale teen core challenges kya hain? :: 1) Capability diversity (models bahut saare tasks karte hain), 2) Output subjectivity (open-ended tasks ke liye koi single "correct" answer nahi), 3) Human evaluation cost (expert review lakhon outputs ke liye scale nahi hoti)

Pass@k formula first principles se derive karo :: Pass@k = P(≥1 correct in k samples) = 1 - P(all k wrong) = 1 - (1-p)^k jahaan p single sample ke correct hone ki probability hai, c/n se estimate kiya jaata hai (correct solutions / total problems)

Concept Map