T5 se pehle, har task ka apna head hota tha: classification softmax over labels use karta tha, span-QA do pointer heads (start/end) use karta tha, regression ek scalar output use karta tha. Har head ko custom code, custom loss, aur custom fine-tuning tricks ki zaroorat thi.
T5 = "Text-To-Text Transfer Transformer" (paanch T's → "T5").
Kyun yeh EK loss SAARE tasks handle karta hai, yeh derive karte hain: Kisi bhi task ka answer ek string y hai. Classification label "positive" ek string hai. Number "3.8" ek string hai. Target string tokens par cross-entropy isliye har task ko same formula se optimize karti hai. Bas yehi poora trick hai.
prefix + input feed karo, decoder ko greedily (ya beam search se) </s> aane tak run karo. Produced string ko answer ki tarah padho. Classification ke liye check karo kaunsa label string produce hua; agar woh off-vocabulary hai, toh woh simply wrong count hota hai.
Recall Feynman: 12-saal ke bachche ko samjhaao
Ek super-smart student ki imagination karo jisे sirf EK cheez karni aati hai: tum unhe ek sentence do, aur woh ek sentence wapas likhte hain. Unse translate karaane ke liye, aage "translate to German:" likh do. Grammar grade karaane ke liye, "is this okay:" likho aur woh "yes" ya "no" likhenge. Seekhne ke liye, woh fill-in-the-blank khelte hain: tum kuch words erase karo, aur woh guess karte hain wahan kya tha. Kyunki sab kuch sirf "ek note padho, ek note likho" hai, wahi student sau alag-alag kaam kar sakta hai.
Har NLP task ko "input string → output string" ke roop mein cast kiya jaata hai, ek model, ek architecture, aur ek cross-entropy loss use karke.
T5 kaun si architecture use karta hai?
Ek standard encoder–decoder Transformer (bidirectional encoder + autoregressive decoder), decoder-only nahi.
T5 bina softmax label head ke classification kaise handle karta hai?
Woh label ko text ke roop mein generate karta hai, jaise "positive"/"unacceptable", aur un target tokens par cross-entropy score karta hai.
T5 regression kaise handle karta hai (jaise STS-B similarity)?
Woh number ko string ke roop mein emit karta hai (nearest 0.2 tak rounded), digits ko tokens ke roop mein same cross-entropy loss se predict karta hai.
T5 ka pretraining objective kya hai?
Span corruption: ~15% tokens ko short contiguous spans mein drop karo, har span ko ek unique sentinel token se replace karo, aur decoder se sentinels ke baad missing text generate karaao.
BERT-style single-token masking ki jagah span corruption kyun?
T5 ko text OUTPUT karna hota hai; puri spans predict karna (generated sequences ke roop mein) iske text-to-text design se match karta hai aur independent per-token classification se richer representations deta hai.
Sentinel tokens (jaise , ) kya hain?
Unique placeholder tokens jo har corrupted span ko mark karte hain, taaki decoder jaane target mein woh kaunsa blank fill kar raha hai.
Input x diye hue target y ke liye T5 ki training loss likhein.
L=−∑t=1mlogp(yt∣y<t,x), yaani teacher forcing ke under token-level cross-entropy.
T5 mein task prefix kaise kaam karta hai?
Ek short instruction string (jaise "translate English to German:") input ke aage prepend ki jaati hai taaki ek hi model ko bataya ja sake kaunsa task perform karna hai.
T5 mein ek loss saare tasks train kyun kar sakta hai?
Har answer ek string hai, isliye target string par cross-entropy classification, regression, translation, aur summarization ko identically optimize karti hai.
