Koi built-in stopping nahi — kuch bhi tree ko purity tak pahunchne se nahi rokta.
Har split training loss ko reduce karta hai (ya waise hi chhod deta hai), isliye pure greed hamesha kehti hai "aur split karo."
Variance explode karta hai: training data mein thodi si bhi change kisi top ke paas ke split ko flip kar sakti hai aur poore tree ko neeche se rewrite kar sakti hai.
Classic signature yeh hai: jaise-jaise tree ki depth badhti hai, training error → 0 monotonically jaata hai, lekin validation error ek U-shape follow karta hai — pehle girta hai, ek sweet spot hit karta hai, phir chadh jaata hai.
Tree ke zyada complex hone se pehle iska growth rok do. WHY it works: yeh capacity ko cap karta hai, isliye tree noise ko isolate nahi kar paata. Common knobs:
max_depth — hard depth limit.
min_samples_split — bahut kam samples wale node ko mat split karo.
min_samples_leaf — har leaf mein ≥ k points hone chahiye (taaki leaves statistically meaningful hon).
min_impurity_decrease — tabhi split karo jab yeh "kaafi" help kare.
Pehle poora tree grow karo, phir branches ko prune karo. Yeh myopia problem se bachata hai kyunki decide karne se pehle poora subtree dekha ja sakta hai.
Kyunki overfitting ek variance problem hai, kaafi trees ka average lo: Random Forests kai deep trees ko bootstrap samples + random feature subsets par train karte hain aur unka average lete hain. n roughly-independent trees ka average karna bias zyada badhaaye bina variance ko ~1/n tak cut kar deta hai.
Sochо ki tum ek test ki taiyari karte ho pichle saal ke paper ke exact answers memorize karke — printing typos samet. Tum uss paper mein ace kar loge lekin is saal fail ho jaoge, kyunki questions badal gaye. Jo decision tree bahut deep tak grow karta hai woh bhi yehi karta hai: practice data ki har thodi si hilchal memorize kar leta hai. Pruning aise hai jaise khud se kaho "idea seekho, typos mat memorize karo" — tum over-specific cheezein kaat dete ho taaki naye questions mein achha karo.
Unki capacity bahut high hoti hai aur yeh greedily split karte rehte hain jab tak leaves pure na ho jaayein, individual noisy points ko isolate karte hain — low bias, bahut high variance.
Bias–variance decomposition mein, overfitting ek bade ___ term se correspond karta hai.
Variance.
Tree depth ke saath overfitting ki visual signature kya hai?
Training error 0 tak girta rehta hai jabki validation error U-shaped hota hai (girta phir badhta hai).
Ek tree ki cost-complexity define karo.
Rα(T)=R(T)+α∣T~∣: training error plus ek penalty α per leaf.
Node t ke liye effective-α (weakest-link) formula do.
α=∣T~t∣−1R(t)−R(Tt).
Denominator ∣T~t∣−1 kyun hai?
Pruning ∣T~t∣ leaves remove karta hai aur 1 wapas add karta hai, net loss ∣T~t∣−1 leaves ka hota hai; α = error rise per leaf removed.
Pre-pruning underfit kyun kar sakta hai (steel-man)?
Yeh greedy/myopic hai: ek split abhi useless lag sakta hai lekin neeche bahut achhe splits enable kar sakta hai (jaise XOR); jaldi rokna useful structure khatam kar deta hai.
Post-pruning us problem se kaise bachta hai?
Yeh pehle poora tree grow karta hai, isliye decide karne se pehle poore subtree ki value judge kar sakta hai.
Trees mein overfitting se ladne ke teen tarike batao.
