5.4.21
Intuition Ek-sentence mental modelEk pandas DataFrame basically ek dictionary of aligned columns hota hai (har column ek NumPy array hai jiske upar ek shared index label chipka hota hai). Pandas jo bhi karta hai — selecting, grouping, merging, pivoting — woh sab data ko label ke basis pe align karne ka ek clever tarika hai, raw position ke basis pe nahi.
NumPy fast arrays deta hai, lekin arrays sirf integer position se address hote hain. Real data ke naam hote hain: ek row "customer 1023" hai, ek column "revenue" hai. Pandas ek label axis (the index) add karta hai taaki:
Alignment automatic ho — do Series add karne par match label se hota hai, position se nahi.Missing data ka ghar ho — NaN wahan fill hota hai jahan labels match nahi karte.Heterogeneous columns saath reh sakein — int, float, string, datetime ek hi table mein.
Definition Series aur DataFrame
Ek Series ek 1-D labelled array hai: do aligned arrays ka pair (values, index).
Ek DataFrame ek 2-D labelled table hai: Series ka collection jo ek row index share karta hai aur column labels rakhta hai.
import pandas as pd s = pd.Series([ 10 , 20 , 30 ], index = [ "a" , "b" , "c" ]) df = pd.DataFrame({ "x" : [ 1 , 2 ], "y" : [ 3 , 4 ]}, index = [ "r0" , "r1" ]) Ek Series ko ek partial function f : Index → Value s1 + s2 karte ho, pandas element 0 ko element 0 se ADD nahi karta. Woh compute karta hai:
( s 1 + s 2 ) [ k ] = { s 1 [ k ] + s 2 [ k ] NaN k ∈ idx ( s 1 ) ∩ idx ( s 2 ) otherwise
Worked example Alignment label se, position se nahia = pd.Series([ 1 , 2 , 3 ], index = [ "x" , "y" , "z" ]) b = pd.Series([ 100 , 200 ], index = [ "y" , "x" ]) a + b # x -> 1 + 200 = 201 (Kyun? dono mein label "x" hai) # y -> 2 + 100 = 102 (Kyun? dono mein label "y" hai) # z -> NaN (Kyun? "z" b mein missing hai → koi match nahi) Yeh step kyun? Pandas pehle indices ka union leta hai, dono Series ko us par reindex karta hai, phir add karta hai. Position irrelevant hai.
.loc[rows, cols] label se select karta hai (slices mein dono endpoints inclusive hote hain)..iloc[rows, cols] integer position se select karta hai (stop exclusive, jaise Python mein).df["col"] label se ek column select karta hai; df[boolean_mask] rows select karta hai.
Worked example Table ko teen tareekon se padhnadf = pd.DataFrame({ "age" :[ 25 , 32 , 47 ], "city" :[ "NY" , "LA" , "SF" ]}, index = [ "u1" , "u2" , "u3" ])
df.loc[ "u2" , "age" ] # 32 (label row, label col) df.iloc[ 1 , 0 ] # 32 (position row, position col) df.loc[ "u1" : "u2" ] # rows u1 AUR u2 (label slice → inclusive!) df.iloc[ 0 : 2 ] # rows 0,1 (position slice → stop excluded) df[df[ "age" ] > 30 ] # boolean mask → u2, u3 .loc inclusive kyun hai:
hai kya
Groupby rows ko ek key ke basis pe buckets mein daalta hai, har bucket par ek function run karta hai, phir answers ko wapas ek labelled object mein jod deta hai. Yeh pattern hai split-apply-combine
result [ g ] = f ( { rows r : key ( r ) = g } )
Worked example Region ke hisaab se mean revenuedf = pd.DataFrame({ "region" : [ "N" , "N" , "S" , "S" , "S" ], "rev" : [ 10 , 30 , 5 , 15 , 10 ]})
df.groupby( "region" )[ "rev" ].mean() # N -> 20.0 (Kyun? (10+30)/2) # S -> 10.0 (Kyun? (5+15+10)/3) Yeh step kyun? groupby("region") index → rows mapping banata hai {N:[0,1], S:[2,3,4]}; .mean() har group par apply hota hai; results combine hote hain group keys ko naye index ke roop mein use karke.
pd.merge(L, R, on="key", how=...) do tables ko matching key values par align karta hai. how decide karta hai ki kaun se keys survive karenge:
inner → keys ka intersection (default)left → L ke saare keys, jahan R missing wahan NaNright → R ke saare keysouter → keys ka union
Worked example Inner vs left joinL = pd.DataFrame({ "id" :[ 1 , 2 , 3 ], "name" :[ "a" , "b" , "c" ]}) R = pd.DataFrame({ "id" :[ 2 , 3 , 4 ], "score" :[ 88 , 90 , 77 ]})
pd.merge(L, R, on = "id" , how = "inner" ) # ids 2,3 survive (Kyun? sirf yahi DONO mein hain)
pd.merge(L, R, on = "id" , how = "left" ) # ids 1,2,3 (Kyun? saara L); id 1 ko score=NaN milta hai (R mein match nahi) Yeh step kyun? Merge har key ke liye, dono sides par matching rows ka Cartesian product compute karta hai, how rule se filter karke.
Definition pivot / pivot_tablepivot long data (ek row per observation) ko ek wide matrix mein reshape karta hai: ek variable rows banta hai, doosra columns, teesra cells fill karta hai.
pivot_table wohi karta hai lekin aggregate bhi karta hai jab (row,col) pairs repeat hote hain.
Worked example Sales by (store × month)df = pd.DataFrame({ "store" :[ "A" , "A" , "B" , "B" ], "month" :[ "Jan" , "Feb" , "Jan" , "Feb" ], "sales" :[ 10 , 20 , 30 , 40 ]})
df.pivot( index = "store" , columns = "month" , values = "sales" ) # Feb Jan # A 20 10 # B 40 30 Yeh step kyun? Har unique store → row label banta hai, har unique month → column label, aur sales cell (store, month) mein jaata hai. Agar do rows same (store, month) share karte, pivot error deta → use karo pivot_table(aggfunc="sum").
.loc[0:2] aur .iloc[0:2] same rows dete hain."
Kyun sahi lagta hai: dono Python slices jaisi lagte hain, aur default 0,1,2,... index ke saath labels positions ke barabar hote hain , isliye accidentally milte hain.
Fix: .loc label se slice karta hai aur stop inclusive hota hai; .iloc position se slice karta hai aur stop exclude hota hai. Frame ko reindex karo taaki labels ≠ positions hon, aur tum dekhoge .loc[0:2] 3 rows deta hai, .iloc[0:2] 2 deta hai.
Common mistake "Chained indexingdf[df.a>0]['b'] = 5 DataFrame ko update karta hai."
Kyun sahi lagta hai: yeh normal Python assignment jaisa read hota hai.
Fix: df[mask] ek copy return kar sakta hai, isliye write lost ho jaata hai (SettingWithCopyWarning). Ek single .loc use karo: df.loc[df.a>0, "b"] = 5.
Common mistake "Merge default mein dono tables ki saari rows rakhta hai."Kyun sahi lagta hai: SQL beginners ek "full" join assume karte hain.
Fix: default how="inner" sirf shared keys rakhta hai. Union ke liye how="outer" use karo.
Common mistake "groupby ek DataFrame return karta hai jise main print kar sakta hoon."Kyun sahi lagta hai: tumne poora expression type kiya.
Fix: df.groupby("k") ek lazy GroupBy object hai; kuch bhi compute nahi hota jab tak tum .mean(), .agg(), .apply(), etc. call nahi karte.
Recall Cover karke answer karo
Q: Series mein kaun se do arrays hote hain? → values + index.
Q: Mismatched indices ke saath a + b NaN se bhara kyun hota hai? → alignment union leta hai, unmatched labels → NaN.
Q: Slice endpoints par .loc vs .iloc? → loc inclusive (label), iloc exclusive (position).
Q: Groupby ke teen steps? → split, apply, combine.
Q: Merge mein default how? → inner.
Q: pivot ke bajay pivot_table kab use karna padta hai? → jab (index,column) pairs repeat hon (aggregation ki zaroorat ho).
Recall Feynman: 12-saal ke bachche ko explain karoSocho ek badi locker chart hai, har locker par ek naam ka tag (the index) hai. Ek Series lockers ka ek column hai; ek DataFrame poori wall hai. Jab tum lockers ke do columns add karte ho, tum unhe naam tag se match karte ho, nahi ki kaun pehla hai — agar ek taraf naam missing hai, woh locker khaali rehta hai (NaN). Groupby bachon ko unki team-shirt ke rang se teams mein sort karna aur har team count karna hai. Merge do charts ko ek shared ID column se chipkana hai, jaise school IDs ko test scores se milana. Pivot ek lambi list ko ek neat grid mein rotate karna hai.
Mnemonic Toolkit yaad rakho"SIG-MP" = S eries (1D) → I ndex selectors (loc/iloc) → G roupby (split-apply-combine) → M erge (join on keys) → P ivot (long↔wide).
Aur joins ke liye: "I'm Left-Out" = I nner, L eft, O uter (teen jo actually use hote hain).
What is a pandas Series structurally? Ek 1-D labelled array = aligned (values array, index array).
Why does adding two Series with different indices produce NaN? Pandas labels ke union se align karta hai; unmatched labels ka koi partner nahi hota, isliye result NaN hota hai.
Difference between .loc and .iloc on a slice? .loc label-based hai aur stop inclusive hai; .iloc position-based hai aur stop exclude karta hai.
What are the three phases of groupby? Rows ko key se split karo, har group par function apply karo, results ko ek labelled object mein combine karo.
What is the default join type in pd.merge? inner (sirf woh keys rakhta hai jo dono tables mein present hain).
When must you use pivot_table instead of pivot? Jab (index, column) combination unique nahi hoti — pivot_table duplicates aggregate karta hai, pivot error raise karta hai.
How do you safely set values on filtered rows? Ek single .loc call use karo: df.loc[mask, "col"] = value (chained indexing avoid karo).
What does how="outer" return in a merge? Dono tables ke keys ka union, jahan ek side ka match nahi wahan NaN fill karta hai.
Is df.groupby("k") eager or lazy? Lazy — yeh ek GroupBy object return karta hai; computation tab hota hai jab aggregation/apply call ki jaati hai.
NumPy — har pandas column ek NumPy array hai; neeche vectorized ops hain.SQL Joins — merge INNER/LEFT/RIGHT/OUTER JOIN semantics mirror karta hai.Split-Apply-Combine — woh general data-analysis pattern jo groupby implement karta hai.Tidy Data — pivot/melt long (tidy) aur wide layouts ke beech convert karte hain.Hash Tables — index O ( 1 ) Missing Data / NaN — alignment NaNs ka main source hai.
NumPy array
DataFrame
GroupBy
