Question bank — Pandas — Series, DataFrame, indexing, groupby, merge, pivot

False — pandas aligns by label (the name), not by slot (the counting position). It takes the union of both indices, reindexes both onto it, then adds. So the value in slot 0 of one Series may pair with the value in slot 5 of the other, if those two share the same label. ::: The whole point of an index is matching by name, not by physical slot. See the figure above.

a = pd.Series([1, 2, 3], index=["x", "y", "z"])
b = pd.Series([100, 200], index=["y", "x"])   # note: reversed order
a + b        # x -> 201, y -> 102, z -> NaN  (matched by NAME, not slot)

same labels but in a different order, a + b still lines them up correctly. True — because alignment is by label, physical order is irrelevant. a indexed ["x","y"] plus b indexed ["y","x"] matches x↔x and y↔y regardless of who is written first. ::: This is exactly why pandas beats raw NumPy (which would add slot 0 to slot 0 blindly).

a = pd.Series([1, 2], index=["x", "y"])
b = pd.Series([10, 20], index=["y", "x"])
(a + b)["x"]     # 1 + 20 = 21, because both call slot-for-x their "x"

df["colname"] returns a copy you can safely write to. False — it usually returns a view/reference into the frame, but chaining a second index on top (df["a"]["b"]) can silently hit a copy. Never trust the writeability of chained access; use one .loc. ::: The ambiguity is the whole reason SettingWithCopyWarning exists.

df["a"]["b"] = 5          # ⚠️ may write to a copy → lost silently
df.loc["b", "a"] = 5      # ✅ one selector → writes back into df

.loc[0:2] and .iloc[0:2] return the same rows. False in general. .loc slices by label and includes the stop; .iloc slices by position (slot) and excludes it. They only coincide by accident when the index labels equal the slots (0,1,2,...). ::: Reindex with string labels and the two diverge immediately.

df = pd.DataFrame({"v":[10,20,30]}, index=["a","b","c"])
df.iloc[0:2]     # slots 0,1 → rows a,b        (stop EXCLUDED → 2 rows)
df.loc["a":"c"]  # labels a..c → rows a,b,c     (stop INCLUDED → 3 rows)

float64. Half-true, and this is the trap. With the classic dtype, yes: NaN is a floating-point value, so the column widens to float64. But pandas also has a nullable integer dtype Int64 (capital I) that stores missing as <NA> and keeps the values integer. ::: Choose Int64 explicitly if you need "integers that can be missing". See Missing Data / NaN.

pd.Series([1, 2, None]).dtype                    # float64  (classic widening)
pd.Series([1, 2, None], dtype="Int64").dtype     # Int64    (stays integer, missing = <NA>)

groupby("k") immediately computes the grouped result. False — it returns a lazy GroupBy object. Nothing is computed until you call .mean(), .sum(), .agg(), .apply(), etc. The object just holds the split plan. ::: Split is planned; apply+combine fire on demand.

g = df.groupby("region")   # <DataFrameGroupBy object> — nothing computed yet
g.mean(numeric_only=True)  # NOW split-apply-combine actually runs

merge keeps all rows from both tables unless you say otherwise. False — the default is how="inner", the intersection of keys. Rows whose key is missing on either side are dropped. You must ask for how="outer" to keep the union. ::: SQL beginners assume "full"; pandas assumes "inner".

L = pd.DataFrame({"id":[1,2,3]}); R = pd.DataFrame({"id":[2,3,4]})
pd.merge(L, R, on="id")                # inner → ids 2,3 only
pd.merge(L, R, on="id", how="outer")   # union → ids 1,2,3,4 (missing → NaN)

pivot and pivot_table are interchangeable. False — pivot errors if any (index, column) pair repeats, because it can't put two values in one cell. pivot_table handles repeats by aggregating (default mean). ::: One reshapes; the other reshapes-and-summarises.

df.pivot(index="store", columns="month", values="sales")            # errors if (store,month) repeats
df.pivot_table(index="store", columns="month", values="sales",
               aggfunc="sum")                                        # repeats → summed

df[df.age > 30]["city"] = "Unknown" — what's wrong and how to fix it? This is chained indexing: df[mask] may return a copy, so the write lands on a throwaway and vanishes (SettingWithCopyWarning). Fix: one selector — df.loc[df.age > 30, "city"] = "Unknown". ::: A single .loc writes back into the real frame.

df[df.age > 30]["city"] = "Unknown"        # ⚠️ writes to a copy → no effect
df.loc[df.age > 30, "city"] = "Unknown"    # ✅ one .loc → updates df

pd.merge(L, R) with no on= and two shared columns joins on the wrong thing. Without on=, merge joins on all commonly-named columns. If L and R share id and date by coincidence, it silently joins on both, shrinking the result. Fix: always state on="id" explicitly. ::: Implicit keys are a classic silent bug — see SQL Joins.

pd.merge(L, R)             # ⚠️ joins on EVERY shared column name
pd.merge(L, R, on="id")    # ✅ join key stated explicitly

pd.merge(L, R, on="id"), a non-key column called val appears twice as val_x and val_y. Why, and how to rename? Both L and R had a column named val that is not a join key, so merge can't overwrite one with the other. It disambiguates by appending the default ==suffixes ("_x", "_y")== (left gets _x, right gets _y). Fix: pass your own, e.g. suffixes=("_L", "_R"). ::: Overlapping non-key names always get suffixed — never silently dropped.

pd.merge(L, R, on="id")                             # → val_x, val_y
pd.merge(L, R, on="id", suffixes=("_L", "_R"))      # → val_L, val_R

df.groupby("region").mean() throws or drops columns unexpectedly. Why, and what controls it? .mean() tries to run on every remaining column; non-numeric columns (strings) can't be averaged. Recent pandas silences the guesswork with an explicit ==numeric_only== flag: numeric_only=True drops strings quietly, False raises on them. Cleanest fix: select the numeric column first. ::: df.groupby("region")["rev"].mean() sidesteps the whole issue.

df.groupby("region").mean(numeric_only=True)   # drops string cols on purpose
df.groupby("region")["rev"].mean()             # ✅ narrow first, no ambiguity

df.loc[1] on a frame indexed [10, 20, 30] — what happens? It raises a KeyError: .loc looks up the label 1, which does not exist. The programmer meant slot 1 — that's .iloc[1], which returns the row labelled 20. ::: .loc never falls back to position.

df = pd.DataFrame({"v":[10,20,30]}, index=[10,20,30])
df.loc[1]    # KeyError: label 1 not found
df.iloc[1]   # ✅ slot 1 → the row labelled 20

df.pivot(index="store", columns="month", values="sales") errors with "Index contains duplicate entries". Cause? Two rows share the same (store, month), so pivot can't place two sales figures in one cell. Fix: pivot_table(..., aggfunc="sum") to combine them. ::: Duplicate cell → you must aggregate.

.loc slicing inclusive of its endpoint while .iloc is exclusive? Labels aren't guaranteed to be numbers, so "stop minus one" is meaningless — pandas therefore includes the named endpoint. .iloc uses positions (slots), which are genuine integers, so it follows Python's usual "stop excluded" rule. ::: Different address types demand different slice rules.

Because split-apply-combine stitches one answer per group back together, and the natural label for each answer is the key that defined the group. So groupby("region").mean() is indexed by region. ::: The key that split the data becomes the label of the summary.

NaN even though you kept all of the left table? how="left" keeps every left key, but for a left key with no match on the right, the right-hand columns have nothing to supply — so pandas fills them with NaN. ::: NaN marks "kept the row, found no partner".

df.groupby("k").apply(f) sometimes run f on all columns and sometimes one? Because apply hands f whatever slice you grouped: groupby("k") gives a full sub-DataFrame per group, while groupby("k")["rev"] gives a single Series per group. The shape of f's input depends on that upstream selection. ::: Select the column before apply to control what f sees.

For each key, merge forms the Cartesian product of matching rows on each side. Three left rows and two right rows sharing a key produce 3 × 2 = 6 output rows. ::: Duplicate keys multiply, they don't just line up.

a + b give for a label present in a but missing in b? NaN — the union of indices includes that label, but b has no value there, so there is no partner to add. This is alignment doing its job, not an error. ::: Missing partner → NaN, by design.

df.groupby("k").mean() do with a group that contains NaN values? By default .mean() skips NaN (skipna=True), averaging only the present values in that group. A group that is entirely NaN returns NaN. ::: Reductions ignore missing values unless told otherwise.

no key values at all? An empty DataFrame with the correct columns but zero rows — the intersection of keys is empty, so nothing survives. ::: Empty-but-well-typed, not an error.

groupby a key column that contains NaN in some rows? Rows whose key is NaN are dropped from the grouping by default (dropna=True) — they belong to no group. Pass dropna=False to keep a dedicated NaN group. ::: A missing key means "no team", so those rows sit out.

pivot_table on data where a (row, col) cell has no observations — what fills it? NaN, because there is nothing to aggregate for that combination. You can override with fill_value=0 to display zero instead. ::: Absent combinations become NaN, not an error.

df[[]] (empty column list) or df.loc[[]] (empty row list) — legal? Yes — both return an empty-but-valid DataFrame preserving the other axis. df[[]] keeps all rows with no columns; df.loc[[]] keeps all columns with no rows. ::: Empty selections are valid degenerate cases, useful for building up filters.