Pandas DataFrames and Series basics: Worked examples

This page is a drill. The parent note built the ideas: a Series is a labelled 1-D column, a DataFrame is a stack of Series that all share one row-index. Here we throw every kind of input at those ideas — clean data, missing data, mismatched labels, empty frames, the lot — and work each one to the end.

Every code block below assumes these two imports at the top of the file:

import pandas as pd
import numpy as np

pd is the nickname for pandas, np for numpy — the library that supplies np.nan, the missing-value marker.

Before line one, one reminder in plain words:

index = the row names (like the leftmost column in a spreadsheet you cannot type into by accident).
columns = the names across the top.
NaN = "Not a Number", Pandas' way of writing a hole where a value should be. It is a real float value that means "missing"; in code it is written np.nan.
dtype = the type Pandas decided a column holds: int64 (whole numbers), float64 (decimals, and the only type that can hold NaN), object (text or mixed). In code the dtype prints as the string 'int64', 'float64', etc.

Prereq refreshers if any word above feels new: 1.4.01-NumPy-arrays-and-vectorization, 1.4.02-Python-data-structures.

The scenario matrix

Every example below is tagged with the cell of this table it covers. By the last example every cell is filled.

Cell	Case class	What makes it tricky
A	Clean construction	baseline — build & read shape
B	Label vs position access	`.loc` vs `.iloc` disagree
C	Missing keys / ragged rows	holes become `NaN`, dtype flips
D	Index misalignment in arithmetic	two Series, different labels
E	Boolean filter — normal + empty result	condition true for none
F	Degenerate inputs	empty frame, single element, all-NaN
G	dtype edge — int column touched by NaN	silent `int64 → float64`
H	Real-world word problem	sales, group-and-average
I	Exam twist	`df[1]` vs `df.iloc[1]` trap

Each numbered case class maps to at least one worked example (A→Ex1, B→Ex2, C→Ex3, D→Ex4, E→Ex5, F→Ex6, G→Ex7, H→Ex8, I→Ex9).

Ex1 — Clean construction (Cell A)

Worked example Build a DataFrame and read its shape

data = {'name': ['Alice','Bob','Charlie'],
        'score': [85, 92, 78]}
df = pd.DataFrame(data)

Forecast: guess df.shape and df.index before reading on.

Step 1 — count rows and columns. Two keys → 2 columns. Each list has length 3 → 3 rows. Why this step? Shape is always (n_rows, n_cols) = (len of a column, number of keys). $shape = (3, 2)$

Step 2 — read the auto index. We gave no index=, so Pandas invents 0,1,2. Why this step? Every row needs a label; with none supplied Pandas uses a RangeIndex starting at 0.

Verify: df.shape == (3, 2) and list(df.index) == [0,1,2]. ✅ 3 rows counted, 2 keys counted.

Look at the figure: the dict becomes two vertical Series glued to one shared index column.

Figure — Pandas DataFrames and Series basics

Ex2 — Label vs position access (Cell B)

Worked example

.loc and .iloc give different answers on purpose

df = pd.DataFrame(
    {'score':[85,92,78]},
    index=['s3','s1','s2'])   # note: NOT sorted, NOT 0-based

Forecast: what is df.loc['s1','score']? what is df.iloc[1,0]? Same or different?

Step 1 — .loc uses the name. .loc['s1', ...] hunts for the row labelled 's1'. That is the second row (position 1), value 92. Why this step? .loc is a hash lookup on the index labels; order in the table is irrelevant.

Step 2 — .iloc uses the position. .iloc[1,0] means "second row, first column" by counting, ignoring names. Second row is s1, value 92. Why this step? .iloc indexes the raw NumPy array underneath — pure counting from 0.

Step 3 — make them disagree, right now. Ask two different questions of the SAME row. .loc['s3','score'] reads the row named 's3' — which sits at the top (position 0) — giving 85. But .iloc[0,0] reads position 0, also 85: they happen to agree here. Now the sharp one: .loc['s2','score'] reads the row named 's2' = 78 (bottom row, position 2), whereas .iloc[1,0] reads position 1 = 92. Same-looking indices, different answers — .loc['s2']→78 but .iloc[1]→92, because the labels s3,s1,s2 are not in position order. Why this step? This is the whole point: once the index is not 0,1,2 in order, "the row called X" and "the row at position X" part ways. Ex9 shows the nastier version where the labels are numbers.

Verify: df.loc['s1','score'] == 92, df.iloc[1,0] == 92, and the disagreement df.loc['s2','score'] == 78 while df.iloc[1,0] == 92. ✅

Ex3 — Ragged list-of-dicts → holes become NaN (Cell C)

Worked example One record is missing a key

records = [{'name':'Alice','score':85},
           {'name':'Bob'}]          # no 'score'
df = pd.DataFrame(records)

Forecast: what fills Bob's score, and what dtype is the score column now?

Step 1 — columns = union of all keys. Keys seen anywhere: name, score. So 2 columns. Why this step? Row-oriented data lets each record differ; Pandas takes the union so no data is lost.

Step 2 — the missing cell. Bob has no score, so Pandas writes NaN at df.loc[1,'score']. Why this step? A rectangle must be filled; the "no value" placeholder is NaN (written np.nan).

Step 3 — dtype flips to float. NaN is a float. A column holding 85 and NaN cannot stay int64, so the whole column becomes float64; Alice's 85 is now 85.0. Why this step? int64 has no way to store "missing"; only float64 does. Pandas upcasts the entire column.

Verify: df.loc[0,'score'] == 85.0, pd.isna(df.loc[1,'score']) == True, and df['score'].dtype == 'float64'. ✅

Ex4 — Index misalignment in arithmetic (Cell D)

Worked example Adding two Series with different labels

a = pd.Series({'x':1,'y':2,'z':3})
b = pd.Series({'y':10,'z':20,'w':30})
total = a + b

Forecast: what is total['x']? what is total['y']? which labels appear?

Step 1 — align by label, not by position. Pandas lines up 'y' with 'y', 'z' with 'z' — never position with position. Look at the figure below: the blue column (a) and orange column (b) are matched label to label, and the dashed grey cells mark a label that lives in only one of them. Why this step? Labels stick to data; that is the whole promise of Pandas. a's y (=2) meets b's y (=10).

Step 2 — the shared labels add cleanly. $total [^{'} y^{'}] = 2 + 10 = 12, total [^{'} z^{'}] = 3 + 20 = 23.$ In the figure these are the two green result cells with real numbers.

Step 3 — labels present in only one Series → NaN. 'x' exists only in a, 'w' only in b. Each has no partner, so the sum is unknown → NaN — the red result cells in the figure. Why this step? $1 + (missing)$ is not 1; it is undefined, so Pandas returns NaN (use a.add(b, fill_value=0) if you want 1).

Verify: total['y']==12, total['z']==23, pd.isna(total['x']), pd.isna(total['w']). ✅

Ex5 — Boolean filter, including the empty result (Cell E)

Worked example Filter rows, then a filter that matches nobody

df = pd.DataFrame({'score':[85,92,78,60]})
passed   = df[df['score'] >= 80]
perfect  = df[df['score'] == 100]   # nobody scored 100

Forecast: how many rows in passed? how many in perfect?

Step 1 — the condition is a Boolean Series. df['score'] >= 80 → [True, True, False, False], sharing the same index as df. Why this step? Filtering works by aligning a mask; the mask must have the same labels as the frame.

Step 2 — keep the True rows. df[mask] returns rows where the mask is True: positions 0 and 1 → 2 rows (85, 92). Why this step? Boolean indexing selects, it never reorders.

Step 3 — empty case is legal, not an error. Nobody scored 100 → mask all False → perfect is an empty DataFrame with shape == (0, 1). Columns survive, rows vanish. Why this step? An empty result is a valid answer to a question; downstream .mean() on it gives NaN, not a crash.

Verify: passed.shape[0] == 2 and perfect.shape == (0,1). ✅

Ex6 — Degenerate inputs (Cell F)

Worked example Empty frame, single-value Series, all-NaN column

empty  = pd.DataFrame()
one    = pd.Series([42], index=['only'])
holes  = pd.Series([np.nan, np.nan])

Forecast: empty.shape? one.mean()? holes.mean()?

Step 1 — empty frame. No rows, no columns → shape == (0, 0). Why this step? The degenerate table still obeys (n_rows, n_cols); both are zero.

Step 2 — single element mean. Mean of one number is that number: one.mean() == 42.0. Why this step? $\frac{42}{1} = 42$ ; there is no special case, the formula just has $n = 1$ .

Step 3 — all-NaN mean skips NaN by default. .mean() ignores NaN. With every value missing there is nothing to average → result NaN (not 0). Why this step? Averaging zero real numbers is undefined; Pandas returns NaN rather than dividing by zero.

Verify: empty.shape==(0,0), one.mean()==42.0, pd.isna(holes.mean()). ✅

Ex7 — dtype edge: int column silently becomes float (Cell G)

Worked example Assigning NaN into an int column

df = pd.DataFrame({'count':[1,2,3]})   # starts int64
before = str(df['count'].dtype)
df.loc[1,'count'] = np.nan
after = str(df['count'].dtype)

Forecast: what is before? what is after? what is df.loc[0,'count'] afterwards?

Step 1 — clean start. All whole numbers → before == 'int64'. Why this step? With no missing values and no decimals Pandas picks the compact integer type.

Step 2 — inject a hole. Writing np.nan into one cell forces the column to a type that can store NaN. Why this step? int64 cannot represent "missing"; only float64 can.

Step 3 — the whole column upcasts. after == 'float64', and the untouched 1 is now stored as 1.0. Why this step? A column has ONE dtype. One float forces all of them to float.

Verify: before=='int64', after=='float64', df.loc[0,'count']==1.0. ✅

Ex8 — Real-world word problem (Cell H)

Worked example Average monthly sales per region

Two regions, three months each. Find each region's mean sales. Sales are in rupees (the ₹ symbol just means "rupees", the currency — treat it as a plain unit tag on the numbers).

df = pd.DataFrame({
    'region': ['N','N','N','S','S','S'],
    'sales' : [100, 200, 300, 40, 50, 60]})
means = df.groupby('region')['sales'].mean()

Forecast: guess means['N'] and means['S'] in rupees, keeping the unit.

Step 1 — split by the key. groupby('region') gathers rows sharing each region label: N-group = {100,200,300}, S-group = {40,50,60}. Why this step? "Per region" means we must partition before we summarise.

Step 2 — average each group. $N = \frac{100 + 200 + 300}{3} = 200, S = \frac{40 + 50 + 60}{3} = 50.$ Why this step? Mean = sum ÷ count, applied inside each group independently.

Step 3 — result is a Series indexed by region. means has index ['N','S'] and values [200, 50] (rupees per month). Why this step? One number per group → the group labels become the new index.

Verify (units + sanity): sales in rupees, mean stays in rupees. means['N']==200.0, means['S']==50.0. Sanity: 200 sits inside [100,300] ✅, 50 inside [40,60] ✅. Grand mean of all six = (600+150)/6 = 125, and $\frac{200 \cdot 3 + 50 \cdot 3}{6} = 125$ agrees. ✅

Related next step: turning group means into model inputs — 2.1.03-Feature-engineering-basics; plotting them — 1.5.01-Data-visualizationmatplotlib-basics; cleaning the raw table first — 1.4.07-Pandas-data-cleaningand-manipulation.

Ex9 — Exam twist: `df[1]` vs `df.iloc[1]` (Cell I)

Worked example The classic trap

df = pd.DataFrame({'a':[10,20,30]}, index=[5,6,7])

Forecast: does df[1] return the second row? What does df[1] try to do?

Step 1 — df[key] selects a column, not a row. df[1] asks for the column named 1. There is no such column → KeyError. Why this step? Square-bracket-on-a-DataFrame is column access (dict-of-Series), never positional row access.

Step 2 — position needs .iloc. df.iloc[1] = the second row = the row whose label is 6, value a=20. Why this step? .iloc counts from 0 regardless of the strange [5,6,7] labels.

Step 3 — label needs .loc. df.loc[6, 'a'] also returns 20, but it found it by the name 6, not by counting. Why this step? Here the label 6 and the position 1 point to the same row — but .loc[1] would raise KeyError (no row labelled 1). Different tools, different questions.

Verify: df.iloc[1,0]==20 and df.loc[6,'a']==20; and asking for column 1 raises KeyError. ✅

Recall Rapid self-test

df[x] on a DataFrame selects a ___ (row/column)? ::: column Which of .loc / .iloc uses names? ::: .loc What dtype can hold NaN? ::: float64 (not int64) Adding two Series, a label present in only one gives ___? ::: NaN Mean of an all-NaN Series is ___? ::: NaN

Coverage check

Recall Every matrix cell hit?

A→Ex1, B→Ex2, C→Ex3, D→Ex4, E→Ex5, F→Ex6, G→Ex7, H→Ex8, I→Ex9. All nine cells covered ✅