Jupyter notebooks workflow — Hinglish notes

1.4.9 · AI-ML › Python & Scientific Computing

What Problem Does Jupyter Solve?

Traditional Python scripts tumhe ek rigid cycle mein force karte hain:

Apna saara code likho
Poori file run karo
Terminal output ya log files check karo
Edit karo aur repeat karo

Ye data science ke liye painful hai kyunki:

Tum sirf ek line test karne ke liye expensive computations dobara run karne mein time waste karte ho
Script khatam hote hi intermediate results kho jaate hain
Tum data ko us code ke saath visualize nahi kar sakte jo usne create kiya
Collaborators tumhari reasoning process nahi dekh sakte

Jupyter ise cell-based execution se solve karta hai: har code block independently run hota hai, apne results memory mein rakhta hai. Tum apna analysis step-by-step build karte ho, outputs turant dekhte ho, zaroorat padne par backtrack karte ho, aur decisions inline document karte ho.

Core Workflow: The Cell Execution Model

How Jupyter Manages State

Jab tum ek Jupyter notebook launch karte ho:

Kernel starts: Ek fresh Python interpreter spin up hota hai
Persistent memory: Saari variables cell executions ke beech memory mein rehti hain
Out-of-order execution: Tum cells ko kisi bhi order mein run kar sakte ho (sirf top-to-bottom nahi)
Outputs are saved: Last run ke results .ipynb file mein store hote hain

The Five Cell Operations

Operation	Shortcut	What It Does	When to Use
Run cell	`Shift+Enter`	Current cell execute karo, next par jao	Normal workflow
Run in place	`Ctrl+Enter`	Execute karo, current cell mein raho	Same cell ko test/debug karna
Run all above	-	Top se current tak saare cells execute karo	Kernel restart ke baad state reproduce karna
Restart kernel	`00` (twice)	Python process kill karo, fresh start karo	Saari variables aur imports clear karna
Restart & run all	-	Restart + top se bottom tak execute karo	Verify karo ki notebook clean run hoti hai

Worked example

Example 1: Step-by-Step Analysis Build Karna Cell 1 (imports & data loading):

import pandas as pd
import numpy as np
data = pd.read_csv('sales.csv')
data.head()

Ye step kyun? Pehle dependencies aur data load karo. Kyunki data.head() cell mein last expression hai, iska output automatically cell ke neeche render hota hai, confirm karta hai ki file sahi se load hui.

Cell 2 (exploration):

display(data.describe())      # explicit display taaki render ho
data['revenue'].plot(kind='hist')  # last expression → plot dikhta hai

Ye step kyun? Ek cell mein sirf last expression auto-display hoti hai. describe() AUR histogram dono dikhane ke liye, hum describe() ko display(...) (Jupyter ki rich display function) mein wrap karte hain, jabki plot render hoti hai kyunki ye final line hai. Quick statistics aur visualization tumhe modeling se pehle data distribution samajhne mein help karte hain.

Cell 3 (feature engineering):

data['log_revenue'] = np.log1p(data['revenue'])
data['log_revenue'].plot(kind='hist')  # last expression → plot dikhta hai

Ye step kyun? Ek transformation test karna. Kyunki data abhi bhi Cell 1 se memory mein hai, tum ise modify kar sakte ho aur turant naya distribution dekh sakte ho. Plot render hoti hai kyunki ye last expression hai. Agar ye galat lagta hai, tum CSV dobara load kiye bina alag transformation ke saath re-run kar sakte ho.

Cell 4 (modeling):

from sklearn.linear_model import LinearRegression
X = data[['ad_spend', 'season']]
y = data['log_revenue']
model = LinearRegression().fit(X, y)
print(f"R² = {model.score(X, y):.3f}")

Ye step kyun? Model training alag ki gayi hai taaki tum Cell 3 mein features tweak kar sako aur libraries dobara import kiye ya data reload kiye bina re-train kar sako. Note karo ki hum yahan print() use karte hain—f-strings aur mid-cell logic ke andar, explicit print()/display() output dikhane ka reliable tarika hai.

Worked example

Example 2: Cell Execution Se Debugging

Tum ek neural network build kar rahe ho aur NaN losses aa rahi hain. Traditional script approach:

train.py mein jagah jagah print statements add karo
Poora training loop run karo (20 minutes)
Check karo kahan NaNs aate hain
Code edit karo, repeat karo

Jupyter approach: Cell 1: Model aur data load karo (ek baar run karo)

model = create_model()
train_loader = get_data()
batch = next(iter(train_loader))  # Ek batch lo

Cell 2: Forward pass test karo (kai baar run karo)

output = model(batch['input'])
print(f"Output range: {output.min():.3f} to {output.max():.3f}")
print(f"Any NaN? {torch.isnan(output).any()}")

Ye step kyun? Tum sirf ek batch ka forward pass test kar rahe ho. 20 minutes nahi, 1 second lagta hai. Tum discover karte ho ki outputs [-1e20, 1e20] hain—overflow!

Cell 3: Fix karo aur re-test karo (modify karo, Cell 2 dobara run karo)

model.final_layer = nn.Linear(512, 10)
torch.nn.init.xavier_uniform_(model.final_layer.weight)
# Ab Cell 2 dobara run karo verify karne ke liye

Ye step kyun? Tumne model memory mein change kiya. Cell 2 ka model variable same object ko reference karta hai, isliye Cell 2 dobara run karne par tumhara fix turant test ho jaata hai.

Workflow Best Practices

1. Cell Organization Principles

Formula

Cell Granularity Heuristic

Har cell ko ek conceptual kaam karna chahiye. Quantitatively: $Cell length \approx 5 - 15 lines$

$Execution time < 30 seconds (ideally < 5 s)$

Kyun?

Chhote cells = debug karna aasaan (jab cell fail ho, tum exactly jaante ho kya toota)
Fast cells = tum coffee break liye bina re-run kar sakte ho
Lambe cells aksar kai kaam karte hain, single-responsibility principle violate karte hain Derivation: Agar ek cell data load karti hai (2 min), use clean karti hai (3 min), aur ek model train karti hai (10 min), aur model fail ho jaata hai, tum 5 minutes un data steps ko dobara run karne mein waste karte ho jo tum jaante ho kaam karte hain. Teen cells mein split karo:

$Time to debug = time to re-run broken step + time to re-run dependencies$

Split cells ke saath, dependencies memory mein hain—sirf model cell dobara run karne ki cost sirf 10 min hai, 15 nahi.

Good cell structure:

# Cell 1: Imports
import pandas as pd
import matplotlib.pyplot as plt
 
# Cell 2: Load data
data = pd.read_csv('data.csv')
 
# Cell 3: Explore
data.describe()
 
# Cell 4: Clean
data_clean = data.dropna()
 
# Cell 5: Visualize
plt.scatter(data_clean['x'], data_clean['y'])

Bad cell structure:

# Ek giant cell
import pandas as pd
import matplotlib.pyplot as plt
data = pd.read_csv('data.csv')
data.describe()
data_clean = data.dropna()
plt.scatter(data_clean['x'], data_clean['y'])
# Agar plot fail ho jaaye, tum sab kuch dobara run karte ho including CSV load

2. Kernel Restart Hygiene

Common mistake

Mistake: "Mera notebook mere liye kaam karta hai lekin doosron ke liye nahi" Ye sahi kyun lagta hai: Tumne cells ko order 1→5→3→4→2 mein run kiya. Cell 2 ne Cell 4 mein define ki gayi variable use ki, jo memory mein thi. Notebook successfully save hui, toh tumhe lagta hai ye theek hai.

Trap: Jab tumhara collaborator top-to-bottom run karta hai, Cell 2 Cell 4 se pehle execute hoti hai, NameError cause karta hai. .ipynb file sirf last outputs save karti hai, execution order nahi jo unhe reproduce karne ke liye chahiye.

Fix: Share karne se pehle "Restart & Run All" karo. Ye ek fresh reader ke experience ko simulate karta hai. Agar koi cell fail hoti hai, tumne ek hidden dependency dhundh li hai.

Ye kyun kaam karta hai: Restart & Run All ye constraint enforce karta hai: $\forall i < j : cell_{i} must not depend on cell_{j}$

Ye single execution order hai jo doosron ke liye kaam karne ki guarantee hai.

3. Markdown Documentation Strategy

Markdown cells use karo taaki:

Explain why: # Why use log transform? Revenue is right-skewed, model assumes normality
Section headers: # Data Loading, # Feature Engineering, etc.
Decisions: # Chose RandomForest over XGBoost: RF trains faster, accuracy difference<1%

Worked example

Example 3: Modeling Decision Document Karna

## Model Selection
 
Validation set par teen models test kiye:
- Logistic Regression: 0.82 AUC
- Random Forest: 0.89 AUC
- XGBoost: 0.90 AUC
**Decision**: Random Forest use kar rahe hain thoda lower AUC ke bawajood.
**Reason**: XGBoost training 10x zyada time leti hai (30 min vs 3 min) sirf 1% improvement ke liye. Is weekly retraining pipeline ke liye, speed zyada matter karti hai.

Ye step kyun? Teen mahine baad, tum (ya tumhara teammate) sochega "Humne XGBoost kyun nahi use kiya?" Ye markdown cell tumhari reasoning preserve karti hai. Note karo ye ek markdown cell hai (code nahi), isliye saara text formatted prose ke roop mein render hota hai—koi execution needed nahi.

Advanced Workflow: Magics and Shell Commands

Jupyter mein special commands (magics) shamil hain jo % (line magic) ya %% (cell magic) se shuru hote hain:

Formula

Useful Magics

Magic	Purpose	Example
`%timeit`	Code benchmark karo	`%timeit sum(range(1000))` → "10 µs ± 0.5 µs"
`%%time`	Poora cell time karo	`%%time` first line par → execution time print karta hai
`%matplotlib inline`	Plots embed karo	Notebook mein plots dikhane ke liye ek baar run karo
`!`	Shell command	`!ls -lh data/` files list karta hai
`%load_ext`	Extension load karo	`%load_ext autoreload` modules auto-reloading ke liye
`%timeit` accuracy ki Derivation:

%timeit tumhara code $n$ times run karta hai aur $r$ repetitions ka best leta hai:

$time_{reported} = min_{i = 1}^{r} (\frac{1}{n} \sum_{j = 1}^{n} time_{i, j})$

Default: $r = 7$ , $n$ is tarah choose kiya jaata hai ki total time ≈ 0.2 seconds ho.

Min kyun aur mean kyun nahi? Minimum background processes ke noise ko filter karta hai. Tumhare code ki true speed woh hai jitni fast wo run kar sakta hai jab kuch interfere nahi karta.

Worked example

Example 4: Data Structure Choose Karne ke Liye Profiling

Tumhe fast membership testing chahiye. List ya set? Fair benchmark ke liye, dono containers mein same number of items hone chahiye (yahan 10 000)—warna comparison meaningless hai.

Cell 1:

data_list = list(range(10000))
data_set  = set(range(10000))   # List ke SAME size ka

Cell 2:

%timeit 9999 in data_list   # worst case: end tak scan karta hai

Output: ~90 µs ± 5 µs per loop

Cell 3:

%timeit 9999 in data_set

Output: ~40 ns ± 2 ns per loop

Ye step kyun? Identical sizes (10 000 each) ke saath, comparison fair hai, aur set lookup phir bhi hazaaron guna faster hai kyunki ye $O (1)$ hai (hash lookup) list ke $O (n)$ linear scan ke versus. Side-by-side %timeit tumhe hard numbers deta hai choice justify karne ke liye, aur tum in cells ko documentation ke roop mein rakh sakte ho.

Common Workflows

Data Science Project Typical Structure

1. Setup & Imports
2. Load Data
3. Exploratory Analysis (plots ke saath multiple cells)
4. Data Cleaning
5. Feature Engineering
6. Train-Test Split
7. Model Training
8. Evaluation
9. Error Analysis
10. Conclusions (markdown)

Prototyping → Production Workflow

Jupyter mein Prototype karo: Freely experiment karo, alag approaches try karo
Successful cells consolidate karo: Jab tum jaano kya kaam karta hai, related cells combine karo
.py mein Export karo: File → Download as → Python Script
Refactor karo: Notebook code ko functions mein convert karo, proper error handling add karo
Notebook ko documentation ki tarah rakho: Original notebook dikhata hai kyun tumne choices ki

Common mistake

Mistake: "Main baad mein apna notebook clean kar loonga" Ye sahi kyun lagta hai: Exploration ke dauran, tumhara notebook messy ho jaata hai—dead-end experiments, debugging cells, same analysis ke multiple versions. Tum sochte ho "Jab khatam ho jaaunga toh junk delete kar doonga." Trap: "Baad mein" kabhi nahi aata. Tum ek 200-cell notebook share karte ho jisme 50 failed experiments mix hote hain. Collaborators nahi bata sakte kya important hai.

Fix: Ek daily ya weekly "gardening" habit use karo:

Failed experiments jaise hi tum unhe abandon karo, delete karo
Jab tum naya approach start karo, markdown headers add karo
Agar kisi cell ki output important hai, toh ek markdown cell add karo jo explain kare kya show ho raha hai

Alternatively, jab tum koi approach finalize karo, sirf successful path ke saath ek naya notebook create karo, reference ke liye messy "lab notebook" se link karo.

Keyboard Shortcuts (Efficiency)

Mode	Shortcut	Action
Command mode (blue)	`A`	Upar cell insert karo
	`B`	Neeche cell insert karo
	`DD`	Cell delete karo
	`M`	Markdown mein convert karo
	`Y`	Code mein convert karo
Edit mode (green)	`Esc`	Command mode mein switch karo
	`Ctrl+/`	Comment/uncomment karo
Both	`Shift+Enter`	Run karo aur advance karo

Recall

Ek 12-saal ke bacche ko samjhao

Imagine karo tum math homework kar rahe ho, lekin galti hone par poora problem dobara karne ki zaroorat nahi, sirf ek step fix karo aur baaki saara kaam rakho. Yahi hai Jupyter!

Ek regular Python program ek poora essay ek baar mein likhne jaisa hai—agar page 5 par typo hai, toh tum saare 5 pages reprint karte ho. Jupyter mein har paragraph ek alag page par hota hai: tum page 3 edit kar sakte ho aur pages 1-2 aur 4-5 unchanged rakh sakte ho.

Har "page" ko cell kehte hain. Tum thoda sa code har cell mein dalte ho. Jab tum cell run karte ho, computer yaad rakhta hai kya hua. Phir tum agla cell likh sakte ho jo computer ne yaad rakha hai use karte hue. Agar kuch toot jaaye, tum sirf woh ek cell fix karte ho aur phir try karte ho—tumhara pehle ka kaam nahi jaata.

Tricky part: kyunki tum cells kisi bhi order mein run kar sakte ho, tum accidentally Step 5 pehle Step 2 se run kar sakte ho. Computer tumhe rokega nahi! Isliye ek "restart" button hota hai—ye Etch A Sketch ko shake karne jaisa hai sab kuch clear karne aur fresh start karne ke liye.

Connections

1.4.08-Virtual-environments: Dependencies isolate karne ke liye hamesha Jupyter ko virtual environment ke andar run karo
1.4.10-NumPy-arrays-basics: NumPy operations Jupyter mein array visualizations ke saath acchi tarah display hote hain
1.5.01-Pandas-DataFrames: DataFrames Jupyter mein HTML tables ke roop mein render hote hain, exploration natural banate hain
1.6.01-Matplotlib-basics: %matplotlib inline ke saath plots inline dikhte hain
2.3.01-Exploratory-data-analysis: EDA workflows Jupyter ke interactive model ke around design kiye gaye hain
4.1.05-Debugging-strategies: Jupyter ki cell execution step-by-step debugging ko natural banati hai

#flashcards/ai-ml

Jupyter notebook kya hai?

Ek interactive document (.ipynb) jo code cells, markdown text, aur outputs mix karta hai, browser mein Python kernel se connected hokar run karta hai

Jupyter aur Python script mein key difference kya hai?

Jupyter cell-based execution use karta hai persistent memory ke saath—tum code chunks mein run kar sakte ho aur intermediate results rakh sakte ho, scripts ke unlike jo ek baar top-to-bottom run hoti hain

Execution number In [5] tumhe kya batata hai?

Cell current kernel session mein 5th execute ki gayi thi (notebook mein uski position necessarily nahi)

Notebook share karne se pehle kernel restart kyun karo?

Ye verify karne ke liye ki ye top-to-bottom out-of-order execution se hidden dependencies ke bina run hoti hai

Multiple expressions wale code cell mein kaunsa auto-display hota hai?

Sirf last expression; pehle walo ko render karne ke liye explicit print() ya display() chahiye

Current cell run karne aur advance karne ka keyboard shortcut kya hai?

Shift+Enter

Current cell ke neeche cell kaise insert karo?

Command mode mein (blue border) B press karo

Recommended cell length kya hai?

5-15 lines jo ek conceptual kaam kare, ideally < 30 seconds execution time

%timeit kya karta hai?

Code ko kai baar run karke minimum average execution time report karke benchmark karta hai

%timeit mein mean ki jagah minimum kyun use karte hain?

Minimum background noise filter karta hai—tumhare code ki true speed woh hai jitni fast wo uninterrupted run kar sakta hai

List vs set lookup benchmark karte waqt dono containers same size kyun hone chahiye?

Warna comparison unfair hai—mismatched sizes measured speedup distort karti hain; sirf equal sizes O(1) vs O(n) difference isolate karte hain

Jupyter cell mein !ls kya karta hai?

Shell command ls run karta hai (! prefix shell commands execute karta hai)

Out-of-order execution ka khatre kya hai?

Baad wale cells pehle wale cells par depend kar sakte hain jo abhi run nahi hue, notebook ko doosron ke liye top-to-bottom fail banate hain

"Restart & Run All" kya verify karta hai?

Ki notebook hidden state dependencies ke bina cleanly top se bottom tak run hoti hai

Keyboard shortcuts ke saath cells insert/delete karne wala mode kaunsa hai?

Command mode (blue border, enter karne ke liye Esc press karo)

%matplotlib inline kya karta hai?

Matplotlib ko configure karta hai ki plots seedha notebook mein us cell ke neeche display hon jo unhe create karta hai