Types of data (numerical, categorical, ordinal, text)

2.1.1 · AI-ML › Data Preprocessing & Feature Engineering

Overview

Data types ko samajhna feature engineering ki buniyad hai. Har ML algorithm apne input data ke baare mein kuch assumptions karta hai—galat type do toh model ya toh crash karega ya chup-chaap bakwaas results dega. Yeh note explain karta hai ki har data type kya represent karta hai, kyun yeh distinction algorithms ke liye matter karti hai, aur kaise har type ko pehchano aur handle karo.

Core Data Types

1. Numerical Data

WHY yeh distinction matter karti hai:

• Continuous data → gradient-based models ke liye normalize/standardize karo (large-scale features ko dominate karne se rokta hai)
• Discrete data → kabhi kabhi better hota hai categorical treat karna agar values kam hoon (e.g., "number of bedrooms: 1, 2, 3" tree models ke liye one-hot encoded better kaam kare)

Hum data ko transform karna chahte hain taaki uska mean = 0, standard deviation = 1 ho.

Step 1: Mean hatao (centering) $x_{\text{centered}}=x-\mu$ jahan $\mu =\frac{1}{n}{\sum }_{i=1}^n{x}_i$ mean hai.

WHY? Centering distribution ko shift karta hai taaki uska center zero par aa jaye. Isse location bias remove hoti hai.

Step 2: Standard deviation se scale karo $x_{\text{scaled}}=\frac{x-\mu }{\sigma }$ jahan $\sigma =\sqrt{\frac{1}{n}{\sum }_{i=1}^n(x_i-\mu {)}^2}$ standard deviation hai.

WHY? $\sigma$ se divide karne par spread (variance) 1 ho jaata hai. Ab saare features same scale par hain.

Kab use karo: Linear regression, logistic regression, SVMs, neural networks (koi bhi algorithm jo distance metrics ya gradient descent use kare).

Step 1: Mean calculate karo $\mu =\frac{50+75+60+90+55}{5}=\frac{330}{5}=66$

WHY yeh step? Hume distribution ka center chahiye taaki use zero par shift kar sakein.

Step 2: Standard deviation calculate karo $\sigma =\sqrt{\frac{(50-66{)}^2+(75-66{)}^2+(60-66{)}^2+(90-66{)}^2+(55-66{)}^2}{5}}$ $=\sqrt{\frac{256+81+36+576+121}{5}}=\sqrt{\frac{1070}{5}}=\sqrt{214}\approx 14.63$

WHY yeh step? Standard deviation spread measure karta hai. Hum ise scale normalize karne ke liye use karte hain.

Step 3: Har value ko scale karo $x_1=\frac{50-66}{14.63}=\frac{-16}{14.63}\approx -1.09$ $x_2=\frac{75-66}{14.63}=\frac{9}{14.63}\approx 0.62$ $x_3=\frac{60-66}{14.63}\approx -0.41$ $x_4=\frac{90-66}{14.63}\approx 1.64$ $x_5=\frac{55-66}{14.63}\approx -0.75$

WHY yeh step? Ab har value represent karta hai "mean se kitne standard deviations door"—ek universal scale.

Result: Scaled values: [-1.09, 0.62, -0.41, 1.64, -0.75] Saari values ab mean ≈ 0, std ≈ 1 hain, jo unhe doosre scaled features ke saath comparable banata hai.

2. Categorical Data

KEY INSIGHT: Tum "colors ka mean" ya "USA - India" calculate nahi kar sakte. Yeh labels hain, numbers nahi.

Problem: ML algorithms ko numbers chahiye. Categories ko numbers mein kaise convert karein bina fake order imply kiye?

Solution: Har category ke liye ek binary column banao.

Step 1: Saari unique categories list karo Categories: {Cat A, Cat B, Cat C}

Step 2: Har sample ke liye ek binary vector banao

• Agar sample Cat A hai: [1, 0, 0]
• Agar sample Cat B hai: [0, 1, 0]
• Agar sample Cat C hai: [0, 0, 1]

WHY yeh kaam karta hai: Har category ko apna dimension milta hai. Koi bhi category numeric space mein doosre ke "paas" nahi hoti. Koi bhi do one-hot vectors ke beech distance same hota hai (Euclidean space mein √2).

Mathematical representation: $k$ categories ke liye, category $i$ ko encode karo as: ${\mathbf{e}}_i=[0,0,\dots ,1,\dots ,0]\in {\mathbb{R}}^k$ jahan 1 position $i$ par hai.

Step 1: Unique categories identify karo Categories: {UPI, Credit Card, Cash} → 3 categories

WHY yeh step? Hume jaanna hai ki kitne binary columns banana hain.

Step 2: Binary columns banao Columns banao: is_UPI, is_CreditCard, is_Cash

Step 3: Har row encode karo

WHY yeh encoding?

• Transaction 1 UPI hai → UPI column on karo, baaki off karo
• Koi bhi category numerically doosre ke "beech" nahi hai
• UPI se Cash ki distance = UPI se Credit Card ki distance = √2

Tree models ke liye alternative: Label encoding (UPI=0, Credit Card=1, Cash=2) theek kaam karta hai kyunki trees individual values par split karte hain, distances par nahi.

Kyun sahi lagta hai: "Maine text ko numbers mein convert kar diya, model handle kar lega!"

Kyun galat hai: Model ab sochta hai ki Cash (2) Credit Card (1) se "double" hai, aur Credit Card UPI aur Cash ke "beech" hai. Woh $w\cdot 2$ jaisi weights seekhega Cash ke liye, jo aisi mathematical relationships imply karta hai jo exist hi nahi karti.

Fix: One-hot encoding use karo algorithms ke liye jo numeric relationships assume karte hain (linear models, neural nets). Label encoding SIRF tree-based models ke liye use karo (jo "is equal to X" par split karte hain) ya ordinal data ke liye.

Steel-man the mistake: Label encoding memory bachata hai (1 column vs. k columns) aur kaam kar sakta hai agar model fake ordering ignore karna seekh le. Lekin yeh risky hai—jab one-hot safer hai toh bias kyun introduce karo?

3. Ordinal Data

KEY DISTINCTION categorical se: Order matter karta hai. PhD > Master's meaningful hai. Lekin High School se Bachelor's ka "gap" ≠ Master's se PhD ka gap.

Constraint: Hume order preserve karna hai: ${\text{category}}_1<{\text{category}}_2<\dots <{\text{category}}_k$

Solution: Categories ko integers mein map karo order respect karte hue: $f:\{{\text{cat}}_1,{\text{cat}}_2,\dots ,{\text{cat}}_k\}\to \{0,1,\dots ,k-1\}$ jahan ${\text{cat}}_i<{\text{cat}}_j\Rightarrow f({\text{cat}}_i)<f({\text{cat}}_j)$

WHY yeh kaam karta hai: Algorithms jo $<,>$ comparisons use karte hain (tree splits: "is education ≥ 2?") ordering preserve karte hain. Distance-based algorithms (KNN, linear models) uniform spacing assume karenge—acceptable hai agar ordinal gaps roughly similar hain, nahi toh problematic hai.

Step 1: Ordering define karo High School < Bachelor's < Master's < PhD

WHY yeh step? Encoding se pehle hume explicitly order specify karna hoga.

Step 2: Integer codes assign karo

• High School → 0
• Bachelor's → 1
• Master's → 2
• PhD → 3

Step 3: Encoding apply karo

WHY yeh encoding?

• Ek decision tree split kar sakta hai: "education_encoded ≥ 2" → Master's aur PhD capture karta hai
• Order preserved hai: 0 < 1 < 2 < 3 reality se match karta hai
• Linear model equal spacing assume karega (problem agar PhD "bahut zyada" impactful hai Master's se)

Spacing ke baare mein kab concern karo: Agar salary predict kar rahe ho aur PhD holders Bachelor's holders se 3× earn karte hain (3× increment nahi), toh consider karo:

• Multiple binary features banana: has_bachelors, has_masters, has_phd
• Target encoding use karna (har category ki mean target value se replace karo)

4. Text Data

KEY CHALLENGE: Text high-dimensional, variable-length, aur context-dependent hota hai. "Bank" (financial institution) vs. "river bank" ke liye context samajhna padta hai.

Goal: Ek document ko fixed-length vector of numbers mein convert karo.

Step 1: Vocabulary build karo (saare documents mein saare unique words ka set) $V=\{w_1,w_2,\dots ,w_n\}$

WHY? Hume ek fixed set of dimensions chahiye. Vocabulary size = number of dimensions.

Step 2: Har document $d$ ke liye, har word ke occurrences count karo ${\mathbf{x}}_d=[c_1,c_2,\dots ,c_n]$ jahan $c_i=\text{count of word }{w}_i\text{ in document }d$

WHY yeh kaam karta hai: Har document ${\mathbb{R}}^n$ space mein ek point ban jaata hai. Similar documents (similar words use karte hue) is space mein paas hote hain.

Mathematical form: Vocabulary $V$ of size $n$ ke liye, document $d$ hai: ${\mathbf{x}}_d\in {\mathbb{R}}^n,({\mathbf{x}}_d{)}_i=\#\{w_i\text{ appears in }d\}$

Limitation: Word order ignore karta hai. "Dog bites man" = "Man bites dog" (same BoW vector).

1. "The movie was good"
2. "The movie was bad"
3. "Great movie"

Step 1: Vocabulary build karo Unique words extract karo: {the, movie, was, good, bad, great} Vocabulary size = 6

WHY yeh step? Hamare feature space dimensions define karta hai.

Step 2: Har document ke liye word occurrences count karo

Document 1: "The movie was good"

• the: 1, movie: 1, was: 1, good: 1, bad: 0, great: 0
• Vector: [1, 1, 1, 1, 0, 0]

WHY yeh step? Text ko numbers mein convert karta hai counting ke zariye.

Document 2: "The movie was bad"

• the: 1, movie: 1, was: 1, good: 0, bad: 1, great: 0
• Vector: [1, 1, 1, 0, 1, 0]

Document 3: "Great movie"

• the: 0, movie: 1, was: 0, good: 0, bad: 0, great: 1
• Vector: [0, 1, 0, 0, 0, 1]

Result Matrix:

Interpretation:

• D1 aur D2 similar hain ("the movie was" share karte hain) → dot product = 3
• D3 different hai (alag words) → D1 ke saath dot product = 1

Real use ke liye next step: Stop words hatao ("the", "was"), common words ko down-weight karne ke liye TF-IDF apply karo.

Kyun sahi lagta hai: "Model unknown words ko ignore kar lega."

Kyun galat hai: Agar test set mein "excellent" extensively use hoti hai positive reviews indicate karne ke liye, toh model use all-zeros dekhta hai (missing signal). Performance drop hoti hai kyunki tum key information ke liye blind ho.

Fix:

1. Training ke dauran: Vocabulary mein ek <UNK> (unknown) token rakho
2. Testing ke dauran: Unseen words ko <UNK> se map karo
3. Better: Subword tokenization use karo (BPE, WordPiece) taaki "excellent" "excell" + "ent" mein toot jaaye, learned pieces reuse karte hue
4. Best: Pre-trained embeddings use karo (Word2Vec, BERT) jo already millions of words jaante hain

Steel-man the mistake: Unknowns ignore karna simple hai aur kabhi kabhi kaam karta hai agar test set training se similar ho. Lekin production mein language evolve hoti hai—nayi slang, product names, events. Unknowns handle karna robustness ke liye critical hai.

Advanced Considerations

80/20 Rule: 80% performance 4 core types ko correctly handle karne se aati hai. Yahan time lagao exotic feature engineering se pehle.

Data Type Decision Tree

Is the data text in sentences?
├─ YES → Text Data (BoW, TF-IDF, embedings)
└─ NO → Is it numbers?
    ├─ YES → Can you compute meaningful average?
    │   ├─ YES → Numerical (continuous or discrete)
    │   └─ NO → Probably categorical (e.g., zip codes look numeric but aren't)
    └─ NO → Is there a meaningful order?
        ├─ YES → Ordinal (education, ratings)
        └─ NO → Categorical (colors, countries)

Connections

• Normalization and Scaling → Min-max, z-score, robust scaling mein deep dive
• One-Hot Encoding vs Label Encoding → Kaunsi encoding strategy kab use karni hai
• TF-IDF and Text Vectorization → Advanced text feature extraction
• Feature Engineering from Datetime → Timestamps se features extract karna
• Handling High-Cardinality Categorical → Embeddings, target encoding, hashing
• Curse of Dimensionality → Kyun one-hot encoding feature space explode kar sakti hai
• Feature Scaling Impact on Gradient Descent → Mathematical proof ki scaling kyun matter karti hai

Numerical data pencils count karne jaisi hai—tum keh sakte ho "mere paas 5 pencils hain" aur math kar sakte ho (3 aur jodne par = 8 total). Tum apni class ka average nikaal sakte ho.

Categorical data tumhari pencil colors jaisi hai—red, blue, green. Tum nahi keh sakte "red + blue = ?" ya "average color." Yeh bas alag types hain, koi math nahi.

Ordinal data medal rankings jaisi hai: bronze < silver < gold. Ek order hai (gold behtar hai), lekin tum nahi keh sakte "gold exactly silver se double as good hai." Order matter karta hai, lekin ranks ke beech spacing equal nahi hoti.

Text data tumhari diary entries jaisi hai—poore sentences aur paragraphs. Computers English seedhi nahi samajhte, isliye hume pehle use numbers mein convert karna padta hai. Hum count kar sakte hain ki words kitni baar aate hain: "homework" 3 baar aata hai, "fun" 0 baar aata hai (sadly!).

Yeh kyun matter karta hai? Agar tum calculator ko words do, woh toot jaata hai. Agar tum apni pencil colors ko numbers ki tarah treat karo ("red = 1, blue = 2"), calculator sochta hai blue "twice" red hai, jo bakwaas hai. Data type ko tool se match karo!

#flashcards/ai-ml

ML mein char fundamental data types kaun se hain? :: Numerical (continuous/discrete), Categorical (no order), Ordinal (ordered categories), Text (natural language)

Concept Map