The perceptron model and history — Hinglish notes

3.1.1 · AI-ML › Neural Network Fundamentals

Overview

Perceptron sabse simple artificial neuron hai aur sabhi modern neural networks ki foundation hai. Frank Rosenblatt ne isse 1957 mein banaya tha, aur yeh pehla algorithm tha jo data se seekh sakta tha aur inputs ko do categories mein classify kar sakta tha. Perceptron ko samajhna zaroori hai kyunki har deep learning architecture—simple feedforward networks se lekar transformers tak—yahin se introduce hue fundamental ideas par build karta hai.

Historical Context

Timeline:

1943: McCulloch & Pitts biological neurons ka mathematical model propose karte hain
1957: Rosenblatt Cornell Aeronautical Laboratory mein perceptron invent karta hai
1958: Pehla hardware implementation (Mark I Perceptron) — simple patterns recognize kar sakta tha
1969: Minsky & Papert "Perceptrons" publish karte hain, fundamental limitations dikhate hain (XOR solve nahin ho sakta)
mid-1970s: Pehla "AI Winter" shuru hota hai — major funding cuts aur neural-network research mein bhari girawat
late 1980s: Early expert-system hype khatam hone ke baad doosra AI Winter aata hai
1986: Backpropagation (Rumelhart, Hinton, Williams ka popularization) multi-layer training solve karke neural networks ko dobara zinda karta hai
Aaj: Perceptrons billion-parameter models ke andar building blocks hain

AI Winter isliye aaya kyunki perceptron ki limitations ko galat tarike se interpret kiya gaya — logon ne socha ki neural networks kabhi kaam nahin karenge, jabki unhe sirf multiple layers ki zaroorat thi.

The Perceptron Model

Jahan:

$x = [x_{1}, x_{2}, \dots, x_{n}]$ input vector hai (features)
$w = [w_{1}, w_{2}, \dots, w_{n}]$ weights hain (har feature ki importance)
$b$ bias term hai (activation ke liye threshold)
$\overset{y}{^} \in {0, 1}$ prediction hai

Threshold convention (fixed): Is poore note mein hum $\overset{y}{^} = 1$ use karte hain jab $z \geq 0$ ho aur $\overset{y}{^} = 0$ jab $z < 0$ ho. Toh $z = 0$ ek tie hai jo class 1 par resolve hota hai, class 0 par nahin. Hum ise neeche consistently apply karte hain.

Biological Inspiration

Real neurons mein hote hain:

Dendrites (inputs) → Input features $x_{i}$
Synapses (connection strengths) → Weights $w_{i}$
Cell body (signals sum karta hai) → Weighted sum $\sum w_{i} x_{i} + b$
Axon (fire karta hai ya nahin) → Activation function (step function)

YEH MODEL KYUN? Biological neurons incoming signals ko sum karte hain, aur agar sum ek threshold se zyada ho jaaye, toh woh ek electrical spike "fire" karte hain. Perceptron mathematically is all-or-nothing behavior ko mimic karta hai.

Deriving the Perceptron from First Principles

GOAL: Ek aisa function banao jo data points ke do classes ko separate kare.

Step 1: Linear Combination Multiple inputs ko combine karne ka sabse basic tarika ek weighted sum hai: $z = w_{1} x_{1} + w_{2} x_{2} + \dots + w_{n} x_{n} + b$

KYUN? Har weight $w_{i}$ batata hai "is decision ke liye feature $i$ kitna important hai?" Positive weight = feature class 1 ki taraf push karta hai; negative weight = class 0 ki taraf push karta hai.

Step 2: Decision Boundary Humein continuous value $z$ ko binary decision mein convert karna hai. Sabse simple rule: $\overset{y}{^} = {10 if z \geq 0 if z < 0$

YEH THRESHOLD KYUN? Line $z = 0$ (jo $w \cdot x + b = 0$ ke equivalent hai) input space ko do half-spaces mein divide karti hai. Ek taraf ke saare points ko label 1 milta hai, doosri taraf ke sabko label 0 milta hai.

Step 3: Geometric Interpretation Equation $w \cdot x + b = 0$ ek hyperplane define karti hai (2D mein line, 3D mein plane, aur aage bhi). Weight vector $w$ is hyperplane ke perpendicular hota hai.

KYUN? Dot product $w \cdot x$ measure karta hai ki $x$ kitna $w$ ki direction mein point karta hai. Hyperplane ke ek taraf ke points ka dot product positive hota hai, doosri taraf ke points ka negative.

Equivalently (2D mein): $w_{1} x_{1} + w_{2} x_{2} + b = 0$ $x_{2} = - \frac{w _{1}}{w _{2}} x_{1} - \frac{b}{w _{2}}$

Yeh ek straight line hai jiska slope $- w_{1} / w_{2}$ hai aur y-intercept $- b / w_{2}$ hai.

The Perceptron Learning Algorithm

Perceptron KAISE seekhta hai? Galtiyan karne par weights adjust karke.

Jahan $η$ learning rate hai (typically 0.01 se 1 tak).

Derivation — YEH RULE KYUN?

Case 1: True label $y = 1$ lekin humne predict kiya $\overset{y}{^} = 0$ (false negative)

Error: $y - \overset{y}{^} = 1 - 0 = + 1$
Update: $w \leftarrow w + η x$
Effect: $w \cdot x$ badh jaata hai, agle baar prediction 1 hone ki zyada probability

Case 2: True label $y = 0$ lekin humne predict kiya $\overset{y}{^} = 1$ (false positive)

Error: $y - \overset{y}{^} = 0 - 1 = - 1$
Update: $w \leftarrow w - η x$
Effect: $w \cdot x$ ghata jaata hai, agle baar prediction 0 hone ki zyada probability

Case 3: Sahi prediction

Error: $y - \overset{y}{^} = 0$
Update: Koi change nahin (weights same rehte hain)

YEH CONVERGE KYUN KARTA HAI? Perceptron Convergence Theorem (1962) prove karta hai ki agar data linearly separable hai, toh algorithm finite steps mein ek separating hyperplane zaroor dhundh lega.

Worked Examples

Truth table (False=0, True=1 maan kar):

$x_{1}$	$x_{2}$	$y$ (AND)
0	0	0
0	1	0
1	0	0
1	1	1

Solution — Manually Weights Dhundhna:

Humein chahiye: $w_{1} x_{1} + w_{2} x_{2} + b \geq 0$ jab $x_{1} = x_{2} = 1$ ho, aur $< 0$ otherwise.

Try karo $w_{1} = 1, w_{2} = 1, b = - 1.5$ :

$(0, 0)$ : $1 (0) + 1 (0) - 1.5 = - 1.5 < 0$ → predicts 0 ✓
$(0, 1)$ : $1 (0) + 1 (1) - 1.5 = - 0.5 < 0$ → predicts 0 ✓
$(1, 0)$ : $1 (1) + 1 (0) - 1.5 = - 0.5 < 0$ → predicts 0 ✓
$(1, 1)$ : $1 (1) + 1 (1) - 1.5 = 0.5 \geq 0$ → predicts 1 ✓

YEH WEIGHTS KYUN? Har input ka weight ~1 hona zaroori hai kyunki activate karne ke liye dono "on" hone chahiye. Bias $- 1.5$ ensure karta hai ki akela ek input ( $z = 1 - 1.5 = - 0.5$ ) kaafi nahin hai.

Decision boundary: $x_{1} + x_{2} = 1.5$ (ek line jo (1,0)/(0,1) aur (1,1) ke beech se guzarti hai)

Try karo $w_{1} = 1, w_{2} = 1, b = - 0.5$ :

$(0, 0)$ : $- 0.5 < 0$ → predicts 0 ✓
$(0, 1)$ : $1 - 0.5 = 0.5 \geq 0$ → predicts 1 ✓
$(1, 0)$ : $1 - 0.5 = 0.5 \geq 0$ → predicts 1 ✓
$(1, 1)$ : $2 - 0.5 = 1.5 \geq 0$ → predicts 1 ✓

KYUN? Sirf ek input ka 1 hona kaafi hai, isliye bias $- 0.5$ ek lower threshold hai.

Email	$x_{1}$	$x_{2}$	$y$
1	2	5	1
2	0	1	0
3	3	7	1
4	0	0	0

Step-by-step training ( $η = 0.1$ , convention $\overset{y}{^} = 1$ jab $z \geq 0$ ):

Initialize karo: $w_{1} = 0, w_{2} = 0, b = 0$

Iteration 1, Email 1: $x = [2, 5], y = 1$

Prediction: $z = 0 (2) + 0 (5) + 0 = 0 \geq 0$ → $\overset{y}{^} = 1$ (hamare convention ke saath consistent)
Error: $1 - 1 = 0$ → koi update nahin (pehle se sahi hai)
KYUN? Kyunki $z = 0$ class 1 par resolve hota hai, aur true label bhi 1 hai, toh yeh example pehle hi pass mein sahi classify ho jaata hai.

Email 2: $x = [0, 1], y = 0$

Prediction: $z = 0 (0) + 0 (1) + 0 = 0 \geq 0$ → $\overset{y}{^} = 1$
Error: $0 - 1 = - 1$
Update: $w_{1} = 0 + 0.1 (- 1) (0) = 0$
Update: $w_{2} = 0 + 0.1 (- 1) (1) = - 0.1$
Update: $b = 0 + 0.1 (- 1) = - 0.1$
KYUN? Humne spam predict kiya lekin yeh spam nahin tha, toh "!" marks ka influence kam karo aur bias ko lower karo.

Email 3: $x = [3, 7], y = 1$

Prediction: $z = 0 (3) + (- 0.1) (7) + (- 0.1) = - 0.8 < 0$ → $\overset{y}{^} = 0$
Error: $1 - 0 = + 1$
Update: $w_{1} = 0 + 0.1 (1) (3) = 0.3$
Update: $w_{2} = - 0.1 + 0.1 (1) (7) = 0.6$
Update: $b = - 0.1 + 0.1 (1) = 0$
KYUN? Hum ek real spam email miss kar gaye, toh in high-count features ke liye weights ko positive direction mein strengthen karo.

Baaki examples aur multiple epochs ke liye tab tak continue karo jab tak ek full pass mein koi galti na ho...

Convergence ke baad: Perceptron in features ke basis par spam aur non-spam emails ko separate karne wali ek line dhundh leta hai.

Limitations and the XOR Problem

XOR truth table:

$x_{1}$	$x_{2}$	$y$ (XOR)
0	0	0
0	1	1
1	0	1
1	1	0

Classes separate karne wali ek line dhundhne ki koshish karo:

Points $(0, 1)$ aur $(1, 0)$ "1" side par hone chahiye
Points $(0, 0)$ aur $(1, 1)$ "0" side par hone chahiye

EK SINGLE STRAIGHT LINE SE YEH IMPOSSIBLE HAI! Koi bhi linear boundary in points ko separate nahin kar sakti.

Yeh galti sahi kyun lagti hai: "Perceptron AND aur OR seekh sakta hai, toh zaroor koi bhi logic function seekh sakta hoga?" Fix: XOR ke liye ek non-linear decision boundary chahiye (ek curve ya multiple lines). Solution: multi-layer perceptrons (hidden layers wale neural networks) XOR solve kar sakte hain.

Limitation ko sahi perspective mein dekhna: Perceptron "broken" nahin hai — woh woh problem solve kar raha hai jiske liye design kiya gaya tha (linearly separable data). Asli galti thi ek linear model se non-linear problems solve karne ki umeed rakhna.

Yeh sahi kyun lagta hai: Zyada training aksar noisy data ya poor initialization ke saath help karti hai.

Fix: Agar data linearly separable nahin hai, toh infinite training bhi kaam nahin karega. Perceptron convergence theorem tabhi apply hota hai jab koi solution exist karta ho. Pehle check karo ki tumhare problem ko non-linear model ki zaroorat hai ya nahin.

Modern Relevance

2026 mein perceptrons kyun padhein?

Building block: Modern networks (BERT, GPT, ResNet) ke har neuron mein ek perceptron hota hai jiske baad ek non-linear activation aata hai
Initialization: Linear separability samajhna debug karne mein help karta hai ki networks fail kyun hote hain
Interpretability: Perceptron weights hi ek fully interpretable neural model hain
SVMs: Support Vector Machines margin optimization wale sophisticated perceptrons hain
Historical literacy: Backpropagation ki appreciation nahin ho sakti jab tak yeh samajh na aaye ki usne kya solve kiya

Perceptron ki simplicity hi uski strength hai — yeh minimal viable learning algorithm hai.

Recall 12 Saal Ke Bachche Ko Samjhao

Imagine karo tum ek club ke bouncer ho, aur tumhe decide karna hai kaun andar aayega. Tumhare paas har insaan ke baare mein do cheezein hain: woh kitna bada lagta hai (feature 1) aur uske paas ticket hai ya nahin (feature 2).

Tum ek simple rule banate ho: "Agar (age × 3) + (has ticket × 10) 15 se zyada ho, toh andar jaane do."

Pehle tumhara rule bakwaas hai (shayad tumne 15 ki jagah 5 set kar diya), kuch bachche andar aa jaate hain aur kuch adults reject ho jaate hain. Har galti par tum apne numbers thoda adjust karte ho. Agar tumne kisi bachche ko galti se andar aane diya, toh rule ko strict banao (15 badhao). Agar kisi adult ko reject kar diya, toh thoda loosen karo.

Bahut logon ko check karne aur har baar adjust karne ke baad, tumhara rule bahut achha ho jaata hai! Bilkul aise hi ek perceptron seekhta hai — woh ek bouncer hai jo apne rules ko galtiyon ke basis par adjust karta hai.

XOR problem aisi hai: "Logon ko andar jaane do jo EITHER bade hain YA jinke paas ticket hai, LEKIN DONO NAHIN." Ek single straight-line rule yeh nahin kar sakta — tumhe do alag rules ya ek curved rule chahiye.

Key Formulas Summary

Formula	Purpose	Key Insight
$z = w \cdot x + b$	Weighted sum	Saare inputs ko importance weights ke saath combine karta hai
$\overset{y}{^} = 1$ if $z \geq 0$ else $0$	Activation	Sum ko binary decision mein convert karta hai
$w \leftarrow w + η (y - \overset{y}{^}) x$	Learning rule	Weights ko error ki direction mein correct karta hai
$w \cdot x + b = 0$	Decision boundary	Classes ko separate karne wala hyperplane

Connections

McCulloch-Pitts Neuron - 1943 ka predecessor jisme learning nahin thi
Linear Separability - Perceptron convergence ke liye mathematical condition
Activation Functions - Step function vs modern smooth functions
Gradient Descent - Modern learning algorithm jo perceptron rule ki jagah aaya
Multi-Layer Perceptron - Perceptrons ko stack karke XOR kaise solve hota hai
Support Vector Machines - Margin optimization wale perceptrons
Logistic Regression - Sigmoid activation aur probabilistic interpretation wala perceptron
Backpropagation - Woh algorithm jo AI winter ke baad neural networks ko dobara zinda kar gaya

#flashcards/ai-ml

Perceptron kya hota hai?

Sabse simple artificial neuron; ek binary linear classifier jo inputs ke weighted sum aur bias ko step function se pass karke data ko do categories mein classify karta hai.

Perceptron ke teen main components kya hain?

1) Weights (w) jo input importance scale karte hain, 2) Bias (b) jo decision threshold shift karta hai, 3) Activation function (step function) jo binary output produce karta hai.

Perceptron decision formula likho

\overset{y}{^} = 1

w \cdot x + b \geq 0

, else

\overset{y}{^} = 0

Perceptron learning rule kya hai?

w \leftarrow w + η (y - \overset{y}{^}) x

aur

b \leftarrow b + η (y - \overset{y}{^})

jahan

η

learning rate hai. Weights sirf galtiyon par update hote hain, error kam karne wali direction mein move karte hue.

Weight vector w geometrically kya represent karta hai?

Woh vector jo decision boundary hyperplane ke perpendicular (normal) hota hai. Yeh class 1 wali direction mein point karta hai.

Linear separability kya hota hai?

Ek dataset linearly separable hota hai agar koi ek hyperplane exist karta ho jo ek class ke saare examples ko doosri class ke saare examples se perfectly separate kar sake.

Perceptron Convergence Theorem bolo

Agar training data linearly separable hai, toh perceptron learning algorithm finite steps mein ek solution (converge) dhundh lene ki guarantee deta hai.

Ek single perceptron XOR kyun solve nahin kar sakta?

XOR linearly separable nahin hai — koi single straight line true cases (0,1) aur (1,0) ko false cases (0,0) aur (1,1) se separate nahin kar sakti. Iske liye non-linear decision boundary chahiye.

XOR problem ne kaunsi historical event mein contribute kiya?

Minsky aur Papert ki "Perceptrons" (1969) ki publication, jo is limitation ko highlight karti thi, pehle AI Winter mein contribute ki, jo mid-1970s mein major funding cuts aur reduced neural-network research ke saath shuru hua.

Perceptron biological neurons se kaise related hai?

Inputs (dendrites) ko weight kiya jaata hai (synapse strengths), sum kiya jaata hai (cell body), aur agar sum threshold se zyada ho, toh neuron "fire" karta hai (axon) — jo step activation function se model hota hai.

Perceptron weight update mein kya hota hai jab true label 1 ho lekin prediction 0 ho?

y - \overset{y}{^} = 1

, toh

w \leftarrow w + η x

. Yeh dot product

w \cdot x

badhata hai, jisse perceptron future mein is input ke liye 1 predict karne ki zyada likelihood ho.

Perceptron decision boundary ki equation kya hai?

w \cdot x + b = 0

, jo ek hyperplane (2D mein line, 3D mein plane) define karta hai jo do classes ko separate karta hai.

Bias term b ka kya role hai?

Bias decision boundary ko origin se door shift karta hai. Bias ke bina, hyperplane ko (0,0) se pass karna padta hai, jo expressiveness ko bahut limit kar deta hai.

Perceptron galtiyon par hi weights kyun update karta hai?

Agar prediction sahi hai (

y = \overset{y}{^}

), toh

y - \overset{y}{^} = 0

, isliye update term zero hota hai. Yeh efficient hai — pehle se sahi classify hue examples par koi computation waste nahin hoti.

Perceptron training mein learning rate η kya hoti hai?

Ek hyperparameter (typically 0.01 se 1 tak) jo weight updates ki step size control karta hai. Bahut badi hogi toh oscillation hogi; bahut choti hogi toh convergence slow hogi.

Weights [1, 1] aur bias -1.5 wala perceptron kaunsa problem solve kar sakta hai?

AND gate. Decision boundary

x_{1} + x_{2} = 1.5

require karta hai ki 1 output karne ke liye dono inputs 1 hon.

Frank Rosenblatt ne perceptron kab invent kiya?

1957 mein Cornell Aeronautical Laboratory mein. Pehla hardware implementation (Mark I Perceptron) 1958 mein banaya gaya tha.