Worked examples — The perceptron model and history

This is a Deep Dive child of the perceptron topic note. The parent gave you the machinery: the weighted sum, the step decision, the update rule. Here we do the opposite of theory — we hammer the perceptron with every kind of input it can meet, one at a time, until no scenario surprises you.

Before anything else, let us re-anchor the three symbols so this page stands on its own.

Look at the figure. The blue arrow is $\mathbf{w}$. The gray line is the fence $z=0$. Everything shaded blue scores $z\ge 0$; everything shaded orange scores $z<0$. Every example below is just "which side did the point land on, and did we want it there?"

---

The scenario matrix

Here is every kind of situation a 2-input perceptron can be handed. Each example that follows is tagged with the cell it covers.

Cell	Scenario class	What's tricky about it
A	Point clearly on class-1 side ($z>0$)	baseline, should predict 1
B	Point clearly on class-0 side ($z<0$)	baseline, should predict 0
C	Point exactly on the line ($z=0$)	the tie-break — must give 1
D	Zero input $\mathbf{x}=[0,0]$	only the bias decides
E	Degenerate weights $\mathbf{w}=[0,0]$	there is no direction; bias sign alone rules (both $b<0$ and $b>0$)
F	A learning step, false positive ($y=0,\hat{y}=1$)	weights must move away from point
G	A learning step, false negative ($y=1,\hat{y}=0$)	weights must move toward point
H	Not-linearly-separable (XOR)	perceptron cannot converge — limiting failure
I	Real-world word problem	translate words → $\mathbf{x},\mathbf{w},b$
J	Exam twist: same line, flipped weights	negating $\mathbf{w},b$ swaps all classes

We now walk cells A→J. Together they touch every sign, the zero/degenerate inputs, the limiting failure, a word problem, and an exam trap.

---

Cells A, B, C — the three signs of $z$

Forecast: guess the label of each before reading. Which side of the arrow does each sit on?

1. Compute $z$ for P. $z=2(2)+(-1)(1)-1=4-1-1=2$. Why this step? The whole decision lives in the single number $z$; nothing else matters.
2. Read the sign. $z=2\ge 0\Rightarrow \hat{y}=1$. (cell A: clearly positive.) Why this step? The step function turns the continuous score into a label; a positive score means "front of the arrow", which is class 1 by definition.
3. Compute $z$ for Q. $z=2(0)+(-1)(3)-1=-4$. Sign is negative $\Rightarrow \hat{y}=0$. (cell B.) Why this step? Q is behind the arrow — negative score is exactly what "behind" feels like numerically.
4. Compute $z$ for R. $z=2(1)+(-1)(1)-1=0$. Exactly zero. Why this step? R lands on the fence. This is the case textbooks skip.
5. Apply the tie-break. $z=0\ge 0\Rightarrow \hat{y}=1$. R is class 1, not class 0. (cell C.) Why this step? Our fixed convention uses $\ge$, so a point on the line is forced to class 1 rather than left undefined — the boundary belongs to class 1.

Verify: P scored $+2$ (front of arrow → 1 ✓), Q scored $-4$ (behind → 0 ✓), R scored $0$ (fence → 1 by our fixed convention ✓). The next figure plots all three: P sits in the blue region, Q in the orange, R exactly on the gray line.

---

Cells D, E — degenerate inputs

Forecast: when the input is all zeros, what can possibly decide the answer? And when the weights are all zeros — does the sign of the bias flip the whole plane's label?

1. Case D: plug in the origin. $z=5(0)+(-3)(0)+0.4=0.4$. Why this step? Every weighted term dies because it's multiplied by $0$. Only $b$ survives.
2. Decide D. $z=0.4\ge 0\Rightarrow \hat{y}=1$. Why this matters: at the origin the perceptron is the bias. The sign of $b$ alone labels the origin.
3. Case E-neg: plug in the point with $b=-2$. $z=0(100)+0(-7)+(-2)=-2<0\Rightarrow \hat{y}=0$. Why this step? With $\mathbf{w}=[0,0]$ there is no "class-1 direction" — the arrow has vanished, so $z$ collapses to $b$ for every point.
4. Case E-pos: same point with $b=+2$. $z=0(100)+0(-7)+2=+2\ge 0\Rightarrow \hat{y}=1$. Why this matters: dead weights make a constant classifier, and the sign of the bias picks which constant. $b<0$ → everything is class 0; $b>0$ (or $b=0$, by the tie-break) → everything is class 1. The whole plane flips with the bias sign.

Verify: D gives $z=b=0.4$ (positive → 1 ✓, matches "origin's label is decided by $b$"). E-neg gives $z=b=-2$ for any $\mathbf{x}$ → always 0 ✓. E-pos gives $z=b=+2$ for the same $\mathbf{x}$ → always 1 ✓. All three degenerate cases collapse to "read the sign of the bias".

Forecast: the input is zero and the bias is zero — so $z$ is exactly $0$. Does the machine refuse to answer, or does the tie-break rescue it?

1. Compute $z$. $z=5(0)+(-3)(0)+0=0$. Why this step? Both weighted terms and the bias are zero, so this is the only place all three vanish at once — the purest tie.
2. Apply the tie-break. $z=0\ge 0\Rightarrow \hat{y}=1$. Why this matters: without the $\ge$ convention this point would be undefined; the fixed rule forces it to class 1, so the classifier is total (defined everywhere).

Verify: $z=0$ and $0\ge 0$, so $\hat{y}=1$ ✓ — the origin-on-the-fence resolves to class 1, exactly like point R in Example 1.

---

Cells F, G — one learning step each

Recall the update from the parent: $\mathbf{w}\leftarrow \mathbf{w}+\eta (y-\hat{y})\mathbf{x},b\leftarrow b+\eta (y-\hat{y})$ The factor $(y-\hat{y})$ is $+1$ on a false negative, $-1$ on a false positive, $0$ when correct. This is the engine of learning by error.

Forecast: the machine thinks this point is class 1 but it isn't. Which way should the arrow move — toward the point or away?

1. Predict. $z=0.5(3)+0.5(1)+0=2\ge 0\Rightarrow \hat{y}=1$. Why this step? We update only on mistakes, so we must first find out we made one.
2. Error signal. $y-\hat{y}=0-1=-1$. It's a false positive. Why: the sign of this number is the direction of the whole correction.
3. Update weights. $\mathbf{w}\leftarrow [0.5,0.5]+0.1(-1)[3,1]=[0.5-0.3,0.5-0.1]=[0.2,0.4]$. Why this step? Subtracting a multiple of $\mathbf{x}$ tilts the arrow away from $\mathbf{x}$, lowering its score next time.
4. Update bias. $b\leftarrow 0+0.1(-1)=-0.1$. Why: lowering $b$ shifts the whole fence so the point is more likely behind it.

Verify: re-score $\mathbf{x}$ with new params: $z=0.2(3)+0.4(1)-0.1=0.6+0.4-0.1=0.9$. Hmm — still positive, one step wasn't enough, but note the score fell from $2.0$ to $0.9$ — it moved the right way. That "score decreased" is the guaranteed effect of a false-positive step. ✓

Forecast: the machine says class 0, truth says class 1. Now which way does the arrow move?

1. Predict. $z=-0.2(1)+0.1(2)-0.5=-0.2+0.2-0.5=-0.5<0\Rightarrow \hat{y}=0$. Why this step? Just like cell F, learning only fires on an error, so we first evaluate the current label — here $z$ is negative, so the machine wrongly says class 0.
2. Error signal. $y-\hat{y}=1-0=+1$. False negative. Why: $+1$ means "pull toward the point."
3. Update weights. $\mathbf{w}\leftarrow [-0.2,0.1]+0.5(+1)[1,2]=[-0.2+0.5,0.1+1.0]=[0.3,1.1]$. Why this step? Adding $\eta \mathbf{x}$ swings the arrow toward $\mathbf{x}$, raising its score.
4. Update bias. $b\leftarrow -0.5+0.5(+1)=0$.

Verify: re-score: $z=0.3(1)+1.1(2)+0=0.3+2.2=2.5\ge 0\Rightarrow \hat{y}=1$. The point is now correctly class 1, and its score jumped from $-0.5$ to $+2.5$. A false-negative step raises the score — exactly opposite to cell F. ✓

---

Cell H — the limiting failure (XOR)

Forecast: guess whether any straight line can put the two "1" corners on one side and the two "0" corners on the other.

1. Write the four inequalities. Class 1 needs $z\ge 0$, class 0 needs $z<0$:
   • $[0,0]$: $b<0$
   • $[0,1]$: $w_2+b\ge 0$
   • $[1,0]$: $w_1+b\ge 0$
   • $[1,1]$: $w_1+w_2+b<0$ Why this step? If no numbers satisfy all four at once, no perceptron exists.
2. Add the two "class 1" lines. $(w_1+b)+(w_2+b)\ge 0\Rightarrow w_1+w_2+2b\ge 0$. Why: combining constraints exposes a contradiction.
3. Compare with the last two. From step 1, $w_1+w_2+b<0$ and $b<0$, so $w_1+w_2+2b=(w_1+w_2+b)+b<0$. Why: a negative plus a negative is negative.
4. Contradiction. Step 2 says $w_1+w_2+2b\ge 0$; step 3 says it is $<0$. Impossible.

Verify: the two conclusions $\ge 0$ and $<0$ about the same quantity cannot both hold, so no single perceptron classifies XOR. This is the historic 1969 wall; the escape is stacking neurons — see the MLP and how to train it. Geometrically, XOR's 1s and 0s sit on opposite diagonals — no straight line separates diagonals.

---

Cell I — a real-world word problem

Forecast: income pushes "approve", the existing loan pushes "reject". Which wins?

1. Translate words to numbers. $\mathbf{x}=[4,1]$. Why this step? The perceptron only eats numbers; features must become coordinates.
2. Read the weight signs as a story. $w_1=+3$: each income unit adds to approval. $w_2=-4$: each existing loan subtracts. That matches the bank's intent.
3. Compute $z$. $z=3(4)+(-4)(1)-6=12-4-6=2$. Why: one number decides the whole policy.
4. Decide. $z=2\ge 0\Rightarrow \hat{y}=1=$ approve. Why this step? A non-negative score means the applicant lands on the "class-1" side of the policy line, which the bank has defined as approve; the step function converts the score into the actual yes/no action.

Verify (units + sanity): income term $3\times 4=12$ "approval points", loan penalty $-4$, base threshold $-6$; net $+2$ approval points → approve ✓. Sanity check the edge: an applicant with the same income but 2 existing loans gives $z=12-8-6=-2<0\Rightarrow$ reject — one extra loan flips the decision, as a threshold model should. This is the same machinery behind a linear score classifier.

---

Cell J — the exam twist

Forecast: negating $\mathbf{w}$ and $b$ negates $z$. But watch the tie — does R survive the flip cleanly?

1. Relate the scores. $z_B=(-2)x_1+(1)x_2+1=-(2x_1-x_2-1)=-z_A$. Why this step? Every term flipped sign, so $B$'s score is exactly minus $A$'s score.
2. P: $z_A=+2\Rightarrow z_B=-2<0\Rightarrow {\hat{y}}_B=0$ (was 1). Flipped. Why this step? A strictly positive $A$-score becomes strictly negative for $B$, so the label crosses the boundary.
3. Q: $z_A=-4\Rightarrow z_B=+4\ge 0\Rightarrow {\hat{y}}_B=1$ (was 0). Flipped. Why this step? Same logic mirrored: a strictly negative $A$-score becomes strictly positive for $B$.
4. R (the trap): $z_A=0\Rightarrow z_B=-0=0\ge 0\Rightarrow {\hat{y}}_B=1$ — and $A$ also gave R the label 1! Why this matters: because the tie-break is $\ge 0$, a point on the line stays class 1 for both machines. Negating parameters flips every strict case but not the tie. That asymmetry is the exam trap.

Verify: P: $1\to 0$ ✓ flipped. Q: $0\to 1$ ✓ flipped. R: $1\to 1$ ✓ not flipped (the $z=0$ tie clings to class 1 on both sides). So negation is almost a mirror — it flips all strict labels but pins the fence-sitters to class 1. This subtlety separates a careful margin-based thinker from a careless one.

---

Recall

Recall Which quantity alone decides the label at

$\mathbf{x}=[0,0]$? The bias $b$ ::: because both weighted terms vanish, so $z=b$ and $\hat{y}=1$ iff $b\ge 0$.

Recall With

$\mathbf{w}=[0,0]$, how does the whole plane's label depend on $b$? Sign of $b$ ::: $b<0$ labels every point class 0; $b\ge 0$ labels every point class 1 (constant classifier).

Recall On a false positive (

$y=0,\hat{y}=1$), does the point's score $z$ go up or down after the update? Down ::: the update subtracts $\eta \mathbf{x}$ from $\mathbf{w}$ and $\eta$ from $b$, lowering $z$ next time.

Recall Why can no single perceptron solve XOR?

Its two class-1 points and two class-0 points sit on opposite diagonals ::: no straight line separates them, and the four inequalities are algebraically contradictory.

Recall If you negate

$\mathbf{w}$ and $b$, which points do NOT switch class? Points exactly on the line ($z=0$) ::: the tie-break $\ge 0$ keeps them class 1 for both machines.

---

Connections

• Parent: 3.1.1 The Perceptron Model and History
• Hinglish version: 3.1.01 The perceptron model and history (Hinglish)
• Roots: McCulloch-Pitts Neuron · geometry: Linear Separability · the step rule: Activation Functions
• Learning engine: Gradient Descent · the XOR escape: Multi-Layer Perceptron, Backpropagation
• Cousins that fix the tie/margin issues: Support Vector Machines, Logistic Regression