Visual walkthrough — Neuro-symbolic AI

Prerequisite bridges (open if a word is new): the network that produces our numbers is a standard neural network; the logic rules live in a knowledge graph; the whole point of readable rules connects to explainable AI; and needing fewer examples ties to few-shot learning.

---

Step 1 — What a network actually outputs: a bar of confidence

WHAT. Before any logic, we need the raw material. A trained network, when shown one X-ray $x$, does not shout "PNEUMONIA!". It emits a number between 0 and 1 for each thing it could detect. Call one such number $p$. Read $p$ as "how sure am I, from 0 (no way) to 1 (certain)".

WHY this and not a yes/no. A hard yes/no ($0$ or $1$) has no slope. Gradient descent — the only way we train networks — needs a quantity that can nudge a little up or down. A continuous confidence $p\in [0,1]$ gives us that gap to move through. This is the single most important reason the whole method is possible.

PICTURE. Two confidence bars for our two facts.

---

Step 2 — Turning an English rule into an inequality

WHAT. Our rule is "pneumonia ⟹ opacity". In the language of confidences this becomes a demand on the two bars: the pneumonia bar must not stand taller than the opacity bar.

$p_{\text{pneu}}\le p_{\text{opac}}$

• $p_{\text{pneu}}$ — "how sure of the cause" (left term).
• $\le$ — "must not exceed" (this is the logical arrow ⟹ in disguise).
• $p_{\text{opac}}$ — "how sure of the required consequence" (right term).

WHY "$\le$" encodes "implies". If $A$ implies $B$, then whenever you believe $A$ strongly you are forced to believe $B$ at least that strongly — you can't be 90% sure of pneumonia while only 30% sure of the opacity it must produce. So "cause-confidence sits under consequence-confidence" is exactly the arrow, rewritten as a height comparison.

PICTURE. The green (allowed) zone where the pneumonia bar hides under the opacity bar, and the red (forbidden) zone where it pokes above.

---

Step 3 — Measuring how badly the rule is broken

WHAT. A true/false verdict ("rule broken: yes") is useless for training — no slope again. We want a number that is 0 when obeyed and grows the more the rule is violated. That number is the overshoot: how far the pneumonia bar pokes above the opacity bar.

WHY the $\max (0,\cdot )$ clamp. Without it, an obeyed rule ($p_{\text{pneu}}$ far below $p_{\text{opac}}$) would give a big negative penalty, which gradient descent would chase forever, distorting predictions for no reason. The clamp says: once you're legal, we stop caring.

PICTURE. The violation as a hinge — flat at zero, then rising with slope 1.

---

Step 4 — Weighting the rule and adding it to the loss

WHAT. A network already has a data loss ${\mathcal{L}}_{\text{data}}$ — one number saying "how wrong were your labels on the training X-rays". We now graft our violation onto it, scaled by a knob $\lambda$ (or a per-rule weight $w_{\varphi }$) that says how much we care about this rule versus fitting the data.

Using the parent's $\lambda =2.0$ and violation $=0.5$: ${\mathcal{L}}_{\text{total}}={\mathcal{L}}_{\text{data}}+2.0\times 0.5={\mathcal{L}}_{\text{data}}+1.0$

WHY add rather than multiply. Addition keeps the two goals separable and readable: the training report literally says "your label error, PLUS 1.0 of logic-breaking." A big $\lambda$ makes logic nearly sacred; $\lambda =0$ recovers a plain network. It's a dial, not a cliff.

PICTURE. Two stacked contributions summing to one bar.

---

Step 5 — Why this shape lets gradients fix the network

WHAT. Training moves each output in the direction that lowers ${\mathcal{L}}_{\text{total}}$. Because our violation is $p_{\text{pneu}}-p_{\text{opac}}$ (in the broken region), its slopes are dead simple:

$\frac{\partial \text{violation}}{\partial p_{\text{pneu}}}=+1,\frac{\partial \text{violation}}{\partial p_{\text{opac}}}=-1$

• $+1$ on $p_{\text{pneu}}$ — "raising pneumonia confidence raises the penalty", so gradient descent is told push pneumonia down.
• $-1$ on $p_{\text{opac}}$ — "raising opacity confidence lowers the penalty", so gradient descent is told push opacity up.

WHY it's exactly the fix we wanted. The two forces are precisely the two anatomically-valid repairs the parent listed: decrease $p_{\text{pneu}}$ OR increase $p_{\text{opac}}$ — until the bars line up and the violation flattens to zero. The math literally implements the doctor's common sense.

PICTURE. Two amber arrows: pneumonia pulled down, opacity pushed up, converging.

---

Step 6 — Edge case A: the rule is already obeyed (the flat zone)

WHAT. Suppose the network says $p_{\text{pneu}}=0.2$, $p_{\text{opac}}=0.7$. Then the raw gap is $0.2-0.7=-0.5$, and $\max (0,-0.5)=0$.

WHY it matters. Here the violation is flat — slope zero on both outputs. Gradient descent feels no force from the logic term at all; only the data loss steers. This is the correct behaviour: a rule you already satisfy should never yank your predictions around. Many beginners forget the hinge is flat, not sloping, in the legal zone.

PICTURE. Same hinge as Step 3, now with a dot sitting in the flat green zone, zero slope.

---

Step 7 — Edge case B: the exact corner, and AND/OR beyond one rule

WHAT — the corner. Right where $p_{\text{pneu}}=p_{\text{opac}}$, the hinge has a kink: slope jumps from $0$ to $1$. The gradient is technically undefined at that single point, so libraries pick $0$ (a subgradient). In practice you never land exactly on it, and if you do, "no push" is a safe choice — you're already legal.

WHAT — many rules at once. Real systems have several rules. To combine them we relax discrete logic into arithmetic (from the parent's product t-norm):

$\underset{\text{AND}}{\underbrace{a\wedge b}}\approx a\cdot b,\underset{\text{OR}}{\underbrace{a\vee b}}\approx a+b-a\cdot b,\underset{\text{NOT}}{\underbrace{\neg a}}=1-a$

• $a\cdot b$ for AND — a product is high only if both are high; drop either toward 0 and the product dies, exactly like "both must hold."
• $a+b-a\cdot b$ for OR — high if either is high; the $-a\cdot b$ just stops double-counting the overlap so it never exceeds 1.
• $1-a$ for NOT — flips confidence, since being sure of $a$ means being unsure of "not $a$."

WHY these formulas. They are smooth (differentiable everywhere except isolated kinks) and they snap back to true/false as inputs approach 0 or 1 — so between examples you still get gradients, but at the extremes you recover honest logic.

PICTURE. The AND-surface $z=a\cdot b$ over the unit square, showing it hugs 0 unless both inputs are near 1.

Recall Check yourself

Why must the network output confidences in $[0,1]$ instead of hard yes/no? ::: Hard values have zero slope everywhere, so gradient descent gets no direction to improve; continuous confidence gives a gap to move through. What does $\max (0,p_{\text{pneu}}-p_{\text{opac}})$ equal when the rule is obeyed? ::: Exactly $0$ — the clamp floors any negative gap, giving a flat, force-free legal zone. In the broken region, which two moves does the gradient prescribe? ::: Push $p_{\text{pneu}}$ down (slope $+1$) and push $p_{\text{opac}}$ up (slope $-1$). Why is AND relaxed to $a\cdot b$? ::: A product is large only when both factors are large, mirroring "both premises must hold," and it stays differentiable.

---

The one-picture summary

Everything above collapses into a single pipeline: raw X-ray → confidence bars → compare to the rule → measure overshoot → scale and add to data loss → gradients repair the bars. The amber loop is the learning signal flowing back.

Recall The whole walkthrough, retold like Feynman

Imagine the network as an eager but sometimes reckless intern who reports two hunches as bars: "80% pneumonia," "30% cloudiness." A senior doctor knows a law of the body: pneumonia can't exist without cloudiness, so the pneumonia bar must never rise above the cloudiness bar. We measure how far the intern broke that law — here the bar pokes out by $0.5$. We can't just say "wrong" because that gives no direction to improve; instead we hand back a number that grows with the overshoot, but sits perfectly flat at zero once the law is respected (so we never nag a correct answer). We add that number, dialed by our trust $\lambda$, onto the ordinary "did you match the labels" score. Then the training machinery reads the slopes — push the pneumonia hunch down, pull the cloudiness hunch up — and repeats until the bars obey the law. Stack many such laws by turning AND into multiply and OR into probabilistic-add, and the intern gradually learns to think in ways that are not just accurate, but logically legal. That's semantic loss: teaching a pattern-matcher to respect the rules of the world, one differentiable nudge at a time.

Related directions once this clicks: rules can come from an knowledge graph, the readable penalties feed explainability, attention-based module routing borrows from attention mechanisms, and reasoning over actions extends into advanced RL and causal inference. Back to the hub: Neuro-symbolic AI.