Neuro-symbolic AI: Exercises — AI-ML deep dive

Level 1 — Recognition

Recall Solution L1.1

Recall the two columns from the parent note. Neural nets are great at perception but weak at logical consistency, sample efficiency, and systematic reasoning. Symbolic systems are great at exact rules but weak at perception and robustness to noise.

(a) symbolic-only weakness — symbolic engines can't read pixels.
(b) neural-only weakness — no built-in logical consistency, so mutually-exclusive labels can both fire.
(c) neural-only weakness — a symbolic rule for addition needs zero examples; the neural net needs many.
(d) symbolic-only weakness — hand-coded rules are brittle to noise; neural nets are the robust half.

Recall Solution L1.2

Using the glossary taxonomy above: the neural part runs first and hands symbols to the reasoner, which then reasons over them. That is Neural-Symbolic. It is not Symbolic-Neural (symbolic knowledge constraining neural training), and not Hybrid (bidirectional coupling where both inform each other every step).

Level 2 — Application

Recall Solution L2.1

Here $max (0, \cdot)$ is a gate: it lets the penalty through only when the sum exceeds 1 (a real violation), and clamps to 0 otherwise so satisfied constraints add nothing. $p (c_{1}) + p (c_{2}) - 1 = 0.7 + 0.6 - 1 = 0.3.$ Since $0.3 > 0$ , the gate passes it: $violation = 0.3$ . $w_{ϕ} \cdot violation = 3.0 \times 0.3 = 0.9 .$ Gradient descent will now push the network to lower $p (c_{1})$ or $p (c_{2})$ until their sum drops to $1$ .

Recall Solution L2.2

An implication $P \Rightarrow O$ is violated exactly when $P$ is confident but $O$ is not, i.e. $p (P) > p (O)$ . The gap measures how much. $p (P) - p (O) = 0.9 - 0.4 = 0.5 > 0 \Rightarrow violation = 0.5.$ $added loss = 2.0 \times 0.5 = 1.0 .$ Backprop can reduce it by (i) decreasing $p (P)$ (be less sure of pneumonia) or (ii) increasing $p (O)$ (raise opacity confidence) — whichever the data loss allows.

Recall Solution L2.3

These are chosen because they are smooth (differentiable), and they reduce to real logic at the corners ${0, 1}$ — try $a, b \in {0, 1}$ and you get back $min$ / $max$ style truth tables.

AND: $a \cdot b = 0.8 \times 0.5 = 0.40$ .
OR: $a + b - ab = 0.8 + 0.5 - 0.40 = 0.90$ .
NOT $a$ : $1 - 0.8 = 0.20$ .

Level 3 — Analysis

Recall Solution L3.1

The AND (product) — for each intermediate $y$ we need both premises true, so we multiply:

via $b_{1}$ : $0.9 \times 0.5 = 0.45$
via $b_{2}$ : $0.6 \times 0.8 = 0.48$

The inner max (existential "there exists some $y$ ") — we only need one connecting middle person, so we take the best path: $max (0.45, 0.48) = 0.48$ .

The outer max (OR with what we already knew) — combine the new derivation with the old value: $max (0, 0.48) = 0.48$ .

So $μ_{S (a, c)}^{(1)} = 0.48$ . Notice the weaker link in each path caps its strength (product punishes uncertainty), and the whole rule reports the strongest available chain.

Worked example Exercise L3.2 — Read the NMN composition graph

A Neural Module Network answers "What color is the object left of the cube?" by chaining modules. The numeric attention values are given here in the text (the figure just visualises them, so you can solve the problem even if the image fails to load):

After find(cube): attention peaks on the cube at horizontal position $x \approx 7$ , with peak weight $μ = 0.9$ .
After left_of: attention peaks on the object to the left at $x \approx 3.5$ , with peak weight $μ = 0.8$ .

Figure alt-text: two bell-shaped attention curves on a horizontal-position axis (0 to 10). The orange curve peaks at x≈7 (weight 0.9) marking the cube; the blue curve peaks at x≈3.5 (weight 0.8) marking the object left of the cube; a red arrow shows the peak shifting leftward from the cube to that object.

(a) After find(cube), where is attention concentrated and with what weight? (b) After left_of, where does the peak move, and why? (c) If we swapped left_of for right_of with everything else unchanged, would we need to retrain the find or query_color modules?

Recall Solution L3.2

(a) find(cube) peaks attention on the cube at $x \approx 7$ with weight $μ = 0.9$ — it is a detector answering "where is the cube?". (b) left_of shifts the attention mass to the left, so the new peak lands on the object sitting left of the cube (at $x \approx 3.5$ , weight $μ = 0.8$ ). It answers "given this reference region, where is left of it?". (c) No retraining needed. The whole point of modularity is reuse: find and query_color are independent, trained-once modules. Only the composition (which spatial module we slot in) changes. Swapping left_of→right_of just picks a different pre-trained spatial module.

Recall Solution L3.3

We relax logic because we want to train with gradient descent, and gradient descent needs a slope everywhere. The crisp $min$ has a flat top and a kink: at $min (0.8, 0.5) = 0.5$ the derivative with respect to the larger input $0.8$ is exactly $0$ . A zero gradient means backprop sends no learning signal to that input — it is "invisible" to the optimizer. The product $a \cdot b$ has $\partial (ab) / \partial a = b = 0.5 \neq = 0$ and $\partial (ab) / \partial b = a = 0.8 \neq = 0$ : both inputs get a signal, proportional to the other's confidence. That is why the product t-norm is preferred for learning even though $min$ is a valid logical AND for inference.

Level 4 — Synthesis

Recall Solution L4.1

(a) Compute each violation with its gate:

$viol_{1} = max (0, 0.6 + 0.7 - 1) = max (0, 0.3) = 0.3$ .
$viol_{2} = max (0, 0.5 - 0.9) = max (0, - 0.4) = 0$ (rule 2 is already satisfied!). $L_{total} = 0.5 + 1.0 (0.3) + 4.0 (0) = 0.5 + 0.3 + 0 = 0.8 .$ (b) Only rule 1 contributes. Its penalty is $1.0 \cdot (p (c_{1}) + p (c_{2}) - 1)$ while active, so $\frac{\partial L}{\partial p ( c _{1} )} = 1.0, \frac{\partial L}{\partial p ( c _{2} )} = 1.0.$ Both partials equal $1.0$ . Decreasing either $p (c_{1})$ or $p (c_{2})$ lowers the loss at rate $1.0$ per unit. (The big weight $w_{2} = 4.0$ is a red herring — its rule is inactive.)

Recall Solution L4.2

Two candidate middle people $y \in {Bea, Dan}$ :

via Bea: $0.95 \times 0.90 = 0.855$
via Dan: $0.40 \times 0.70 = 0.280$

Existential max: $μ_{gp (Ann, Cy)} = max (0.855, 0.280) = 0.855$ . The Ann→Bea→Cy chain wins because both of its links are strong; the product punishes the weak $0.40$ link on the other path.

Level 5 — Mastery

Worked example Exercise L5.1 — Full mini-pipeline: perceive, constrain, reason

A Neural-Symbolic vision system processes an image in three stages. Carry the numbers through all three.

Stage A (Neural perception). A detector outputs, for one region, $p (ball) = 0.85$ and $p (cube) = 0.30$ .

Stage B (Symbolic constraint at train time). A rule forbids a region being both ball and cube: mutual exclusivity $max (0, p (ball) + p (cube) - 1)$ , weight $w = 5.0$ . Compute this training penalty.

Stage C (Differentiable reasoning at inference). After training the region is confidently a ball, $μ_{ball} = 0.9$ . A rule says "IF ball( $x$ ) AND red( $x$ ) THEN throwable( $x$ )", with $μ_{red} = 0.8$ . Using product-AND ( $a \land b = a \cdot b$ ), compute $μ_{throwable}$ . Then a second rule "throwable( $x$ ) OR heavy( $x$ ) → actionable( $x$ )" with $μ_{heavy} = 0.2$ ; using the product-t-norm OR ( $a \lor b = a + b - ab$ ), compute $μ_{actionable}$ .

Recall Solution L5.1

Stage B. $p (ball) + p (cube) - 1 = 0.85 + 0.30 - 1 = 0.15 > 0$ , so violation $= 0.15$ . Penalty $= 5.0 \times 0.15 = 0.75$ . (This large penalty is exactly what teaches the detector to stop claiming "both ball and cube here".)

Stage C, step 1 (AND = product). The rule needs both ball and red to be true, so we multiply the two truth degrees: $μ_{throwable} = μ_{ball} \cdot μ_{red} = 0.9 \times 0.8 = 0.72 .$

Stage C, step 2 (OR = $a + b - ab$ ). The next rule fires if throwable or heavy holds, so we use the t-norm OR with $a = μ_{throwable} = 0.72$ and $b = μ_{heavy} = 0.2$ : $μ_{actionable} = 0.72 + 0.2 - (0.72 \times 0.2) = 0.92 - 0.144 = 0.776 .$ The pipeline moved from raw pixels ( $p (ball) = 0.85$ ) → a consistency-corrected detector → chained logical conclusions, each step differentiable so the whole thing trains end-to-end. That is the entire promise of Neuro-symbolic AI in one worked example.

Recall Solution L5.2

(a) $a \cdot b = 0 \times 0.7 = 0$ . A false premise kills the AND — matches crisp logic. (b) $a + b - ab = 1 + 0.3 - (1) (0.3) = 1.3 - 0.3 = 1.0$ . A true disjunct makes the OR certainly true — matches crisp logic. (c) The inner $max$ over an empty set is taken as $0$ (nothing derived), so $μ_{S}^{(t + 1)} = max (0, 0) = 0$ . With no connecting $y$ , no new fact is created — correct. (d) $\neg a = 1 - 0.5 = 0.5$ . Maximum uncertainty is its own negation — the fixed point of NOT, exactly where a truth value is least informative. (e) At the boundary the gap is $p (c_{1}) + p (c_{2}) - 1 = 0.4 + 0.6 - 1 = 0$ , so $violation = max (0, 0) = 0$ . This is the hinge of $max (0, \cdot)$ : for sums $\leq 1$ the penalty is flat at zero (rule satisfied); the instant the sum crosses $1$ the penalty rises linearly with slope $1$ . So the constraint is "free" right up to the equality line and only starts pulling once the two probabilities genuinely over-commit past a total of $1$ .

Recall Self-test: state the rule from memory

AND (product t-norm) ::: $a \land b \approx a \cdot b$ OR (product t-norm) ::: $a \lor b \approx a + b - ab$ NOT ::: $\neg a = 1 - a$ Implication $P \Rightarrow O$ violation ::: $max (0, p (P) - p (O))$ Mutual-exclusivity violation ::: $max (0, p (c_{1}) + p (c_{2}) - 1)$ Why product AND over $min$ for learning ::: $min$ gives zero gradient to the larger input; product gives nonzero gradient to both Total loss shape ::: $L_{total} = L_{data} + \sum_{ϕ} w_{ϕ} violation (ϕ)$

See also: 6.4.02-Explainable-AI (why symbolic outputs are interpretable) · 6.5.01-Few-Shot-Learning (symbolic priors buy sample efficiency) · 6.2.04-Knowledge-Graphs (where the facts live) · 4.3.05-Attention-Mechanisms (the find module's heatmaps) · 6.5.07-Reinforcement-Learning-Advanced · 6.5.10-Causal-Inference · 6.1.03-Neural-Network-Architectures.