Intuition The One Core IdeaNeuro-symbolic AI glues together two machines that are each half-blind: a neural network that sees patterns in messy data but can't reason , and a symbolic engine that reasons with crisp logic rules but can't see . The whole field is about building a shared language so the seeing-machine's guesses can flow into the reasoning-machine's rules — and to do that, every crisp true/false must become a soft number between 0 and 1 so a training signal (called a loss , defined in §2) can flow backward and improve both halves at once.
Before you can read the parent note, you need to own every squiggle it uses. Below, each symbol is built from nothing: plain words → a picture → why the topic can't live without it. Read top to bottom; each one leans on the one above.
Definition Neural network — the pattern-matcherA neural network is a big adjustable function. You feed it raw numbers (pixels, sound samples, word codes) at one end; numbers come out the other end (a probability). Inside are millions of little dials called weights , and we collect all of those dials into one bundle written θ θ θ
Definition The network as a functionf θ f θ θ x f θ ( x ) θ f θ ( x ) f θ
Definition Symbolic system — the rule-followerA symbolic system stores facts and rules like "IF cat THEN mammal" and mechanically chains them: given some facts, it grinds out new facts. It never guesses; it only follows rules exactly. But it cannot look at a photo — it only handles clean labelled symbols.
Figure s01 — the red bridge. The two boxes are the two machines. The neural box (left) turns a raw photo into a symbol with a confidence, "cat" 0.8. That number rides across the red arrow — the differentiable bridge — into the symbolic box (right), which then reasons with rules to reach a conclusion. Every remaining section of this page builds the machinery that makes that red arrow work: the soft numbers, the soft logic, and the loss that trains it. Keep this picture in mind — it is the whole topic in one diagram.
See how the two are complementary: neural architectures provide the eyes, and structured knowledge like knowledge graphs provides the rulebook.
Before any formula, a few tiny notations appear everywhere.
x i i x i i x 1 , x 2 , x 3 x 2 i x i i
Definition Training examples( x i , y i ) y i worked examples : each is a pair, an input x i i y i label (the true "cat"/"dog" tag a human wrote down). So x i y i i x i y i
Definition A loss — the "how wrong are we?" scoreA loss is a single number that measures how badly the network's output disagrees with the truth. Small loss = good; big loss = bad. We write the per-example loss as ℓ ( f θ ( x i ) , y i ) ℓ f θ ( x i ) y i whole training loss is the average of these over every example — that is what §7 assembles.
Picture: a scorecard; the closer the guess to the answer key, the lower the number.
i = 1 ∑ N ∑ i N ∑ i = 1 3 x i = x 1 + x 2 + x 3 Why the topic needs it: the training loss averages the per-example score ℓ every example. That "over every example" is exactly what ∑
Definition Set-builder braces{ } ∪ { a , b , c } a b c ∪ { a , b } ∪ { b , c } = { a , b , c } Why the topic needs it: forward chaining (§5) writes F ( t + 1 ) = F ( t ) ∪ ( new facts )
∑ ⋃ ∑ adds numbers ; ⋃ collects things . Same "do it for every item" spirit, different glue.
We meet predicates before probability, because the soft truth value in §4 attaches a number to exactly these statements.
P ( X , Y ) parent ( X , Y ) template meaning "X Y X , Y , Z blanks (variables) you can fill with real names. Fill them: parent ( Alice , Bob ) predicate , and we write it P ( ⋅ ) Q ( ⋅ )
A predicate is either true or false once its blanks are filled — that is what §4 will soften into a number. This relational, slot-filling view is exactly the thinking behind knowledge graphs .
p ( ⋅ ) p ( cat ) = 0.8 [ 0 , 1 ] 0 1
Why does this matter so much? Because a symbolic rule wants a crisp TRUE/FALSE, but a neural network only ever hands you a slider position . The entire trick of the field is: treat that slider position AS a truth value. A truth of 0.8
Figure s02 — probability as a slider. The black track runs from 0 1 red bar fills the track up to the value 0.8 p ( cat ) = 0.8 μ p
Definition Fuzzy truth degreeμ predicate from §3. The parent note writes μ P ( x , y ) ∈ [ 0 , 1 ] truth degree of the statement "P x y μ μ parent ( Alice , Bob ) = 0.8
Classical logic uses connectors that glue predicates together. Learn them as plain English first.
∧ ∨ ¬
a ∧ b a b both are true.a ∨ b a b at least one is true.¬ a a
Now the problem the topic solves: those are jumps — true suddenly flips to false. Jumps have no slope, and neural learning needs slope (see §7). So we relax each operator into a smooth version that works on the slider values of §4:
Figure s03 — why we soften. The black line is the hard AND, min ( a , b ) kink at b = 0.7 red line is the soft AND, a ⋅ b min max
Common mistake "Isn't OR justa + b a = 0.8 b = 0.7 a + b = 1.5 [ 0 , 1 ] − a ⋅ b [ 0 , 1 ] 0.8 + 0.7 − 0.56 = 0.94
Definition Soft implication (→ a → b a b a b a b ¬ a ∨ b a b a → b ≡ ¬ a ∨ b ≈ ( 1 − a ) + b − ( 1 − a ) b = 1 − a + a b Check the corners: premise true, conclusion false (a = 1 , b = 0 1 − 1 + 0 = 0 a = 0 1 − 0 + 0 = 1 a = 1 , b = 1 1 − 1 + 1 = 1
max min max ( a , b ) min hard logic, AND = min = max X , Y , Z
There exists (∃ max y [ … ] strongest case over all y For all (∀ min y [ … ] max y [ … ] ∃ ∀ min
max min max y min y smooth stand-in . For max log-sum-exp / softmax trick:
max y u y ≈ τ 1 log ∑ y e τ u y τ max τ every candidate y min max
F F bag of facts we currently believe — the set (§2's braces) of all predicate-statements known so far, each with its truth degree μ F ( t ) time stamp (not a power): F ( 0 ) F ( 1 ) F ( 2 ) parent ( X , Y ) ∧ parent ( Y , Z ) → grandparent ( X , Z ) X Y Y Z X Z Y F fixed point — is forward chaining , written F ( t + 1 ) = F ( t ) ∪ ( newly derived facts )
Here is the recipe the parent note assumes but never spells out: how a fuzzy rule becomes a number you can minimise .
w ϕ ϕ w ϕ ≥ 0 importance weight . A rule you insist on (a hard anatomical fact) gets a big w ϕ w ϕ = 0
Worked example A one-rule penaltyRule ϕ a → b a = p ( pneumonia ) = 0.8 b = p ( opacity ) = 0.3 μ ϕ = 1 − a + ab = 1 − 0.8 + 0.8 ⋅ 0.3 = 0.44 1 − 0.44 = 0.56 w ϕ = 2 2 × 0.56 = 1.12 b a
This penalty-from-logic construction is what makes symbolic knowledge learnable , and it is the same soft-constraint idea that powers the interpretability goals of explainable AI and the data-thrift of few-shot learning .
Everything above bends toward one goal. Here is the chain of reasoning.
L data L total loss — one number saying "how wrong are the predictions right now." Big = bad; learning = making it small. It is the average, over the N ( x i , y i ) ℓ L data = N 1 ∑ i = 1 N ℓ ( f θ ( x i ) , y i ) f θ ( x i ) i y i ℓ ∑ / N
Definition Gradient & gradient descent — rolling downhillThe gradient is the slope of L θ Gradient descent nudges every dial in θ The catch: slopes only exist for smooth curves. A logic operator that jumps has no slope — the ball has nothing to roll down. That is why §5 relaxed AND/OR/implication (and the quantifiers) into smooth versions.
Common mistake The hidden danger of the product t-norm: vanishing gradientsSoft AND multiplies truths, and multiplying many numbers below 1 shrinks fast : chaining ten premises each at 0.5 0. 5 10 ≈ 0.001 tiny gradient too, so learning through a long rule-chain crawls or stalls — the vanishing-gradient problem. Two standard fixes: (1) work in log-space — replace products ∏ a k ∑ log a k Łukasiewicz AND max ( 0 , a + b − 1 ) 0 ε
Recall Why the product t-norm, and not another?"Soft AND = a ⋅ b t-norm (a family of soft-AND functions). Common alternatives: Gödel (AND = min = max Łukasiewicz (AND = max ( 0 , a + b − 1 ) 0 product t-norm is popular because it is smooth everywhere and its gradient with respect to one input equals the other input (∂ ( ab ) / ∂ a = b
Numbers and sliders 0 to 1
Soft AND OR NOT implication
Gradient descent on theta
Read the map bottom-right: the two arms — a differentiable objective and soft forward chaining — both feed the parent topic Neuro-symbolic AI . Every box above is a symbol you now own.
Test yourself. Cover the right side and answer each.
What does x i The i i
What is a label y i The correct answer (the human-written tag) for training example number i
What is a loss, and what does ℓ ( f θ ( x i ) , y i ) A number scoring badness; ℓ f θ ( x i ) y i
What do θ f θ θ f θ θ
What does i = 1 ∑ N Add the thing on its right for every i N
What is the difference between ∑ ⋃ ∑ ⋃
What is a predicate P ( X , Y ) A fill-in-the-blank fact template whose blanks X , Y
What range must a probability p [ 0 , 1 ] 0 1
What is μ P ( x , y ) The soft truth degree in [ 0 , 1 ] P ( x , y )
Give the soft (product t-norm) forms of AND, OR, NOT. a ∧ b ≈ a ⋅ b a ∨ b ≈ a + b − ab ¬ a = 1 − a
Derive the soft OR from soft AND and NOT. De Morgan: a ∨ b = ¬ (( ¬ a ) ∧ ( ¬ b )) ≈ 1 − ( 1 − a ) ( 1 − b ) = a + b − ab
Give the soft form of implication a → b ¬ a ∨ b ≈ 1 − a + ab
How do ∃ ∀ max / min ∃ → max y ∀ → min y τ
What is F F ( t ) F F ( t ) t
How do you turn a rule ϕ w ϕ Penalise w ϕ ( 1 − μ ϕ ) w ϕ ≥ 0
Write L data N 1 ∑ i = 1 N ℓ ( f θ ( x i ) , y i )
Why can't we train with hard logic (min/max jumps)? Jumps/kinks have no slope, and gradient descent needs a slope to roll downhill.
What is the vanishing-gradient danger of the product t-norm, and one fix? Multiplying many sub-1 truths shrinks toward 0 with tiny gradients; fix by computing in log-space (sum of logs) or switching t-norm.
What does λ L data + λ L symbolic The volume knob balancing rule