Intuition The one-line ideaA condition number measures how much the output of a problem amplifies a small wiggle in the input . It is a property of the problem itself , not of any algorithm. If a tiny relative error in the data can blow up into a huge relative error in the answer, the problem is ill-conditioned — and no clever code can save you.
Intuition Why this mattersEvery real number you put into a computer carries error: rounding (≈ 10 − 16 \approx 10^{-16} ≈ 1 0 − 16 not "is my answer exact?" but:
(error in answer) ≤ (condition number) × (error in input) . \text{(error in answer)} \;\le\; \text{(condition number)} \times \text{(error in input)}. (error in answer) ≤ (condition number) × (error in input) . gain knob between input and output error. Knowing it tells you whether your answer can be trusted to 10 digits or to 2.
Think of a problem as a function y = f ( x ) y = f(x) y = f ( x ) given x x x f ( x ) f(x) f ( x ) . We perturb the input x → x + Δ x x \to x + \Delta x x → x + Δ x Δ y = f ( x + Δ x ) − f ( x ) \Delta y = f(x+\Delta x) - f(x) Δ y = f ( x + Δ x ) − f ( x )
Definition Absolute condition numberThe absolute condition number measures how an absolute input change maps to an absolute output change:
κ abs = lim Δ x → 0 ∣ Δ y ∣ ∣ Δ x ∣ = ∣ f ′ ( x ) ∣ . \kappa_{\text{abs}} = \lim_{\Delta x \to 0} \frac{|\Delta y|}{|\Delta x|} = |f'(x)|. κ abs = lim Δ x → 0 ∣Δ x ∣ ∣Δ y ∣ = ∣ f ′ ( x ) ∣.
Definition Relative condition numberThe relative condition number measures how a relative input change maps to a relative output change:
κ rel = lim Δ x → 0 ∣ Δ y ∣ / ∣ y ∣ ∣ Δ x ∣ / ∣ x ∣ = ∣ x ∣ ∣ f ′ ( x ) ∣ ∣ f ( x ) ∣ . \kappa_{\text{rel}} = \lim_{\Delta x \to 0} \frac{|\Delta y|/|y|}{|\Delta x|/|x|} = \frac{|x|\,|f'(x)|}{|f(x)|}. κ rel = lim Δ x → 0 ∣Δ x ∣/∣ x ∣ ∣Δ y ∣/∣ y ∣ = ∣ f ( x ) ∣ ∣ x ∣ ∣ f ′ ( x ) ∣ . dimensionless — this is the one people usually mean by "the condition number," because data error is naturally relative (10 − 16 10^{-16} 1 0 − 16
Definition Ill- vs well-conditionedA problem is well-conditioned if κ rel \kappa_{\text{rel}} κ rel O ( 1 ) O(1) O ( 1 ) 10 3 10^{3} 1 0 3 ill-conditioned if κ rel \kappa_{\text{rel}} κ rel ≫ 1 \gg 1 ≫ 1 you lose log 10 κ rel \log_{10}\kappa_{\text{rel}} log 10 κ rel
We never memorise; we build.
Intuition The "digits lost" rule — where it comes fromIf input relative error is ε in \varepsilon_{\text{in}} ε in ε out ≈ κ rel ε in \varepsilon_{\text{out}} \approx \kappa_{\text{rel}}\,\varepsilon_{\text{in}} ε out ≈ κ rel ε in log 10 \log_{10} log 10 log 10 ε out ≈ log 10 κ rel + log 10 ε in . \log_{10}\varepsilon_{\text{out}} \approx \log_{10}\kappa_{\text{rel}} + \log_{10}\varepsilon_{\text{in}}. log 10 ε out ≈ log 10 κ rel + log 10 ε in . ≈ − log 10 ε \approx -\log_{10}\varepsilon ≈ − log 10 ε lose about log 10 κ rel \log_{10}\kappa_{\text{rel}} log 10 κ rel .
f ( x ) = x f(x)=\sqrt{x} f ( x ) = x f ′ ( x ) = 1 2 x f'(x)=\frac{1}{2\sqrt{x}} f ′ ( x ) = 2 x 1 κ rel = ∣ x ⋅ 1 2 x x ∣ = ∣ x 2 x ∣ = 1 2 . \kappa_{\text{rel}} = \left|\frac{x\cdot \tfrac{1}{2\sqrt x}}{\sqrt x}\right| = \left|\frac{x}{2x}\right| = \frac12. κ rel = x x ⋅ 2 x 1 = 2 x x = 2 1 . Why this step? We plug f f f f ′ f' f ′ halved — square-rooting is beautifully stable. (Generally x p x^p x p κ rel = ∣ p ∣ \kappa_{\text{rel}}=|p| κ rel = ∣ p ∣
f ( x ) = tan ( x ) f(x)=\tan(x) f ( x ) = tan ( x ) x = π / 2 x=\pi/2 x = π /2 f ′ ( x ) = sec 2 x f'(x)=\sec^2 x f ′ ( x ) = sec 2 x x = π / 2 x=\pi/2 x = π /2 tan x → ∞ \tan x\to\infty tan x → ∞ sec 2 x → ∞ \sec^2 x\to\infty sec 2 x → ∞ κ rel = ∣ x sec 2 x tan x ∣ = ∣ x sin x cos x ∣ = ∣ 2 x sin 2 x ∣ . \kappa_{\text{rel}} = \left|\frac{x\sec^2 x}{\tan x}\right| = \left|\frac{x}{\sin x \cos x}\right| = \left|\frac{2x}{\sin 2x}\right|. κ rel = t a n x x s e c 2 x = s i n x c o s x x = s i n 2 x 2 x . x → π / 2 x\to\pi/2 x → π /2 sin 2 x → 0 \sin 2x\to 0 sin 2 x → 0 κ rel → ∞ \kappa_{\text{rel}}\to\infty κ rel → ∞ Why this step? We rewrote sec 2 / tan = 1 / ( sin cos ) \sec^2/\tan = 1/(\sin\cos) sec 2 / tan = 1/ ( sin cos ) sin 2 x \sin 2x sin 2 x
Worked example 3. Subtraction of nearly-equal numbers — catastrophic cancellationLet f ( x , y ) = x − y f(x,y)=x-y f ( x , y ) = x − y y y y x x x f ′ = 1 f'=1 f ′ = 1 κ rel = ∣ x x − y ∣ . \kappa_{\text{rel}} = \left|\frac{x}{x-y}\right|. κ rel = x − y x . x = 1.00001 x=1.00001 x = 1.00001 y = 1.00000 y=1.00000 y = 1.00000 x − y = 10 − 5 x-y=10^{-5} x − y = 1 0 − 5 ∣ x ∣ ≈ 1 |x|\approx 1 ∣ x ∣ ≈ 1 κ rel ≈ 10 5 \kappa_{\text{rel}}\approx 10^{5} κ rel ≈ 1 0 5 Why this step? The denominator x − y x-y x − y x x x is the disaster. You lose ~5 digits. This is the famous cancellation problem; the problem is ill-conditioned, so rearrange the algebra to avoid the subtraction.
Worked example 4. Linear systemsA x = b Ax=b A x = b A x = b Ax=b A x = b matrix condition number
κ ( A ) = ∥ A ∥ ∥ A − 1 ∥ = σ max σ min (in 2-norm) , \kappa(A) = \|A\|\,\|A^{-1}\| \;=\; \frac{\sigma_{\max}}{\sigma_{\min}}\ \text{(in 2-norm)}, κ ( A ) = ∥ A ∥ ∥ A − 1 ∥ = σ m i n σ m a x (in 2-norm) , σ \sigma σ b → b + Δ b b\to b+\Delta b b → b + Δ b A Δ x = Δ b A\Delta x=\Delta b A Δ x = Δ b Δ x = A − 1 Δ b \Delta x = A^{-1}\Delta b Δ x = A − 1 Δ b ∥ Δ x ∥ ≤ ∥ A − 1 ∥ ∥ Δ b ∥ \|\Delta x\|\le\|A^{-1}\|\|\Delta b\| ∥Δ x ∥ ≤ ∥ A − 1 ∥∥Δ b ∥ ∥ b ∥ ≤ ∥ A ∥ ∥ x ∥ \|b\|\le\|A\|\|x\| ∥ b ∥ ≤ ∥ A ∥∥ x ∥ ∥ Δ x ∥ ∥ x ∥ ≤ ∥ A ∥ ∥ A − 1 ∥ ⏟ κ ( A ) ∥ Δ b ∥ ∥ b ∥ . \frac{\|\Delta x\|}{\|x\|} \le \underbrace{\|A\|\|A^{-1}\|}_{\kappa(A)}\,\frac{\|\Delta b\|}{\|b\|}. ∥ x ∥ ∥Δ x ∥ ≤ κ ( A ) ∥ A ∥∥ A − 1 ∥ ∥ b ∥ ∥Δ b ∥ . Why this step? We bounded numerator and denominator separately, then combined — the product ∥ A ∥ ∥ A − 1 ∥ \|A\|\|A^{-1}\| ∥ A ∥∥ A − 1 ∥ A A A σ min ≈ 0 \sigma_{\min}\approx 0 σ m i n ≈ 0
Common mistake "My answer is bad, so my algorithm is bad."Why it feels right: when results are garbage, blaming the code is natural.
The fix: Conditioning is about the problem , stability is about the algorithm . An ill-conditioned problem gives bad answers even with a perfect algorithm. Diagnose κ \kappa κ backward stable .
Common mistake "A large absolute condition number means ill-conditioned."Why it feels right: "large number = bad" is a tempting shortcut.
The fix: κ abs \kappa_{\text{abs}} κ abs dimensionless relative κ rel \kappa_{\text{rel}} κ rel f ( x ) = 1000 x f(x)=1000x f ( x ) = 1000 x κ abs = 1000 \kappa_{\text{abs}}=1000 κ abs = 1000 κ rel = 1 \kappa_{\text{rel}}=1 κ rel = 1
det ( A ) ≈ 0 \det(A)\approx 0 det ( A ) ≈ 0 A A A Why it feels right: singular matrices have zero determinant.
The fix: Determinant scales with size and units; 0.1 I 100 0.1\,I_{100} 0.1 I 100 10 − 100 10^{-100} 1 0 − 100 κ = 1 \kappa=1 κ = 1 κ ( A ) = σ max / σ min \kappa(A)=\sigma_{\max}/\sigma_{\min} κ ( A ) = σ m a x / σ m i n not the determinant.
Recall Feynman: explain to a 12-year-oldImagine a magnifying glass. The input error is a tiny smudge on the page. The condition number is how strong the magnifying glass is. A weak glass (small condition number) — the smudge stays small, your answer is clear. A super-strong glass (huge condition number) — that tiny smudge becomes a giant blurry blob and you can't read the answer. The magnifying glass strength belongs to the problem , not to your eyes (the algorithm). Some questions are just naturally "blurry" — like measuring how a see-saw tips when it's balanced perfectly on the edge: the tiniest push sends it crashing.
"CRAB" — C ondition = R elative A mplification of B lunders.
And for the formula: "x f-prime over f" → κ rel = ∣ x f ′ f ∣ \kappa_{\text{rel}}=\left|\dfrac{xf'}{f}\right| κ rel = f x f ′ × \times × ÷ \div ÷
What does a condition number measure? How much a relative (or absolute) change in the input is amplified in the output — a property of the problem, not the algorithm.
Formula for absolute condition number of y = f ( x ) y=f(x) y = f ( x ) κ abs = ∣ f ′ ( x ) ∣ \kappa_{\text{abs}}=|f'(x)| κ abs = ∣ f ′ ( x ) ∣ , derived as the limit
∣ Δ y ∣ / ∣ Δ x ∣ |\Delta y|/|\Delta x| ∣Δ y ∣/∣Δ x ∣ from Taylor expansion.
Formula for relative condition number? κ rel = ∣ x f ′ ( x ) f ( x ) ∣ \kappa_{\text{rel}}=\left|\dfrac{x f'(x)}{f(x)}\right| κ rel = f ( x ) x f ′ ( x ) , dimensionless.
Which condition number decides ill-conditioning, and why? The relative one — it is dimensionless, so it's not fooled by scaling/units.
How many digits do you lose for condition number κ \kappa κ About
log 10 κ \log_{10}\kappa log 10 κ decimal digits.
Condition number of f ( x ) = x p f(x)=x^p f ( x ) = x p κ rel = ∣ p ∣ \kappa_{\text{rel}}=|p| κ rel = ∣ p ∣ (e.g.
x \sqrt{x} x gives
1 / 2 1/2 1/2 ).
Why is subtracting nearly-equal numbers dangerous? κ rel = ∣ x / ( x − y ) ∣ \kappa_{\text{rel}}=|x/(x-y)| κ rel = ∣ x / ( x − y ) ∣ blows up when
x ≈ y x\approx y x ≈ y — catastrophic cancellation.
Condition number of solving A x = b Ax=b A x = b κ ( A ) = ∥ A ∥ ∥ A − 1 ∥ = σ max / σ min \kappa(A)=\|A\|\|A^{-1}\|=\sigma_{\max}/\sigma_{\min} κ ( A ) = ∥ A ∥∥ A − 1 ∥ = σ m a x / σ m i n (2-norm).
Difference between conditioning and stability? Conditioning = sensitivity of the problem; stability = quality of the algorithm. Both needed for an accurate result.
Is large det ( A ) ≈ 0 \det(A)\approx 0 det ( A ) ≈ 0 No — determinant scales with size/units; use
σ max / σ min \sigma_{\max}/\sigma_{\min} σ m a x / σ m i n instead.
Catastrophic cancellation — the classic ill-conditioned subtraction.Floating-point arithmetic — source of the ∼ 10 − 16 \sim 10^{-16} ∼ 1 0 − 16 Backward stability — algorithm-side partner to conditioning.Singular Value Decomposition — gives κ ( A ) = σ max / σ min \kappa(A)=\sigma_{\max}/\sigma_{\min} κ ( A ) = σ m a x / σ m i n Taylor series — the engine behind the derivation of κ abs \kappa_{\text{abs}} κ abs Newton-Raphson method — root finding near f ′ = 0 f'=0 f ′ = 0
dimensionless, usual meaning
large, much bigger than 1
property of problem, not algorithm
Absolute condition number
Relative condition number
Intuition Hinglish mein samjhoDekho, condition number ka matlab simple hai: agar tum input me thoda sa error daalo (jo
har computer me hota hi hai, kyunki rounding ~10 − 16 10^{-16} 1 0 − 16 problem ki property
hai, tumhare code ki nahi. Chahe tum world ka best algorithm likho, agar problem hi ill-conditioned
hai to answer kharab hi aayega.
Do flavour hote hain. Absolute condition number = ∣ f ′ ( x ) ∣ =|f'(x)| = ∣ f ′ ( x ) ∣ Relative condition number
= ∣ x f ′ ( x ) / f ( x ) ∣ =|x f'(x)/f(x)| = ∣ x f ′ ( x ) / f ( x ) ∣ log 10 κ \log_{10}\kappa log 10 κ κ = 10 5 \kappa = 10^5 κ = 1 0 5
Classic example hai nearly-equal numbers ka subtraction — jaise 1.00001 − 1.00000 1.00001 - 1.00000 1.00001 − 1.00000 κ r e l = ∣ x / ( x − y ) ∣ \kappa_{rel}=|x/(x-y)| κ r e l = ∣ x / ( x − y ) ∣ A x = b Ax=b A x = b κ ( A ) = σ m a x / σ m i n \kappa(A)=\sigma_{max}/\sigma_{min} κ ( A ) = σ ma x / σ min determinant zero hone se
ill-conditioned mat samajh lena — woh scaling pe depend karta hai, galat signal deta hai.
Hamesha singular values ka ratio dekho.
