4.8.21 · D4 · HinglishNumerical Methods

ExercisesEigenvalue computation — power method, inverse iteration

3,417 words16 min read↑ Read in English

4.8.21 · D4 · Maths › Numerical Methods › Eigenvalue computation — power method, inverse iteration

Poore page mein hamaari workhorse matrix yeh hai: Ise hum baar baar use karte hain taaki tum usi matrix ko alag-alag sawalon ka jawab dete dekh sako.

Shuru karne se pehle, plain words mein chaar reminders:

  • Ek eigenpair satisfy karta hai : matrix sirf ko stretch karti hai, kabhi rotate nahi karti. Dekho Spectral Decomposition.
  • Identity matrix ek "kuch-nahi-karta" matrix hai: har vector ke liye (2D mein, ). Humein yeh isliye chahiye kyunki " minus ek number " ka koi matlab nahi — tum ek number ko ek matrix se subtract nahi kar sakte — isliye hum ke copies of the do-nothing matrix subtract karte hain, likha jaata hai , jiska seedha matlab hai "har diagonal entry se subtract karo."
  • Kisi vector ka Rayleigh quotient yeh number hai — isko aise padho: "agar ek eigenvector hota, toh uska stretch factor kya hota?" Dekho Rayleigh Quotient Iteration.
  • Normalize ka matlab hai: ek vector ko uski length se divide karo taaki uski length ho jaaye. Hum yeh har step pe karte hain taaki numbers explode na karein.

Poora page ek picture mein. Algebra se pehle, us geometry ko dekho jisme yeh exercises rehte hain. Neeche ka figure hamaara unit vector dikhata hai jo se shuru hota hai (seedha daayein point karta hua) aur step by step red dominant-eigenvector line ki taraf swing karta hai. Dashed quarter-circle unit vectors ka set hai (length exactly 1), aur angle marks dikhate hain ki har step red line se angle ko kitna kam karta hai. Har L1–L3 exercise basically yeh sawaal hai: "kaali arrow red line tak kitni tezi se pahunchi, aur main stretch factor ko usse kaise padhunga?" Jab tum calculate karo toh yeh picture dimag mein rakhna.

Figure — Eigenvalue computation — power method, inverse iteration

Do cheezein notice karo jo tum baad mein numerically verify karoge: (1) har kaali arrow pichli wali se red line ke zyada paas padti hai, aur (2) consecutive arrows ke beech ke gaps chhote hote jaate hain — woh shrinking hi Exercise 3.1 ka linear convergence hai.


L1 — Recognition

Exercise 1.1 (L1)

Inn vectors mein se kaun sa ka eigenvector hai, aur kis eigenvalue ke saath?

Recall Solution 1.1

WHAT karna hai: ek eigenvector woh hai jahan sirf ka ek plain multiple aata hai — same direction, bas scaled.

test karo: . Kya , ka multiple hai? Nahi — doosri entry se ho gayi. Isliye eigenvector NAHI hai.

test karo: . Kya yeh ka multiple hai? Pehli entries ka ratio check karo: . Doosra check karo: . Same factor → haan! .

Answer: ek eigenvector hai eigenvalue ke saath; nahi hai.

Exercise 1.2 (L1)

Power method ko aur mein convert karta hai. Blanks ko words mein bharo.

Recall Solution 1.2

se shuru karo.

  • Dono sides ko se multiply karo: , isliye . Eigenvector unchanged rehta hai; eigenvalue flip hokar ho jaata hai.
  • se subtract karo: (yaad raho do-nothing matrix hai, isliye ). Eigenvector unchanged; eigenvalue se slide karta hai.

Yeh single fact — eigenvectors dono operations mein survive karte hain, eigenvalues predictably transform hote hain — inverse aur shifted iteration ka poora engine hai.


L2 — Application

Exercise 2.1 (L2)

pe se do power-method steps chalao. (normalized) aur Rayleigh estimates report karo, consistent rule use karke (har ki length 1 hai).

Recall Solution 2.1

Convention (inconsistency theek karna): Rayleigh estimate usi normalized vector se belong karta hai jo hum abhi hold kar rahe hain. Isliye same index ke liye. Hum ek per vector report karte hain: for , for , for .

WHY har Rayleigh quotient se pehle normalize karein: do reasons. (1) Numbers tame rehte hain. Rescaling ke bina, grow karta hai jaisa aur overflow ho jaata hai; length se divide karne se har unit circle pe rehta hai. (2) Quotient formula simple ho jaata hai. Jab hota hai toh denominator , isliye mein koi division nahi lagti — jo number tum compute karte ho wahi estimate hai.

WHY Rayleigh quotient dominant eigenvalue pe converge karta hai: jaise kaali arrow red line ki taraf swing karti hai (shuruwaat ka figure dekho), , isliye . Isse quotient mein dalo: . Isliye arrow jitna zyada red ke paas hoga, utna hi ke paas hoga — jo exactly woh monotone climb hai jo hum neeche compute karte hain.

Rayleigh at seed (length pehle se 1 hai):

Step 1 — WHAT karte hain: lagao, phir length 1 pe rescale karo. WHY: vector ko dominant direction ki taraf tilt karta hai; rescaling growth ko remove karta hai taaki sirf direction bachti hai. Rayleigh at : pehle , phir

Step 2: Rayleigh at : , phir

Answer: , , aur — ek clean monotone climb ki taraf.

Exercise 2.2 (L2)

Chhoti eigenvalue target karte hue ek shifted-inverse step karo: , . solve karo, normalize karo, aur Rayleigh estimate do.

Recall Solution 2.2

WHAT karna hai: unknown ke liye linear system solve karo — inverse matrix mat banao. Yahi LU Decomposition ka message hai: ke liye find karne ke liye, ko ek baar factor karo aur forward/back-substitution chalao, kabhi mat banao.

System set up karo. Yahan (har diagonal entry se subtract karo — yahi karta hai). Hum solve karte hain: Elimination se solve karo (yahi LU under the hood karta hai). Row 1 se: . Row 2 se: , yaani . Row 1 mein substitute karo: Back-substitute: . Toh kabhi compute kiye bina mila, exactly jaise preached.

Normalize: , isliye . Rayleigh: . Pehle , phir

Answer: , — practically ek hi step mein.


L3 — Analysis

Exercise 3.1 (L3)

Hamare ke liye, power-method error ki tarah shrink hoti hai jahan . compute karo. Error ko ke factor se cut karne ke liye kitne steps chahiye? Isse shuruwaat ke figure mein shrinking arrow-gaps se relate karo.

Recall Solution 3.1

WHY yeh ratio: derivation ko factor out karta hai, bacha hua component se scale hota hai. Woh bacha hua error hi hai, isliye steps ke baad uski size hai. Dekho Convergence Rates. Humein chahiye . Log lo (log = "mujhe kitni baar multiply karna hoga yahan pahunchne ke liye?"): Answer: ; teen decimal digits gain karne ke liye roughly steps. Yeh linear convergence hai — har digit ke liye steps ki fixed number. Shuruwaat ke figure mein, yahi explain karta hai ki har kaali arrow ka red line se angular gap times pichle gap ke barabar hota hai — arrows visibly red line ke against pile up hoti hain.

Exercise 3.2 (L3)

ko target karte hue ke saath shifted inverse iteration ke liye, rate hai . Ise compute karo aur Exercise 3.1 se contrast karo.

Recall Solution 3.2

WHY yeh do gaps: shifting aur inverting ke baad, target eigenvalue bada wala ban jaata hai, aur nearest competitor hai (doosra sirf ek eigenvalue). Rate "distance to target" aur "distance to competitor" ka ratio hai: Answer: — ek step mein hi digits gain ho jaate hain, plain power method ke ke comparison mein. ko target ke paas choose karna hi inverse iteration ko explosive banata hai: shrink karo aur rate plummet kar jaata hai.


L4 — Synthesis

Exercise 4.1 (L4)

ka dominant eigenvector hai. Doosra eigenvector hai ( ka eigenvector). Maano main power method seed se shuru karta hoon — yaani exactly direction se. Exact arithmetic mein, power method kahan converge karega, aur kyun? Real computation mein kya rescue karta hai? Shuruwaat ke figure ki geometry use karke explain karo.

Recall Solution 4.1

Check karo ki seed sach mein hai: . Kya woh hai? Compute karo . Exact match — isliye genuinely direction mein hai, ke saath zero component ke saath, yaani .

WHY yeh fail karta hai: derivation ko chahiye tha with . Agar , toh hamesha — yeh pe converge karta hai aur report karta hai, nahi. Power method sabse bade present component ko amplify karta hai; agar dominant wala absent hai toh woh use invent nahi kar sakta.

Geometric picture: shuruwaat ke figure mein red line hai. Ek seed jo exactly perpendicular eigen-direction pe lie karta hai woh ek kaali arrow hai jise seedha wapas apne aap pe map karta hai — woh kabhi red line ki taraf swing nahi karta, kyunki koi red-line component amplify karne ke liye hai hi nahi. Yeh ek knife-edge balance point hai.

WHAT rescue karta hai: floating-point rounding. Har real multiplication ek tiny error inject karta hai, jisme almost surely component nonzero hota hai. Woh ka seed phir se amplify hota hai aur eventually takeover kar leta hai — kaali arrow, knife-edge se nudge hoke, phir bhi red line pe slide ho jaati hai. Lesson: ek random seed mein probability 1 ke saath hota hai, isliye practice mein hum kabhi worry nahi karte.

Exercise 4.2 (L4)

Tumhe ka woh eigenvalue chahiye jo ke sabse paas ho. Method design karo, shift choose karo, aur convergence rate predict karo. (Eigenvalues: .)

Recall Solution 4.2

WHAT banana hai: ke saath shifted inverse iteration (shift exactly target region pe rakho).

  • , . Woh equidistant hain! do eigenvalues ka midpoint hai.
  • ke eigenvalues aur hain — equal magnitude, opposite sign. Rate

WHY yeh worst case hai: rate matlab koi convergence nahi — method stall ho jaata hai kyunki koi bhi transformed eigenvalue dominate nahi karta. Fix: shift ko off-center nudge karo, jaise : tab , , aur closer eigenvalue hai, isliye yeh pe rate se converge karta hai. Answer: ko kabhi do eigenvalues ke exactly beech mein mat rakho; jis wale ko chahiye uski taraf bias karo.


L5 — Mastery

Exercise 5.1 (L5)

Prove karo ki hamare symmetric ke liye, Rayleigh quotient error second order hai: agar (true unit eigenvector ka ek chhota perturbation ek unit direction mein), toh hai, nahi. Hamare ke liye ke saath numerically verify karo.

Recall Solution 5.1

WHY symmetry matter karti hai: symmetric ke liye, eigenvectors orthogonal hote hain, aur ek smooth function hai jiska gradient har eigenvector pe vanish karta hai (har eigenvector Rayleigh quotient ka ek critical point hai — yahi Rayleigh Quotient Iteration insight hai). Ek function apne critical point pe sirf quadratically change karta hai.

Proof. likho with , , . Numerator expand karo, use karke aur — crucially — symmetry use karke, jo deti hai : Denominator: Linear-in- term cancel ho gaya (symmetry ki wajah se ki zaroorat thi). Isliye .

Numeric check (): unit dominant eigenvector lo aur unit orthogonal direction lo (). Form karo aur compute karo . Result deta hai , jabki vector error hai. Ratio — ek clean (constant hai, aur hai, order match karta hai). Ek linear error deta, paanch gunga zyada bada.

Exercise 5.2 (L5)

Transformation rules use karke explain karo ki Rayleigh Quotient Iteration (shift ko har step pe current Rayleigh estimate update karna) symmetric matrices ke liye cubically converge kyun karta hai, aur plain shifted inverse iteration (fixed ) sirf linear kyun hai. QR Algorithm aur Characteristic Polynomial se connect karo.

Recall Solution 5.2

Fixed shift (linear): constant rehne pe, har step error ko fixed ratio se multiply karta hai. Constant ratio → error jaisi → linear. Dekho Convergence Rates.

Adaptive shift (cubic) — poori chain: set karo, current vector ka Rayleigh quotient, aur current vector error ho.

  1. Shift accuracy (Exercise 5.1 se): kyunki Rayleigh quotient symmetric eigenvector pe second-order accurate hai, shift ke andar land karta hai true eigenvalue se. Yahi crucial input hai — shift error quadratically small hai, sirf linearly small nahi.
  2. Uss shift ke saath ek inverse-iteration step: transformed target eigenvalue hai, aur nearest competitor hai with fixed distance door. Isliye one-step contraction factor hai:
  3. Dono ko compound karo: nayi error purani error times contraction factor hai: Yahi cubic convergence haisahi digits ki sankhya har step mein triple hoti hai (3, 9, 27, …). Engine hai feedback loop: accurate vector → quadratically-accurate shift → quadratically-strong contraction → aur bhi accurate vector.

QR aur characteristic polynomial se concrete link:

  • QR Algorithm disguise mein shifted inverse iteration hai jo sab eigenvectors pe ek saath chalti hai. Woh Rayleigh-type Wilkinson shifts use karta hai (essentially current matrix ka ek corner Rayleigh quotient) exactly isi cubic speed ko symmetric problems mein inherit karne ke liye — isiliye practical libraries QR use karti hain, un definitions se nahi jinse humne shuru kiya tha.
  • Hum Characteristic Polynomial route () se bilkul bachte hain: ke liye us polynomial ke koi closed-form roots nahi hain (Abel–Ruffini), aur numerically bhi uske roots coefficient errors ke liye wildly sensitive hain. Iteration polynomial ko sidestep karta hai: har eigenvalue shifting aur solving se mili, aur Rayleigh feedback ise cubically fast banata hai. Slogan: "polynomial root mat karo — shift karo, solve karo, aur Rayleigh ko answer chase karne do."

Recall Poore page ka one-line recap

Power method biggest stretch ride karta hai; inverse iteration chhote ki taraf flip karta hai; shifting kahin bhi aim karta hai; Rayleigh quotient eigenvalue quadratically padhta hai (symmetric case); aur shift ko us quotient ke peeche chase karne dene se cubic speed milti hai — QR Algorithm ka seed.