Unsupervised Learning

Q1. (3 marks)

• Unsupervised learning: learning patterns/structure from unlabeled data (no target output). (2)
• Difference: supervised learning uses labeled input–output pairs to predict targets; unsupervised has no labels and seeks structure (clusters, low-dim representations). (1)

Q2. (5 marks)

• Assignment step: assign each point to nearest centroid. (1.5)
• Update step: recompute each centroid as the mean of its assigned points. (1.5)
• Objective (WCSS): (2) $J = \sum_{k=1}^{K}\sum_{x_i \in C_k} \lVert x_i - \mu_k \rVert^2$

Q3. (6 marks)

• Assignment with $c_1=3, c_2=11$: (3)
  • 2 → $c_1$ (|2−3|=1 < |2−11|=9)
  • 4 → $c_1$ (|4−3|=1 < |4−11|=7)
  • 10 → $c_2$ (|10−11|=1 < |10−3|=7)
  • 12 → $c_2$ (|12−11|=1 < |12−3|=9)
  • Cluster 1 = {2,4}, Cluster 2 = {10,12}
• Update: (3)
  • $c_1 = (2+4)/2 = 3$
  • $c_2 = (10+12)/2 = 11$
• New centroids: $c_1 = 3$, $c_2 = 11$ (already converged). (state result)

Q4. (6 marks)

• Elbow method: plot WCSS vs $K$; choose $K$ at the "elbow" where the rate of decrease sharply slows (diminishing returns). (2)
• Silhouette score: measures how well a point fits its cluster vs the nearest other cluster; average over all points, higher (max 1) is better. (2)
• Formula for point $i$: (2) $s(i) = \frac{b(i) - a(i)}{\max\{a(i), b(i)\}}$ where $a(i)$ = mean intra-cluster distance, $b(i)$ = mean distance to nearest other cluster.

Q5. (4 marks)

• Idea: choose the first centroid randomly, then choose each subsequent centroid with probability proportional to squared distance $D(x)^2$ from the nearest already-chosen centroid. (2)
• Preferred because it spreads initial centroids apart → faster convergence, better/more consistent solutions, avoids poor local minima from random init. (2)

Q6. (6 marks)

• Agglomerative: bottom-up; start with each point as its own cluster and merge closest pairs iteratively. (1.5)
• Divisive: top-down; start with one cluster containing all points and recursively split. (1.5)
• Linkage methods (any three, 1 mark each): (3)
  • Single linkage: distance = minimum pairwise distance between clusters.
  • Complete linkage: distance = maximum pairwise distance.
  • Average linkage: distance = mean of all pairwise distances.
  • (Ward: merges to minimize increase in total within-cluster variance.)

Q7. (6 marks)

• $\varepsilon$ (eps): radius of neighborhood around a point. (1)
• minPts: minimum number of points required within $\varepsilon$ to form a dense region. (1)
• Core point: has ≥ minPts points (including itself) within $\varepsilon$. (1.5)
• Border point: within $\varepsilon$ of a core point but not itself core (fewer than minPts neighbors). (1.5)
• Noise point: neither core nor border (not reachable from any core point). (1)

Q8. (6 marks)

• $\Sigma$ is diagonal → eigenvalues are $\lambda_1 = 4$, $\lambda_2 = 1$. (2)
• First PC = eigenvector of largest eigenvalue = $(1,0)^T$ (the x-axis direction). (2)
• Proportion of variance explained by PC1: (2) $\frac{\lambda_1}{\lambda_1+\lambda_2} = \frac{4}{4+1} = 0.8 = 80\%$

Q9. (4 marks) (any two, 2 marks each)

• PCA is linear and preserves global structure/variance; t-SNE is nonlinear and preserves local neighborhood structure.
• PCA gives a deterministic projection usable for new data; t-SNE is stochastic, mainly for visualization, and doesn't easily embed new points.
• PCA distances/variances are meaningful; t-SNE cluster sizes and inter-cluster distances are not reliably interpretable.

Q10. (4 marks) (any two, 2 marks each)

• Statistical / z-score: flag points beyond a threshold (e.g. $|z| > 3$) from the mean.
• Isolation Forest: anomalies are isolated in fewer random splits → shorter path length flags them.
• One-Class SVM: learns a boundary around normal data; points outside are anomalies.
• LOF (Local Outlier Factor): compares a point's local density to neighbors' densities; much lower density flags it.
• DBSCAN: points labeled as noise are treated as anomalies.

[
  {"claim": "Q3 new centroids are 3 and 11", "code": "c1=(2+4)/2; c2=(10+12)/2; result = (c1==3 and c2==11)"},
  {"claim": "Q8 eigenvalues of diag(4,1) are 4 and 1", "code": "M=Matrix([[4,0],[0,1]]); ev=sorted(M.eigenvals().keys()); result = (ev==[1,4])"},
  {"claim": "Q8 PC1 explained variance is 0.8", "code": "result = Rational(4,4+1)==Rational(4,5) and float(Rational(4,5))==0.8"},
  {"claim": "Q3 assignment: 2,4 to c1 and 10,12 to c2 given c1=3,c2=11", "code": "pts=[2,4,10,12]; assign=[1 if abs(p-3)<abs(p-11) else 2 for p in pts]; result = (assign==[1,1,2,2])"}
]