Noesis

AI-ML

Learning from data — linear algebra to modern deep nets.

notes
398notes
chapters
28chapters
tests
146tests
words
1.0Mwords
Test bank →Learning path →Deep dives → (361 topics)

Phase 1 — Mathematical & Programming Foundations

Beginner • 6-8 weeks4 chapters
1.1

Linear Algebra Essentials

18 topics
  1. 1.1.1Scalars, vectors, matrices, and tensors definitions
  2. 1.1.2Vector addition, scalar multiplication, and geometric interpretation
  3. 1.1.3Dot product and its geometric meaning (projection, angle)
  4. 1.1.4Vector norms (L1, L2, L-infinity, Lp)
  5. 1.1.5Matrix multiplication rules and dimensionality
  6. 1.1.6Identity, diagonal, symmetric, and orthogonal matrices
  7. 1.1.7Matrix transpose and properties
  8. 1.1.8Matrix inverse and conditions for invertibility
  9. 1.1.9Determinant computation and meaning
  10. 1.1.10Rank, column space, null space
  11. 1.1.11Linear independence and basis vectors
  12. 1.1.12Solving linear systems (Gaussian elimination)
  13. 1.1.13Eigenvalues and eigenvectors
  14. 1.1.14Eigendecomposition of matrices
  15. 1.1.15Singular Value Decomposition (SVD) intuition and computation
  16. 1.1.16Trace operator and properties
  17. 1.1.17Positive definite and semidefinite matrices
  18. 1.1.18Quadratic forms
1.2

Calculus & Optimization Basics

14 topics
  1. 1.2.1Functions, limits, and continuity
  2. 1.2.2Derivatives and rules (product, quotient, chain)
  3. 1.2.3Partial derivatives
  4. 1.2.4Gradients and directional derivatives
  5. 1.2.5The Jacobian matrix
  6. 1.2.6The Hessian matrix
  7. 1.2.7Taylor series approximation
  8. 1.2.8Convex vs non-convex functions
  9. 1.2.9Local vs global minima - maxima
  10. 1.2.10Critical points and saddle points
  11. 1.2.11Lagrange multipliers for constrained optimization
  12. 1.2.12Gradient descent intuition and update rule
  13. 1.2.13Learning rate effects on convergence
  14. 1.2.14Chain rule for multivariate functions (backprop foundation)
1.3

Probability & Statistics

21 topics
  1. 1.3.1Sample spaces, events, and axioms of probability
  2. 1.3.2Conditional probability
  3. 1.3.3Bayes' theorem and applications
  4. 1.3.4Independence and mutual exclusivity
  5. 1.3.5Random variables (discrete and continuous)
  6. 1.3.6Probability mass and density functions
  7. 1.3.7Cumulative distribution functions
  8. 1.3.8Expectation, variance, and standard deviation
  9. 1.3.9Covariance and correlation
  10. 1.3.10Common distributions (Bernoulli, Binomial, Poisson)
  11. 1.3.11Gaussian - Normal distribution properties
  12. 1.3.12Uniform, Exponential, and Beta distributions
  13. 1.3.13Joint, marginal, and conditional distributions
  14. 1.3.14Law of large numbers
  15. 1.3.15Central limit theorem
  16. 1.3.16Maximum likelihood estimation (MLE)
  17. 1.3.17Maximum a posteriori estimation (MAP)
  18. 1.3.18Entropy and KL divergence
  19. 1.3.19Cross-entropy concept
  20. 1.3.20Hypothesis testing and p-values
  21. 1.3.21Confidence intervals
1.4

Python & Scientific Computing

12 topics
  1. 1.4.1Python syntax, data types, control flow
  2. 1.4.2Functions, classes, and modules
  3. 1.4.3List - dict comprehensions and generators
  4. 1.4.4NumPy arrays and vectorized operations
  5. 1.4.5NumPy broadcasting rules
  6. 1.4.6Pandas DataFrames and Series basics
  7. 1.4.7Data loading (CSV, JSON, parquet)
  8. 1.4.8Matplotlib and Seaborn visualization
  9. 1.4.9Jupyter notebooks workflow
  10. 1.4.10Virtual environments and pip - conda
  11. 1.4.11Git version control basics
  12. 1.4.12Reading documentation and debugging

Phase 2 — Data Preprocessing & Classical Machine Learning

Beginner→Intermediate • 8-10 weeks6 chapters
2.1

Data Preprocessing & Feature Engineering

15 topics
  1. 2.1.1Types of data (numerical, categorical, ordinal, text)
  2. 2.1.2Handling missing values (deletion, imputation strategies)
  3. 2.1.3Outlier detection and treatment
  4. 2.1.4Feature scaling - normalization vs standardization
  5. 2.1.5Min-max scaling and z-score normalization
  6. 2.1.6One-hot encoding and label encoding
  7. 2.1.7Ordinal and target encoding
  8. 2.1.8Binning and discretization
  9. 2.1.9Log and power transformations
  10. 2.1.10Feature creation and interaction terms
  11. 2.1.11Handling imbalanced datasets (SMOTE, undersampling)
  12. 2.1.12Train - validation - test splitting
  13. 2.1.13Data leakage identification and prevention
  14. 2.1.14Exploratory data analysis (EDA) workflow
  15. 2.1.15Correlation analysis and multicollinearity
2.2

Linear & Logistic Regression

16 topics
  1. 2.2.1Simple linear regression model
  2. 2.2.2Multiple linear regression
  3. 2.2.3Ordinary least squares derivation
  4. 2.2.4Cost function (MSE) and gradient descent fitting
  5. 2.2.5Normal equation closed-form solution
  6. 2.2.6Polynomial regression
  7. 2.2.7Assumptions of linear regression
  8. 2.2.8R-squared and adjusted R-squared
  9. 2.2.9Logistic regression and the sigmoid function
  10. 2.2.10Log-loss - binary cross-entropy
  11. 2.2.11Decision boundaries
  12. 2.2.12Multinomial - softmax regression
  13. 2.2.13L1 (Lasso) regularization
  14. 2.2.14L2 (Ridge) regularization
  15. 2.2.15Elastic Net regularization
  16. 2.2.16Interpreting model coefficients
2.3

Tree-Based & Instance Methods

17 topics
  1. 2.3.1Decision tree structure and terminology
  2. 2.3.2Entropy and information gain
  3. 2.3.3Gini impurity
  4. 2.3.4Tree pruning techniques
  5. 2.3.5Overfitting in decision trees
  6. 2.3.6Bagging and bootstrap aggregating
  7. 2.3.7Random forest algorithm
  8. 2.3.8Feature importance from trees
  9. 2.3.9Out-of-bag error estimation
  10. 2.3.10Boosting concept and intuition
  11. 2.3.11AdaBoost algorithm
  12. 2.3.12Gradient Boosting Machines
  13. 2.3.13XGBoost fundamentals and tuning
  14. 2.3.14LightGBM and CatBoost overview
  15. 2.3.15K-Nearest Neighbors algorithm
  16. 2.3.16Distance metrics (Euclidean, Manhattan, cosine)
  17. 2.3.17Choosing K and the curse of dimensionality
2.4

SVM, Naive Bayes & Probabilistic Models

11 topics
  1. 2.4.1Support Vector Machine maximum margin concept
  2. 2.4.2Hard vs soft margin classifiers
  3. 2.4.3The kernel trick
  4. 2.4.4Linear, polynomial, and RBF kernels
  5. 2.4.5Hyperparameters C and gamma
  6. 2.4.6Support vectors interpretation
  7. 2.4.7Naive Bayes assumption
  8. 2.4.8Gaussian Naive Bayes
  9. 2.4.9Multinomial and Bernoulli Naive Bayes
  10. 2.4.10Laplace smoothing
  11. 2.4.11Naive Bayes for text classification
2.5

Unsupervised Learning

13 topics
  1. 2.5.1K-Means clustering algorithm
  2. 2.5.2Choosing K (elbow method, silhouette score)
  3. 2.5.3K-Means++ initialization
  4. 2.5.4Hierarchical clustering (agglomerative - divisive)
  5. 2.5.5Dendrograms and linkage methods
  6. 2.5.6DBSCAN density-based clustering
  7. 2.5.7Gaussian Mixture Models and EM algorithm
  8. 2.5.8Principal Component Analysis (PCA) theory
  9. 2.5.9PCA via eigendecomposition and SVD
  10. 2.5.10Explained variance and choosing components
  11. 2.5.11t-SNE for visualization
  12. 2.5.12UMAP for dimensionality reduction
  13. 2.5.13Anomaly detection methods
2.6

Model Evaluation & Selection

16 topics
  1. 2.6.1Bias-variance tradeoff
  2. 2.6.2Underfitting vs overfitting diagnosis
  3. 2.6.3Training, validation, and test error
  4. 2.6.4K-fold cross-validation
  5. 2.6.5Stratified and leave-one-out cross-validation
  6. 2.6.6Confusion matrix interpretation
  7. 2.6.7Accuracy, precision, recall, F1-score
  8. 2.6.8Precision-recall tradeoff and curves
  9. 2.6.9ROC curve and AUC
  10. 2.6.10Regression metrics (MAE, MSE, RMSE, MAPE)
  11. 2.6.11Log-loss and calibration
  12. 2.6.12Learning curves analysis
  13. 2.6.13Grid search and random search
  14. 2.6.14Bayesian hyperparameter optimization
  15. 2.6.15Cross-validation pitfalls and nested CV
  16. 2.6.16Ensemble methods (voting, stacking, blending)

Phase 3 — Neural Networks & Deep Learning

Intermediate • 10-12 weeks5 chapters
3.1

Neural Network Fundamentals

13 topics
  1. 3.1.1The perceptron model and history
  2. 3.1.2Multi-layer perceptron architecture
  3. 3.1.3Forward propagation computation
  4. 3.1.4Activation functions - sigmoid, tanh
  5. 3.1.5ReLU and variants (Leaky ReLU, ELU, GELU)
  6. 3.1.6Softmax for output layers
  7. 3.1.7Universal approximation theorem
  8. 3.1.8Loss functions - MSE, cross-entropy
  9. 3.1.9Backpropagation algorithm derivation
  10. 3.1.10Computational graphs and autograd
  11. 3.1.11Vanishing and exploding gradients
  12. 3.1.12Weight initialization (Xavier, He)
  13. 3.1.13Bias terms and their role
3.2

Training Deep Networks

15 topics
  1. 3.2.1Stochastic gradient descent (SGD)
  2. 3.2.2Mini-batch gradient descent
  3. 3.2.3Momentum and Nesterov momentum
  4. 3.2.4AdaGrad and RMSprop
  5. 3.2.5Adam and AdamW optimizers
  6. 3.2.6Learning rate scheduling
  7. 3.2.7Learning rate warmup
  8. 3.2.8Batch normalization
  9. 3.2.9Layer normalization
  10. 3.2.10Dropout regularization
  11. 3.2.11Early stopping
  12. 3.2.12L1 - L2 weight decay in deep nets
  13. 3.2.13Data augmentation strategies
  14. 3.2.14Gradient clipping
  15. 3.2.15Hyperparameter tuning for deep nets
3.3

Deep Learning Frameworks

11 topics
  1. 3.3.1PyTorch tensors and operations
  2. 3.3.2Autograd and computational graphs in PyTorch
  3. 3.3.3Building models with nn.Module
  4. 3.3.4Datasets and DataLoaders
  5. 3.3.5Training loops from scratch
  6. 3.3.6GPU acceleration and device management
  7. 3.3.7Saving and loading models (checkpoints)
  8. 3.3.8TensorFlow - Keras basics
  9. 3.3.9Mixed precision training
  10. 3.3.10TensorBoard - Weights & Biases logging
  11. 3.3.11Distributed training overview
3.4

Convolutional Neural Networks

15 topics
  1. 3.4.1Convolution operation and filters
  2. 3.4.2Stride, padding, and dilation
  3. 3.4.3Pooling layers (max, average)
  4. 3.4.4Feature maps and receptive fields
  5. 3.4.5CNN architecture design
  6. 3.4.6LeNet and AlexNet
  7. 3.4.7VGG networks
  8. 3.4.8Inception - GoogLeNet
  9. 3.4.9ResNet and skip connections
  10. 3.4.10DenseNet and EfficientNet
  11. 3.4.11Transfer learning and fine-tuning
  12. 3.4.12Image classification pipeline
  13. 3.4.13Object detection (R-CNN, YOLO, SSD)
  14. 3.4.14Semantic segmentation (U-Net, FCN)
  15. 3.4.15Data augmentation for images
3.5

Sequence Models

14 topics
  1. 3.5.1Recurrent Neural Networks (RNN) architecture
  2. 3.5.2Backpropagation through time
  3. 3.5.3Vanishing gradients in RNNs
  4. 3.5.4Long Short-Term Memory (LSTM) cells
  5. 3.5.5Gated Recurrent Units (GRU)
  6. 3.5.6Bidirectional RNNs
  7. 3.5.7Sequence-to-sequence models
  8. 3.5.8Encoder-decoder architecture
  9. 3.5.9The attention mechanism intuition
  10. 3.5.10Bahdanau and Luong attention
  11. 3.5.11Word embeddings (Word2Vec, GloVe)
  12. 3.5.12Handling variable-length sequences
  13. 3.5.13Teacher forcing
  14. 3.5.14Beam search decoding

Phase 4 — Transformers, LLMs & Generative Models

Advanced • 10-12 weeks5 chapters
4.1

Transformer Architecture

14 topics
  1. 4.1.1Limitations of RNNs motivating transformers
  2. 4.1.2Self-attention mechanism in detail
  3. 4.1.3Query, key, value matrices
  4. 4.1.4Scaled dot-product attention
  5. 4.1.5Multi-head attention
  6. 4.1.6Positional encodings (sinusoidal)
  7. 4.1.7Rotary positional embeddings (RoPE)
  8. 4.1.8Feed-forward network sublayers
  9. 4.1.9Residual connections and layer norm placement
  10. 4.1.10Encoder vs decoder vs encoder-decoder
  11. 4.1.11Masked attention for autoregression
  12. 4.1.12The original - Attention is All You Need - architecture
  13. 4.1.13Computational complexity of attention
  14. 4.1.14Flash attention and efficient attention
4.2

Tokenization & Language Modeling

10 topics
  1. 4.2.1Tokenization fundamentals
  2. 4.2.2Byte-Pair Encoding (BPE)
  3. 4.2.3WordPiece and SentencePiece
  4. 4.2.4Vocabulary size tradeoffs
  5. 4.2.5Embedding layers and tied weights
  6. 4.2.6Causal language modeling objective
  7. 4.2.7Masked language modeling (BERT)
  8. 4.2.8Next sentence prediction
  9. 4.2.9Perplexity as a metric
  10. 4.2.10Context window and sequence length
4.3

Pretraining & Fine-Tuning LLMs

14 topics
  1. 4.3.1GPT family architecture evolution
  2. 4.3.2BERT and encoder models
  3. 4.3.3T5 and text-to-text framework
  4. 4.3.4Pretraining data curation and cleaning
  5. 4.3.5Self-supervised pretraining objectives
  6. 4.3.6Full fine-tuning vs feature extraction
  7. 4.3.7Parameter-efficient fine-tuning (PEFT)
  8. 4.3.8LoRA and QLoRA
  9. 4.3.9Adapter layers and prefix tuning
  10. 4.3.10Instruction tuning
  11. 4.3.11Supervised fine-tuning (SFT)
  12. 4.3.12Catastrophic forgetting
  13. 4.3.13Quantization (INT8, INT4, GPTQ)
  14. 4.3.14Knowledge distillation
4.4

Alignment, Prompting & RAG

16 topics
  1. 4.4.1Reinforcement Learning from Human Feedback (RLHF)
  2. 4.4.2Reward modeling
  3. 4.4.3Proximal Policy Optimization for LLMs
  4. 4.4.4Direct Preference Optimization (DPO)
  5. 4.4.5Constitutional AI overview
  6. 4.4.6Zero-shot and few-shot prompting
  7. 4.4.7Chain-of-thought prompting
  8. 4.4.8Self-consistency and tree-of-thought
  9. 4.4.9In-context learning mechanisms
  10. 4.4.10Prompt engineering best practices
  11. 4.4.11Retrieval-Augmented Generation (RAG) architecture
  12. 4.4.12Vector databases and embeddings
  13. 4.4.13Chunking strategies for retrieval
  14. 4.4.14Reranking and hybrid search
  15. 4.4.15Hallucination mitigation
  16. 4.4.16Evaluation of LLMs (benchmarks, LLM-as-judge)
4.5

Generative Models

17 topics
  1. 4.5.1Generative vs discriminative models
  2. 4.5.2Autoencoders fundamentals
  3. 4.5.3Variational Autoencoders (VAE) theory
  4. 4.5.4Reparameterization trick
  5. 4.5.5ELBO objective and KL term
  6. 4.5.6Generative Adversarial Networks (GAN) framework
  7. 4.5.7Generator and discriminator dynamics
  8. 4.5.8GAN training instability and mode collapse
  9. 4.5.9DCGAN, WGAN, StyleGAN
  10. 4.5.10Diffusion models forward - reverse process
  11. 4.5.11Denoising diffusion probabilistic models (DDPM)
  12. 4.5.12Noise scheduling
  13. 4.5.13Score-based generative models
  14. 4.5.14Classifier-free guidance
  15. 4.5.15Latent diffusion (Stable Diffusion)
  16. 4.5.16Text-to-image conditioning (CLIP)
  17. 4.5.17Evaluating generative models (FID, IS)

Phase 5 — Reinforcement Learning & MLOps

Advanced • 8-10 weeks3 chapters
5.1

Reinforcement Learning Foundations

13 topics
  1. 5.1.1Agent, environment, state, action, reward
  2. 5.1.2Markov Decision Processes (MDP)
  3. 5.1.3Policies and value functions
  4. 5.1.4State-value and action-value functions
  5. 5.1.5Bellman equations
  6. 5.1.6Discount factor and returns
  7. 5.1.7Exploration vs exploitation tradeoff
  8. 5.1.8Epsilon-greedy strategy
  9. 5.1.9Dynamic programming (value - policy iteration)
  10. 5.1.10Monte Carlo methods
  11. 5.1.11Temporal Difference learning
  12. 5.1.12SARSA algorithm
  13. 5.1.13Q-learning algorithm
5.2

Deep & Advanced RL

14 topics
  1. 5.2.1Deep Q-Networks (DQN)
  2. 5.2.2Experience replay
  3. 5.2.3Target networks
  4. 5.2.4Double DQN and Dueling DQN
  5. 5.2.5Policy gradient methods
  6. 5.2.6REINFORCE algorithm
  7. 5.2.7Actor-critic methods
  8. 5.2.8Advantage Actor-Critic (A2C - A3C)
  9. 5.2.9Proximal Policy Optimization (PPO)
  10. 5.2.10Trust Region Policy Optimization (TRPO)
  11. 5.2.11Soft Actor-Critic (SAC)
  12. 5.2.12Multi-agent reinforcement learning
  13. 5.2.13Reward shaping and sparse rewards
  14. 5.2.14Model-based RL overview
5.3

MLOps & Deployment

18 topics
  1. 5.3.1ML project lifecycle
  2. 5.3.2Experiment tracking and reproducibility
  3. 5.3.3Model versioning and registries
  4. 5.3.4Data versioning (DVC)
  5. 5.3.5Feature stores
  6. 5.3.6Model serving (REST APIs, FastAPI)
  7. 5.3.7Batch vs real-time inference
  8. 5.3.8Containerization with Docker
  9. 5.3.9Kubernetes for ML workloads
  10. 5.3.10Model serving frameworks (TorchServe, Triton)
  11. 5.3.11CI - CD pipelines for ML
  12. 5.3.12Model monitoring and observability
  13. 5.3.13Data drift and concept drift detection
  14. 5.3.14A - B testing for models
  15. 5.3.15Model retraining pipelines
  16. 5.3.16Cost optimization and inference latency
  17. 5.3.17Edge deployment and ONNX
  18. 5.3.18LLM serving (vLLM, quantized inference)

Phase 6 — Cutting-Edge Topics: Scaling, Agents & Safety

Expert • 8-10 weeks5 chapters
6.1

Scaling & Efficient Architectures

13 topics
  1. 6.1.1Neural scaling laws (Chinchilla, compute-optimal)
  2. 6.1.2Compute-data-parameter tradeoffs
  3. 6.1.3Emergent abilities in large models
  4. 6.1.4Mixture-of-Experts (MoE) architecture
  5. 6.1.5Sparse routing and gating networks
  6. 6.1.6Load balancing in MoE
  7. 6.1.7Model parallelism (tensor, pipeline)
  8. 6.1.8Data parallelism and ZeRO optimization
  9. 6.1.9FSDP and sharded training
  10. 6.1.10Long-context architectures
  11. 6.1.11State-space models (Mamba, S4)
  12. 6.1.12Speculative decoding
  13. 6.1.13KV-cache optimization
6.2

AI Agents & Tool Use

10 topics
  1. 6.2.1Agent architectures and reasoning loops
  2. 6.2.2ReAct (reasoning + acting) framework
  3. 6.2.3Tool use and function calling
  4. 6.2.4Planning and task decomposition
  5. 6.2.5Memory systems for agents
  6. 6.2.6Multi-agent collaboration
  7. 6.2.7Agentic frameworks (LangChain, LlamaIndex)
  8. 6.2.8Code-generation agents
  9. 6.2.9Autonomous agent evaluation
  10. 6.2.10Guardrails and constrained generation
6.3

Interpretability & Explainability

11 topics
  1. 6.3.1Importance of interpretability
  2. 6.3.2Feature attribution (SHAP, LIME)
  3. 6.3.3Saliency maps and Grad-CAM
  4. 6.3.4Attention visualization and limitations
  5. 6.3.5Probing classifiers
  6. 6.3.6Mechanistic interpretability
  7. 6.3.7Circuits and superposition
  8. 6.3.8Sparse autoencoders for features
  9. 6.3.9Activation patching
  10. 6.3.10Counterfactual explanations
  11. 6.3.11Concept-based explanations
6.4

AI Safety & Alignment

15 topics
  1. 6.4.1The alignment problem definition
  2. 6.4.2Outer vs inner alignment
  3. 6.4.3Reward hacking and specification gaming
  4. 6.4.4Goal misgeneralization
  5. 6.4.5Scalable oversight
  6. 6.4.6Red-teaming language models
  7. 6.4.7Jailbreaks and adversarial prompts
  8. 6.4.8Adversarial examples and robustness
  9. 6.4.9Bias, fairness, and discrimination metrics
  10. 6.4.10Privacy (differential privacy, membership inference)
  11. 6.4.11Data poisoning and backdoor attacks
  12. 6.4.12Watermarking and provenance
  13. 6.4.13AI governance and regulation (EU AI Act)
  14. 6.4.14Existential and catastrophic risk frameworks
  15. 6.4.15Responsible AI deployment practices
6.5

Research Frontiers & Practice

12 topics
  1. 6.5.1Reading and reproducing ML papers
  2. 6.5.2Implementing models from scratch
  3. 6.5.3Benchmark design and evaluation rigor
  4. 6.5.4Self-supervised and contrastive learning (SimCLR, CLIP)
  5. 6.5.5Multimodal models (vision-language)
  6. 6.5.6World models and embodied AI
  7. 6.5.7Continual and lifelong learning
  8. 6.5.8Federated learning
  9. 6.5.9Neuro-symbolic AI
  10. 6.5.10Open problems and future directions
  11. 6.5.11Contributing to open-source ML
  12. 6.5.12Building a portfolio and research roadmap