Wenqing's Machine Learning Blog

Foundational Concepts in Machine Learning

• Linear Algebra

  • Inverse matrix
  • Determinants
  • Linear Transformations
  • Linear Systems
  • Change Of Basis
  • Eigenvector
  • Singular Vector
  • Eigenvalue
  • Singular Value
  • Singular Value Decomposition (SVD)

• Calculus

  • Calculus

• Statistical Measures

  • Standard Deviation vs Standard Error
  • Margin of Error
  • Statistical Inference
  • Confidence Intervals
  • Statistical Power
  • P-values
  • Statistical Significance
  • Variance&Standard Deviation
  • Covariance
  • Correlation
  • Joint Variability
  • Bessel’s Correction
  • Degrees of Freedom

• Model Evaluation

  • Underfitting (High Bias) & Overfitting (High Variance)
  • AUC (Area Under Curve)
  • Confusion Matrix
  • Precision, Recall and F1 score

• Linear Regression

  • Overview
  • Least Squares
  • Assumptions of Linear Regression
    • Linearity
    • Homoscedasticity (Constant Variance)
    • Normal Distribution of Errors
    • No Multicollinearity
    • Independence of Errors(Non-Random Residuals)
  • Performance Metrics for Linear Regression
    • R-squared
    • P-values
    • Statistical Significance
    • Prediction Interval
    • Standard Error of the Estimate

• Logistic Regression

  • Overview

• T-SNE

  • Overview

• Optimization Algorithms

  • Overview
  • Gradient Descent
    • Overview
    • Batch Gradient Descent
      • Overview
    • Stochastic Gradient Descent (SGD)
      • Overview
    • Mini-Batch Gradient Descent
      • Overview
  • Advanced Optimization Algorithms
    • Momentum
    • Adam
    • RMSProp
    • AdaGrad

• Dimensionality Reduction

  • Principal Component Analysis (PCA)

Deep Learning Concepts

Basic Neural Network Concepts

• Neural Network
  • Overview
• Feed Forward Neural Network (FFNN)
  • Overview
  • MLP (Multi-Layer Perceptron)
    • Overview
• Backpropagation
  • Overview
• Activation Functions
  • Overview
  • ReLU
  • Sigmoid
  • Tanh
  • Softmax

Sequence Models

• Overview
• Language Model Evaluation Metrics
  • Perplexity
  • [Other relevant metrics...]
• RNN (Recurrent Neural Networks)
  • Standard RNN
  • LSTM (Long Short-Term Memory)
• CNN (Convolutional Neural Networks)
  • Overview
• Transformers
  • Original Transformer
  • BERT (Bidirectional Encoder Representations from Transformers)
    • Overview
  • GPT (Generative Pre-trained Transformer)
    • Overview
  • T5 (Text-to-Text Transfer Transformer)
    • Overview
• Model Comparisons: CNN vs. RNN vs. Transformer

Word Representations

• Embedding
  • Overview