Introduction to Machine Learning

This course is an introduction to the theory and techniques of machine learning for creating data-driven learning models.

Course Objectives

Knowledge and Understanding

Acquire a basic understanding of the most common techniques in supervised and unsupervised learning, as well as the ability to quantify and evaluate the performance of various models.

Application of Knowledge

Develop the skills to create a machine learning model from scratch, starting with the preprocessing of raw data and culminating in the presentation of final results in regression, classification, or clustering analysis.

Communication Skills

Gain the ability to present the results of developed machine learning models, including an explanation of the motivations behind their choices.

Learning Skills

Be able to navigate the machine learning literature, compare different models, and improve upon the chosen models.

Course Topics

1. Basics, Linear and Logistic Regression

   • Introduction to fundamental concepts in machine learning, including supervised and unsupervised learning.
   • Detailed exploration of linear regression for continuous target variables and logistic regression for binary classification problems.
   • Practical applications and implementation of these models in real-world scenarios.

2. Model Complexity, Bias-Variance Tradeoff, Regularization, Model Evaluation

   • Understanding the concept of model complexity and its impact on model performance.
   • Detailed discussion of the bias-variance tradeoff and strategies to balance them.
   • Techniques for regularizing models to prevent overfitting, including L1 and L2 regularization.
   • Methods for evaluating model performance, including cross-validation, precision, recall, and F1 score.

3. Decision Trees and K-NN Classifiers

   • Introduction to decision trees for both regression and classification tasks.
   • Understanding the concepts of entropy, information gain, and tree pruning.
   • Exploring k-Nearest Neighbors (K-NN) classifiers and their applications in pattern recognition and data classification.

4. PCA and Dimensionality Reduction, Probability Density Estimation, Clustering Methods

   • Techniques for Principal Component Analysis (PCA) to reduce the dimensionality of datasets while preserving important information.
   • Introduction to probability density estimation methods for understanding the distribution of data.
   • Overview of clustering methods such as K-means, hierarchical clustering, and DBSCAN for grouping similar data points.

5. Support Vector Machines and Kernel Methods

   • In-depth study of Support Vector Machines (SVMs) for classification and regression tasks.
   • Understanding the mathematical foundations of SVMs and the role of the hyperplane.
   • Exploration of kernel methods to handle non-linear decision boundaries.

6. Neural Networks and Deep Learning

   • Comprehensive introduction to neural networks, including perceptrons, feedforward networks, and backpropagation.
   • Study of advanced neural network architectures such as Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs).
   • Exploration of deep learning techniques and frameworks for building and training deep neural networks.
   • Practical applications of deep learning in areas such as image recognition, natural language processing, and reinforcement learning.

Repository Content

This repository contains four projects completed during the year, reflecting the concepts listed above. These projects are:

• Challenge 0: Basics, Linear and Logistic Regression
• Challenge 1: Application of different Supervised Learning and Unsupervised Learning methods, studying bias-variance trade-off and evaluation methods
• Challenge 2: Implementation of Support Vector Machines and Principal Component Analysis with kernel methods, applying the studied notions
• Challenge 3: Neural Network, comparison between Fully Connected and Convolutional Networks