ml_engineer.md

Roadmap

data-engineer-roadmap

Resources

ML System Design

• Architecting a Machine Learning Pipeline by Semi Koen
• Uber's ML Platform - Michelangelo
• Architecture of a real-world Machine Learning system by Louis Dorard
• How to deploy Machine Learning models by Christopher Samiullah

Miscellaneous

• ML in Production by Luigi Patruno
• Chip Huyen's Blog on ML

Technology Stack

Just some of the skills ML Engineering professionals use. One may not need to know all of them, but one technology from each area would make you well-rounded.

Software Engineering / Application Development

• Python
• Java
• Scala

REST API

• Flask

Databases / Data Stores

• SQL / Relational
• NoSQL
• Graph (not critically important)

Analysis / Query

• SQL

Parallel Processing / Distributed Computing

• Spark
• Pandas / Dask

Libraries

• Pandas
• numpy
• scikit-learn

Cloud

• AWS: S3, Lambda, DynamoDB, Kinesis, Batch ...
• Serverless frameworks, eg. AWS CDK, serverless, chalice

Workflow Management / Job Orchestration

• Airflow
• Luigi
• Kubeflow Pipelines

ML Frameworks

• PyTorch
• TensorFlow
• Keras
• FastAPI
• MLFlow
• Sagemaker

Deployment

• Docker
• Kubernetes
• KFServing (Kubeflow)
• TF-serving (Tensorflow)

Topics

• Data Versioning, eg. DVC
• Monitoring Drift, Performance
• Testing: Unit, Integration, Load, ML Robustness

References

ML Engineering skills by Chip Huyen

ML in Production by Made with ML, Goku Mohandas