Contents
Figure 1: Best Courses on Software Engineering for ML — you aced the ML course. Your models converge. Then you join a team and realize: you don't know how to write production code.
You aced the ML course. Your models converge. Your loss curves look beautiful. Then you join a team and realize: you don't know how to write production code. At all.
I have been there. My first PR had 47 comments. "Why is this in the global scope?" "Where are the tests?" "This function does five things." My model worked. My code was a disaster.
ML programs teach math. They do not teach engineering. This article fixes that gap.
Here are the courses, books, and resources that actually teach software engineering for ML practitioners—curated from someone who learned the hard way.
The Core Problem: Two Different Skill Trees
ML Researcher Skills:
- Gradient descent, backprop, attention mechanisms
- Paper reading, experiment design, hyperparameter tuning
- Jupyter notebooks, one-off scripts, "it works on my data"
ML Engineer Skills:
- Clean architecture, dependency injection, SOLID principles
- Testing (unit, integration, property-based), CI/CD
- Packaging, versioning, monitoring, rollback strategies
- Code review, documentation, API design
You need both. Most courses only teach the first column.
Tier 1: Software Engineering Fundamentals (Language-Agnostic)
Software Engineering at Google (Book + Free Chapters)
Authors: Titus Winters, Tom Manshreck, Hyrum Wright
Cost: ~$40 print, free chapters online
Time: 20-30 hours
Why it's essential: Written by people who run one of the world's largest codebases. Covers: code review culture, testing philosophy, dependency management, deprecation, scale.
Key chapters for ML folks:
- Chapter 4: Testing (not "how to write pytest" but "why test")
- Chapter 5: Code Review (the Google way)
- Chapter 11: Dependency Management (why requirements.txt isn't enough)
- Chapter 18: Deprecation (critical for model APIs)
My take: Read this before any other resource. It changes how you think about code.
A Philosophy of Software Design by John Ousterhout
Cost: ~$30
Time: 8-12 hours
Core thesis: Complexity is the enemy. Deep modules beat shallow ones.
ML relevance: Your model code tends toward "spaghetti with a side of notebook." This book teaches you to design modules that hide complexity.
Key concepts:
- Deep modules: Simple interface, complex implementation (your model wrapper)
- Information hiding: Don't expose
model.encoder.layer[3].attention - Pull complexity down: Push decisions to the leaves
- Define errors out of existence: Type system > runtime checks
Read this. Then refactor your training script. You'll see the difference.
Designing Data-Intensive Applications by Martin Kleppmann
Cost: ~$50
Time: 30-40 hours
Not ML-specific. But ML is data-intensive.
Covers: Storage engines, replication, partitioning, consistency, batch/stream processing. Every ML platform team lives this.
Chapters that matter for ML:
- Chapter 3: Storage & Retrieval (why Parquet > CSV, LSMTrees)
- Chapter 6: Partitioning (sharding your training data)
- Chapter 10: Batch Processing (MapReduce → Spark → your data pipeline)
- Chapter 11: Stream Processing (real-time inference pipelines)
Heavy but worth it. Skip if you only do research.
Tier 2: Python-Specific Engineering
Effective Python: 90 Specific Ways to Write Better Python by Brett Slatkin
Cost: ~$40
Time: 15-20 hours
Edition: Get 2nd edition (Python 3.8+)
Not a tutorial. 90 independent items. Dip in, level up.
Items every ML engineer needs:
- Item 14: Prefer dataclasses over dictionaries (your config objects)
- Item 19: Use
@dataclassfor lightweight containers (model outputs) - Item 23: Consider
@propertyinstead of getters/setters (clean APIs) - Item 31: Use
functools.lru_cachefor memoization (expensive preprocessing) - Item 47: Use
pathlibfor path manipulation (cross-platform data paths) - Item 59: Consider
contextlibfor resource management (GPU memory, file handles) - Item 72: Use
typing.Protocolfor structural subtyping (plug-and-play models) - Item 83: Test with
unittest.mock(mock your wandb, S3, API calls)
Keep it on your desk. Reference, don't memorize.
Python Packaging User Guide (Official, Free)
URL: packaging.python.org
Time: 5-10 hours
You will package code. Do it right once.
Learn:
pyproject.toml(modern standard, killsetup.py)build + twine(don'tpython setup.py sdist)- Versioning (SemVer, CalVer, why
0.0.1is fine for private) - Optional dependencies (
[dev],[gpu],[docs]) - Publishing to PyPI / private index
My pyproject.toml template (steal it):
[build-system]
requires = ["setuptools>=68", "wheel"]
build-backend = "setuptools.build_meta"
[project]
name = "my-ml-lib"
version = "0.1.0"
description = "My ML library"
readme = "README.md"
requires-python = ">=3.10"
dependencies = [
"torch>=2.1",
"transformers>=4.35",
"numpy>=1.24",
]
optional-dependencies = {
dev = ["pytest", "ruff", "mypy", "pre-commit"],
gpu = ["flash-attn"],
docs = ["mkdocs", "mkdocstrings"],
}
[tool.setuptools.packages.find]
where = ["src"]
[tool.ruff]
line-length = 100
target-version = "py310"
[tool.mypy]
python_version = "3.10"
strict = true
Copy this. Adapt. Done.
Architecture Patterns with Python by Harry Percival & Bob Gregory
Cost: Free online (cosmicpython.com) or ~$40 print
Time: 20-30 hours
Teaches: Domain-driven design, dependency injection, repository pattern, unit of work—in Python, with tests.
Why ML engineers need this: Your training script is a domain. Data loading, preprocessing, augmentation, model, loss, optimizer, logging—each is a separable concern.
Build their example project. Then refactor your training loop to use:
- Repository pattern for data access
- Service layer for training logic
- Dependency injection for model/optimizer/scheduler
- Events for logging/checkpointing (decouple from wandb/TensorBoard)
Your tests will actually test logic, not mocks.
Tier 3: ML-Specific Engineering Courses
Full Stack Deep Learning (UC Berkeley, Free)
URL: fullstackdeeplearning.com
Instructors: Sergey Karayev, Pieter Abbeel, Josh Tobin
Time: 40-60 hours (lectures + assignments)
The gold standard for "ML → Production."
Covers:
- Project setup: Config, logging, reproducibility, CI/CD
- Data: Versioning, pipelines, quality checks, labeling
- Training: Distributed, mixed precision, checkpointing, hyperparam tuning
- Deployment: Containers, model servers (Triton, TorchServe), monitoring
- MLOps: Feature stores, experiment tracking, model registry
Assignments are real. You build a complete system. Not a toy.
Prereqs: Solid PyTorch, comfortable with CLI, basic Docker.
My take: If you take one ML engineering course, make it this one. Free. World-class.
MLOps Zoomcamp (DataTalks.Club, Free)
URL: github.com/DataTalksClub/mlops-zoomcamp
Time: 10 weeks, ~8 hrs/week
Cohort-based. Live lectures + homework + capstone project. Community is the value.
Stack: Prefect, MLflow, Evidently, Docker, Kubernetes, Terraform, AWS/GCP.
Week-by-week:
- MLflow + experiment tracking
- Model deployment (FastAPI, Docker, Cloud Run)
- Orchestration (Prefect, scheduling, retries)
- Model monitoring (Evidently, drift detection)
- CI/CD (GitHub Actions, testing, deployment)
- Kubernetes (Kubeflow, KServe, scaling)
- Feature stores (Feast)
- Best practices (testing, documentation, security)
Capstone: End-to-end project you can show in interviews.
Downside: Tool-heavy. Some tools change fast (Kubeflow especially). Principles transfer.
Verdict: Best free structured program. Join a cohort.
Made With ML (Goku Mohandas, Free)
URL: madewithml.com
Time: 20-30 hours
Focus: Software engineering practices for ML. Not MLOps tools.
Topics:
- Project structure (modular, testable)
- Configuration (Hydra, OmegaConf)
- Logging (structlog, not print)
- Testing (pytest, fixtures, parametrize, property-based)
- CI/CD (GitHub Actions, pre-commit, dependabot)
- Documentation (MkDocs, type hints, docstrings)
- Reproducibility (seeds, environments, data versioning)
- Packaging (pyproject.toml, versioning, publishing)
Code-first. Every lesson has a PR you can study.
My take: Closest to "how I actually work." Opinionated in the right ways.
Deep Learning Engineering (NYU / Yann LeCun's Course, Free)
URL: cds.nyu.edu/deep-learning-engineering/
Time: Semester course (~40 hours lectures)
More academic. Covers systems: memory management, kernel fusion, distributed training, quantization, compilation (TVM, XLA).
Watch if: You want to understand why your GPU OOMs, why DDP is faster than DP, why torch.compile exists.
Skip if: You just want to deploy models.
Tier 4: Open Source Practice
Contributing to Open Source (Free Resources)
First-timers only: firsttimersonly.com, goodfirstissue.dev
Real talk: Don't start with "good first issue" labels. They're often stale or trivial.
Better approach:
- Use the library daily. Find a bug you hit. Fix it.
- Read the
CONTRIBUTING.md. Every serious project has one. - Run tests locally first.
pytest/tox/nox. - Small PR > big PR. One fix, one test, clear description.
- Respond to review fast. That's how you get invited back.
Projects Worth Contributing To (ML Ecosystem)
| Project | Difficulty | Good For Learning |
|---|---|---|
| Hugging Face Transformers | Medium | Large codebase, CI, testing patterns |
| PyTorch Lightning | Medium | Plugin architecture, callbacks |
| Weights & Biases SDK | Low-Med | API design, Python packaging |
| DVC | Medium | CLI design, cloud integrations |
| LangChain | Low | Rapid iteration, plugin system |
| FastAPI | Low | Modern Python, type hints, async |
| Pydantic | Medium | Validation, serialization, performance |
| Rich / Textual | Low | Terminal UX, rendering |
Pick one you use. Contribute to fix your own pain.
Open Source Guides (GitHub, Free)
URL: opensource.guide
Time: 2-3 hours
Covers: Starting a project, finding contributors, licensing, governance, code of conduct, funding.
Read before you open-source your research code. Saves headaches.
Tier 5: Specialized Deep Dives
Testing ML Systems (No Single Course—Build Your Own Curriculum)
Resources:
- Property-based testing:
hypothesis.readthedocs.io— test properties not examples - Mutation testing:
mutmut— your tests catch bugs? - Contract testing:
schemathesis— your API matches OpenAPI spec? - ML-specific:
pytest-ml(custom),deepchecks(data/model validation)
Pattern I use:
# test_model.py
import hypothesis.strategies as st
from hypothesis import given
@given(st.integers(1, 100), st.integers(1, 100))
def test_model_output_shape(batch_size, seq_len):
model = MyModel()
x = torch.randn(batch_size, seq_len, 512)
out = model(x)
assert out.shape == (batch_size, seq_len, vocab_size)
# Property: loss decreases after one step (usually)
@given(st.data())
def test_training_step_decreases_loss(data):
model, optimizer, batch = setup()
loss_before = train_step(model, optimizer, batch)
loss_after = train_step(model, optimizer, batch)
# Not guaranteed but good smoke test
assert loss_after <= loss_before * 1.1
API Design for ML Models
Resources:
- FastAPI docs (
fastapi.tiangolo.com) — best modern Python API framework - Google API Design Guide (
cloud.google.com/apis/design) — resource-oriented design - Stripe API docs — gold standard for DX
Patterns for ML APIs:
# Good: async, batching, streaming, cancellation
@app.post("/generate")
async def generate(request: GenerateRequest) -> GenerateResponse:
# Validate
# Queue (if busy)
# Stream tokens via SSE
# Handle client disconnect
pass
# Bad: sync, no timeout, no batching, blocks event loop
def generate(prompt: str) -> str:
return model.generate(prompt)
Monitoring & Observability
Courses:
- Grafana Labs: Free courses on Prometheus, Loki, Tempo, Grafana
- Datadog: Free fundamentals certification
- Honeycomb: "Observability Engineering" book (free PDF)
ML-specific metrics to track:
- Latency: p50, p95, p99 (not average)
- Throughput: req/s, tokens/s
- Error rate: by type (OOM, timeout, validation)
- Model quality: drift, accuracy on shadow traffic
- GPU utilization: memory, compute, SM occupancy
My Recommended Learning Path (2024)
If you have 3 months, 10 hrs/week:
| Month | Focus | Resources |
|---|---|---|
| 1 | Python packaging, testing, CI/CD | Effective Python (Ch 14,19,23,31,47,59,72,83), Python Packaging Guide, GitHub Actions docs |
| 2 | Architecture, config, logging | Architecture Patterns with Python, Made With ML (all), Hydra/OmegaConf docs |
| 3 | ML systems, deployment, monitoring | Full Stack Deep Learning (lectures 1-15), MLOps Zoomcamp (weeks 1-5), FastAPI tutorial |
If you have 1 month, 20 hrs/week:
| Week | Focus |
|---|---|
| 1 | Read A Philosophy of Software Design + Effective Python (key items) |
| 2 | Complete Made With ML + Python Packaging Guide |
| 3 | Full Stack Deep Learning lectures 1-10 + build one assignment |
| 4 | Contribute 3 PRs to a project you use |
If you have 1 weekend:
| Time | Activity |
|---|---|
| Saturday AM | Read A Philosophy of Software Design |
| Saturday PM | Complete Python Packaging Guide + set up pyproject.toml + ruff + mypy + pre-commit on your project |
| Sunday AM | Watch Full Stack Deep Learning lectures 1, 2, 5 (project setup, data, training) |
| Sunday PM | Add GitHub Actions CI (test, lint, type-check) to your repo |
What Not to Waste Time On
| Resource | Why Skip |
|---|---|
| Coursera "ML Engineering" specializations | Too academic, tool-focused, outdated |
| Kaggle "MLOps" courses | Toy environments, no real infrastructure |
| Vendor certifications (AWS/GCP/Azure ML) | Teach their tools, not principles |
| "Clean Code" by Robert Martin | Some good ideas, but dogmatic; Ousterhout is better |
| Design Patterns (GoF book) | Java-centric, over-applied; Architecture Patterns with Python is better |
The "Senior Engineer" Checklist
Before you call yourself an ML Engineer, you should be able to:
- Write a
pyproject.tomlthat builds a wheel + publishes to PyPI - Set up
ruff+mypy+pytest+pre-commitin 10 minutes - Design a config system (Hydra/OmegaConf) with dev/staging/prod overrides
- Write unit tests that run in <5 seconds (no GPU, no network)
- Write integration tests that spin up real services (Testcontainers)
- Design a REST API with proper status codes, validation, OpenAPI docs
- Containerize with multi-stage Dockerfile (<100 MB final image)
- Write GitHub Actions workflow: test → lint → type-check → build → deploy
- Implement structured logging (JSON, correlation IDs, log levels)
- Add metrics (Prometheus) + traces (OpenTelemetry) to a model server
- Do a code review that catches design issues, not style nits
- Write a useful README + CONTRIBUTING.md + CHANGELOG.md
- Debug a production issue using logs + metrics + traces (not print statements)
- Contribute a merged PR to an open-source ML library
How many can you check today? That's your roadmap.
Final Thoughts
Software engineering is a craft. You learn by doing, reviewing, breaking, fixing.
Courses give you vocabulary. Practice gives you intuition.
My advice: Pick one project. Apply one new practice per week. This week: add ruff + mypy. Next week: add pytest + one integration test. Week after: add GitHub Actions.
In 6 months, your code will be unrecognizable. In a good way.
And when you open that PR to Transformers or Lightning or DVC—that's when you're really an ML engineer. 🚀
Questions? Resources I missed? Horror stories from your first code review? Drop them below. I still remember my 47-comment PR. Happy to compare scars. 😅
Sources
[1]Google, Software Engineering at Google — Titus Winters et al.: abseil.io/resources/swe-book/html
[2]John Ousterhout, A Philosophy of Software Design: web.stanford.edu/~ouster/cgi-bin/book.php[3]Martin Kleppmann, Designing Data-Intensive Applications: dataintensiveapplications.com[4]Brett Slatkin, Effective Python (2nd ed.): effectivepython.com[5]Python Packaging User Guide: packaging.python.org[6]Harry Percival & Bob Gregory, Architecture Patterns with Python: cosmicpython.com[7]Full Stack Deep Learning: fullstackdeeplearning.com[8]DataTalks.Club, MLOps Zoomcamp: github.com/DataTalksClub/mlops-zoomcamp[9]Goku Mohandas, Made With ML: madewithml.com[10]NYU CDS, Deep Learning Engineering: cds.nyu.edu/deep-learning-engineering[11]Open Source Guides: opensource.guide[12]Hypothesis Documentation: hypothesis.readthedocs.io[13]FastAPI Documentation: fastapi.tiangolo.com[14]Google API Design Guide: cloud.google.com/apis/design[15]Grafana Labs Training: grafana.com/training[16]Honeycomb, Observability Engineering: honeycomb.io/observability-engineering