Sam Austinon November 7, 202620 min read

Best Courses on Software Engineering and Open-Source Practice for ML

Best Courses on Software Engineering and Open-Source Practice for ML
Contents

Figure 1: Best Courses on Software Engineering for ML — you aced the ML course. Your models converge. Then you join a team and realize: you don't know how to write production code.

You aced the ML course. Your models converge. Your loss curves look beautiful. Then you join a team and realize: you don't know how to write production code. At all.

I have been there. My first PR had 47 comments. "Why is this in the global scope?" "Where are the tests?" "This function does five things." My model worked. My code was a disaster.

ML programs teach math. They do not teach engineering. This article fixes that gap.

Here are the courses, books, and resources that actually teach software engineering for ML practitioners—curated from someone who learned the hard way.

The Core Problem: Two Different Skill Trees

ML Researcher Skills:

  • Gradient descent, backprop, attention mechanisms
  • Paper reading, experiment design, hyperparameter tuning
  • Jupyter notebooks, one-off scripts, "it works on my data"

ML Engineer Skills:

  • Clean architecture, dependency injection, SOLID principles
  • Testing (unit, integration, property-based), CI/CD
  • Packaging, versioning, monitoring, rollback strategies
  • Code review, documentation, API design

You need both. Most courses only teach the first column.


Tier 1: Software Engineering Fundamentals (Language-Agnostic)

Software Engineering at Google (Book + Free Chapters)

Authors: Titus Winters, Tom Manshreck, Hyrum Wright
Cost: ~$40 print, free chapters online
Time: 20-30 hours

Why it's essential: Written by people who run one of the world's largest codebases. Covers: code review culture, testing philosophy, dependency management, deprecation, scale.

Key chapters for ML folks:

  • Chapter 4: Testing (not "how to write pytest" but "why test")
  • Chapter 5: Code Review (the Google way)
  • Chapter 11: Dependency Management (why requirements.txt isn't enough)
  • Chapter 18: Deprecation (critical for model APIs)

My take: Read this before any other resource. It changes how you think about code.


A Philosophy of Software Design by John Ousterhout

Cost: ~$30
Time: 8-12 hours

Core thesis: Complexity is the enemy. Deep modules beat shallow ones.

ML relevance: Your model code tends toward "spaghetti with a side of notebook." This book teaches you to design modules that hide complexity.

Key concepts:

  • Deep modules: Simple interface, complex implementation (your model wrapper)
  • Information hiding: Don't expose model.encoder.layer[3].attention
  • Pull complexity down: Push decisions to the leaves
  • Define errors out of existence: Type system > runtime checks

Read this. Then refactor your training script. You'll see the difference.


Designing Data-Intensive Applications by Martin Kleppmann

Cost: ~$50
Time: 30-40 hours

Not ML-specific. But ML is data-intensive.

Covers: Storage engines, replication, partitioning, consistency, batch/stream processing. Every ML platform team lives this.

Chapters that matter for ML:

  • Chapter 3: Storage & Retrieval (why Parquet > CSV, LSMTrees)
  • Chapter 6: Partitioning (sharding your training data)
  • Chapter 10: Batch Processing (MapReduce → Spark → your data pipeline)
  • Chapter 11: Stream Processing (real-time inference pipelines)

Heavy but worth it. Skip if you only do research.


Tier 2: Python-Specific Engineering

Effective Python: 90 Specific Ways to Write Better Python by Brett Slatkin

Cost: ~$40
Time: 15-20 hours
Edition: Get 2nd edition (Python 3.8+)

Not a tutorial. 90 independent items. Dip in, level up.

Items every ML engineer needs:

  • Item 14: Prefer dataclasses over dictionaries (your config objects)
  • Item 19: Use @dataclass for lightweight containers (model outputs)
  • Item 23: Consider @property instead of getters/setters (clean APIs)
  • Item 31: Use functools.lru_cache for memoization (expensive preprocessing)
  • Item 47: Use pathlib for path manipulation (cross-platform data paths)
  • Item 59: Consider contextlib for resource management (GPU memory, file handles)
  • Item 72: Use typing.Protocol for structural subtyping (plug-and-play models)
  • Item 83: Test with unittest.mock (mock your wandb, S3, API calls)

Keep it on your desk. Reference, don't memorize.


Python Packaging User Guide (Official, Free)

URL: packaging.python.org
Time: 5-10 hours

You will package code. Do it right once.

Learn:

  • pyproject.toml (modern standard, kill setup.py)
  • build + twine (don't python setup.py sdist)
  • Versioning (SemVer, CalVer, why 0.0.1 is fine for private)
  • Optional dependencies ([dev], [gpu], [docs])
  • Publishing to PyPI / private index

My pyproject.toml template (steal it):

[build-system]
requires = ["setuptools>=68", "wheel"]
build-backend = "setuptools.build_meta"

[project]
name = "my-ml-lib"
version = "0.1.0"
description = "My ML library"
readme = "README.md"
requires-python = ">=3.10"
dependencies = [
    "torch>=2.1",
    "transformers>=4.35",
    "numpy>=1.24",
]
optional-dependencies = {
    dev = ["pytest", "ruff", "mypy", "pre-commit"],
    gpu = ["flash-attn"],
    docs = ["mkdocs", "mkdocstrings"],
}

[tool.setuptools.packages.find]
where = ["src"]

[tool.ruff]
line-length = 100
target-version = "py310"

[tool.mypy]
python_version = "3.10"
strict = true

Copy this. Adapt. Done.


Architecture Patterns with Python by Harry Percival & Bob Gregory

Cost: Free online (cosmicpython.com) or ~$40 print
Time: 20-30 hours

Teaches: Domain-driven design, dependency injection, repository pattern, unit of work—in Python, with tests.

Why ML engineers need this: Your training script is a domain. Data loading, preprocessing, augmentation, model, loss, optimizer, logging—each is a separable concern.

Build their example project. Then refactor your training loop to use:

  • Repository pattern for data access
  • Service layer for training logic
  • Dependency injection for model/optimizer/scheduler
  • Events for logging/checkpointing (decouple from wandb/TensorBoard)

Your tests will actually test logic, not mocks.


Tier 3: ML-Specific Engineering Courses

Full Stack Deep Learning (UC Berkeley, Free)

URL: fullstackdeeplearning.com
Instructors: Sergey Karayev, Pieter Abbeel, Josh Tobin
Time: 40-60 hours (lectures + assignments)

The gold standard for "ML → Production."

Covers:

  • Project setup: Config, logging, reproducibility, CI/CD
  • Data: Versioning, pipelines, quality checks, labeling
  • Training: Distributed, mixed precision, checkpointing, hyperparam tuning
  • Deployment: Containers, model servers (Triton, TorchServe), monitoring
  • MLOps: Feature stores, experiment tracking, model registry

Assignments are real. You build a complete system. Not a toy.

Prereqs: Solid PyTorch, comfortable with CLI, basic Docker.

My take: If you take one ML engineering course, make it this one. Free. World-class.


MLOps Zoomcamp (DataTalks.Club, Free)

URL: github.com/DataTalksClub/mlops-zoomcamp
Time: 10 weeks, ~8 hrs/week

Cohort-based. Live lectures + homework + capstone project. Community is the value.

Stack: Prefect, MLflow, Evidently, Docker, Kubernetes, Terraform, AWS/GCP.

Week-by-week:

  1. MLflow + experiment tracking
  2. Model deployment (FastAPI, Docker, Cloud Run)
  3. Orchestration (Prefect, scheduling, retries)
  4. Model monitoring (Evidently, drift detection)
  5. CI/CD (GitHub Actions, testing, deployment)
  6. Kubernetes (Kubeflow, KServe, scaling)
  7. Feature stores (Feast)
  8. Best practices (testing, documentation, security)

Capstone: End-to-end project you can show in interviews.

Downside: Tool-heavy. Some tools change fast (Kubeflow especially). Principles transfer.

Verdict: Best free structured program. Join a cohort.


Made With ML (Goku Mohandas, Free)

URL: madewithml.com
Time: 20-30 hours

Focus: Software engineering practices for ML. Not MLOps tools.

Topics:

  • Project structure (modular, testable)
  • Configuration (Hydra, OmegaConf)
  • Logging (structlog, not print)
  • Testing (pytest, fixtures, parametrize, property-based)
  • CI/CD (GitHub Actions, pre-commit, dependabot)
  • Documentation (MkDocs, type hints, docstrings)
  • Reproducibility (seeds, environments, data versioning)
  • Packaging (pyproject.toml, versioning, publishing)

Code-first. Every lesson has a PR you can study.

My take: Closest to "how I actually work." Opinionated in the right ways.


Deep Learning Engineering (NYU / Yann LeCun's Course, Free)

URL: cds.nyu.edu/deep-learning-engineering/
Time: Semester course (~40 hours lectures)

More academic. Covers systems: memory management, kernel fusion, distributed training, quantization, compilation (TVM, XLA).

Watch if: You want to understand why your GPU OOMs, why DDP is faster than DP, why torch.compile exists.

Skip if: You just want to deploy models.


Tier 4: Open Source Practice

Contributing to Open Source (Free Resources)

First-timers only: firsttimersonly.com, goodfirstissue.dev

Real talk: Don't start with "good first issue" labels. They're often stale or trivial.

Better approach:

  1. Use the library daily. Find a bug you hit. Fix it.
  2. Read the CONTRIBUTING.md. Every serious project has one.
  3. Run tests locally first. pytest / tox / nox.
  4. Small PR > big PR. One fix, one test, clear description.
  5. Respond to review fast. That's how you get invited back.

Projects Worth Contributing To (ML Ecosystem)

Project Difficulty Good For Learning
Hugging Face Transformers Medium Large codebase, CI, testing patterns
PyTorch Lightning Medium Plugin architecture, callbacks
Weights & Biases SDK Low-Med API design, Python packaging
DVC Medium CLI design, cloud integrations
LangChain Low Rapid iteration, plugin system
FastAPI Low Modern Python, type hints, async
Pydantic Medium Validation, serialization, performance
Rich / Textual Low Terminal UX, rendering

Pick one you use. Contribute to fix your own pain.


Open Source Guides (GitHub, Free)

URL: opensource.guide
Time: 2-3 hours

Covers: Starting a project, finding contributors, licensing, governance, code of conduct, funding.

Read before you open-source your research code. Saves headaches.


Tier 5: Specialized Deep Dives

Testing ML Systems (No Single Course—Build Your Own Curriculum)

Resources:

  • Property-based testing: hypothesis.readthedocs.io — test properties not examples
  • Mutation testing: mutmut — your tests catch bugs?
  • Contract testing: schemathesis — your API matches OpenAPI spec?
  • ML-specific: pytest-ml (custom), deepchecks (data/model validation)

Pattern I use:

# test_model.py
import hypothesis.strategies as st
from hypothesis import given

@given(st.integers(1, 100), st.integers(1, 100))
def test_model_output_shape(batch_size, seq_len):
    model = MyModel()
    x = torch.randn(batch_size, seq_len, 512)
    out = model(x)
    assert out.shape == (batch_size, seq_len, vocab_size)

# Property: loss decreases after one step (usually)
@given(st.data())
def test_training_step_decreases_loss(data):
    model, optimizer, batch = setup()
    loss_before = train_step(model, optimizer, batch)
    loss_after = train_step(model, optimizer, batch)
    # Not guaranteed but good smoke test
    assert loss_after <= loss_before * 1.1

API Design for ML Models

Resources:

  • FastAPI docs (fastapi.tiangolo.com) — best modern Python API framework
  • Google API Design Guide (cloud.google.com/apis/design) — resource-oriented design
  • Stripe API docs — gold standard for DX

Patterns for ML APIs:

# Good: async, batching, streaming, cancellation
@app.post("/generate")
async def generate(request: GenerateRequest) -> GenerateResponse:
    # Validate
    # Queue (if busy)
    # Stream tokens via SSE
    # Handle client disconnect
    pass

# Bad: sync, no timeout, no batching, blocks event loop
def generate(prompt: str) -> str:
    return model.generate(prompt)

Monitoring & Observability

Courses:

  • Grafana Labs: Free courses on Prometheus, Loki, Tempo, Grafana
  • Datadog: Free fundamentals certification
  • Honeycomb: "Observability Engineering" book (free PDF)

ML-specific metrics to track:

  • Latency: p50, p95, p99 (not average)
  • Throughput: req/s, tokens/s
  • Error rate: by type (OOM, timeout, validation)
  • Model quality: drift, accuracy on shadow traffic
  • GPU utilization: memory, compute, SM occupancy

If you have 3 months, 10 hrs/week:

Month Focus Resources
1 Python packaging, testing, CI/CD Effective Python (Ch 14,19,23,31,47,59,72,83), Python Packaging Guide, GitHub Actions docs
2 Architecture, config, logging Architecture Patterns with Python, Made With ML (all), Hydra/OmegaConf docs
3 ML systems, deployment, monitoring Full Stack Deep Learning (lectures 1-15), MLOps Zoomcamp (weeks 1-5), FastAPI tutorial

If you have 1 month, 20 hrs/week:

Week Focus
1 Read A Philosophy of Software Design + Effective Python (key items)
2 Complete Made With ML + Python Packaging Guide
3 Full Stack Deep Learning lectures 1-10 + build one assignment
4 Contribute 3 PRs to a project you use

If you have 1 weekend:

Time Activity
Saturday AM Read A Philosophy of Software Design
Saturday PM Complete Python Packaging Guide + set up pyproject.toml + ruff + mypy + pre-commit on your project
Sunday AM Watch Full Stack Deep Learning lectures 1, 2, 5 (project setup, data, training)
Sunday PM Add GitHub Actions CI (test, lint, type-check) to your repo

What Not to Waste Time On

Resource Why Skip
Coursera "ML Engineering" specializations Too academic, tool-focused, outdated
Kaggle "MLOps" courses Toy environments, no real infrastructure
Vendor certifications (AWS/GCP/Azure ML) Teach their tools, not principles
"Clean Code" by Robert Martin Some good ideas, but dogmatic; Ousterhout is better
Design Patterns (GoF book) Java-centric, over-applied; Architecture Patterns with Python is better

The "Senior Engineer" Checklist

Before you call yourself an ML Engineer, you should be able to:

  • Write a pyproject.toml that builds a wheel + publishes to PyPI
  • Set up ruff + mypy + pytest + pre-commit in 10 minutes
  • Design a config system (Hydra/OmegaConf) with dev/staging/prod overrides
  • Write unit tests that run in <5 seconds (no GPU, no network)
  • Write integration tests that spin up real services (Testcontainers)
  • Design a REST API with proper status codes, validation, OpenAPI docs
  • Containerize with multi-stage Dockerfile (<100 MB final image)
  • Write GitHub Actions workflow: test → lint → type-check → build → deploy
  • Implement structured logging (JSON, correlation IDs, log levels)
  • Add metrics (Prometheus) + traces (OpenTelemetry) to a model server
  • Do a code review that catches design issues, not style nits
  • Write a useful README + CONTRIBUTING.md + CHANGELOG.md
  • Debug a production issue using logs + metrics + traces (not print statements)
  • Contribute a merged PR to an open-source ML library

How many can you check today? That's your roadmap.


Final Thoughts

Software engineering is a craft. You learn by doing, reviewing, breaking, fixing.

Courses give you vocabulary. Practice gives you intuition.

My advice: Pick one project. Apply one new practice per week. This week: add ruff + mypy. Next week: add pytest + one integration test. Week after: add GitHub Actions.

In 6 months, your code will be unrecognizable. In a good way.

And when you open that PR to Transformers or Lightning or DVC—that's when you're really an ML engineer. 🚀

Questions? Resources I missed? Horror stories from your first code review? Drop them below. I still remember my 47-comment PR. Happy to compare scars. 😅


Sources

What are You Looking For?

esc