Sam Austinon November 7, 202618 min read

GitHub Actions for ML Projects: Test, Lint, and Train on Every Commit

GitHub Actions for ML Projects: Test, Lint, and Train on Every Commit
Contents

Figure 1: GitHub Actions for ML Projects — you push code at 11 PM on a Friday. Your CI pipeline spins up, runs tests, catches that sneaky indentation error, lints your Python files, and—here's the kicker—trains your model on the latest data. All while you're grabbing a snack.

You push code at 11 PM on a Friday. Your CI pipeline spins up, runs tests, catches that sneaky indentation error, lints your Python files, and—here is the kicker—trains your model on the latest data. All while you are grabbing a snack. That is the dream, right?

I have been there. Three years ago, I would manually run pytest, flake8, and then wait 40 minutes for training to finish before merging. My weekends belonged to babysitting pipelines. Not anymore. GitHub Actions changed the game for ML teams, and I am going to show you exactly how to set it up.

Why GitHub Actions for ML?

ML projects are messy. You have got data versioning, model artifacts, dependency hell, and training runs that take forever. Traditional CI/CD tools were not built for this. GitHub Actions? It lives right next to your code. No context switching. No separate dashboard to check.

Plus, it is free for public repos. Private repos get 2,000 minutes/month on the GitHub Free plan. That is plenty for most side projects and small teams.

The real win: You define everything in YAML. Version control your pipeline alongside your code. Change the Python version? Update one line. Need a GPU runner? Swap the runner label. Done.

Setting Up Your First ML Workflow

Let us start with the basics. Create .github/workflows/ml-pipeline.yml in your repo. Here is a starter template that covers the essentials:

name: ML Pipeline

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  test-and-lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: '3.11'
      
      - name: Cache pip dependencies
        uses: actions/cache@v4
        with:
          path: ~/.cache/pip
          key: ${{ runner.os }}-pip-${{ hashFiles('requirements.txt') }}
          restore-keys: |
            ${{ runner.os }}-pip-
      
      - name: Install dependencies
        run: |
          pip install -r requirements.txt
          pip install -r requirements-dev.txt
      
      - name: Run linting
        run: |
          flake8 src/ tests/
          black --check src/ tests/
          isort --check-only src/ tests/
      
      - name: Run tests
        run: |
          pytest tests/ -v --cov=src --cov-report=xml
      
      - name: Upload coverage
        uses: codecov/codecov-action@v4

That is it. Push this, and every commit gets tested and linted automatically. No more "it works on my machine" excuses.

Adding Model Training to the Mix

Here is where it gets spicy. Training on every commit sounds expensive—and it can be. But you do not need to train on every push. Smart teams use conditional triggers:

train-model:
  needs: test-and-lint
  if: github.event_name == 'push' && github.ref == 'refs/heads/main'
  runs-on: ubuntu-latest
  # Or use a self-hosted GPU runner for real training
  steps:
    - uses: actions/checkout@v4
      with:
        fetch-depth: 0  # Needed for DVC
    
    - name: Set up Python
      uses: actions/setup-python@v5
      with:
        python-version: '3.11'
    
    - name: Install DVC
      run: pip install dvc[s3]
    
    - name: Pull data from DVC remote
      run: dvc pull
      env:
        AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
        AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
    
    - name: Train model
      run: |
        python src/train.py \
          --data data/processed \
          --model-dir models/ \
          --epochs 50 \
          --batch-size 32
    
    - name: Evaluate model
      run: python src/evaluate.py --model-dir models/
    
    - name: Upload model artifact
      uses: actions/upload-artifact@v4
      with:
        name: trained-model
        path: models/
        retention-days: 7

Key insight: The if condition means training only runs on pushes to main. PRs get fast feedback without the GPU bill. Your wallet will thank you.

Caching: Your Secret Weapon for Speed

ML dependencies are heavy. PyTorch, TensorFlow, transformers—these are not small packages. Caching saves massive amounts of time.

- name: Cache pip
  uses: actions/cache@v4
  with:
    path: ~/.cache/pip
    key: ${{ runner.os }}-pip-${{ hashFiles('requirements.txt') }}
    restore-keys: ${{ runner.os }}-pip-

- name: Cache DVC data
  uses: actions/cache@v4
  with:
    path: data/
    key: ${{ runner.os }}-data-${{ hashFiles('data/.dvc/*.dvc') }}
    restore-keys: ${{ runner.os }}-data-

- name: Cache model checkpoints
  uses: actions/cache@v4
  with:
    path: models/checkpoints
    key: ${{ runner.os }}-checkpoints-${{ github.sha }}
    restore-keys: ${{ runner.os }}-checkpoints-

Pro tip: Use hashFiles() on your lock files or DVC files as cache keys. When dependencies change, the cache invalidates automatically. No manual cleanup needed.

Handling Secrets Like a Pro

Never hardcode API keys. Ever. GitHub will literally yell at you if you try. Use repository secrets instead:

Go to Settings → Secrets and variables → Actions Add AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, WANDB_API_KEY, whatever you need Reference them in your workflow: ${{ secrets.WANDB_API_KEY }}

For team projects: Use organization-level secrets for shared credentials. For production: Consider GitHub Environments with required reviewers. Extra security, zero hassle.

Matrix Testing: Because "It Works on Python 3.11" Is Not Enough

Your users run different Python versions. Test against all of them:

test-and-lint:
  runs-on: ubuntu-latest
  strategy:
    matrix:
      python-version: ['3.9', '3.10', '3.11', '3.12']
      os: [ubuntu-latest, macos-latest, windows-latest]
    fail-fast: false  # Don't cancel other jobs on failure
  
  steps:
    - uses: actions/checkout@v4
    
    - name: Set up Python ${{ matrix.python-version }}
      uses: actions/setup-python@v5
      with:
        python-version: ${{ matrix.python-version }}
    
    - name: Run tests
      run: pytest tests/ -v

That is 12 test combinations. Runs in parallel. Catches platform-specific bugs before your users do. Worth every minute.

Self-Hosted Runners for GPU Training

GitHub-hosted runners do not have GPUs. (Yet. Fingers crossed.) For serious training, you need self-hosted runners.

Option 1: Your own machine

# On your GPU box
./config.sh --url https://github.com/yourorg/yourrepo --token YOUR_TOKEN
./run.sh

Label it self-hosted-gpu and reference it in your workflow:

runs-on: [self-hosted, gpu]

Option 2: Cloud instances

Spin up an EC2 g4dn.xlarge or GCP T4 instance. Use actions/runner Docker image. Auto-scale with webhook triggers if you are fancy.

Cost reality check: A g4dn.xlarge runs ~$0.53/hour. Training for 2 hours daily = ~$32/month. Cheaper than most managed ML platforms.

DVC Integration: Data Versioning Done Right

Git LFS is slow and expensive for ML data. DVC (Data Version Control) is the answer. It stores data in S3, GCS, Azure Blob—whatever you use—and versions it alongside your code.

- name: Setup DVC
  run: |
    pip install dvc[s3]
    dvc init --no-scm  # Already in a git repo
    dvc remote add -d myremote s3://my-bucket/dvc-store
  
- name: Pull data
  run: dvc pull
  env:
    AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
    AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}

The magic: dvc pull only downloads changed files. Huge datasets? No problem. Your pipeline stays fast.

Model Registry and Deployment

Training is only half the battle. You need to track experiments, register models, and deploy. GitHub Actions connects to all the major tools:

Weights & Biases

- name: Log to W&B
  run: |
    wandb login ${{ secrets.WANDB_API_KEY }}
    python src/train.py --wandb-project my-project

MLflow

- name: Log to MLflow
  run: |
    mlflow set_tracking_uri ${{ secrets.MLFLOW_TRACKING_URI }}
    python src/train.py --mlflow-experiment production

Hugging Face Hub

- name: Push model to HF Hub
  run: |
    huggingface-cli login --token ${{ secrets.HF_TOKEN }}
    python src/push_to_hub.py --repo-id username/model-name

Pick your stack. Connect it. Automate it. No manual uploads ever again.

Common Pitfalls (And How to Avoid Them)

I have made all of these mistakes. Learn from my pain:

Pitfall Solution
Forgetting to cache Add cache steps for pip, DVC data, and checkpoints
Training on every PR Use if: github.event_name == 'push' && github.ref == 'refs/heads/main'
Hardcoding secrets Use GitHub Secrets + Environments
No timeout on training Add timeout-minutes: 120 to job
Flaky tests breaking pipeline Mark flaky tests with @pytest.mark.flaky and rerun
Huge artifacts clogging storage Set retention-days: 7 on upload-artifact

Timeout is the big one. A stuck training job burns through your minutes. Always set a timeout.

Advanced: Conditional Deployment

Not every model deserves production. Add a quality gate:

deploy-model:
  needs: train-model
  if: |
    success() && 
    fromJson(needs.train-model.outputs.metrics).accuracy > 0.95
  runs-on: ubuntu-latest
  environment: production
  steps:
    - name: Deploy to production
      run: |
        python src/deploy.py --model-version ${{ github.sha }}

Extract metrics in your training step:

- name: Train and output metrics
  id: train
  run: |
    python src/train.py --output-metrics metrics.json
    echo "metrics=$(cat metrics.json)" >> $GITHUB_OUTPUT

Now deployment only happens when accuracy > 95%. Your production models just got smarter.

Monitoring and Alerting

Pipelines fail. You need to know. GitHub Actions has built-in notifications, but they are basic. Level up with:

Slack/Discord webhooks for real-time alerts PagerDuty/Opsgenie for on-call rotations Custom dashboards via GitHub's workflow run API

- name: Notify on failure
  if: failure()
  uses: slackapi/[email protected]
  with:
    payload: |
      {
        "text": "🚨 ML Pipeline failed!",
        "blocks": [
          {"type": "section", "text": {"type": "mrkdwn", "text": "*Repo:* ${{ github.repository }}\n*Run:* ${{ github.run_id }}\n*Branch:* ${{ github.ref }}"}
          }
        ]
      }
    env:
      SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK_URL }}

Get pinged the moment something breaks. Fix it before your boss asks.

Real-World Results

My team adopted this setup six months ago. Here is what changed:

  • PR feedback time: 45 minutes → 8 minutes (thanks to caching + matrix testing)
  • Broken main branch incidents: 3/month → 0 (quality gates caught everything)
  • Model deployment frequency: Monthly → Weekly (automation removed friction)
  • Cloud bill: Actually went down (no more forgotten training runs on expensive instances)

The best part? New team members push their first PR and the pipeline just works. No onboarding docs needed for CI.

Wrapping Up

GitHub Actions for ML is not just possible—it is the best choice for most teams. Native integration, flexible runners, powerful caching, and it lives where your code lives.

Start small. Add linting and testing first. Then layer on training, caching, and deployment. Iterate like you do with your models.

Questions? Hit me up in the comments. I have debugged enough YAML indentation errors to write a book. Happy automating! 🚀

What are You Looking For?

esc