Contents
Figure 1: Best Dependency and Security Scanners for Python ML Projects — four layers of an ML stack, and the one scanner everyone runs only covers the first
Your ML project pulls in 300 packages, three CUDA-flavored Docker layers, and a model checkpoint from a stranger on the internet. Then someone asks, "Did we scan it?" You say yes, because pip-audit passed. But pip-audit never looked at the checkpoint, the container, or the typosquatted package that slipped into your requirements file last Tuesday.
The biggest mistake is not picking the wrong scanner — it is trusting one scanner to cover all four attack surfaces. ML projects have more of them than web apps do, and each surface needs a different kind of tool.
The Four Attack Surfaces in an ML Project
Every Python ML project exposes four layers:
- Python dependencies — your direct and transitive packages, pinned or floating.
- Your own code — unsafe patterns, injection bugs, and leaked secrets in scripts and notebooks.
- Containers and system libraries — base images, CUDA layers, and OS packages.
- Model and data artifacts — checkpoints, pickles, downloaded weights, and dataset files.
Most scanner roundups cover layer one and stop. Ever shipped a clean pip-audit report on a project that loads unvetted pickles? That gap is what this article closes, and it is why the tool list below is organised by layer instead of by popularity.
Layer 1: Python Dependency Scanners
What these tools cannot do
Before the list, one limit matters. CVE-based scanners only know about published advisories. A brand-new malicious PyPI package, a typosquat with a plausible name, or a compromised maintainer account has no CVE yet, so pip-audit, Safety, and OSV-Scanner all see nothing. Keep that in mind: this layer catches known-bad versions, not bad intent.
pip-audit
pip-audit is my default first gate. It is maintained under the Python Packaging Authority on GitHub, licensed Apache-2.0, and currently at version 2.10.1 [1]. It reads a requirements file or the installed environment and checks it against the PyPI advisory database, with JSON and SARIF output for CI.
pip install pip-audit
# Audit a requirements file
pip-audit -r requirements.txt
# Audit the installed environment
pip-audit
# Second advisory source (OSV instead of PyPI)
pip-audit --service osv
# Machine-readable output for CI
pip-audit -r requirements.txt --format json --output audit.json
One correction to a common assumption: pip-audit already exits non-zero when it finds a vulnerability. The --strict flag is narrower than most guides imply — it makes the run fail on dependency resolution errors too, which is useful in CI but is not what turns the scan "on".
OSV-Scanner
OSV-Scanner is Google's open-source client for OSV.dev, an aggregated advisory database that pulls from PyPI, GitHub, and the NVD among others. Aggregation matters: a vulnerability disclosed on GitHub but never filed as a CVE still shows up, which is exactly the gap single-source scanners leave.
The current CLI is the v2 line, which splits scanning by target [2]:
# Scan a lockfile or manifest
osv-scanner scan source -r .
# Scan a built container image
osv-scanner scan image my-ml-image:latest
Run it beside pip-audit rather than instead of it. The overlap costs seconds and occasionally finds something the other missed — different databases, different ingestion lag.
Dependabot and Renovate
Dependabot does not just scan; it opens pull requests. It keeps dependencies current rather than only patching known vulnerabilities, and it is free on GitHub but only on GitHub. If you host elsewhere, Renovate is the cross-platform equivalent.
Automated update PRs can break your stack. ML libraries pin each other through CUDA and ABI constraints, so always let the full test suite gate the merge. A green Dependabot PR on a CPU-only test suite proves almost nothing about a GPU training job.
Snyk and Socket
These two solve different problems, and it is worth separating them:
- Snyk is a commercial platform whose selling point is reachability analysis — it checks whether your code actually calls the vulnerable function, which cuts noise on a 300-package tree. It also bundles licence compliance. Verify current pricing and free-tier limits yourself.
- Socket is behaviour-based rather than advisory-based. It flags what a package does on install: postinstall scripts, network calls, obfuscated code. That is the layer CVE scanners structurally cannot see. It is a complement, not a replacement.
Layer 2: Your Own Code
Dependency scanners never read your code. That is where Bandit, Semgrep, and the secret scanners come in.
Bandit
Bandit is a Python-specific static analyser for security patterns: eval, subprocess with shell=True, pickle.loads, weak hashing, and hardcoded passwords. It is Apache-2.0 and currently at 1.9.4.
pip install bandit
bandit -r src/ -ll # only medium and high severity
bandit -r src/ -f json -o bandit.json
-ll is the flag that decides whether your team keeps reading the output. Default Bandit reports a long tail of low-severity style findings, and a report nobody reads is worse than no report.
Semgrep
Semgrep does rule-based scanning across many languages, with a large public ruleset and the option to write your own patterns. It earns its place in an ML repo for the custom rules: flagging torch.load(...) without weights_only=True, blocking trust_remote_code=True, or catching a notebook cell that writes a token into a file.
semgrep --config auto src/
semgrep --config p/python --error src/
Secrets: gitleaks and TruffleHog
ML projects leak cloud and Hugging Face tokens constantly, usually through notebooks, because notebook outputs and .ipynb history are committed far more casually than .py files.
- gitleaks scans the working tree and Git history for credential patterns.
- TruffleHog adds verification, so you learn whether a found key is still live.
gitleaks detect --source . --redact
Scan history, not just the current tree. A token removed in a later commit is still in the repository, and anyone who clones gets it.
Notebook hygiene
Strip outputs before committing. Outputs carry API keys in tracebacks, sample rows from private data, and absolute paths that reveal usernames and directory structure. nbstripout as a pre-commit hook handles this mechanically, which is the only approach that survives deadlines.
Layer 3: Containers and System Libraries
ML images are enormous: a CUDA runtime, a few hundred system libraries, Python, and your code in one layer cake. Most of that surface is not Python and is invisible to layer 1.
Trivy is the common default here. It scans filesystems and built images, reports OS packages and language dependencies, and runs as a single binary:
trivy fs . # project files and lockfiles
trivy image my-ml-image:latest # the built image
trivy image --severity HIGH,CRITICAL my-ml-image:latest
OSV-Scanner can scan images too, so a team already running it may not need a second container tool [2]. What you should not skip is the cadence:
Rebuild and rescan on a schedule, not only when your code changes. Base-image vulnerabilities accumulate while your Dockerfile sits untouched. A training image built in March carries March's libssl whether or not you committed anything since.
Pin the base image by digest, not by tag. nvidia/cuda:12.4.1-runtime-ubuntu22.04 can be repushed; @sha256:... cannot. This is the container equivalent of committing a lockfile, and I cover the wider habit in Docker for research repos.
Layer 4: Models and Checkpoints
This is the layer most guides skip, and the one that makes ML special. It is also the layer where a scanner is a speed bump rather than a wall.
Why pickle is dangerous
torch.save and torch.load use pickle. Pickle is a serialization format that can encode function calls, not just arrays — so loading an untrusted file is closer to running an untrusted script than to opening a data file. The safetensors project puts it in four words: pickle is "unsafe, runs arbitrary code" [4].
The ecosystem is full of .bin, .pt, .pth, and .ckpt files that are pickles in disguise. A checkpoint named pytorch_model.bin tells you nothing about its contents.
Hugging Face knows this, and runs a security scanner over uploads that inspects pickle imports and displays the list of dangerous functions on the model page [3]. Their docs are refreshingly honest about the limits: it is "not 100% foolproof", and the safe/unsafe import lists are maintained on a best-effort basis.
Safetensors first
Safetensors stores tensors and metadata with no executable payload, which is why it exists at all. My rule: accept safetensors by default for anything you did not create yourself, and treat every other format as untrusted input.
GGUF, ONNX, H5, and SavedModel are different containers with different risks — not automatically safe, just not pickle. Check each format's parser history before you trust a random file.
The model scanners
Three tools cover this layer today, with different scopes:
| Tool | Scope | Notes |
|---|---|---|
| picklescan | pickle files | The narrow, fast check for dangerous imports; 431 stars on GitHub, still active in November 2026 |
| ModelScan | pickle, H5, SavedModel | Protect AI's multi-format scanner, Apache-2.0, version 0.8.8 |
| ModelAudit | 40+ scanners: pickle, PyTorch, safetensors, ONNX, TF, Keras, GGUF | Static — never loads or unpickles the model; MIT licensed, version 0.2.56 |
| fickling | pickle | Static analyser and interpreter for pickle bytecode, LGPL |
# picklescan
pip install picklescan
picklescan --path downloads/pytorch_model.bin
# ModelScan
pip install modelscan
modelscan -p downloads/model.pkl
# ModelAudit
pip install "modelaudit[all]"
modelaudit downloads/model.pkl
modelaudit ./models/ --format json --output modelaudit.json
ModelAudit is the one I would reach for first on a mixed repo, because it covers formats the pickle-only tools ignore and emits SARIF for CI [5]. Two caveats before you point it at client artifacts: it is a young 0.x project, and its own docs disclose that it includes telemetry for usage analytics. Read the privacy section, and run it on trusted artifacts in isolated environments.
Ignore self-reported detection tables. Every model-scanner vendor publishes a benchmark where it wins, usually against the previous release of a competitor. There is no neutral third-party leaderboard for this class of tool yet.
Scanners can fail
This is the honest part: scanners reduce risk, they do not eliminate it.
In early 2025, researchers documented malicious Hugging Face models that hid their payload inside a 7z-compressed pickle, slipping past scanners that only inspected standard pickle streams. In July 2026 a single researcher published a proof of concept claiming that legacy non-zip PyTorch .bin files evaded picklescan 1.0.5 and ModelScan 0.8.8 — and both of those are still the current PyPI versions, which makes the claim plausible rather than stale. It is one researcher's report, so verify it before repeating it, but the pattern is the point: the file format is an attack surface, and scanners parse formats.
So treat scanning as one layer. The layers that actually hold:
- Use safetensors wherever you have the choice.
- Load untrusted models in an isolated container or VM with no network access and no cloud credentials mounted.
- Pin and verify hashes for every downloaded artifact, and record them in the repo.
- Download from known publishers, and read the model card and file history before the weights.
- Avoid
trust_remote_code=Trueunless you have read the code it downloads. - Gate uploads with policy in a private mirror or registry, so a blocked model never reaches a developer laptop.
Point 2 is the one people skip. A malicious pickle inside a network-isolated container with no credentials is an incident; the same pickle on your workstation is a breach.
Quick Comparison
- pip-audit: best free first gate for Python dependencies; PyPA-maintained.
- OSV-Scanner: best second advisory source, and it covers lockfiles, source trees, and images.
- Safety: best if you want a curated database and a paid support path.
- Dependabot or Renovate: best for automated update PRs; Dependabot is GitHub-only.
- Snyk: best for reachability analysis and enterprise reporting.
- Socket: best complement for malicious-package behaviour.
- Bandit and Semgrep: best for scanning your own code; Semgrep if you want custom rules.
- gitleaks and TruffleHog: best for leaked secrets, including Git history.
- Trivy: best for container and filesystem scanning.
- picklescan, ModelScan, ModelAudit: best for model files — with safetensors as the real fix.
A Practical Pipeline
For a small ML team, this is the setup I would start with:
- Pre-commit:
gitleaksplusnbstripout, so secrets and notebook outputs never enter history. - Every pull request:
pip-audit -r requirements.txtandosv-scanner scan source -r .against the lockfile. - Every pull request:
bandit -r src/ -llorsemgrep --config auto --error src/. - On image build:
trivy image --severity HIGH,CRITICALagainst the training and serving images. - Automated updates: Dependabot or Renovate, with the full test suite gating merges.
- Model gate: accept safetensors by default, scan other formats, record hashes, isolate anything untrusted.
- Weekly schedule: rescan everything, because advisories arrive after you ship.
- Lockfiles: commit them, so the scan describes what you deploy.
# .github/workflows/security.yml (excerpt)
name: security
on: [pull_request, schedule]
jobs:
deps:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: pip install pip-audit
- run: pip-audit -r requirements.txt --format sarif --output audit.sarif
- run: |
curl -sSfL https://raw.githubusercontent.com/google/osv-scanner/main/install.sh | sh
osv-scanner scan source -r .
Your lockfile matters more than your scanner. A scan of loose requirements.txt ranges tells you about whatever the resolver picks today, not what runs in production. This is the same reproducibility argument as pinning a research repo's environment, applied to security.
Handling Findings Without Losing Your Mind
Scanners produce noise. Triage in this order:
- Reachability first. Does your code call the vulnerable function? A vulnerable XML parser in a package you import for one utility is not the same finding as one on your request path.
- Severity plus exposure. A high-severity bug in a training-only dependency on an isolated machine matters less than a medium one in an internet-facing serving container.
- Transitive dependencies count. Most exploitable vulnerabilities live deep in the tree, not in the package you typed into
requirements.txt. - Upgrade or mitigate. If upgrading breaks your model code, isolate the service, restrict its network, and document the exception with an expiry date.
- Old research code. Archived repos often carry known vulnerabilities by design. Keep those environments off the network and away from secrets — the approach I describe in running old ML repos without local setup.
Never suppress a finding without an expiry date. A permanent ignore file is how a team ends up with 40 accepted risks and no memory of accepting them.
Common Pitfalls
- Scanning only
requirements.txt. Scan lockfiles, images, and artifacts too. - Trusting one tool. Different scanners cover different databases and different layers.
- Assuming CVE scanners catch malicious packages. They structurally cannot; that needs behaviour analysis.
- Loading unknown pickles on your main machine. Isolate first, ask questions second.
- Treating a model scan as a guarantee. Bypasses exist and are published.
- Merging automated updates without tests. ML stacks break on version bumps more than web stacks do.
- Ignoring secrets in notebooks. Outputs and history leak keys that the code never contained.
- Never rescanning. Yesterday's clean report says nothing about tomorrow's advisory.
- Alert fatigue. Tune severity thresholds, or the team stops reading reports and the tool becomes decoration.
Most of these are the same mistake wearing different hats: trusting a single signal for a problem that has four layers.
Best Book to Read
| Cover | Book | Description | Get it |
|---|---|---|---|
![]() |
Software Supply Chain Security | SBOMs, provenance, dependency trust, and the organisational side of deciding what you are willing to ship. | View on Amazon |
As an Amazon Associate I earn from qualifying purchases. Links on this page are affiliate links, and I may earn a commission at no extra cost to you.
Read it for the process, not the tool list. The chapters on inventory and provenance explain why a lockfile plus recorded artifact hashes beats a scanner you run once.
Wrapping Up
Layer your defences: dependency scanners for known vulnerabilities, behaviour tools for malicious packages, static analysis and secret scanning for your code, container scanners for images, and safetensors plus isolation for models. No single tool covers an ML project, and the free tools cover most of the ground.
Set up the cheap version this week. Run pip-audit and osv-scanner on your lockfile, scan your Docker image with Trivy, and list every checkpoint your project loads with its format. If any of them is a pickle from an unknown source, you just found your first real finding. If everything is clean, schedule the rescan for next week — that is when the new advisories arrive.
Frequently Asked Questions
Is pip-audit enough to secure a Python ML project?
No. pip-audit checks installed packages against published advisories, so it never looks at your own code, your container base image, or the pickle checkpoint you downloaded. It is a good first gate, not a complete one.
Why are pickle model files dangerous?
Pickle is a serialization format that can encode function calls, not just data, so loading a malicious .pt, .pth, .bin, or .ckpt file can run arbitrary code with your privileges. The safetensors project lists pickle's flaw plainly: it is unsafe and runs arbitrary code.
Does Hugging Face scan uploaded models for malware?
Yes. Hugging Face runs a security scanner on the Hub that inspects pickle imports and shows the list of dangerous functions on the model page. Its docs are explicit that this is not foolproof and that the import auditing is best-effort.
Which model scanner should I use?
Start with picklescan for pickle files, ModelScan if you need H5 and SavedModel coverage, and ModelAudit if you want one static scanner across pickle, PyTorch, safetensors, ONNX, Keras, and GGUF. None of them is a guarantee, so pair scanning with isolation.
How often should I rescan an ML project?
Weekly at minimum, and on every pull request and image build. Advisories are published after you ship, so a clean report from last month says nothing about this month's dependency tree.
Sources
[1]GitHub, pypa/pip-audit — the PyPA-maintained scanner, Apache-2.0, v2.10.1: github.com/pypa/pip-audit[2]Google, OSV-Scanner documentation — source, lockfile, container image, and licence scanning: google.github.io/osv-scanner[3]Hugging Face, Pickle and the Hub security scanner — why pickle executes code and how the Hub inspects imports: huggingface.co/docs/hub/security-pickle[4]Hugging Face, safetensors README — format comparison, including "Pickle: Unsafe, runs arbitrary code": github.com/huggingface/safetensors[5]PyPI, ModelAudit 0.2.56 — 40+ static scanners, SARIF output, telemetry disclosure; picklescan and ModelScan are the narrower alternatives: pypi.org/project/modelaudit
