Contents
Figure 1: Best GPU Cloud Services — you found the paper, the GitHub link works, but your laptop fan sounds like a jet engine. The right cloud GPU changes everything.
You found the paper. The GitHub link works. The README says "just run python train.py." You clone it, hit enter, and… your laptop fan sounds like a jet engine. Three hours later: OOM error. Sound familiar?
I have reproduced 40+ papers. Some took 20 minutes on a cloud GPU. Others took three weekends of debugging dependency hell on my 2080 Ti. The difference was not the code—it was the compute.
Let me save you the trial-and-error. Here is the honest breakdown of every GPU cloud service worth your time (and money) in 2026.
The Decision Matrix: What Actually Matters
Before we compare providers, define your constraints:
| Factor | Questions to Ask |
|---|---|
| Budget | $50/month hobby? $500/month grant? Corporate card? |
| Reproducibility | Need exact CUDA/PyTorch versions? Frozen environments? |
| Data | Public datasets (fast pull)? Private TB-scale (slow upload)? |
| Workflow | Interactive notebooks? Batch training? Hyperparameter sweeps? |
| Team | Solo? 3-person lab? 20-person org with SSO? |
No single winner exists. The "best" depends entirely on your row in this table. Anyone telling you otherwise is selling something.
Tier 1: The Big Three (AWS / GCP / Azure)
Enterprise grade. Infinite scale. Painful UX.
AWS (p3 / p4 / p5 instances)
The default choice for grants and companies.
| Instance | GPU | VRAM | On-Demand | Spot (≈70% off) |
|---|---|---|---|---|
| p3.2xlarge | 1× V100 | 16 GB | $3.06/hr | ~$0.90/hr |
| p3.8xlarge | 4× V100 | 64 GB | $12.24/hr | ~$3.60/hr |
| p4d.24xlarge | 8× A100 | 320 GB | $32.77/hr | ~$11.00/hr |
| p5.48xlarge | 8× H100 | 640 GB | $98.32/hr | N/A (rare) |
Pros: Spot instances = massive savings. S3 + FSx for Lustre = fast data. SageMaker = managed notebooks/training. Every paper's AWS script works.
Cons: Console is a maze. Permissions (IAM) eat hours. Spot interruptions kill long runs unless you checkpoint religiously. Data egress fees = surprise bills.
My take: Use if your lab has AWS credits or you need SageMaker's experiment tracking. Otherwise, the UX tax is not worth it.
GCP (A2 / A3 / G2 instances)
Better UX. Better TPU access. Slightly pricier.
| Instance | GPU | VRAM | On-Demand | Spot (≈60-70% off) |
|---|---|---|---|---|
| a2-highgpu-1g | 1× A100 | 40 GB | $2.93/hr | ~$0.85/hr |
| a2-highgpu-8g | 8× A100 | 320 GB | $23.46/hr | ~$7.00/hr |
| a3-highgpu-8g | 8× H100 | 640 GB | $30.88/hr | ~$10.00/hr |
| g2-standard-4 | 1× L4 | 24 GB | $0.75/hr | ~$0.25/hr |
Pros: Vertex AI beats SageMaker for notebooks. Preemptible VMs are more predictable than AWS spot. TPU v4/v5 if you are into JAX. Billing is per-second (not per-minute like AWS).
Cons: Fewer regions with GPU quota. Quota increases take days. No true "spot fleet" equivalent—managing preemptible groups is manual.
My take: Best all-rounder for research teams. Vertex AI Workbench + preemptible A100s = my current default.
Azure (ND / NC / NV series)
Only if you are already in the Microsoft ecosystem.
| Instance | GPU | VRAM | On-Demand | Spot |
|---|---|---|---|---|
| ND96asr_v4 | 8× A100 | 320 GB | $22.88/hr | ~$7.00/hr |
| NC24ads_A100_v4 | 1× A100 | 80 GB | $3.67/hr | ~$1.10/hr |
Pros: Azure ML studio is genuinely good. NDv4-series has InfiniBand for multi-node. Great if your uni has Azure credits.
Cons: Spot eviction rate feels higher. Portal is… Azure. Avoid unless you have a reason.
Tier 2: ML-Focused Cloud (The Sweet Spot)
Built for researchers. Opinionated UX. Pricier per GPU-hour, cheaper per human-hour.
Lambda Cloud
My personal favorite. Built by ML engineers, for ML engineers.
| GPU | VRAM | Price/hr | Notes |
|---|---|---|---|
| RTX 6000 Ada | 48 GB | $0.99 | Best $/VRAM |
| A100 (80GB) | 80 GB | $1.50 | SXM, NVLink |
| H100 (80GB) | 80 GB | $2.50 | SXM, NVLink |
| 8× H100 | 640 GB | $18.00 | Full node |
Why it wins:
- Launch in 30 seconds:
lambda launch --gpu a100 --jupyter - Pre-installed: PyTorch, TensorFlow, JAX, NeMo, DeepSpeed, FlashAttention, xFormers. Zero setup.
- Persistent storage (home dir survives restarts)
- No spot interruptions. You pay for reserved, you get reserved.
- Team sharing: One account, multiple SSH keys, shared volumes.
Downsides: Limited regions (US East/West, EU Central). No multi-node InfiniBand (yet). No managed hyperparameter tuning.
Verdict: Best for solo researchers and small teams. I have run 200+ experiments here. Zero environment bugs.
RunPod
Cheapest on-demand. Community templates. Wild West reliability.
| GPU | VRAM | Secure Cloud | Community Cloud |
|---|---|---|---|
| RTX 3090 | 24 GB | $0.44/hr | $0.19/hr |
| RTX 4090 | 24 GB | $0.69/hr | $0.34/hr |
| A100 (80GB) | 80 GB | $1.64/hr | $1.19/hr |
| H100 (80GB) | 80 GB | $2.69/hr | $2.29/hr |
Secure Cloud = RunPod's own data centers. Community Cloud = random people's GPUs (cheaper, less reliable).
Pros: Insane $/VRAM on 4090s. 50+ community templates (llama.cpp, ComfyUI, Stable Diffusion, Axolotl). Pods start in ~10s. Pay per second.
Cons: Community pods disappear mid-training. No persistent storage by default (must mount network volume). No SLA. Support is Discord tickets.
My take: Great for inference, fine-tuning, quick experiments. Would I run a 2-week pretraining run on Community Cloud? Absolutely not. Secure Cloud A100s? Maybe.
Modal
Serverless GPUs. Write Python, get GPUs. Magic for batch jobs.
# modal_train.py
import modal
image = modal.Image.debian_slim().pip_install("torch", "transformers", "accelerate")
app = modal.App("train-llama", image=image)
@app.function(gpu="A100-80GB:4", timeout=86400, volumes={"/data": modal.Volume.from_name("datasets")})
def train():
# Your training code here
pass
@app.local_entrypoint()
def main():
train.remote()
Run: modal run modal_train.py → 4× A100 spin up, run, shut down. Billed per second.
Pros: Zero infra management. Automatic cold starts (~30s). Scales to 1000s of GPUs. Built-in secrets, volumes, web endpoints. Best for: hyperparameter sweeps, batch inference, CI/CD.
Cons: Not interactive. No SSH. No persistent Jupyter. Debugging = add logs, re-deploy. Expensive for long-running training (no reserved pricing).
Verdict: Revolutionary for batch workloads. Useless for interactive exploration.
Replicate
Model hosting + training API. Not a general compute platform.
import replicate
training = replicate.trainings.create(
version="meta/llama-2-7b:...",
train_data="https://my-bucket/data.jsonl",
num_train_epochs=3,
)
Pros: Easiest fine-tuning API. Handles LoRA, quantization, push-to-Hub. Pay per training run.
Cons: Closed ecosystem. Limited model architectures. No arbitrary code. Expensive at scale.
Verdict: Use for: "I need LoRA on Llama-2 by tonight." Skip for: everything else.
Tier 3: Academic / Free / Cheap
For when budget = $0.
Google Colab (Free / Pro / Pro+)
| Tier | GPU | VRAM | Max Runtime | Cost |
|---|---|---|---|---|
| Free | T4 / K80 | 16 GB | 12 hr | $0 |
| Pro | T4 / V100 / A100 | 16-40 GB | 24 hr | $10/mo |
| Pro+ | V100 / A100 | 40-80 GB | 24 hr | $50/mo |
Reality check: Free tier = T4 if you are lucky, K80 usually. A100s on Pro/Pro+ are not guaranteed. "Priority access" means "maybe."
Pros: Zero setup. Drive integration. Great for: teaching, demos, <4hr experiments.
Cons: No persistence. Session dies → files gone (unless Drive). No background execution. UI freezes on heavy output. Quota is opaque.
My take: Keep a Colab notebook for quick tests. Do not build your research on it.
Kaggle Kernels
Similar to Colab. 30hr/week free GPU (T4/P100). Better for: competition workflows, dataset integration. Worse for: custom environments, large private data.
Vast.ai
Marketplace for random GPUs. Cheapest raw compute. Highest friction.
| GPU | Typical $/hr | Reliability |
|---|---|---|
| RTX 3090 | $0.15-0.25 | 🎲 |
| RTX 4090 | $0.30-0.50 | 🎲 |
| A100 40GB | $0.80-1.20 | 🎲 |
| A100 80GB | $1.50-2.50 | 🎲 |
Pros: Insane prices. Huge inventory. Bid/ask market.
Cons: Machines vanish. No SLA. Random driver/CUDA versions. SSH key management is manual. Data transfer = your problem.
My take: Only for: fault-tolerant batch jobs (hyperparam sweeps with checkpointing). Never for: single-run experiments you cannot afford to lose.
University / Lab Clusters
Free. Sometimes fast. Always painful.
Pros: $0. Sometimes H100s / InfiniBand. Admin handles hardware.
Cons: Queue times (hours to days). Ancient drivers (CUDA 11.1 in 2024?). No root. No internet on compute nodes. Slurm scripts from 2015. One admin who "does not support that package."
My take: Use for: large-scale runs once code is bulletproof. Develop elsewhere.
The Reproducibility Checklist (Regardless of Provider)
You chose a provider. Now make it reproducible.
1. Lock Your Environment
# Dockerfile
FROM nvidia/cuda:12.1.1-cudnn8-devel-ubuntu22.04
# Pin everything
ARG PYTHON_VERSION=3.10.13
ARG PYTORCH_VERSION=2.1.2
ARG TORCHVISION_VERSION=0.16.2
RUN pip install --no-cache-dir \
torch==${PYTORCH_VERSION} \
torchvision==${TORCHVISION_VERSION} \
--index-url https://download.pytorch.org/whl/cu121
# Your deps
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
Build once. Push to registry (GHCR, Docker Hub). Pull everywhere. Same hash = same results.
2. Use a Lock File
# Generate once
pip freeze > requirements.lock.txt
# Or better: use uv / poetry / pixi
uv pip compile requirements.in -o requirements.lock.txt
Commit the lock file. Review it in PRs.
3. Config as Code
# config/train.yaml (Hydra / OmegaConf / YAML)
trainer:
max_epochs: 100
precision: bf16-mixed
accumulate_grad_batches: 4
model:
_target_: my_model.LlamaForCausalLM
hidden_size: 4096
num_layers: 32
data:
train_path: ${oc.env:DATA_DIR}/train.jsonl
seq_length: 4096
batch_size: 4
No hardcoded paths. No magic numbers. Environment variables for paths.
4. Checkpoint Everything
# Every N steps + on preemption signal
checkpoint = {
'model': model.state_dict(),
'optimizer': optimizer.state_dict(),
'scheduler': scheduler.state_dict(),
'step': global_step,
'epoch': epoch,
'rng_state': torch.get_rng_state(),
'cuda_rng_state': torch.cuda.get_rng_state_all(),
'config': OmegaConf.to_container(cfg),
}
torch.save(checkpoint, f"checkpoint_step_{global_step}.pt")
Handle SIGTERM (spot/preemptible):
import signal
import sys
def save_and_exit(signum, frame):
save_checkpoint()
sys.exit(0)
signal.signal(signal.SIGTERM, save_and_exit)
signal.signal(signal.SIGINT, save_and_exit)
AWS Spot gives 2-minute warning. Lambda/RunPod give ~30s. Handle both.
5. Track Experiments
# wandb.init(project="llama-pretrain", config=OmegaConf.to_container(cfg))
# mlflow.log_params(flat_config)
# clearml.Task.init(project_name="llama", task_name="pretrain-4k")
Pick one. Use it religiously. Artifacts = model checkpoints + logs + config.
My Personal Workflow (2024)
After 200+ cloud runs, this is what I actually do:
| Phase | Tool | Why |
|---|---|---|
| Prototyping | Lambda Cloud (A100 80GB) | 30s start, persistent $HOME, no spot kills |
| Debugging | Local RTX 4090 (24GB) | Free, interactive, VS Code remote SSH |
| Hyperparam sweeps | Modal (A100×4-8) | Serverless, scales to 50 parallel, per-second billing |
| Final training | GCP Preemptible A100×8 + Vertex AI | Cheap at scale, managed checkpoints, TPU option |
| Inference/serving | RunPod Secure (A100) or Modal | Cheap, fast cold start, web endpoints |
| Reproducing papers | Lambda or GCP | Exact env via Docker, persistent storage |
Total monthly spend: ~$300-800 depending on project phase. Grant-funded, so I optimize for human time, not GPU $/hr.
Quick Decision Guide
| If you need… | Start here |
|---|---|
| Interactive dev, solo/small team | Lambda Cloud |
| Cheapest interactive (tolerate flakiness) | RunPod Community |
| Batch sweeps, CI/CD, serverless | Modal |
| Fine-tune Llama/Mistral via API | Replicate |
| Multi-node, InfiniBand, enterprise | GCP (A3) or AWS (P5) |
| Zero budget, fault-tolerant | Vast.ai + checkpointing |
| Zero budget, interactive | Colab Pro ($10/mo) |
| Academic cluster access | Use it for final runs only |
The "Hidden Cost" Nobody Talks About
Data Transfer
| Scenario | Cost |
|---|---|
| Pull public dataset (LAION, Common Crawl) | Free (provider peering) |
| Push 1TB from your laptop | $0.09/GB = $90 (AWS/GCP egress) |
| Pull 1TB from S3 to Lambda | Free (same region) |
| Push 1TB results to S3 | $90 |
Fix: Keep data and compute in the same region. Same provider. Use rsync --partial or rclone for resumable transfers.
I learned this the hard way: $400 egress bill for one dataset move. Do not repeat my mistake.
Final Thoughts
No provider is perfect. The "best" changes every 6 months as new GPUs launch and pricing shifts.
My advice: Pick two. One for interactive (Lambda/RunPod), one for batch (Modal/GCP Spot). Dockerize your env. Checkpoint religiously. Track experiments.
Then stop thinking about infra and go reproduce that paper. 🚀
Questions? Horror stories? Better workflows? Drop them below. I have probably made your mistake already. Happy to compare scars. 😅
Frequently Asked Questions
Which GPU cloud service is cheapest for interactive development?
RunPod Community Cloud offers the lowest $/VRAM (RTX 4090 at ~$0.34/hr), but instances can disappear. Lambda Cloud offers the best balance of price, reliability, and UX for interactive work.
Which service is best for large-scale distributed training?
GCP (A3/A3 Ultra) or AWS (P5) with Megatron-LM/FSDP. For NVIDIA-only stacks at massive scale, NeMo-RL on NVIDIA DGX Cloud. Azure NDv4-series for InfiniBand.
How do I make my cloud training reproducible?
Dockerize with pinned CUDA/PyTorch versions, commit lock files (requirements.lock), use config-as-code (Hydra/OmegaConf), checkpoint everything including RNG states, handle SIGTERM for spot/preemptible interruptions.
Which GPU cloud is cheapest for training?
Vast.ai community cloud ($0.15-0.50/hr for RTX 3090/4090) but unreliable. RunPod Community Cloud ($0.19-0.34/hr for RTX 3090/4090). Lambda Cloud reserved A100 80GB at $1.50/hr is best reliable price.
How do I avoid surprise cloud bills?
Keep data and compute in same region/provider (no egress fees). Set timeout-minutes on jobs. Use spot/preemptible with checkpointing. Monitor with billing alerts.
Sources
[1]Lambda Cloud, Documentation — GPU instances, pricing, CLI: lambda.cloud/docs[2]RunPod, Documentation — Secure/Community Cloud, templates: docs.runpod.io[3]Modal, Documentation — Serverless GPUs, modal.com: modal.com/docs[4]Replicate, Training API — fine-tuning Llama, LoRA: replicate.com/docs[5]Google Cloud, Vertex AI — managed notebooks, training: cloud.google.com/vertex-ai