Sam Austinon November 10, 202613 min read

Learn from Kaggle Winners: The Best GitHub Repos of Competition Solutions

Learn from Kaggle Winners: The Best GitHub Repos of Competition Solutions
Contents

Figure 1: Learn from Kaggle Winners: The Best GitHub Repos of Competition Solutions — a leaderboard is the last page of the story; the repos below hold the earlier drafts

You can read ten tutorials about gradient boosting, or you can read how someone beat 4,000 other teams with it. The second option teaches you more. Kaggle winners publish their code and write-ups, and GitHub holds a large share of it — the hard part is knowing which repos to open and how to read them without drowning.

A warning before the list: winning code is written to score, not to teach. It is full of dead branches, hard-coded paths, and last-day hacks. That is exactly why it is valuable, and exactly why you need a reading method before you start cloning.

Why Study Winning Solutions?

Tutorials show you the clean path. Winning solutions show you the messy one. You see the validation scheme that survived, the features that did nothing, the post-processing that added the last 0.003, and the experiments that were abandoned. That mess is where the real skill hides.

Winners also tell you what actually moved the leaderboard. A top-finisher repo answers questions no generic guide can: which features mattered, how cross-validation was set up, how much of the gain came from the ensemble rather than the model. Those answers are worth more than another explanation of how XGBoost works.

But a repo is evidence, not a curriculum. Nobody in that repo knows what you already understand. You have to supply the order, the baseline, and the honesty about what you actually reproduced.

Start With the Indexes

Do not hunt through forum threads one competition at a time. Two repos do the cataloguing for you.

kaggle-solutions by Farid Rashidi

This is the master index. The repo describes itself as a curated collection of solutions, ideas, and insights from top performers across hundreds of competitions [1]. At the time of writing it had 6,551 stars, an MIT license, and a last push in November 2026, so it is genuinely maintained rather than abandoned after one good year [1].

You can browse by category — computer vision, NLP, tabular, time series — or search for a technique. The companion site at kaggle.farid.one is easier to skim than raw Markdown.

Its reading advice is better than most blog posts on the subject. For each competition it suggests working through the description, the evaluation metric, the dataset characteristics, and the rules first; then the top finishers' leaderboard profiles, the solution discussions, the most-upvoted notebooks, and the ensemble strategies; then feature engineering, model selection, validation, and post-processing [1]. Follow that order. If you jump straight to the code, the metric is usually the thing that explains why the code looks so strange.

git clone --depth 1 https://github.com/faridrashidi/kaggle-solutions
cd kaggle-solutions && npm install && npm run dev

Link to the original, not a fork. Several mirrors exist, and a stale copy will send you to dead notebook links.

kaggle_competition_solutions by anuj0456

A different approach: this repo compiles links to solutions that winners and top performers posted in the discussion forums, built from Kaggle's Meta Kaggle dataset inside a Jupyter notebook [2]. You get a notebook with links rather than a polished site.

It is small — 16 stars, Apache-2.0, last push September 2024 [2]. Treat it as a backup index when the main one misses a competition, not as your primary source, and check that the links still resolve before you plan a study session around one.

Repos With Full Winning Code

Links to write-ups help. Runnable code helps more. These repos give you both, and they cover three very different eras of competition ML.

ASHRAE Great Energy Predictor III: five winners, one problem

This repo packages the top five solutions to the same competition, with code, documentation, and a comparative analysis [3]. The 2019 contest drew 4,370 participants across 3,614 teams from 94 countries, who submitted 39,403 predictions [3].

Five winners on one problem is the most useful format on this list. You can see where they agreed — the same meter-level quirks, the same leakage traps — and where they diverged. Agreement across independent teams is a much stronger signal than one confident write-up.

The comparison covers workflow, computation time, and score distribution by meter type, site, and primary space usage [3]. Reproduction instructions live in the repo wiki, and the README links a conference video walkthrough. That structure is rare: most solution repos give you a notebook and a shrug.

pairjudge: a gold-medal LLM judge you can install

If you care about LLM competitions, start here. pairjudge generalizes the 4th-place gold-medal solution to Kaggle's LMSYS Chatbot Arena Human Preference Predictions, a competition with 1,849 teams [4].

It returns A-wins / B-wins / tie probabilities, plus position-bias and swap diagnostics, and it ships a small trained example model you can load after installing [4]. The methods are the interesting part: multi-turn packing under a budget, position-bias correction, and pseudo-label distillation.

pip install pairjudge

Check the claims against the code. The README states the methods originated in that gold solution and points to the original write-up; that is the author describing his own work, so verify the details yourself before you cite them as fact.

labelbank: silver, and clearer because of it

labelbank comes from a silver-medal solution to Kaggle's Eedi — Mining Misconceptions in Mathematics [4]. It does retrieve-and-rerank over a closed label bank: LLM bi-encoders, self-mined hard negatives, and a generative listwise reranker.

The original competition scripts, configs, logs, and write-up are preserved in a competition folder, with the cleaned library alongside. That separation is the thing to copy in your own repos: raw solution on one side, reusable idea on the other.

Do not skip silver medals. A silver team often explains its reasoning more carefully, because it could not hide behind a leaderboard gap.

Recent Winners Worth Studying

Fresh solutions show where the field is heading right now. Three NVIDIA write-ups are worth your evening.

ARC Prize 2025: first place without a giant model

Ivan Sorokin and Jean-Francois Puget, competing as team NVARC, took first place on the Kaggle ARC Prize 2025 public leaderboard with 27.64% on ARC-AGI-2 [5]. Their winning solution was a fine-tuned 4B-parameter model that cost about 20 cents per task, built on synthetic data generation and test-time training rather than brute-force search [5].

Two lessons hide in that result. First, the compute budget is part of the problem statement — heavyweight reasoning methods could not fit Kaggle's runtime, so the team moved the expensive work offline. Second, the public and private splits disagreed (27.64% public, 24.03% private), which is normal and is exactly why you should never tune against the public leaderboard.

AIMO-2: 34 out of 50, on four L4 GPUs

NVIDIA's NemoSkills team won the AI Mathematical Olympiad Progress Prize 2 by answering 34 of 50 Olympiad problems within a five-hour limit, using a cluster of four L4 GPUs against more than 2,200 teams [5]. The model combines natural-language reasoning with Python code execution.

Their optimizations were integrated into the NeMo-Skills pipelines on GitHub [5]. That is where you should look for reusable code — the competition notebook shows the winning configuration, not the machinery that made it affordable.

The churn win that was mostly a loop

In March 2026, three LLM agents generated over 600,000 lines of code and ran 850 experiments on the way to a first-place finish in a Kaggle playground churn competition [5]. The final ensemble was a large stack — roughly 150 models — combining hill climbing, stacking, cuDF, and cuML.

The transferable lesson is not the stack size. It is the loop: generate an idea, run it, keep the delta, repeat. If you have one GPU, copy the discipline and shrink everything else.

How to Study a Winning Solution

Reading code passively teaches you almost nothing. This is the process I use:

  1. Read the problem and the metric first. The metric shapes every decision in the repo, including the ones that look irrational.
  2. Read the write-up before the code. Understand the strategy, then go looking for where it lives in the files.
  3. Map the validation scheme. Cross-validation design explains more of the score than the model choice does.
  4. Rebuild a simplified version. One model, the top three features, the same metric. Get a number, even a bad one.
  5. Compare against your own baseline. A winner's 0.94 means nothing until you know your baseline was 0.91.
  6. Write down what surprised you. Surprises mark the edge of what you actually understand.

The index repo gives the same advice: try implementing winning solutions on your own [1]. Copying code feels productive, but it builds almost no skill. If you are earlier in the journey, pair this with a from-scratch ML repo list so the terms are not new twice over.

Common Pitfalls

  • Copying the code without the metric. A solution tuned for MAE will hurt you in an AUC competition.
  • Trusting a repo that never states its rank. "Kaggle solution" in a description is not a medal. Look for the placement.
  • Running 2019 code with 2026 libraries. Pin the versions or translate the idea; do not debug someone else's ancient environment for a weekend.
  • Reproducing a 150-model stack on one GPU. Learn the pipeline, then simplify it until it fits your hardware.
  • Missing the leakage and post-processing. Some gains only exist because of how that competition's test set was built.
  • Skipping the write-up. The code shows what they did; the write-up shows why, and the why is the part that transfers.

One more trap: collecting repos instead of studying them. Starring a solution teaches you nothing. The same rubric I use to judge a GitHub ML repo applies here — maintenance, honest claims, and evidence of real work beat star counts.

Best Book to Read

CoverBookDescriptionGet it
Cover of “The Kaggle Book” The Kaggle Bookby Konrad Banachewicz and Luca Massaron how competitions are actually won: validation design, feature engineering, ensembling, and the competition mindset that the repos assume you already have. View on Amazon

As an Amazon Associate I earn from qualifying purchases. Links on this page are affiliate links, and I may earn a commission at no extra cost to you.

Read it alongside one competition, not before all of them. The book gives you the vocabulary; a single repo gives you the practice.

Wrapping Up

You do not have to win a competition to learn from one. Index repos point you to the right write-ups, full-code repos let you reproduce what worked, and recent winners show where the field is moving next.

Pick one competition in your favorite domain this weekend, read the winner's write-up, and rebuild the core idea from scratch. You will learn more from your own wrong first attempt than from ten more summaries. And if the competition is a hiring signal you are chasing, the same study habit feeds the ML interview prep repos that get you through the loop.

Frequently Asked Questions

Which Kaggle solutions repo should I start with?

faridrashidi/kaggle-solutions. It is MIT-licensed, actively updated, and organizes competitions by category with links to write-ups, discussions, and top notebooks. Use the smaller repos when you want runnable code for one specific competition.

Is it allowed to reuse code from a winning Kaggle solution?

Check the repository license and the competition rules before you copy anything. Many repos are MIT or Apache-2.0, but licenses vary and some solutions were never open-sourced. Use the code to learn, and credit the author if you build on it.

How long should studying one winning solution take?

Plan three to six hours: one hour on the write-up and metric, one to two hours tracing the code, and one to two hours rebuilding a simplified version. Reading the code alone takes less time and teaches far less.

Do I need a GPU to reproduce a Kaggle winner's solution?

Not for most tabular competitions. Gradient-boosted trees on a CPU still win plenty of them. LLM and vision competitions usually need a GPU, and some winning stacks assume hardware you do not have, so reproduce the core idea instead of the full ensemble.

Are older Kaggle solutions still worth reading?

Yes, for technique. Validation schemes, feature ideas, and leakage traps age slowly. Library calls, GPU assumptions, and leaderboard tricks age fast, so translate the idea into current tooling instead of running the old notebook.

Sources

What are You Looking For?

esc