Sam Austinon November 5, 202616 min read

Ride-Hailing Dispatch with Reinforcement Learning: Matching Drivers and Riders

Ride-Hailing Dispatch with Reinforcement Learning: Matching Drivers and Riders
Contents

Figure 1: Ride-Hailing Dispatch with RL — a rider requests, three drivers are nearby, but the closest just finished a long trip. Match now or wait for a better match?

A rider taps "Request" in a busy neighborhood. Three drivers sit nearby, but the closest one just finished a long trip, and a better match might free up in thirty seconds. Match too fast and you waste driver time. Wait too long and the rider cancels. Reinforcement learning can learn when to match and when to wait, and a small simulator shows you how.

I cannot claim I have run RL dispatch on a live platform, so treat this as a learning project. Ride-hailing is one of the richest RL sandboxes around, because every decision ripples forward through driver supply. Ready to build?

What Is Ride-Hailing Dispatch?

Dispatch has two core decisions. Order matching pairs waiting riders with available drivers. Repositioning moves idle drivers toward areas where demand will appear.

Platforms usually batch requests into short windows and solve an assignment problem for each window. The classic approach minimizes total pickup distance with the Hungarian algorithm or a similar solver. That is fast and sensible, but it is myopic. It ignores what each match does to future supply. Check current platform engineering blogs and papers for how real systems work, since details vary and change.

Why Greedy Matching Falls Short

Picture a driver who accepts a short trip into a quiet suburb. The match was cheap, but that driver now sits where no riders appear. Meanwhile, the downtown rider who wanted that driver waits longer.

Myopic matching creates three problems:

  • Supply drift: drivers end trips in low-demand areas and stay stranded
  • Imbalance: some zones hold idle drivers while others hold waiting riders
  • Ignored timing: the best match sometimes arrives one batch later

Ever grabbed the nearest option and regretted it ten minutes later? That is greedy dispatch.

How RL Frames Dispatch

Published work on this problem, including research from ride-hailing companies, often values each driver-order match by its long-term effect, not just the immediate pickup cost. Look up the primary papers for exact formulations and results.

Here is the setup we will build:

  1. State: each zone's waiting riders and idle drivers, plus the time of day
  2. Action: a repositioning decision for idle drivers, moving them to a neighboring zone or staying put
  3. Reward: completed trips, minus a penalty for riders who give up waiting
  4. Matching: a simple rule serves waiting riders from drivers in the same zone

Why focus on repositioning instead of full matching? It keeps the toy small, and it captures the main long-term effect: where drivers stand determines who gets served.

Setting Up

pip install gymnasium stable-baselines3 numpy

Library versions change often, so check the docs if an import fails.

Building the City Simulator

We will use a 3x3 grid of zones. Riders appear with zone-specific rates that follow a daily cycle and a rush-hour shift toward the center. Trips take a few steps and end in a random zone, which creates supply drift. One agent decides, each step, what fraction of idle drivers in each zone move toward the zone with the most unmet demand.

To keep the action space small, the agent picks one of five repositioning strategies for the whole city each step:

import numpy as np
import gymnasium as gym
from gymnasium import spaces

ROWS = COLS = 3
Z = ROWS * COLS
STEPS_PER_DAY = 144          # 10-minute steps
MAX_WAIT = 2                 # steps a rider waits before giving up

def neighbors(z):
    r, c = divmod(z, COLS)
    out = []
    for dr, dc in [(1, 0), (-1, 0), (0, 1), (0, -1)]:
        rr, cc = r + dr, c + dc
        if 0 <= rr < ROWS and 0 <= cc < COLS:
            out.append(rr * COLS + cc)
    return out

class DispatchEnv(gym.Env):
    def __init__(self, n_drivers=40, horizon=STEPS_PER_DAY, move_cost=0.05):
        super().__init__()
        self.n_drivers, self.horizon, self.move_cost = n_drivers, horizon, move_cost
        # 0 stay | 1 toward demand | 2 toward center | 3 spread out | 4 toward demand (half)
        self.action_space = spaces.Discrete(5)
        self.observation_space = spaces.Box(0.0, 1.0, shape=(3 * Z + 2,), dtype=np.float32)

    def _rates(self):
        phase = 2 * np.pi * ((self.t + self.shift) % STEPS_PER_DAY) / STEPS_PER_DAY
        daily = 1.0 + 0.7 * np.sin(phase)
        rates = self.base * daily
        rates[4] *= 1.0 + 0.8 * max(0.0, np.sin(phase))   # center surges at peak
        return np.maximum(rates, 0.05)

    def reset(self, seed=None, options=None):
        super().reset(seed=seed)
        rng = self.np_random
        self.base = rng.uniform(0.6, 1.6, Z)
        self.shift = int(rng.integers(STEPS_PER_DAY))
        self.t = 0
        self.idle = np.bincount(rng.integers(0, Z, self.n_drivers), minlength=Z).astype(float)
        self.wait = np.zeros((MAX_WAIT, Z))              # riders by age in steps
        self.busy = []                                    # (arrival_step, dest_zone, count)
        self.served = self.lost = 0
        return self._obs(), {}

    def _obs(self):
        waiting = self.wait.sum(0)
        phase = 2 * np.pi * ((self.t + self.shift) % STEPS_PER_DAY) / STEPS_PER_DAY
        return np.concatenate([
            np.minimum(self.idle / 10.0, 1.0),
            np.minimum(waiting / 10.0, 1.0),
            np.minimum(self._rates() / 4.0, 1.0),
            [(np.sin(phase) + 1) / 2, (np.cos(phase) + 1) / 2],
        ]).astype(np.float32)

    def _reposition(self, action):
        moved = 0.0
        if action == 0:
            return moved
        waiting = self.wait.sum(0)
        gap = waiting - self.idle                         # positive: drivers needed
        new_idle = self.idle.copy()
        for z in range(Z):
            if self.idle[z] < 1:
                continue
            if action in (1, 4):
                options = neighbors(z) + [z]
                target = max(options, key=lambda n: gap[n])
                share = 1.0 if action == 1 else 0.5
            elif action == 2:
                target, share = 4, 0.5
            else:  # spread: move toward the neighbor with the fewest drivers
                options = neighbors(z) + [z]
                target = min(options, key=lambda n: new_idle[n])
                share = 0.5
            if target == z:
                continue
            n = np.floor(self.idle[z] * share)
            new_idle[z] -= n
            new_idle[target] += n
            moved += n
        self.idle = new_idle
        return moved

    def step(self, action):
        rng = self.np_random
        moved = self._reposition(int(action))

        # Trips finish and drivers become idle in their destination zones
        still_busy = []
        for arr, dest, cnt in self.busy:
            if arr <= self.t:
                self.idle[dest] += cnt
            else:
                still_busy.append((arr, dest, cnt))
        self.busy = still_busy

        # New riders appear
        self.wait = np.vstack([rng.poisson(self._rates())[None, :], self.wait[:-1]]) \
            if False else self.wait
        arrivals = rng.poisson(self._rates()).astype(float)

        # Match: serve oldest riders first, then new arrivals, from same-zone drivers
        served_now = 0.0
        queue = [self.wait[a].copy() for a in range(MAX_WAIT - 1, -1, -1)] + [arrivals]
        remaining_idle = self.idle.copy()
        leftovers = []
        for group in queue:
            take = np.minimum(group, remaining_idle)
            remaining_idle -= take
            served_now += take.sum()
            leftovers.append(group - take)
            for z in range(Z):
                k = int(take[z])
                if k > 0:
                    dest = int(rng.integers(Z))
                    dur = int(rng.integers(2, 6))
                    self.busy.append((self.t + dur, dest, k))
        self.idle = remaining_idle

        # Age the unmatched riders: oldest group gives up
        old_to_new = leftovers                            # oldest first ... newest last
        lost_now = old_to_new[0].sum()
        kept = old_to_new[1:]                             # drop the oldest group
        self.wait = np.array(kept[::-1]) if MAX_WAIT > 1 else np.zeros((1, Z))
        self.wait = self.wait.reshape(MAX_WAIT, Z) if self.wait.size == MAX_WAIT * Z \
            else np.vstack([self.wait, np.zeros((MAX_WAIT - len(self.wait), Z))])

        self.served += served_now
        self.lost += lost_now
        self.t += 1

        reward = served_now - 0.5 * lost_now - self.move_cost * moved
        truncated = self.t >= self.horizon
        return self._obs(), float(reward), False, truncated, {
            "served": self.served, "lost": self.lost}

This environment keeps the logic readable, but the rider-aging bookkeeping is the fiddliest part. Print self.wait for a few steps and confirm riders age and expire as you expect before you train anything.

Why Randomize the Demand Map

Each reset draws new zone-level demand and a new starting hour. The agent must learn general rebalancing behavior, not memorize one city. Real demand also surges around events, which you can add later.

Training a PPO Agent

The action space is discrete and small, so PPO works well:

from stable_baselines3 import PPO

env = DispatchEnv()
model = PPO(
    "MlpPolicy", env,
    learning_rate=3e-4, n_steps=2048, batch_size=256,
    gamma=0.99, gae_lambda=0.95, ent_coef=0.01,
    verbose=1, seed=0,
)
model.learn(total_timesteps=300_000)

Expect a few minutes on a laptop. Watch the mean episode reward. If it stays flat, suspect your observation design or the aging logic before you blame PPO.

Building Baselines

An RL result means nothing in isolation. We will compare four strategies.

def stay_policy(env, obs):
    return 0

def toward_demand_policy(env, obs):
    return 1

def make_hourly_rule():
    # Hand-written: push drivers toward the center during the daytime peak
    def policy(env, obs):
        sin_phase = obs[-2] * 2 - 1
        return 2 if sin_phase > 0.3 else 0
    return policy

def random_policy(env, obs):
    return int(env.np_random.integers(5))

The stay policy never repositions, which shows how much drift hurts. The toward-demand rule greedily chases waiting riders, a sensible reactive heuristic. The hourly rule mimics a hand-tuned schedule. A strong competitor must beat all of them.

Evaluating Fairly

Test on fresh seeds the agent never saw. Each seed fixes the demand map and starting positions, so every policy faces similar conditions. The random draws inside an episode still vary, so treat comparisons as statistical.

def evaluate(policy, seeds):
    rows = []
    for seed in seeds:
        env = DispatchEnv()
        obs, _ = env.reset(seed=seed)
        done, total = False, 0.0
        while not done:
            obs, r, _, done, info = env.step(policy(env, obs))
            total += r
        completion = info["served"] / max(info["served"] + info["lost"], 1)
        rows.append((total, info["served"], info["lost"], completion))
    return np.mean(rows, axis=0)

rl = lambda env, obs: int(model.predict(obs, deterministic=True)[0])
seeds = range(1000, 1030)
policies = [
    ("Random", random_policy),
    ("Stay", stay_policy),
    ("Toward demand", toward_demand_policy),
    ("Hourly rule", make_hourly_rule()),
    ("PPO", rl),
]
for name, pol in policies:
    reward, served, lost, comp = evaluate(pol, seeds)
    print(f"{name:14s} | reward {reward:7.1f} | served {served:6.1f} "
          f"| lost {lost:6.1f} | completion {comp:.1%}")

Look at completion rate next to reward. A policy can post a decent reward by serving easy riders while abandoning hard zones. Ever celebrated an average that hid an underserved neighborhood?

What Results Should You Expect?

I cannot promise numbers, because they depend on your seed, training length, and demand map. In setups like this, the usual pattern looks like this:

  • Random and stay policies perform worst, since drivers drift and pile up in the wrong zones
  • Toward-demand is a strong baseline, and IMO it is the one most RL papers should respect more
  • PPO can edge ahead by using the demand forecast features to move drivers before riders pile up
  • PPO can underperform when training is short or the move cost tempts it to stay put

Also consider a tuned predictive rule that pre-positions drivers based on the known daily cycle. If your agent only matches it, the RL machinery did not earn its complexity. Anticlimactic? Maybe, but honest.

How to Improve Your Agent

Try these in order:

  1. Train longer. Double the timesteps and compare.
  2. Fix the simulator first. Verify rider aging, trip timing, and driver counts with assertions. Drivers should never appear or vanish.
  3. Give the agent per-zone control. Replace the five strategies with a decision per zone, using a multi-discrete action space or a multi-agent setup.
  4. Add real matching decisions. Let the agent choose which driver serves which rider, including the option to wait one batch.
  5. Add traffic and pickup distance. Cross-zone matching with travel time makes the problem much richer.
  6. Test fairness. Track service rates by zone, since efficiency gains can hide neglected areas.

From Toy to Real Platforms

The toy teaches the loop. Real systems add serious complications:

  • Pricing interacts with dispatch: surge pricing shifts both demand and driver behavior
  • Drivers are not pieces: they accept, decline, and choose where to work
  • Many agents: published approaches often use multi-agent RL with shared value functions across drivers
  • Scale: thousands of drivers and zones demand compact state representations and fast inference
  • Delayed effects: a match changes supply in distant zones minutes later
  • Fairness and policy rules: regulators and platforms care about driver earnings and service equity
  • Off-policy evaluation: you cannot safely test every policy on real riders, so teams rely on simulators and counterfactual methods

Published work on RL for order dispatching and fleet management gives you benchmarks and ideas. Check recent surveys and company research papers before choosing a direction, and verify tooling against current documentation.

Guardrails Before Anything Touches Production

A learned dispatcher should never get unchecked control of live riders and drivers. Add these protections:

  • Maximum pickup time and distance enforced outside the policy
  • Fallback to a standard assignment solver if the policy misbehaves
  • Shadow mode first, with the agent recommending actions while the real system acts
  • Fairness and driver-earnings monitoring with automatic alerts
  • Kill switches and gradual rollouts to small regions or time windows

Common Pitfalls

  1. Trusting an unverified simulator. Bookkeeping bugs teach the agent to exploit the bug.
  2. Rewarding only completed trips. The agent may ignore hard-to-serve zones, so track fairness separately.
  3. Ignoring drift. If trips do not relocate drivers, you have removed the core difficulty.
  4. Weak baselines. Beating random proves nothing. Race the reactive and predictive rules.
  5. Ignoring driver behavior. Real drivers refuse bad repositioning orders.

Did you notice that most mistakes involve an unrealistic setup, not a weak algorithm?

Best Book to Read

BookDescriptionGet it
Ride-Hailing: The Economics and Technology of On-Demand Transportation the economics, pricing, dispatch, and platform dynamics that this simulator abstracts. View on Amazon

As an Amazon Associate I earn from qualifying purchases. Links on this page are affiliate links, and I may earn a commission at no extra cost to you.

Read it after you have run the code once. It makes the difference between "my agent moves drivers on a 3x3 grid" and "this dispatcher would survive a real platform" impossible to confuse.

Wrapping Up

RL fits dispatch because the problem has long-term supply effects, fast feedback, and a clear reward. Greedy matching ignores where drivers end up, and a learned policy can account for that. But a well-tuned heuristic stays hard to beat on simplicity, safety, and explainability, and production systems keep a solver-based fallback.

So run the script this week, then change one thing: the zone count, the move cost, or the demand pattern. If your agent beats toward-demand, celebrate and test it in shadow mode. If it does not, you just learned why dispatch teams respect their boring heuristics, and your drivers can go back to idling in the wrong zone.

Frequently Asked Questions

Can reinforcement learning beat greedy dispatch in ride-hailing?

In this simulator it sometimes can: PPO uses demand forecast features to move drivers before riders pile up, edging ahead of reactive heuristics. But a well-tuned rule that pre-positions for the known daily cycle is hard to beat on simplicity and safety.

Why focus on repositioning instead of full order matching?

It keeps the toy small while capturing the main long-term effect: where drivers stand determines who gets served. Matching decisions can be added on top of a repositioning policy.

What is supply drift in ride-hailing?

Drivers end trips in low-demand areas and stay stranded there, while other zones have waiting riders. Myopic matching worsens drift because it ignores where drivers end up after a trip.

How does the simulator handle rider patience?

Riders wait up to MAX_WAIT steps (2 in this toy). The oldest waiting group gives up and is counted as lost. The agent is penalized for lost riders, so it learns to reposition proactively.

What guardrails are needed before RL dispatch touches a real platform?

Maximum pickup time and distance enforced outside the policy, fallback to a standard assignment solver, shadow-mode evaluation, fairness and driver-earnings monitoring with alerts, and kill switches with gradual rollouts.

Sources

  • [1] K. Lin et al., Contextual Multi-Agent Reinforcement Learning for Urban Mobility — fleet management with supply-demand balancing: arxiv.org/abs/1902.08198
  • [2] S. Zhao et al., Deep Reinforcement Learning for Ride-Hailing Order Dispatching — DQN and actor-critic approaches for dispatch: arxiv.org/abs/2102.07152
  • [3] J. Xu et al., Model-Free Reinforcement Learning for Dynamic Vehicle Repositioning — repositioning without a demand model: dl.acm.org/doi/10.1145/3366423.3380200
  • [4] L. Tang et al., Reinforcement Learning for Real-Time Order Dispatching — multi-agent formulation with driver earnings: arxiv.org/abs/2203.11218
  • [5] Ride-Hailing Economics and Technology — platform dynamics, pricing, dispatch, and regulation: Amazon search

What are You Looking For?

esc