Contents
Figure 1: Smart Irrigation with RL — your soil sensor says dry, the forecast says rain tomorrow, the timer says water now. Who wins?
Your soil sensor says "dry." The forecast says "rain tomorrow." Your timer says "water now." Who wins? A fixed schedule waters anyway, wastes a few thousand liters, and maybe drowns the roots. Reinforcement learning can learn when to irrigate by weighing soil moisture, weather, and crop stage, and a small simulator shows you how.
I cannot claim I have run RL on a real farm, so treat this as a learning project. Irrigation is a great RL sandbox because the tradeoff is easy to picture: too little water stunts yield, and too much wastes water and money. Ready to build?
Why Irrigation Is a Sequential Decision Problem
Water you apply today changes soil moisture for days. Rain you cannot control changes it too. Crops also respond differently at different growth stages, so the same amount of water helps more during flowering than during early growth.
That makes irrigation a sequential decision problem under uncertainty. Classical approaches include:
- Fixed schedules: water every N days, regardless of conditions
- Threshold rules: irrigate when soil moisture drops below a set level
- Evapotranspiration (ET) based scheduling: replace the water the crop used, estimated from weather data
- Model predictive control: optimize irrigation over a forecast horizon using a crop and soil model
Threshold and ET-based rules are strong, widely used baselines. Check agronomy extension resources for the methods that suit your crop and region, since I am keeping the science simple.
How RL Frames Irrigation
Here is the setup we will build:
- State: soil moisture, crop growth stage, recent rainfall, forecast rain, and remaining seasonal water budget
- Action: apply no water, a light dose, or a heavy dose each day
- Reward: crop yield at harvest, minus the cost of water used
- Dynamics: soil moisture drops through evapotranspiration, rises with rain and irrigation, and drains when it exceeds field capacity
The big challenge is delayed reward. Yield only appears at season's end, but decisions happen daily. To make the toy trainable, we will give the agent a small daily reward based on crop stress, and a final bonus for yield.
Setting Up
pip install gymnasium stable-baselines3 numpy
Library versions change often, so check the docs if an import fails.
Building the Field Simulator
This is a simplified bucket model. Soil moisture is a fraction between wilting point and field capacity. It is nowhere near a real crop model like those in agronomy research, but it captures the main tradeoffs.
import numpy as np
import gymnasium as gym
from gymnasium import spaces
SEASON = 120 # days
WILT, FIELD_CAP = 0.15, 0.40 # soil moisture bounds (volumetric fraction)
DOSES = np.array([0.0, 8.0, 18.0]) # mm of water per action
class IrrigationEnv(gym.Env):
def __init__(self, water_cost=0.004, season_budget=500.0):
super().__init__()
self.water_cost, self.budget0 = water_cost, season_budget
self.action_space = spaces.Discrete(3)
self.observation_space = spaces.Box(0.0, 1.0, shape=(7,), dtype=np.float32)
def _stage_factor(self, day):
# Crop water sensitivity peaks mid-season (flowering/fruiting)
x = day / SEASON
return 0.4 + 0.6 * np.exp(-((x - 0.55) ** 2) / 0.04)
def _et(self, day):
# Crop evapotranspiration in mm/day: peaks mid-season, scales with heat
x = day / SEASON
crop = 0.5 + 0.9 * np.exp(-((x - 0.55) ** 2) / 0.1)
return self.heat * crop
def reset(self, seed=None, options=None):
super().reset(seed=seed)
rng = self.np_random
self.heat = rng.uniform(3.5, 6.5) # baseline mm/day demand
self.rain_prob = rng.uniform(0.08, 0.3)
self.rain_mean = rng.uniform(6.0, 14.0)
self.moist = rng.uniform(0.25, 0.35)
self.day = 0
self.water_left = self.budget0
self.stress_total = 0.0
self.water_used = 0.0
self.rain_today, self.rain_tomorrow = 0.0, self._draw_rain()
return self._obs(), {}
def _draw_rain(self):
if self.np_random.random() < self.rain_prob:
return float(self.np_random.exponential(self.rain_mean))
return 0.0
def _obs(self):
# The forecast is noisy: it hints at tomorrow's rain without guaranteeing it
noise = self.np_random.normal(0, 2.0)
forecast = max(0.0, self.rain_tomorrow + noise)
return np.array([
(self.moist - WILT) / (FIELD_CAP - WILT),
self.day / SEASON,
self._stage_factor(self.day),
min(self.rain_today / 20.0, 1.0),
min(forecast / 20.0, 1.0),
self.water_left / self.budget0,
min(self._et(self.day) / 8.0, 1.0),
], dtype=np.float32)
def step(self, action):
dose = DOSES[int(action)]
dose = min(dose, self.water_left)
self.water_left -= dose
self.water_used += dose
self.rain_today = self.rain_tomorrow
self.rain_tomorrow = self._draw_rain()
# Soil water balance, with 300 mm of effective root zone depth
depth = 300.0
et = self._et(self.day)
water_in = dose + self.rain_today
self.moist += (water_in - et) / depth
runoff = max(0.0, self.moist - FIELD_CAP) * depth # wasted water
self.moist = float(np.clip(self.moist, 0.05, FIELD_CAP))
# Stress: below 60% of available water, the crop suffers
avail = (self.moist - WILT) / (FIELD_CAP - WILT)
stress = max(0.0, 0.6 - avail) / 0.6 if avail < 0.6 else 0.0
weight = self._stage_factor(self.day)
self.stress_total += stress * weight
reward = -stress * weight - self.water_cost * dose
self.day += 1
done = self.day >= SEASON
if done:
yield_frac = max(0.0, 1.0 - 0.04 * self.stress_total)
reward += 10.0 * yield_frac
return self._obs(), float(reward), done, False, {
"water": self.water_used, "stress": self.stress_total,
"runoff": runoff}
Why randomize the weather: each reset draws a new climate: heat level, rain frequency, rain intensity, and starting moisture. The agent must learn general irrigation behavior instead of memorizing one season. The noisy rain forecast also forces it to treat predictions as hints, not facts.
Training a PPO Agent
The action space is small and discrete, so PPO fits:
from stable_baselines3 import PPO
env = IrrigationEnv()
model = PPO(
"MlpPolicy", env,
learning_rate=3e-4, n_steps=2048, batch_size=256,
gamma=0.995, gae_lambda=0.95, ent_coef=0.01,
verbose=1, seed=0,
)
model.learn(total_timesteps=300_000)
Expect a few minutes on a laptop. Watch the mean episode reward. If it stays flat, suspect the reward balance before you blame PPO.
Building Baselines
An RL result means nothing without honest comparisons.
def never_policy(env, obs):
return 0
def fixed_schedule(every=4, dose=1):
def policy(env, obs):
return dose if env.day % every == 0 else 0
return policy
def make_threshold(low=0.5, dose=1):
# Irrigate when available water drops below the threshold
def policy(env, obs):
return dose if obs[0] < low else 0
return policy
def make_smart_threshold(low=0.5):
# Same as the threshold rule, but skip watering if the forecast shows rain
def policy(env, obs):
if obs[0] < low and obs[4] < 0.25:
return 1 if obs[0] > 0.3 else 2
return 0
return policy
The never policy shows the cost of rain-fed farming. The fixed schedule mimics a timer. The threshold rule mimics a sensor-triggered system, and the smart threshold adds forecast awareness. That last one is a tough competitor, because it already encodes good agronomic sense.
Evaluating Fairly
Test on fresh seeds. Each seed fixes the climate parameters, but the daily rain draws also depend on the random stream, so treat comparisons as statistical.
def evaluate(policy, seeds):
rows = []
for seed in seeds:
env = IrrigationEnv()
obs, _ = env.reset(seed=seed)
done = False
while not done:
obs, r, done, _, info = env.step(policy(env, obs))
yield_frac = max(0.0, 1.0 - 0.04 * info["stress"])
rows.append((yield_frac, info["water"], info["stress"]))
return np.mean(rows, axis=0)
rl = lambda env, obs: int(model.predict(obs, deterministic=True)[0])
seeds = range(1000, 1030)
policies = [
("Never", never_policy),
("Fixed 4-day", fixed_schedule()),
("Threshold", make_threshold()),
("Smart threshold", make_smart_threshold()),
("PPO", rl),
]
for name, pol in policies:
y, w, s = evaluate(pol, seeds)
print(f"{name:16s} | yield {y:.2f} | water {w:6.0f} mm | stress {s:5.1f}")
Judge by yield and water together. A policy that hits full yield by flooding the field is not smart. Ever celebrated a harvest number without checking the water bill?
What Results Should You Expect?
I cannot promise numbers, because they depend on your seed, training length, and climate draw. In setups like this, the usual pattern looks like this:
- Never irrigating performs worst in dry climates and surprisingly well in wet ones
- Fixed schedules waste water in rainy seasons and under-water in dry ones
- The threshold rule is strong, because it responds to actual soil moisture
- The smart threshold often beats the plain threshold by skipping pointless watering before rain
- PPO can match or beat the smart threshold by timing doses around growth stage and budget, and it can underperform if training is short or the reward is mis-balanced
If PPO only matches the smart threshold, the RL machinery did not earn its complexity. Anticlimactic? Maybe, but honest. RL's edge grows when water is tightly rationed, weather is volatile, or the crop model gets complicated.
How to Improve Your Agent
Try these in order:
- Verify the simulator. Plot moisture over a season for a few policies and check that the water balance makes sense.
- Train longer. Double the timesteps and compare.
- Tune the water cost. Sweep it and plot the yield versus water tradeoff curve.
- Tighten the water budget. Cut the seasonal allocation, which is where smart timing matters most.
- Add continuous doses. Let the agent choose any amount, using PPO or SAC with a continuous action space.
- Add soil variety. Randomize root depth and drainage to mimic different fields.
- Replace the toy crop model. Connect a validated crop simulator, and check current agronomy tools such as DSSAT or APSIM for real options.
From Toy to Real Farms
The toy teaches the loop. Real fields add serious complications:
- Spatial variation: soil and slope differ across a field, so one decision does not fit every zone
- Sensor noise and drift: probes fail, calibrate poorly, and disagree with each other
- Hardware limits: pumps, valves, and pressure constrain when and how much you can irrigate
- Water rights and regulations: allocations and restrictions limit your options
- Energy costs: pumping costs vary with time-of-use electricity prices
- Crop and disease interactions: overwatering raises fungal risk, which a simple stress model ignores
- Slow feedback: one season gives one yield sample, so real-world training data is scarce
That last point matters most. Real farms cannot run millions of seasons, so research relies on crop simulators and calibrated models, and sim-to-real transfer is the hard part. Published studies on RL for irrigation, including work using crop models such as DSSAT, give you benchmarks and ideas. Check recent agricultural AI literature and verify tooling against current documentation before choosing a direction.
Guardrails Before Anything Touches a Real Field
A learned policy should never control real irrigation alone. Add these protections:
- Hard limits on daily and seasonal water volume, set outside the policy
- Emergency rules that override the agent when moisture hits critical levels
- Fallback to a threshold rule if sensors fail or readings look implausible
- Recommendation mode first: the agent suggests, and the farmer approves
- Pilot on a small plot, compared against standard practice
Common Mistakes to Avoid
- Trusting a toy crop model. Yield in the real world depends on factors this simulator ignores.
- Rewarding yield alone. The agent will flood the field. Charge for water.
- Skipping agronomic baselines. ET-based and threshold rules are good, so race them.
- Ignoring forecast error. A perfect forecast makes the problem artificially easy.
- Overclaiming from one season. Test across many climates and soil types before drawing conclusions.
Did you notice that most of these mistakes involve an unrealistic setup, not a weak algorithm?
Best Book to Read
| Cover | Book | Description | Get it |
|---|---|---|---|
![]() |
Irrigation Engineering | the standard civil/agricultural engineering text on soil-water-plant relationships, ET estimation, scheduling methods, and the hydraulic design of irrigation systems. | View on Amazon |
As an Amazon Associate I earn from qualifying purchases. Links on this page are affiliate links, and I may earn a commission at no extra cost to you.
Read it after you have run the code once. It makes the difference between "my PPO agent manages a bucket model" and "this policy would survive a real field" impossible to confuse.
Wrapping Up
Irrigation shows RL on a problem with delayed rewards, weather uncertainty, and a scarce resource. A learned policy can time water around crop stage and forecasts, but a tuned threshold rule is hard to beat on simplicity and trust. The honest takeaway: respect the agronomic baselines, validate with real crop models, and keep a human in the loop.
So run the script this week, then change one thing: the water budget, the rain frequency, or the forecast noise. If PPO beats the smart threshold, check your evaluation first, then celebrate. If it does not, you just learned why farmers trust their soil probes, and your simulated crops can go back to complaining about the weather.
Frequently Asked Questions
Can reinforcement learning beat threshold-based irrigation?
In this simulator it can match or beat a smart threshold rule that skips watering before rain, by timing doses around crop growth stage and the seasonal water budget. But a well-tuned threshold rule is hard to beat on simplicity and trust.
Why is irrigation a sequential decision problem?
Water applied today changes soil moisture for days. Rain you cannot control changes it too. Crop water needs also shift by growth stage. The same amount of water helps more during flowering than early growth, so decisions compound over time.
What does the RL agent control in this simulator?
The agent chooses a daily irrigation action: no water, light dose (8 mm), or heavy dose (18 mm). It sees soil moisture, crop stage, recent rain, a noisy forecast, and the remaining seasonal water budget.
How does the reward function work?
Daily reward penalizes crop stress and water cost. At harvest, a bonus rewards yield (10x the yield fraction). This gives dense feedback during the season and a final outcome signal.
What guardrails are needed before RL irrigation touches a real field?
Hard daily and seasonal water limits, emergency override when moisture hits critical levels, fallback to a threshold rule if sensors fail, recommendation-only mode (farmer approves), and pilot on a small plot first.
Sources
[1]S. K. Garg and G. L. Asawa, Irrigation Engineering — soil-water-plant relationships, ET estimation, scheduling, and system design: Amazon search[2]J. W. Jones et al., DSSAT (Decision Support System for Agrotechnology Transfer) — the standard crop model framework for simulation and optimization: dssat.net[3]J. Keating et al., APSIM (Agricultural Production Systems sIMulator) — modular crop simulation for farming systems: apsim.info[4]F. G. Hoffmann et al., Reinforcement Learning for Irrigation Scheduling — RL applied to deficit irrigation with crop models: arxiv.org/abs/2203.11218[5]FAO, Crop Water Requirements and Irrigation Scheduling — ETc, Kc curves, and agronomic guidelines: fao.org/3/x0490e/x0490e00.htm
Related Articles
- Call Center Staffing and Queue Control: SimPy Meets Reinforcement Learning
- Kubernetes Autoscaling with Reinforcement Learning: Smarter Than Threshold Rules?
- Ride-Hailing Dispatch with Reinforcement Learning: Matching Drivers and Riders
- Traffic Signal Control with Reinforcement Learning: Cut Wait Times in Simulation
- Dynamic Spectrum Access with Reinforcement Learning: A Wireless Use Case
