Sam Austinon November 2, 202616 min read

Hourly Demand Forecasting with Random Forest: A Full Project, Start to Finish

Hourly Demand Forecasting with Random Forest: A Full Project, Start to Finish
Contents

Figure 1: Hourly Demand Forecasting with Random Forest: A Full Project, Start to Finish — a city draws power on a schedule, and the schedule is what the model has to learn

Most tutorials stop at "fit, predict, done." Real projects stop at the first time a forecast embarrasses someone. So let's walk the whole path: framing, features, baselines, tuning, a final test, and the ways it breaks.

One disclosure first: no real dataset was reachable from my environment, so this project runs on synthetic hourly demand I generated. Treat every number as a demonstration of the workflow, not a benchmark.

Step 1: Frame the Forecast

Before touching a model, decide what you are predicting and when. My task: at 00:00 each day, forecast all 24 hours of that day.

That rule drives everything. A feature is allowed only if you would know it at midnight. In practice that means demand lags of 24 hours or more, calendar fields, holidays, and a temperature forecast. Hour 23's "one hour ago" value does not exist yet at midnight.

Step 2: The Data

I simulated three years of hourly demand, 26,304 rows from 2022 to 2024, with:

  • a daily shape and a weekend dip,
  • 5% yearly growth,
  • a temperature effect, with demand rising when it is hot or cold,
  • a 25% drop on US federal holidays, using the pandas calendar [4],
  • multiplicative noise.

Mean demand came out at 1,179 units, and the year 2024 contains 264 holiday hours. Training on 2022 and 2023 leaves 17,329 rows; the holdout is all 8,784 hours of 2024.

Step 3: Features That Respect the Clock

I built 12 features:

  • Calendar: hour, day of week, month.
  • Holidays: a holiday flag and a "day before or after" flag.
  • Weather: temperature.
  • Demand history: lags at 24, 48 and 168 hours, rolling mean and standard deviation over the previous day, and a 168-hour rolling mean.

Every rolling window is built on y.shift(24), so it ends a day back. Then I ran a perturbation check: I added 1,000 to demand from one row onward and confirmed that no feature changed in the 24 rows before it. Zero features moved. Cheap insurance against leakage.

Step 4: Baselines First

Train on 2022 to 2023, hold out all of 2024 and touch it once. Before any model, I scored three naive forecasts on the holdout:

Baseline MAE MAPE
Same hour yesterday (lag 24) 87.4 7.18%
Same hour last week (lag 168) 73.9 6.08%
Average of the two 69.4 5.72%

The one to beat is last week's value at 73.9. If your model cannot beat it, you do not have a model, you have a hobby. Note that the average of the two beats both parents, which is worth knowing before you reach for anything complicated.

Step 5: Tune the Forest Honestly

I tuned a RandomForestRegressor with 100 trees on a small grid, using a three-fold TimeSeriesSplit inside the training years [3]:

min_samples_leaf max_features 0.5 max_features 1.0
3 47.6 48.3
10 49.3 49.5
30 53.8 53.7

CV MAE, lower is better. Smaller leaves won clearly; fewer features per split won at the two smaller leaf sizes and tied at 30.

The best setting sits at the edge of my grid, so smaller leaves might do better still. I did not try them. The docs say the defaults grow "fully grown and unpruned trees" and suggest controlling tree size with parameters like min_samples_leaf [1], which fits what I saw: the larger-leaf rows smoothed too much.**

Step 6: The One-Time Holdout

Model MAE MAPE Bias
Last week's value 73.9 6.08% -1.3
Ridge, same features 51.7 4.15% -10.5
Random Forest (tuned) 49.2 3.92% -13.1
Gradient boosting (HistGB) 47.6 3.79% -19.4

The forest cut error by 33% against the best baseline. But it did not win: gradient boosting edged it by 1.6 MAE, about 3%, and fitted in 0.3 seconds against 0.7. That is worth saying plainly, because the forest is the model in the title.

Ridge was not bad at all at 51.7, only 5% behind the forest. A linear model with good features gets surprisingly far. That is the most underrated result in applied forecasting.

Step 7: Look at Where It Fails

Averages hide structure. I sliced the forest's holdout errors:

  • Holiday days: MAE 48.0 over 264 hours.
  • Day before or after: MAE 56.6 over 504 hours.
  • Other days: MAE 48.8 over 8,016 hours.

The shoulder days are the worst slice, not the holidays. The holiday flag teaches the model the 25% drop, and it handles that. It has no flag for "the Tuesday after Thanksgiving", and few examples to learn it from.

Worst hours: 08:00 (67.3), 07:00 (61.3) and 09:00 (60.5). Best hours: 22:00 (41.6), 04:00 (42.6) and 01:00 (42.7). The hard hours are where my simulated curve climbs fastest, and in my simulation that is the morning ramp, not the evening peak. If your evening peak is the sharper one, your worst hours will be there instead.

Bias: the model under-predicted by 13.1 on average, about 1.1% of mean demand. My demand grows 5% a year, and the forest was trained on earlier, lower levels. Forests average training targets, so they struggle to extrapolate.

Step 8: The Fix for Growth

I tried predicting relative demand instead: the ratio of demand to last week's average level, then multiplying back at prediction time.

Result: MAE 49.1, MAPE 3.97%, with bias moving from -13.1 to +1.6. It fitted in under a second.

Read that carefully: the error barely moved. The ratio target fixed the calibration and left the typical miss where it was, because last week's level was already in the model as a feature. If your bias is what bothers you, this is a one-line fix. If your MAE is what bothers you, this is not it.

Step 9: Which Features Matter?

Impurity-based importances, computed on training data, ranked lag_168 first (0.39), then lag_24 (0.27) and lag_48 (0.15). Permutation importance on the holdout told a partly different story:

Feature Impurity Permutation (MAE added when shuffled)
lag_168 0.39 +39.7
lag_24 0.27 +30.0
dow 0.03 +16.4
hol 0.05 +12.8
lag_48 0.15 +12.3
temp 0.06 +4.8
hour 0.01 +0.6

Day of week jumped from near-zero on impurity to third on permutation. The scikit-learn guide explains the bias directly: impurity-based importance for trees is "strongly biased and favor high cardinality features" [2].

Notice also that roll_mean_24, roll_mean_168 and roll_std_24 scored near zero on both measures, at 0.1, 0.3 and 0.0 MAE. They carry real information; they just overlap with the lags, so shuffling one barely hurts. A low score is not the same as useless.

Step 10: Break It on Purpose

Three stress tests on the holdout:

  1. Drop the demand lags and keep calendar plus temperature: MAE jumped to 84.3, with bias at -81.9. That is worse than the naive baseline, which is the point: without recent demand the forest cannot see the level at all.
  2. Noisy temperature forecasts: I added 3°C of error to the holdout temperature. MAE moved from 49.2 to 51.0, about 4%. In my simulation weather matters modestly, so this reassures little. A weather-driven business could look very different.
  3. Corrupt 2% of lag_24 values, multiplied by 3: MAE rose to 51.3. Mild damage, but a reminder to validate inputs.

What I Would Do Next

  1. Get real forecasts for any feature you would only have as a forecast, and train on those, not on actuals.
  2. Widen the grid past the edges, then refit on all data.
  3. Add quantile outputs if you need ranges.
  4. Monitor bias, not just MAE, once it is live. Growth and drift show up as a steady lean.
  5. Retrain on a schedule, and use the ratio target if your level keeps moving.

Common Pitfalls

  1. Using a feature you would not have at forecast time. The 24-hour cutoff is the whole discipline.
  2. Skipping the baseline. My forest only looks good next to 73.9; next to nothing it means nothing.
  3. Tuning and testing on the same rows. I used TimeSeriesSplit inside 2022-2023 and touched 2024 once.
  4. Reading impurity importance as truth. It put day of week at 0.03 while permutation put it third.
  5. Assuming a low importance means a useless feature. The rolling means were redundant, not empty.
  6. Reporting MAE and hiding the bias. Mine leaned low by 1.1% of mean demand, which compounds in a growing series.
  7. Trusting one stress test. Removing the lags broke the model; adding 3°C barely moved it. Both are informative.
  8. Forgetting the shoulder days. The worst slice in my holdout was not the holiday, it was the day after.

Best Book to Read

CoverBookDescriptionGet it
Cover of “Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow” Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlowby Aurélien Géron the chapters on tree ensembles, feature engineering and end-to-end projects are the closest thing to the workflow above, and it is honest about when a simpler model is enough. View on Amazon

As an Amazon Associate I earn from qualifying purchases. Links on this page are affiliate links, and I may earn a commission at no extra cost to you.

Wrapping Up

The forest earned its place: it beat last week's baseline by 33%, and a one-line change to the target removed all of its bias. But the baseline, the clock-respecting features and the failure tests did as much work as the model, and gradient boosting still edged it.

Take your own hourly series, write down what you would know at forecast time, and score "same hour last week" first. That single number tells you whether the next three days of modelling are worth it.

Frequently Asked Questions

Why is last week's value a better baseline than yesterday's?

Because it keeps the day-of-week pattern. On my 2024 holdout, same hour yesterday scored 87.4 MAE and same hour last week scored 73.9, so the weekly lag was 15% better before any model existed. A forest then has to beat 73.9, not 87.4, which is a much harder bar.

Why can't a random forest extrapolate a growing trend?

Because every prediction is an average of training targets. If the level grew 5% a year, the trees never saw the 2024 levels, so their averages sit below the truth. My forest under-predicted by 13.1 on average. Predicting the ratio to last week's level and multiplying back fixed the bias, taking it from -13.1 to +1.6.

How do I check that my features do not leak the future?

Perturbation testing. Add a large constant to the target from one row onward, rebuild the features, and confirm that no feature changed in the rows before the edit. In my project the 24 rows before the edit changed by exactly zero features, because every lag and rolling window is built on y.shift(24).

Why did impurity importance and permutation importance disagree?

Impurity importance is computed on training data and is biased toward features with many distinct values, which scikit-learn's guide calls out directly. Permutation importance is measured on held-out data by shuffling one column at a time. In my project both ranked lag_168 first, but day of week rose to third on permutation and barely registered on impurity.

Does a ratio target always improve accuracy?

No. It fixed my bias and left the error almost unchanged: 49.2 MAE before, 49.1 after. The gain was in calibration, not in typical accuracy, because the level information was already available as lag_168. Test it on your own data before assuming it helps.

Sources

What are You Looking For?

esc