Contents
Figure 1: Time Series Features: Stop Training on the Future — the model matters less than the features, and the features matter less than whether you cheated
Here is the dirty secret of machine learning on time series: the model matters less than the features, and the features matter less than whether you cheated.
Time series data has one property that separates it from every other tabular dataset: order matters. A feature that peeks one row into the future will give you a spectacular validation score and a useless production model. Every technique in this guide, from lags to rolling windows to cyclical encodings, comes with one rule attached: use only what you would actually know at prediction time.
This guide walks through the full toolkit with pandas and scikit-learn: calendar features, cyclical encodings, lags, rolling statistics, trend features, multi-series data, and leakage-safe validation. I verified the scikit-learn specifics against its official time-related feature engineering example, and the rest comes from my own knowledge and a tutorial I cross-checked. Numbered brackets like [1] point to the sources at the end. The code is mine; I ran it once on the synthetic data below with scikit-learn 1.9.1, pandas 3.0.6 and numpy 2.5.3, and the scores are in the pipeline section. Run it on your data and your library versions.
The Setup: A Toy Dataset
We will use synthetic daily sales with a trend, weekly and yearly seasonality, a promotion flag, and noise. Replace it with your own data:
import numpy as np
import pandas as pd
rng = np.random.default_rng(0)
idx = pd.date_range("2022-01-01", periods=900, freq="D")
t = np.arange(len(idx))
dow = idx.dayofweek.to_numpy()
doy = idx.dayofyear.to_numpy()
promo = (rng.random(len(idx)) < 0.1).astype(int)
sales = (
100 + 0.05 * t
+ 15 * np.sin(2 * np.pi * dow / 7)
+ 20 * np.sin(2 * np.pi * doy / 365.25)
+ 25 * promo
+ rng.normal(0, 5, len(idx))
)
df = pd.DataFrame({"sales": sales, "promo": promo}, index=idx)
Two habits to build now: keep a proper DatetimeIndex, and keep the data sorted chronologically. Every operation below assumes it.
Rule One: Never Use the Future
Before any feature, learn the four ways leakage sneaks in.
| Leakage mode | What it looks like | The fix |
|---|---|---|
| Shuffled splits | Standard k-fold trains on the future and tests on the past [4] | TimeSeriesSplit |
| Centered windows | .rolling(7, center=True) averages rows on both sides [4] |
Shift, then roll backward |
| Today's target in the window | A rolling mean that includes the row you are predicting | shift(1) before .rolling() |
| Preprocessing before the split | Scaling, imputing or encoding on all data [5] | Fit transformers inside a pipeline |
1. Shuffled Splits
Standard shuffled k-fold cross-validation trains on the future and tests on the past. The tutorial I cross-checked calls using standard cross-validation instead of time-aware splitting the most common mistake in time series machine learning [4]. Use TimeSeriesSplit instead.
2. Centered Rolling Windows
A centered window averages values on both sides of each row, so it uses the future. The same tutorial flags .rolling(7, center=True) as leaking [4].
# WRONG: the window includes future rows
df["bad"] = df["sales"].rolling(7, center=True).mean()
# RIGHT: shift first, so the window only sees the past
df["good"] = df["sales"].shift(1).rolling(7).mean()
3. Including Today's Target
If you predict today's sales, a rolling mean that includes today's sales leaks the answer. Shift the series before you roll it, as above.
4. Preprocessing on the Whole Dataset
Scaling, imputing or encoding before you split lets test statistics leak into training. A feature engineering reference names this as a common cause of leakage [5]. Fit transformers on training data only, which scikit-learn pipelines handle for you inside cross-validation.
Causal pandas features like shift and backward-looking rolling are safe to compute on the full series, because each row only looks backward.
Calendar Features
If your data only has a timestamp, expand it into components. The scikit-learn example points to pandas' date component accessors for exactly this: hour of day, day of week, day of month, and month of year [1][3].
def add_calendar(df):
out = df.copy()
out["dow"] = out.index.dayofweek # 0 = Monday
out["month"] = out.index.month
out["dayofyear"] = out.index.dayofyear
out["is_weekend"] = (out.index.dayofweek >= 5).astype(int)
out["is_month_end"] = out.index.is_month_end.astype(int)
return out
Add these when they matter for your domain:
- Holidays and events, from a holiday calendar for your country or business.
- Pay days and billing cycles, like first and last business days.
- Distance to events, such as days until the next holiday.
- Trading days or business days, instead of calendar days.
Events known in advance are safe features, because you know them at prediction time.
Encoding Cyclical Features
Day of week runs 0 to 6 and then wraps back to 0. A raw integer tells a linear model that Sunday is six units away from Monday, when they are neighbours.
The scikit-learn example puts it this way: linear regression does not automatically model non-monotonic relationships, so you must engineer non-linear terms, and a naive min-max encoding of periodic features hurt its linear model badly [1].
The gap is large. In that example, the gradient boosting model reached an average error around 4 to 5% of maximum demand, while linear regression with the naive encoding landed around 14%, more than three times higher [1]. Trees handle ordinal calendar features well, so tree models rarely need the fancy encodings. Linear and neural models do.
| Option | Best for | The catch |
|---|---|---|
| Sine and cosine | Linear and neural models | Needs both columns; sine alone maps two times to one value |
| Periodic splines | Sharper, non-sinusoidal cycles | More features, and you choose the knot count |
| Native categoricals | Tree models | Only available in estimators that support them |
Option 1: Sine and Cosine
def add_cyclical(df, col, period):
out = df.copy()
out[f"{col}_sin"] = np.sin(2 * np.pi * out[col] / period)
out[f"{col}_cos"] = np.cos(2 * np.pi * out[col] / period)
return out
The pair places each value on a circle, so the end of the cycle sits next to the start. Always use both columns, because sine alone maps two different times to the same value.
Option 2: Periodic Splines
Scikit-learn's example introduces SplineTransformer with extrapolation="periodic" as another way to encode cycles [1]. Here is a helper in the style of that example:
from sklearn.preprocessing import SplineTransformer
def periodic_spline_transformer(period, n_splines=None, degree=3):
if n_splines is None:
n_splines = period
n_knots = n_splines + 1
return SplineTransformer(
degree=degree,
n_knots=n_knots,
knots=np.linspace(0, period, n_knots).reshape(n_knots, 1),
extrapolation="periodic",
include_bias=True,
)
Splines can capture sharper, non-sinusoidal patterns than a single sine and cosine pair, such as a weekday-versus-weekend shape. Compare both options on your data.
Option 3: Native Categoricals
The same example also uses HistGradientBoostingRegressor with categorical_features="from_dtype", which treats columns with a categorical dtype as true categories and uses a dedicated splitting rule for them [1]. For tree models, that is often the simplest route for day of week and month.
Lag Features
Lag features give the model recent history. shift(k) moves the series down by k rows, so row t sees the value from t minus k.
Choose lags from two sources:
- Domain knowledge. For daily data, lag 1 (yesterday), lag 7 (same weekday last week), lag 14 and lag 28 often help. The tutorial's comment on lag 7 calls it the same day last week [4].
- Autocorrelation. Plot the autocorrelation function and look for spikes. The right lags depend on the autocorrelation structure of the series [4].
Respect the Forecast Horizon
Here is the subtle part. If you forecast h days ahead, then at prediction time the newest value you know is h days old. A lag of 1 day is only legal when h = 1. Build your features around the horizon:
def add_lags_and_windows(
df, target="sales", horizon=1, lags=(1, 7, 14, 28), windows=(7, 28)
):
out = df.copy()
# the most recent value you'll actually know when predicting
known = out[target].shift(horizon)
for lag in lags:
out[f"{target}_lag_{lag}"] = out[target].shift(horizon - 1 + lag)
for w in windows:
roll = known.rolling(w)
out[f"{target}_roll_mean_{w}"] = roll.mean()
out[f"{target}_roll_std_{w}"] = roll.std()
out[f"{target}_roll_max_{w}"] = roll.max()
out[f"{target}_ewm_14"] = known.ewm(span=14, adjust=False).mean()
out[f"{target}_diff_7"] = known - out[target].shift(horizon + 7)
return out
With horizon=1, lag_1 equals shift(1). With horizon=7, lag_1 becomes shift(7), so it never uses data you would not have.
Rolling and Expanding Window Features
Rolling statistics summarize a sliding window of past observations. The tutorial describes them as capturing local trends, volatility and momentum that individual lag values cannot [4]. Common choices:
- Mean: the local level.
- Standard deviation: recent volatility.
- Min, max and quantiles: the recent range.
- Exponentially weighted mean: weights recent values more heavily, and needs no fixed window.
- Expanding statistics: everything so far, such as a year-to-date maximum, via
.expanding()[5].
Use several window lengths together, like 7, 28 and 90 days, so the model can compare short-term and long-term behaviour.
Trend and Change Features
Features that describe change often beat raw levels:
- Differences:
known - known.shift(7)shows week-over-week change. - Percentage change:
known.pct_change(7)scales the change to the level. - Ratio to a rolling mean: a value well above its recent average signals a spike. The tutorial includes a ratio-to-trend feature along with diffs and percentage changes [4].
- Time index: an elapsed-days counter captures a linear trend for linear models.
Tree models cannot extrapolate beyond the trend range they saw in training, so consider modelling a differenced or detrended target.
Multiple Series (Panel Data)
If your data holds many series, like sales per store, compute features within each group so one store's history does not leak into another's:
g = df.groupby("store")["sales"]
df["sales_lag_7"] = g.shift(7)
df["sales_roll_mean_7"] = g.transform(lambda s: s.shift(1).rolling(7).mean())
Also add the group identifier as a categorical feature, and consider group-level aggregates like each store's historical average, computed on training data only.
Exogenous Features: Known or Unknown at Prediction Time?
External variables help, and they bring a timing question. Ask of every feature: will I have this value when I make the forecast?
| Feature type | Usable at future values? | Examples |
|---|---|---|
| Known ahead | Yes | Calendar, holidays, planned promotions, scheduled price changes |
| Unknown ahead | No, only lagged or forecast | Weather observations, competitor actions, traffic |
A classic trap: training on actual weather and deploying with forecast weather. Train on what production will see, so use historical forecasts when you can get them.
Put It Together With scikit-learn
Compute the causal pandas features on the full series. Then let a scikit-learn pipeline fit scalers and encoders inside each cross-validation fold:
from sklearn.compose import ColumnTransformer
from sklearn.ensemble import HistGradientBoostingRegressor
from sklearn.linear_model import RidgeCV
from sklearn.model_selection import TimeSeriesSplit, cross_validate
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
HORIZON = 1
data = add_lags_and_windows(add_calendar(df), horizon=HORIZON).dropna()
y = data["sales"]
X = data.drop(columns="sales")
lag_cols = [c for c in X.columns if c.startswith("sales_")]
ridge_features = ColumnTransformer(
[
("dow", periodic_spline_transformer(7, n_splines=3), ["dow"]),
("doy", periodic_spline_transformer(366, n_splines=12), ["dayofyear"]),
("lags", StandardScaler(), lag_cols),
],
remainder="passthrough", # promo, is_weekend, is_month_end, month
)
models = {
"ridge": make_pipeline(ridge_features, RidgeCV(alphas=np.logspace(-3, 3, 13))),
"hist_gbm": HistGradientBoostingRegressor(random_state=0),
}
cv = TimeSeriesSplit(n_splits=5, gap=7, test_size=60)
for name, model in models.items():
res = cross_validate(model, X, y, cv=cv, scoring="neg_mean_absolute_error")
print(name, round(-res["test_score"].mean(), 2))
# Seasonal-naive baseline: predict the value from 7 days earlier
naive = np.mean([
(y.iloc[te] - X["sales_lag_7"].iloc[te]).abs().mean()
for _, te in cv.split(X)
])
print("seasonal naive", round(naive, 2))
Notice the validation setup. The scikit-learn example uses TimeSeriesSplit with n_splits, gap, max_train_size and test_size, and in that example a gap of 48 hours separates train and test on hourly data, with 1,000 test points per split [1]. The gap keeps highly autocorrelated rows right next to the boundary out of the comparison, which gives a more honest score.
What the Snippet Actually Scored
On the synthetic data above, the output was:
ridge 4.28
hist_gbm 6.26
seasonal naive 8.51
The spline ridge beat the gradient boosting model, which is not the result most tutorials would predict. The reason is the data: this series is two clean sine waves plus a trend, which is exactly what periodic splines model. Real sales data is rarely that tidy, so do not read this as "ridge beats boosting." Read it as "measure both," and notice that the baseline is the number everything else has to beat.
Why the Baseline Matters
Always beat a simple baseline before you celebrate. A seasonal naive forecast, meaning "same as last week," is hard to beat on strongly seasonal data. If your engineered model cannot beat it, your features are not adding value.
Evaluating Time Series Models
- Use forward-chaining splits, with training always before testing [1].
- Add a gap at least as long as your longest dependence, and at least as long as your forecast horizon minus one.
- Keep a final holdout from the most recent period, and score it once.
- Pick metrics that fit: MAE and RMSE for scale-dependent errors, and percentage-based or scaled errors when you compare series.
- Check errors by horizon, season and segment, not only overall.
- Re-run backtests whenever you change features.
Libraries Worth Knowing (From My Own Knowledge)
I did not verify these in this research, so check their current status:
- statsmodels: autocorrelation plots and classical time series models.
- tsfresh: automated extraction of many time series features. It can generate hundreds of features, so watch for leakage and overfitting.
- sktime, skforecast and Darts: frameworks that wrap the lag-and-window workflow and handle forecasting-specific cross-validation.
Hand-built features teach you what is happening, so build a few yourself before you automate.
Common Pitfalls
- Shuffled cross-validation [4].
- Centered windows [4].
- Forgetting to shift before rolling.
- Ignoring the forecast horizon. A feature that is illegal at horizon 7 inflates your score.
- Fitting scalers on all data [5].
- Dropping rows incorrectly. Lags and windows create NaNs at the start, so drop them after you build features, and never fill them from the future.
- Training on actuals and serving forecasts for exogenous variables.
- Extrapolating trends with trees. They cannot.
- Skipping the baseline.
- Too many correlated features. Lags and rolling means overlap a lot, so check importance and stability.
Your Quick-Start Checklist
- Sort by time, and use a
DatetimeIndex. - Define the forecast horizon, and decide what you will know at prediction time.
- Add calendar and event features.
- Encode cycles with sine and cosine, splines, or native categoricals.
- Build lags and rolling features with
shiftfirst, relative to the horizon. - Add change and trend features.
- Compute group features within each series.
- Use pipelines so transformers fit only on training folds.
- Validate with
TimeSeriesSplitand a gap, and compare against a seasonal naive baseline. - Hold out the most recent period, and score it once.
Wrapping Up
Time series feature engineering comes down to two skills: turning time into signals a model can use, and refusing to let the future leak into the past. Calendar features, cyclical encodings, lags and rolling statistics give you the signals. Shifting before rolling, respecting the horizon and splitting in order keep you honest.
Start with the seasonal naive baseline, add features one group at a time, and check the score after each group. Any feature that makes your validation score jump suspiciously deserves a leakage audit before it earns a celebration.
Frequently Asked Questions
What is target leakage in time series?
Target leakage is any feature that contains information you would not have at prediction time. In time series it usually arrives through shuffled cross-validation, centered rolling windows, a rolling mean that includes today's target, or scaling and imputing before the train/test split. The symptom is a validation score that collapses in production.
Why is TimeSeriesSplit better than KFold for time series?
KFold shuffles rows, so the model trains on future observations and is tested on past ones. TimeSeriesSplit keeps the order: every training fold ends before its test fold begins, and the gap parameter drops the rows closest to the boundary so highly autocorrelated values do not make the score look better than it is.
Do I need sine and cosine encoding for day of week?
Only for models that cannot learn the wrap-around themselves. Tree models handle ordinal calendar features well, and HistGradientBoostingRegressor can treat them as native categoricals. Linear and neural models benefit from sine and cosine pairs or periodic splines, because a raw integer puts Sunday six units away from Monday.
How do I choose lag features?
Start with domain knowledge, such as lag 1 for yesterday and lag 7 for the same weekday last week, then check the autocorrelation and partial autocorrelation functions for spikes. Whatever you choose must respect the forecast horizon: if you predict seven days ahead, a one-day lag is not available at prediction time.
What is a seasonal naive baseline?
It is the forecast that repeats the value from one season ago, such as the same weekday last week. It is cheap, hard to beat on strongly seasonal data, and it turns your validation score into a comparison instead of a number without context.
Sources
[1]scikit-learn, Time-related feature engineering — the official example usingSplineTransformerwithextrapolation="periodic", theTimeSeriesSplitsetup, native categorical support inHistGradientBoostingRegressor, the "linear regression does not automatically model non-monotonic relationships" explanation, the 4 to 5% of maximum demand error for gradient boosting versus around 14% for the naive min-max linear model, and the note that the latter is more than three times higher: scikit-learn.org/stable/auto_examples/applications/plot_cyclical_feature_engineering.html[2]scikit-learn, source of the same example —TimeSeriesSplit(n_splits=5, gap=48, max_train_size=10000, test_size=1000), the comment that 1,000 test datapoints is a bit less than a month and a half of contiguous hourly test data, andcategorical_features="from_dtype"for columns with a categorical dtype: scikit-learn.org/1.9/_downloads/9fcbbc59ab27a20d07e209a711ac4f50/plot_cyclical_feature_engineering.py[3]pandas, Time/date components — the date component accessors the scikit-learn example links to for hour of day, day of week, day of month and month of year. I did not open this page, and the link returned 200 when I checked it on 2026-10-31: pandas.pydata.org/pandas-docs/stable/user_guide/timeseries.html#time-date-components[4]KindaTechnical, Time Series Feature Engineering — the tutorial behind the leakage notes on shuffled splits and.rolling(7, center=True), the "most common mistake in time series ML is using standard cross-validation instead of time-aware splitting" warning, the lag-7 comment about the same day last week, the description of rolling features capturing local trends, volatility and momentum, the autocorrelation guidance on choosing lags, and the ratio-to-trend feature: kindatechnical.com/machine-learning/time-series-feature-engineering.html[5]Hugging Face Space DataScience, feature engineering reference commit — the commit that adds the time series and target leakage sections, with lag, rolling and expanding examples and the preprocessing leakage warning: huggingface.co/spaces/AashishAIHub/DataScience/commit/ba2eb6ff81cb80d33afa8cb849834a87761bffd2