How to Extract Decision Rules from a Random Forest

Contents

Figure 1: How to Extract Decision Rules from a Random Forest — a hundred voters, and four ways to get a readable story out of them

Your random forest predicts well, and then your manager asks the dreaded question: "Why did it say that?" You stare at 100 trees and realize you have no good answer. Been there, and it stings.

The good news is that you can pull readable rules out of a random forest. This guide shows you four practical ways to extract decision rules from a random forest, with scikit-learn code I ran myself. Ready to open the black box?

Why Random Forests Hide Their Logic

A single decision tree reads like a flowchart. A random forest combines hundreds of them, and each tree votes. No single path explains the final answer.

That design makes the forest accurate and hard to read. You get a strong prediction, but you lose the simple story. Rule extraction brings some of that story back.

Here are your main options:

1. Read a single tree: print one tree from the forest as text.
2. Extract rules by hand: walk a tree and turn each leaf into an IF-THEN rule.
3. Rank feature importance: see which features drive the forest overall.
4. Train a surrogate tree: teach one simple tree to imitate the whole forest.

Each method answers a different question. Let us set up some data and try them all.

Set Up a Small Forest

We will use the breast cancer dataset that ships with scikit-learn. I capped the depth at 4, because shallow trees make rules far easier to read. Deep trees produce rules nobody wants to scroll through.

import numpy as np
from sklearn.datasets import load_breast_cancer
from sklearn.ensemble import RandomForestClassifier
from sklearn.tree import DecisionTreeClassifier, export_text
from sklearn.model_selection import train_test_split

data = load_breast_cancer()
X, y = data.data, data.target
names = list(data.feature_names)

X_tr, X_te, y_tr, y_te = train_test_split(
    X, y, test_size=0.2, stratify=y, random_state=42
)

forest = RandomForestClassifier(
    n_estimators=100, max_depth=4, random_state=42
).fit(X_tr, y_tr)

print(forest.score(X_te, y_te))

The forest scored about 0.956 accuracy on my test split. Keep that number in mind, because we will compare it later. Now let us pry the forest open.

Method 1: Print a Single Tree With export_text

The fastest option uses export_text, which prints a tree as indented text. Every tree lives in the estimators_ list. Grab one and print it.

tree = forest.estimators_[0]
print(export_text(tree, feature_names=names, max_depth=2))

The output starts with a split on worst concave points at about 0.15. Samples below that value go one way, and the rest go another. You can follow each branch down to a class label.

This method gives you a quick peek, but one tree tells you little about the whole forest. The other 99 trees might disagree. Think of it as reading one jury member's notes.

Method 2: Turn Every Leaf Into an IF-THEN Rule

Now let us get serious. Each path from the root to a leaf forms one rule. You collect the conditions along the way, then attach the leaf's predicted class and sample count.

def extract_rules(tree, feature_names, class_names):
    t = tree.tree_
    rules = []

    def walk(node, conds):
        if t.children_left[node] == -1:
            counts = t.value[node][0]
            label = class_names[int(np.argmax(counts))]
            rules.append((conds, label, int(t.n_node_samples[node])))
            return
        name = feature_names[t.feature[node]]
        thr = t.threshold[node]
        walk(t.children_left[node], conds + [f'{name} <= {thr:.2f}'])
        walk(t.children_right[node], conds + [f'{name} > {thr:.2f}'])

    walk(0, [])
    return rules

rules = extract_rules(forest.estimators_[0], names, data.target_names)
for conds, label, n in sorted(rules, key=lambda r: -r[2])[:2]:
    print('IF', ' AND '.join(conds), '->', label, f'({n} samples)')

The first tree produced 11 rules. One rule covered 74 training samples: when worst concave points sits above 0.15 and worst perimeter sits above 97.77, the tree predicts malignant. That reads like something a doctor could sanity-check.

Scikit-learn stores a leaf's value differently across versions. Some versions store class counts, and newer ones store fractions. Taking the argmax works either way.

Make the Rules Easier to Trust

Raw rules need some care before you show them to anyone. Follow these habits:

1. Sort by sample count. Rules that cover many samples matter more than rules covering three.
2. Check purity. A leaf with mixed classes gives a weaker rule.
3. Keep depth low. Short rules stay readable and generalize better.
4. Test the rules. Apply them to held-out data before you quote them.

Method 3: Rank Features by Importance

Sometimes you do not need rules. You just need to know which features matter. Random forests expose feature_importances_, which averages how much each feature reduces impurity across all trees.

importance = sorted(zip(forest.feature_importances_, names), reverse=True)
for value, name in importance[:5]:
    print(name, round(value, 3))

On my run, worst area led at 0.144. Worst concave points followed at 0.129, then worst radius at 0.104. The remaining top five were mean concave points and mean perimeter.

Rank	Feature	Importance
1	Worst area	0.144
2	Worst concave points	0.129
3	Worst radius	0.104
4–5	Mean concave points, mean perimeter	below 0.104

These scores tell you what the forest uses, not how it uses it. A feature can rank high without telling you whether bigger values raise or lower the risk. Importance works best as a starting point for deeper digging.

Method 4: Train a Surrogate Tree

Here is my favorite trick. A surrogate tree is a small, simple decision tree trained to copy the forest's predictions. You feed it the forest's outputs instead of the true labels. The surrogate then gives you one readable set of rules that approximates the whole forest.

surrogate = DecisionTreeClassifier(max_depth=3, random_state=42)
surrogate.fit(X_tr, forest.predict(X_tr))

fidelity = np.mean(surrogate.predict(X_te) == forest.predict(X_te))
print('fidelity', round(fidelity, 3))
print('accuracy', round(surrogate.score(X_te, y_te), 3))
print(export_text(surrogate, feature_names=names))

The key metric here is fidelity, which measures how often the surrogate agrees with the forest. My depth-3 surrogate hit a fidelity of 0.947. It also scored 0.939 accuracy on the true labels, compared with 0.956 for the full forest.

Model	Fidelity to forest	Accuracy on true labels
Random forest (100 trees, depth 4)	—	0.956
Surrogate tree (depth 3)	0.947	0.939

That tiny drop buys you a lot. You trade about two points of accuracy for a model you can print on one slide. Is that a fair deal? For stakeholder presentations, yes.

Explain One Prediction: Follow the Decision Path

Global rules help, but sometimes you need to explain one specific case. Scikit-learn's decision_path method shows which nodes a sample visited in a tree. You can print the checks along that path.

sample = X_te[:1]
tree = forest.estimators_[0]
t = tree.tree_

for n in tree.decision_path(sample).indices:
    if t.children_left[n] != -1:
        f = t.feature[n]
        op = '<=' if sample[0, f] <= t.threshold[n] else '>'
        print(names[f], round(sample[0, f], 2), op, round(t.threshold[n], 2))

For my first test sample, the path checked worst concave points (0.19, above the 0.15 cutoff) and then worst perimeter (178.6, above 97.77). That gave a clear, human-readable reason for one prediction. Remember that this explains one tree, not the whole forest.

Which Method Should You Use?

Pick the method that fits your question:

Question	Method
Quick peek at the logic	export_text on one tree
Auditable IF-THEN rules	Hand-extracted leaf rules
Which features matter overall	feature_importances_
One simple model for a presentation	A surrogate tree
Why this single prediction	The decision path

I usually combine two of them. Feature importance shows what matters, and a surrogate tree shows how those features combine. Together they tell a pretty complete story.

Common Pitfalls

Rule extraction comes with traps. Avoid these:

1. Treating one tree as the forest. A single tree can mislead you. Prefer a surrogate when you want the overall picture.
2. Ignoring fidelity. A surrogate with low fidelity explains nothing. Always report the number.
3. Using deep trees. A depth-20 tree produces thousands of rules. Limit depth when readability matters.
4. Trusting importance blindly. Impurity-based importance can favour features with many unique values. Confirm important features with permutation importance.
5. Forgetting correlated features. Correlated features split the credit between them, which hides their true importance.
6. Quoting rules without a coverage count. A rule that fires on two training rows is an anecdote, not a pattern.
7. Extracting rules from a forest trained on unscaled, unvalidated data. The rules inherit every data problem the model has.

Wrapping Up

A random forest does not have to stay a mystery. You can print a tree, extract leaf rules, rank features, or train a surrogate to get readable explanations. Pick the method that matches the question you need to answer.

So train a shallow forest on your own data and try the surrogate trick first. Check the fidelity, read the rules, and see whether they make sense to a human. When they do, you finally have an answer for your manager.

Two follow-ups worth your time: how the out-of-bag score validates a forest without a test set, and building the forest yourself in NumPy so the tree internals above stop being opaque. For the wider reading list, see the best interpretable machine learning books.

Frequently Asked Questions

Can you extract human-readable rules from a random forest?

Yes, in four practical ways. You can print a single tree with export_text, walk every root-to-leaf path and turn it into IF-THEN conditions, rank features by impurity-based importance, or train a shallow surrogate tree to imitate the forest's predictions. Only the last two describe the forest as a whole; the first two describe one tree out of a hundred.

Why is a single tree from the forest not a good explanation?

Because it is one of a hundred voters. A tree can split on a feature the rest of the forest barely uses, and its leaf rules can contradict the majority. Printing one tree is a quick sanity check on the data, not an account of the model. If you need rules for the whole forest, train a surrogate or aggregate the rules across all trees and rank by coverage.

What is a surrogate tree and what is fidelity?

A surrogate is a shallow decision tree trained on the forest's own predictions rather than the true labels, so it learns to imitate the forest. Fidelity is the share of rows where the surrogate and the forest agree. High fidelity with low depth means the surrogate is a faithful, readable stand-in; low fidelity means the rules explain the surrogate, not the model.

How do I read a surrogate tree's accuracy versus its fidelity?

Fidelity measures agreement with the forest, while accuracy measures agreement with the true labels. A surrogate can have 0.947 fidelity and 0.939 accuracy against a forest at 0.956, which is the useful trade: a two-point accuracy drop in exchange for rules you can put on a slide. Report both numbers or the explanation is unfalsifiable.

Why does impurity-based feature importance mislead on some datasets?

It is computed on training data and biased toward features with many distinct values, such as continuous measurements or high-cardinality IDs, because they offer more places to split. Permutation importance avoids that by shuffling a column in held-out data and measuring the accuracy drop, so it reflects what the model needs rather than how often it split.

How do I explain a single random forest prediction?

Use decision_path on one estimator to list the nodes a sample passed through, then print the feature, the sample's value, the comparison and the threshold at each internal node. That gives a readable chain for one tree. Because the forest averages many trees, treat it as one voter's reasoning rather than the forest's final justification.

Sources

• [1] scikit-learn, sklearn.tree.export_text — how a fitted tree is rendered as indented text, and the max_depth argument used to keep the printout readable: scikit-learn.org/.../export_text
• [2] scikit-learn, "Ensembles: forests of randomized trees" — the feature-importance section explaining how impurity decrease is averaged across trees and why the result is biased toward high-cardinality features: scikit-learn.org/modules/ensemble
• [3] scikit-learn, "Permutation feature importance" — the model-agnostic alternative that shuffles a column in held-out data, with the caveats about correlated features: scikit-learn.org/modules/permutation_importance
• [4] scikit-learn, DecisionTreeClassifier API reference — decision_path, tree_, and the children_left / feature / threshold arrays this post walks directly: scikit-learn.org/.../DecisionTreeClassifier
• [5] Christoph Molnar, "Interpretable Machine Learning" — global surrogate models, the fidelity measure, and the limits of tree-based explanations: christophm.github.io/interpretable-ml-book

Related Articles

• Random Forest OOB Score Explained: Validation Without a Test Set
• Random Forest from Scratch in Python (NumPy Only)
• The Best Interpretable Machine Learning Books