SHAP vs LIME: Which Explainability Method Should You Use?

Contents

Figure 1: SHAP vs LIME: Which Explainability Method Should You Use? — a bar chart a stakeholder will trust, and the settings that decide whether it deserves that trust

Your model rejects a loan application, and someone asks you why. You run an explainer, get a bar chart, and read it out with total confidence. How sure are you that the chart is right?

That question separates SHAP from LIME more than any feature table does. I tested both on problems where I already knew the correct answer, by building each method from scratch, and the results changed how I use them.

SHAP and LIME answer different questions, and only one of them promises that the numbers add up to the prediction. Here is what each computes, what broke when I tested them, and how to choose between them.

What SHAP and LIME Actually Compute

LIME fits a small, simple model around a single prediction. The original paper describes it as learning an interpretable model locally around the prediction, and says it can explain the predictions of any classifier. What you get back is a set of coefficients: the slopes of a local linear fit. For a full library walkthrough, see the LIME tutorial.

from lime.lime_tabular import LimeTabularExplainer

explainer = LimeTabularExplainer(X_train, mode="regression")
explanation = explainer.explain_instance(x_row, predict_fn)

SHAP assigns each feature an importance value for a particular prediction. The authors identify a class of additive feature importance measures, and they report a unique solution within that class that satisfies a set of desirable properties. That solution is the Shapley value from cooperative game theory, and the contributions reconcile with the prediction. The library side is covered in SHAP values in Python.

import shap

explainer = shap.Explainer(model.predict, X_background)
values = explainer(x_row)

In short:

• LIME answers: if I nudge the inputs near this point, how does the output change?
• SHAP answers: how much did each feature push this prediction away from the baseline?

Those are different questions, so the numbers differ even when both methods work perfectly. That is the single most common source of confusion when teams compare the two bar charts side by side.

How I Tested Both Methods From Scratch

I could not install either library in my sandbox, so I built both myself. That turned out to be useful, because it forced every assumption into the open instead of hiding it behind a default.

Here is what I did:

1. Pick a model with a known formula. My test model is 3x₀ + 2x₁x₂ − x₃², with a fifth feature x₄ that does nothing at all. Because I wrote the formula, I know the true local slope of every real feature, which is the ground truth most explainer comparisons never have.
2. Build a LIME-style surrogate. Gaussian perturbations around the point, a kernel that weights nearby samples more heavily, then a ridge fit. Mine is a simplified lookalike, not the lime package.
3. Compute exact Shapley values by enumerating every feature subset. That is the quantity SHAP's exact, permutation and kernel explainers are all targeting.
4. Compare both against the truth, not against each other.

def f(x):
    return 3 * x[0] + 2 * x[1] * x[2] - x[3] ** 2   # x[4] is dead weight

# exact Shapley needs one fit per feature subset per background row
from itertools import combinations

subsets = [set(c) for r in range(len(features) + 1) for c in combinations(features, r)]

Everything below describes my two implementations on a toy model, not the published packages. Where a result depends on a setting I chose, I say so.

Different Units, Different Questions

At the point I explained, here is what each method reported:

Feature	True local slope	LIME-style coefficient	Exact Shapley value
x₀	3.00	2.999	3.263
x₁	2.00	1.996	1.554
x₂	3.00	3.000	1.239
x₃	−3.00	−3.000	−1.159
x₄ (dead)	0.00	−0.001	0.000

LIME recovered the true local slopes almost perfectly. It reported 2.999 where the truth is 3.00, and it gave the dead feature a coefficient of −0.001.

The Shapley values look different because they measure contributions relative to a baseline, which here is the average prediction over the background data. My Shapley values summed to 4.896, the base value was −1.146, and the model's actual output was 3.750, so base plus contributions matched exactly.

That exact match is additivity, and additivity is SHAP's headline property. LIME's coefficients carry no such guarantee, and my own conversion attempt (coefficient times deviation from the average) summed to 4.502 against the 4.896 required. That gap of about 0.39 comes from a conversion I invented, so read it as a demonstration of the problem, not a verdict on LIME.

LIME's Kernel Width Is a Real Knob

LIME weights nearby samples more heavily, and a kernel width decides what counts as nearby. On my quadratic model, widths from 0.75 up to 50 gave identical answers, which taught me nothing at all. So I moved to a wiggly model, sin(3x₀) + x₁, at a point where the true slope is 1.865:

Kernel width	LIME-style slope for x₀
0.10	1.818
0.25	1.633
0.75	0.693
1.68	0.134
5.00	0.029

Same model, same point, same data, and the explanation slides from 1.8 down to nearly zero. Narrow widths follow the local slope. Wide widths average over the wiggles. If your model is bumpy, the width you pick decides what you are told.

A second trap came up in the same experiment: ridge shrinkage. On the first test model, width 0.25 collapsed every coefficient toward zero. The kernel weights summed to only about 3.85, so a default ridge penalty of 1 swamped the local fit. Dropping the penalty to 1e-6 restored the correct slopes (2.994, 1.995, 3.010, −3.027).

# weighted local fit: the kernel and the ridge penalty interact
w = np.exp(-(d ** 2) / (2 * kernel_width ** 2))
coef = ridge_fit(X_perturbed, y_perturbed, sample_weight=w, alpha=1e-6)

I did not check whether the real lime package shows the same interaction, so test it on your own data before blaming the library.

The moment this became personal: my first run of the wiggly model used the width 1.68 row, and I wrote a sentence explaining a slope of 0.134 for a feature whose true local slope was 1.865. The number was correct for the settings I chose, and the sentence was wrong for the question I was answering. Printing five widths in one loop would have caught it, and it costs almost nothing.

Both Methods Wobble Across Seeds

Both methods rely on random sampling, so both can give different answers on different runs. I repeated each with 40 seeds at a budget of about 1,000 model evaluations. Slopes and contributions have different units, so raw standard deviations are not comparable. I compared relative wobble instead, meaning standard deviation divided by the mean, for x₀ through x₃:

Feature	LIME-style relative wobble	Permutation Shapley relative wobble
x₀	1.3%	6.4%
x₁	3.1%	16.6%
x₂	2.3%	23.1%
x₃	2.2%	11.4%

On this smooth toy, my LIME-style method wobbled less, which surprised me. I expected the opposite result. Treat it as a property of an easy model rather than a ranking of the two methods.

Published work agrees that this is a general problem. A 2024 AAAI paper reports that Kernel SHAP suffers from high instability, and says that perturbation-based methods, LIME included, are known to be unstable. I read only the abstract and a summary of the introduction, so that finding is theirs, and the detail is worth checking in the paper itself.

The cure is the same for both methods: more samples. My LIME-style spread shrank roughly with the square root of the sample count, from 0.10 at 200 samples to 0.01 at 20,000. That is the same trade every sampling method makes, and it is why a single-run explanation is a claim rather than a measurement.

Cost: Where Exact SHAP Stops Being an Option

Exact Shapley values need one evaluation per feature subset per background row. With 5 features and 300 background rows, that is 32 × 300 = 9,600 model calls, and every extra feature doubles it:

Features	Feature subsets (2^f)	Model calls at 300 background rows
5	32	9,600
10	1,024	307,200
15	32,768	9,830,400
20	1,048,576	314,572,800

That is why SHAP ships several explainers instead of one. The shap docs list a TreeExplainer for tree ensembles, a LinearExplainer, a DeepExplainer for deep learning, and model-agnostic options such as KernelExplainer and PermutationExplainer. The docs describe PermutationExplainer and DeepExplainer as approximating Shapley values, and describe only ExactExplainer as exact enumeration.

import shap

# tree ensembles: the explainer built for the job
explainer = shap.TreeExplainer(rf_model)
shap_values = explainer.shap_values(x_row)

My rule of thumb: for tree models, use the tree explainer and skip the sampling debate entirely. LIME's cost profile is different in kind, because it fits one small model per explanation, so its budget is set by how many perturbations you request at that point rather than by an exponential in the feature count.

If you want the cheaper global view first, partial dependence and ICE plots usually tell you more per second of effort than a single local explanation.

Can SHAP or LIME Be Fooled?

Yes. Slack and colleagues showed a scaffolding trick that hides a classifier's bias. Their abstract states that post hoc explanation techniques relying on input perturbations, such as LIME and SHAP, are not reliable, and that biased models can easily fool both.

I built a toy version to see it happen. Real-looking inputs go to a model that uses only x₀. Off-distribution inputs, which perturbation methods generate constantly, go to a model that uses only x₂. About 86% of my independent perturbations looked off-distribution. Here is how much attribution each method gave to x₂, a feature the model never uses on real data:

Method	Attribution on x₂
Exact Shapley	20%
LIME-style (raw coefficients)	80%

My exact Shapley was fooled far less than my LIME-style surrogate, which may not be what you expected from a comparison like this. Keep the caveats in view: this is one toy, with one detector and one background. x₀ and x₁ were near-copies of each other, so the even split Shapley gave them is not an error. The published attack tunes its scaffold to each method's sampling behaviour and reports that both can be fooled, which is the more durable finding.

Common Pitfalls

1. Reporting an explanation without its settings. The background data, the kernel width and the sample count are part of the explanation, not implementation detail. A reader who cannot see them cannot judge the chart.
2. Expecting LIME coefficients to add up to the prediction. They are local slopes. Only additive methods such as Shapley values carry that guarantee.
3. Comparing a SHAP value and a LIME coefficient directly. They are measured in different units against different baselines, so the ranking can flip while both methods are working correctly.
4. Using a wide kernel on a wiggly model. The explanation then describes an average over the neighbourhood, not the local behaviour you asked about.
5. Trusting a single seed. Rerun with several seeds and check whether the ranking survives. If it does not, you have found a limit of your budget.
6. Treating an explainer as a fairness certificate. An explanation can look perfectly reasonable while the model leans on a proxy feature.
7. Assuming the explainer sees your data distribution. Perturbation methods generate off-distribution points by construction, and that is exactly the attack surface. The wider explainable AI tooling landscape mostly inherits the same assumption, so the caveat travels with the tool.

Check the edition date before you buy. Methodology books age better than library guides, but the code samples in them do not age at all.

Unlock AI That Actually Works

Get lifetime access to GPT-6 Astra, Claude Fable 5.1, Gemini 3.5, Grok 4.5, and more — all in one platform. Build websites, apps, videos, content, and digital products from a single command. No monthly fees. No tool-hopping.

Click here to get GPTAstra Max now — one-time payment, lifetime access.

Frequently Asked Questions

Which is better, SHAP or LIME?

Neither is better in the abstract, because they answer different questions. Use SHAP when you need contributions that add up to the prediction, and especially for tree ensembles, where the tree explainer is fast and computes values directly from the model structure. Use LIME when you want a quick local sketch, particularly for a model with no dedicated explainer. If the decision is high stakes, use one method and then audit it rather than trusting either one on its own.

Why do SHAP and LIME give different answers?

Because they measure different things. LIME fits a local linear surrogate and returns slopes, which tell you how the output moves when you nudge the inputs near that point. SHAP returns contributions measured against a baseline, usually the average prediction over background data, and those contributions sum to the prediction. In my tests the LIME-style slopes matched the true local derivatives almost exactly, while the Shapley values were smaller for most features, because a slope and a contribution are not the same quantity.

When should I use LIME instead of SHAP?

Use LIME when you want a fast, model-agnostic local explanation and you are willing to tune its kernel width, because its cost is set by the number of perturbations at a single point rather than by an exponential in the feature count. It is also a reasonable choice for text and image models, where perturbation is easy to define. Whatever you use it for, run it at two or three widths and check whether the story survives.

Is SHAP always exact?

No. Exact Shapley values need one evaluation per feature subset per background row, and the subset count doubles with every added feature, so 5 features over 300 background rows already costs 9,600 model calls. Most SHAP explainers approximate instead. In the documentation I read, PermutationExplainer and DeepExplainer are described as approximating Shapley values, and only ExactExplainer is described as exact enumeration. TreeExplainer is the practical exception, because it reads values out of the tree structure instead of sampling.

Do SHAP values have to add up to the prediction?

For additive explainers, yes, by construction. The base value plus the feature contributions equals the model output. In my test the contributions summed to 4.896, the base value was minus 1.146 and the model output was 3.750, which matches exactly. That property is what makes SHAP values easy to present, because the parts reconcile with the whole. LIME coefficients carry no such guarantee.

Can SHAP and LIME be fooled?

Yes, and the published attack does not need a bug in either package. Slack and colleagues showed a scaffolding trick in which real-looking inputs go to one model and off-distribution inputs go to another, so the explainer describes behaviour the model never uses on real data. Their abstract states that perturbation-based post hoc methods such as LIME and SHAP are not reliable, and that biased models can easily fool both. The practical defence is to perturb in ways that stay on your data distribution and to check stability across seeds.

Wrapping Up

SHAP gives contributions that add up to the prediction, and LIME gives local slopes whose meaning depends on a width you choose. Both rely on sampling, both depend on settings the reader never sees, and the published research says both can be fooled.

Pick the method that matches your question, then audit it: rerun it, change the knob, and check whether the story survives. If the story does not survive, you have learned something about the model rather than about the explainer — and that is usually the more valuable finding.

Which method are you running in production, and does your explanation survive a rerun?

Related Articles

• LIME Tutorial: Explain Individual Predictions
• SHAP Values in Python: Explain Your Machine Learning Model Predictions
• Partial Dependence and ICE Plots: Feature Effects Explained
• Best Explainable AI and Model Interpretability Tools
• InterpretML: Explainable Boosting Machines