SHAP values: explain a prediction without claiming causation

SHAP methods use Shapley-value ideas to allocate a model prediction among features relative to a baseline. They explain the specified model under a specified background and feature-handling assumption. They do not show what would happen if a person or business changed a feature.

Derive a two-feature attribution exactly

Let the fitted function be f(x1,x2) = 1 + 2*x1 + 3*x2. Use baseline [0,0] and explain point [2,1].

The baseline prediction is 1. The point prediction is 8. For two features, each feature's Shapley value averages its marginal contribution when it enters first and when it enters second.

Feature one contributes 4 and feature two contributes 3. Baseline plus contributions equals the prediction: 1 + 4 + 3 = 8.

from feature_cases import shapley_case

r = shapley_case()
assert r['baseline_prediction'] == 1
assert r['prediction'] == 8
assert r['feature_contributions'] == [4, 3]
assert r['sum_check'] == r['prediction']
print(r)

Run the exact coalition calculation in the feature-engineering lab. It does not invoke the external shap package or claim a TreeSHAP run.

Treat baseline and dependence as part of the result

Change the background and the contributions change. An overall population, recent cohort and customer segment can produce different reference predictions. Record the dataset, weighting and version used.

When features are correlated, “missing” a feature requires an assumption. Interventional approaches can create combinations off observed support; conditional approaches need a dependence model. Neither automatically answers a causal intervention.

A positive attribution means the feature value pushed this model output above the chosen reference under the explainer's rules. It does not mean the feature caused the outcome or that changing it will produce the opposite result. Proxy variables can receive attribution while standing in for another process.

Validate explanations with simple cases, sum checks, sensitivity to background and model-behaviour tests. Use local attribution alongside global error analysis and data support. Do not treat colourful plots as proof of fairness or scientific mechanism.

Exercise: change the baseline to [1,1], recompute exact contributions and explain the new reference. Then create perfectly correlated features and compare interventional replacement with an on-support conditional definition.

NeuraPath's Data Science course teaches model explanation with explicit baselines and assumptions. Attribution describes a model calculation; causal claims require a different design.

Continue learning

This article is part of the Feature engineering and data quality sequence. Use the neighbouring tasks when you need the prerequisite or the next application.

• Review the prerequisite or neighbouring task in Permutation importance with correlated features.
• Continue with Detect training-serving skew with schema contracts.