1) What it is

  • SHAP is a unified framework for explaining predictions of any machine learning model.
  • Based on Shapley values from cooperative game theory (Lloyd Shapley, 1953).
  • Idea: Treat each feature as a “player” in a game; the prediction is the “payout.”
  • SHAP assigns each feature a fair share of contribution to the prediction.

Each feature gets a number: how much it pushed the prediction up or down relative to the baseline.

2) Formula (conceptual)

For a prediction $\hat{y}$​:

$\hat{y} = \phi_0 + \sum_{i=1}^M \phi_i$

  • $\phi_0$​ = baseline value (average prediction if no features are known).
  • $\phi_i$​ = SHAP value for feature $i$ (contribution).

So the prediction is exactly decomposed into contributions from each feature.

3) Example

Model: Loan approval probability = 0.8

  • Baseline (population average) = 0.5
  • SHAP explanation:
    • Income = +0.2
    • Employment history = +0.1
    • Debt ratio = 0
    • Age = 0

$0.8 = 0.5 + 0.2 + 0.1$

Interpretation: Income and employment history raised approval probability.

4) Strengths of SHAP

Consistent: Fair allocation of feature contributions.
Local + Global: Explains individual predictions and overall feature importance.
Model-agnostic or model-specific: Works with tree models, deep nets, linear, etc.
Visualizations: Force plots, summary plots, dependence plots.

5) Visualization Examples

  • Force Plot: shows how each feature pushed a single prediction up/down.
  • Summary Plot: global view of feature impacts across dataset.
  • Dependence Plot: feature interaction effects.

6) Limitations

Computational cost: Exact Shapley values are exponential in features; SHAP uses approximations.
Interpretation risk: Explanations can be misused if not carefully contextualized.
Correlated features: Hard to disentangle their contributions fairly.

7) Python Example

import shap
import xgboost as xgb
from sklearn.datasets import load_boston

X, y = load_boston(return_X_y=True)
model = xgb.XGBRegressor().fit(X, y)

explainer = shap.Explainer(model, X)
shap_values = explainer(X)

# Summary plot (global feature importance)
shap.summary_plot(shap_values, X)

# Force plot (local explanation for 1 instance)
shap.force_plot(explainer.expected_value, shap_values[0], X[0])

Summary

  • SHAP = explanation method based on Shapley values.
  • Fairly attributes contributions of features to predictions.
  • Useful for both local (instance-level) and global (dataset-level) interpretability.
  • Strong theoretical foundation, widely adopted in ML pipelines.