A Unified Approach to Interpreting Model Predictions (SHAP)
0 likes•872 views
The document discusses the SHAP (SHapley Additive exPlanations) framework for interpreting model predictions, emphasizing the importance of explainability in machine learning. It outlines various additive feature attribution methods, their computational techniques, and the theoretical properties that ensure their accuracy and consistency. The conclusion highlights the unique solution provided by the SHAP framework, which balances model accuracy with interpretability.
A Unified Approach to Interpreting Model Predictions (SHAP)
- 1. A Unified Approach to Interpreting Model Predictions Rama Irsheidat Scott M. Lundberg. et al.
- 2. Introduction SHAP Additive Feature Attribution Methods Simple Properties Uniquely Determine Additive Feature Attributions 02 01 04 03 TABLE OF CONTENTS Computational and User Study Experiments 05 Conclusion 06
- 3. Why do we care so much about explainability in ML ?
- 6. Introduction Understanding why a model makes a certain prediction can be as crucial as the prediction’s accuracy in many applications
- 7. Introduction The reason this problem exists is with in Large Datasets complex models tend to be very accurate but hard to interpret
- 8. Introduction Focus on explaining individual predictions one at a time.
- 9. Introduction We are replacing the input to a summation that you would get in a linear model with something that represents the importance of that feature in the complicated model.
- 10. Interprets individual model predictions based on locally approximating the model around a given prediction. Interprets the predictions of deep networks. Recursive prediction explanation method for deep learning. Shapley regression values Quantitative Input Influence Shapley sampling values LIME DeepLIFT Layer-Wise Relevance Propagation Feature importance for linear models. This method requires retraining the model on all features. It assigns an importance value to each feature that represents the effect on the model prediction of including that feature. Explaining any model by Applying sampling approximations to Equation in Shapley reg. values , and Approximating the effect of removing a variable from the model by integrating over samples from the training dataset. Proposes a sampling approximation to Shapley values that is nearly identical to Shapley sampling values.
- 11. Interprets individual model predictions based on locally approximating the model around a given prediction. Interprets the predictions of deep networks. Recursive prediction explanation method for deep learning. Shapley regression values Quantitative Input Influence Shapley sampling values LIME DeepLIFT Layer-Wise Relevance Propagation Feature importances for linear models in the presence of multicollinearity. This method requires retraining the model on all features. It assigns an importance value to each feature that represents the effect on the model prediction of including that feature. Explain any model by Applying sampling approximations to Equation in Shapley reg. values , and Approximating the effect of removing a variable from the model by integrating over samples from the training dataset. Proposes a sampling approximation to Shapley values that is nearly identical to Shapley sampling values. Additive Feature Attribution Methods
- 12. Additive Feature Attribution Methods Interprets individual model predictions based on locally approximating the model around a given prediction. Interprets the predictions of deep networks. Recursive prediction explanation method for deep learning. Shapley regression values Quantitative Input Influence Shapley sampling values LIME DeepLIFT Layer-Wise Relevance Propagation Feature importances for linear models in the presence of multicollinearity. This method requires retraining the model on all features. It assigns an importance value to each feature that represents the effect on the model prediction of including that feature. Explain any model by Applying sampling approximations to Equation in Shapley reg. values , and Approximating the effect of removing a variable from the model by integrating over samples from the training dataset. Proposes a sampling approximation to Shapley values that is nearly identical to Shapley sampling values. Have some better theoretical grounding but slower computation
- 13. Additive Feature Attribution Methods Interprets individual model predictions based on locally approximating the model around a given prediction. Interprets the predictions of deep networks. Recursive prediction explanation method for deep learning. Shapley regression values Quantitative Input Influence Shapley sampling values LIME DeepLIFT Layer-Wise Relevance Propagation Feature importances for linear models in the presence of multicollinearity. This method requires retraining the model on all features. It assigns an importance value to each feature that represents the effect on the model prediction of including that feature. Explain any model by Applying sampling approximations to Equation in Shapley reg. values , and Approximating the effect of removing a variable from the model by integrating over samples from the training dataset. Proposes a sampling approximation to Shapley values that is nearly identical to Shapley sampling values. Have faster estimation but less guarantees
- 14. Interprets individual model predictions based on locally approximating the model around a given prediction. Interprets the predictions of deep networks. Recursive prediction explanation method for deep learning. Shapley regression values Quantitative Input Influence Shapley sampling values LIME DeepLIFT Layer-Wise Relevance Propagation Feature importances for linear models in the presence of multicollinearity. This method requires retraining the model on all features. It assigns an importance value to each feature that represents the effect on the model prediction of including that feature. Explain any model by Applying sampling approximations to Equation in Shapley reg. values , and Approximating the effect of removing a variable from the model by integrating over samples from the training dataset. Proposes a sampling approximation to Shapley values that is nearly identical to Shapley sampling values.
- 15. SHAP
- 16. How should we define importance of each feature (φi (f , x))
- 17. Base rate of loan rejection or how often do people get their loans denied on average?
- 18. Why am I 55 percent ?
- 19. We have to explain is this 35 percent difference here . So how can we do this?
- 20. We should just take the expected value of the output of our model (Base rate), then we can just introduce a term into that conditional expectation. Fact that John's 20, his risk jumps up by 15 percent.
- 21. We should just take the expected value of the output of our model (Base rate), then we can just introduce a term into that conditional expectation. A very risky profession and that jumps the risk up to 70 percent.
- 22. We should just take the expected value of the output of our model (Base rate), then we can just introduce a term into that conditional expectation.
- 23. We should just take the expected value of the output of our model (Base rate), then we can just introduce a term into that conditional expectation. He made a ton of money in the stock market last year. So his capital gains pushes him down to 55 percent.
- 24. We have to explain is this 35 percent difference here . So how can we do this? We've basically divided up how we got from here to here by conditioning one at a time on all the features until we've conditioned on all of them.
- 26. Example:
- 27. We can't just pick a particular order and think that we've solved it so what do we do here? Meaningless
- 28. Simple Properties Uniquely Determine Additive Feature Attributions
- 29. Means the output of the explanation model matches the original model for the prediction being explained. Requires features missing in the original input to have no impact. If you change the original model such that a feature has a larger impact in every possible ordering, then that input's attribution (importance) should not decrease. Shapley Properties Local accuracy Missingness Consistency
- 30. SHAP values arise from averaging the φi values across all possible orderings. Very painful to compute.
- 31. Find an approximate solution
- 32. 1. Model-Agnostic Approximations 1.1 Shapley sampling values 2.1 Kernel SHAP (Linear LIME + Shapley values) Linear LIME (uses a linear explanation model) fit a linear model locally to the original model that we're trying to explain. Shapley values are the only possible solution that satisfies Properties 1-3 – local accuracy, missingness and consistency. This means we can now estimate the Shapley values using linear regression.
- 33. 2. Model-Specific Approximations 2.1 Linear SHAP For linear models, SHAP values can be approximated directly from the model’s weight coefficients. 2.2 Low-Order SHAP 3.2 Max SHAP Calculating the probability that each input will increase the maximum value over every other input.
- 34. 2. Model-Specific Approximations 2.4 Deep SHAP (DeepLIFT + Shapley values) DeepLIFT Recursive prediction explanation method for deep learning that satisfies local accuracy and missingness, we know that Shapley values represent the attribution values that satisfy consistency. Adapting DeepLIFT to become a compositional approximation of SHAP values, leading to Deep SHAP.
- 35. Computational and User Study Experiments
- 36. 1. Computational Efficiency Comparing Shapley sampling, SHAP, and LIME on both dense and sparse decision tree models illustrates both the improved sample efficiency of Kernel SHAP and that values from LIME can differ significantly from SHAP values that satisfy local accuracy and consistency.
- 37. 2. Consistency with Human Intuition (A) Attributions of sickness score (B) Attributions of profit among three men Participants were asked to assign importance for the output (the sickness score or money won) among the inputs (i.e., symptoms or players). We found a much stronger agreement between human explanations and SHAP than with other methods.
- 38. 3. Explaining Class Differences Explaining the output of a convolutional network trained on the MNIST digit dataset. (A) Red areas increase the probability of that class, and blue areas decrease the probability . Masked removes pixels in order to go from 8 to 3. (B) The change in log odds when masking over 20 random images supports the use of better estimates of SHAP values.
- 39. Conclusion • The growing tension between the accuracy and interpretability of model predictions has motivated the development of methods that help users interpret predictions. • The SHAP framework identifies the class of additive feature importance methods (which includes six previous methods) and shows there is a unique solution in this class that adheres to desirable properties. • We presented several different estimation methods for SHAP values, along with proofs and experiments showing that these values are desirable.
- 40. THANKS!