What If? Demystifying AI Decisions with Counterfactuals
- 1. What If? Demystifying AI Decisions with Counterfactuals David Martens admantwerp.github.io
- 2. Advanced AI in our lives
- 3. Black Box? Deep learning: large artificial neural network with massive number of parameters o MobileNetV2: 4 million parameters o GPT-4: estimated 100 trillion parameters Non-linear models
- 4. Complex AI models are creeping into our lives Whose decisions are extremely difficult to explain But Explanations are needed if we want to trust AI
- 5. Explain individual predictions Counterfactual explanation: What needs to change in your data, to reach a different decision?
- 6. Explain individual predictions Example: credit scoring using sociodemo and financial data User xi: Sam, with income $ 32,000 and 39 years old. Sam is denied credit WHY? Counterfactual explanation: What needs to change in your data, to reach a different decision?
- 7. Explain individual predictions Example: credit scoring using sociodemo and financial data User xi: Sam Sam is denied credit WHY? Dieter Brughmans, Pieter Leyman, David Martens (2023) NICE : an algorithm for nearest instance counterfactual explanations. Data mining and knowledge discovery p. 1-39 IF Sam would make 8,000$ more THEN his predicted class would change from denied to granted
- 8. Explain individual predictions Counterfactual Explanation LIME: Linear Interpretable Model-Agnostic Explainer (k=10) Example: gender prediction using movie viewing data User xi: Sam Sam watched 120 movies Sam is predicted as male IF Sam would not have watched {Taxi driver, The Dark Knight, Die Hard, Terminator 2, Now You See Me, Interstellar}, THEN his predicted class would change from male to female WHY? Martens D, Provost F. (2014) Explaining Data-Driven Document Classifications. MIS Quarterly 38(1):73-99. admantwerp.github.io M.T. Ribeiro, S. Singh, C. Guestrin (2016) Model-Agnostic Interpretability of Machine Learning 2016 ICML Workshop on Human Interpretability in Machine Learning (WHI 2016), New York, NY github.com/marcotcr/lime Ramon Y., Martens D., Provost F., Evgeniou T. (2019) Instance-level explanation algorithms SEDC, LIME, SHAP for behavioral and textual data: a counterfactual-oriented comparison, Advances in Data Analysis and Classification 14:4, p. 801-819.
- 9. Complex AI models are creeping into our lives Whose decisions are extremely difficult to explain But explanations are needed if we want to trust AI Counterfactuals explain a decision of a model for an instance
- 10. Counterfactual generating algorithms Yet more algorithms to explain complex algorithms Ill-defined, measures: sparse/short, near, diverse, actionable, feasible, plausible, justified, fast, model- agnostic, data-agnostic, truthful, complete, stable, robust, …
- 11. Karimi et al (2021) A survey of algorithmic recourse: contrastive explanations and consequential recommendations https://arxiv.org/pdf/2010.04050.pdf
- 12. Complex AI models are creeping into our lives Whose decisions are extremely difficult to explain But Explanations are needed if we want to trust AI Counterfactuals explain a decision of a model for an instance What these should look like is ill-defined and open research
- 13. The Counterfactual Explains Wrong Decisions To improve the predictive performance of the model Example o Data: image o Task: predict if missile in image Tom Vermeire, Dieter Brughmans, Sofie Goethals, Raphael Mazzine de Oliveira, David Martens (2022) Explainable Image Classication with Evidence Counterfactual, Pattern Anal Applic 25, 315–335
- 14. The Counterfactual Explains Wrong Decisions To improve the predictive performance of the model Example o Data: image o Task: predict if missile in image o Mainly interested in improving misclassifications o Issue: Lighthouse wrongly classified as missile o Pattern learnt: line of smoke indicates missile
- 15. Complex AI models are creeping into our lives Whose decisions are extremely difficult to explain But Explanations are needed if we want to trust AI Counterfactuals explain a decision of a model for an instance What these should look like is ill-defined and open research Very useful to explain (incorrect or unfair) decisions
- 16. Imagine a world with CF explanations Well, if your wife would also have had a 20+ year relationship with our bank, and would have been regarded as Premium customer at some point in time, she would also receive a 20x credit limit Well, if your wife’s relationship status would have been “husband” instead of “wife”, she would also receive a 20x credit limit We clearly messed up, we’re updating our models now. Ah, ok, thanks for the additional feedback! Glad you found this and react responsibly. It’s how big tech should be in the 21st century.
- 17. Q&A David Martens (2022) Data Science Ethics: Concepts, Techniques and Cautionary Tales Oxford University Press, 272 pages. Play around with CF expanations: admantwerp.github.io Book on Data Science Ethics: www.dsethics.com
Editor's Notes
- #3: I’ll be speaking of how the CF concept can be used to explain the decisions that are being made by very complex AI models. The irony is that in order to explain such complicated models, it seems we’re just adding more complex algorithms. In this presentaiton, I try to answer why we do this, and whether we actually should be doing this.