Copyright © 2020 ML6. All rights reserved. ML6 Confidential Information | 1Copyright © 2019 ML6. All rights reserved. ML6 Confidential Information | 1
Copyright © 2020 ML6. All rights reserved. ML6 Confidential Information | 2
Matthias Feys
Copyright © 2020 ML6. All rights reserved. ML6 Confidential Information | 3
Copyright © 2020 ML6. All rights reserved. ML6 Confidential Information | 4
■Unawareness
■Group fairness
■Individual fairness
→ No “best” definition of fairness
Copyright © 2020 ML6. All rights reserved. ML6 Confidential Information | 5
Don’t include sensitive variable as a feature in the
training data
+ simple to implement
+ doesn’t require knowledge of the sensitive
variable
- proxies of the sensitive variable are not
excluded
Image source: https://research.google.com/bigpicture/attacking-discrimination-in-ml/
Copyright © 2020 ML6. All rights reserved. ML6 Confidential Information | 6
Sensitive variable divides individuals into two groups,
each group should be treated “equal”
+ excludes potential proxy features
- definition of “equal” is contradictory
- prone to positive discrimination (lazy
implementation)
Image source: https://research.google.com/bigpicture/attacking-discrimination-in-ml/
Copyright © 2020 ML6. All rights reserved. ML6 Confidential Information | 7
Similar individuals should be treated similarly,
implemented by analysing pairs of individuals
+ most fine-grained fairness
- ambiguous definition of similarity
Copyright © 2020 ML6. All rights reserved. ML6 Confidential Information | 8
■Optimizing for any definition improves the status quo
■Impossible to optimize for all, possible to calculate all
→ multiple tools exist to help you with this (Google What-if, IBM AI Fairness 360, ...)
■Monitoring fairness is the first step
Copyright © 2020 ML6. All rights reserved. ML6 Confidential Information | 9
■ sample bias
■ prejudicial bias
■ exclusion bias
■ measurement bias
■ algorithmic bias
Copyright © 2020 ML6. All rights reserved. ML6 Confidential Information | 10
■ sample bias
■ prejudicial bias
■ exclusion bias
■ measurement bias
■ algorithmic bias
introduced by the
dataset
introduced by the
model
→
Copyright © 2020 ML6. All rights reserved. ML6 Confidential Information | 11
■ Reducing bias in the data (pre-processing)
■ Reducing bias in the classifier (training)
■ Reducing bias in the predictions (post-processing)
training data model training inference
Copyright © 2020 ML6. All rights reserved. ML6 Confidential Information | 12
■ Sampling or re-weighting the data
■ Modify features or labels (obfuscate
information about protected attributes) training data model training inference
Copyright © 2020 ML6. All rights reserved. ML6 Confidential Information | 13
■ Add a discrimination-aware regularization
term to the learning objective
■Adversarial debiasing: maximize accuracy
while reducing evidence of sensitive
features in the predictions
training data model training inference
Copyright © 2020 ML6. All rights reserved. ML6 Confidential Information | 14
■ Changes predictions from a classifier to
make them more fair
■ Works with black box models and unknown
datasets
training data model training inference
Copyright © 2020 ML6. All rights reserved. ML6 Confidential Information | 15
■3 main types of solutions
■Trade-off between accuracy and fairness
■Typically need sensitive features to achieve fairness
Copyright © 2020 ML6. All rights reserved. ML6 Confidential Information | 16
■ Fairness/bias is on the map
‐ New papers
‐ New tools
‐ Work on legal context
■ “If you can measure it, you can improve it”
vs.
“You get what you measure”
Image source: https://towardsdatascience.com/a-tutorial-on-fairness-in-machine-learning-3ff8ba1040cb

More Related Content

PDF
Matthias Feys (ML6) – Bias in ML: A Technical Intro
PPTX
Responsible AI in Industry (ICML 2021 Tutorial)
PDF
Explainability and bias in AI
PPTX
Explainable AI in Industry (AAAI 2020 Tutorial)
PDF
Towards Human-Centered Machine Learning
PPTX
Interpretable ML
PPTX
Fairness-aware Machine Learning: Practical Challenges and Lessons Learned (WW...
PPTX
Combatting Bias in Machine Learning
Matthias Feys (ML6) – Bias in ML: A Technical Intro
Responsible AI in Industry (ICML 2021 Tutorial)
Explainability and bias in AI
Explainable AI in Industry (AAAI 2020 Tutorial)
Towards Human-Centered Machine Learning
Interpretable ML
Fairness-aware Machine Learning: Practical Challenges and Lessons Learned (WW...
Combatting Bias in Machine Learning

What's hot (20)

PPTX
Fairness-aware Machine Learning: Practical Challenges and Lessons Learned (WS...
PPTX
Fairness in AI (DDSW 2019)
PDF
Fairness in Machine Learning and AI
PDF
Interpretability of machine learning
PDF
Ethics in the use of Data & AI
PPTX
Fairness and Privacy in AI/ML Systems
PDF
Algorithmic Bias - What is it? Why should we care? What can we do about it?
PDF
Fairness-aware Machine Learning: Practical Challenges and Lessons Learned (KD...
PDF
Fairness and Bias in Machine Learning
PDF
AIF360 - Trusted and Fair AI
PDF
Citython presentation
PPTX
Explainable AI in Industry (WWW 2020 Tutorial)
PDF
How do we train AI to be Ethical and Unbiased?
PPTX
A Tutorial to AI Ethics - Fairness, Bias & Perception
PPTX
An Introduction to XAI! Towards Trusting Your ML Models!
PDF
Algorithmic Bias: Challenges and Opportunities for AI in Healthcare
PDF
DC02. Interpretation of predictions
PPTX
Explainable AI in Industry (FAT* 2020 Tutorial)
PPTX
Explainable AI in Industry (KDD 2019 Tutorial)
PPTX
Explainable AI
Fairness-aware Machine Learning: Practical Challenges and Lessons Learned (WS...
Fairness in AI (DDSW 2019)
Fairness in Machine Learning and AI
Interpretability of machine learning
Ethics in the use of Data & AI
Fairness and Privacy in AI/ML Systems
Algorithmic Bias - What is it? Why should we care? What can we do about it?
Fairness-aware Machine Learning: Practical Challenges and Lessons Learned (KD...
Fairness and Bias in Machine Learning
AIF360 - Trusted and Fair AI
Citython presentation
Explainable AI in Industry (WWW 2020 Tutorial)
How do we train AI to be Ethical and Unbiased?
A Tutorial to AI Ethics - Fairness, Bias & Perception
An Introduction to XAI! Towards Trusting Your ML Models!
Algorithmic Bias: Challenges and Opportunities for AI in Healthcare
DC02. Interpretation of predictions
Explainable AI in Industry (FAT* 2020 Tutorial)
Explainable AI in Industry (KDD 2019 Tutorial)
Explainable AI
Ad

Similar to Introduction to Bias in Machine Learning, presented by Matthias Feys, CTO @ ML6 at the Trustworthy & Ethical AI conference on Feb. 13th (20)

PDF
02 simon logghe_ml6
PPTX
Data Done Right: Ensuring Information Integrity
PDF
M Information Systems 2nd Edition Baltzan Test Bank
PDF
Data-Ed Online Presents: Data Warehouse Strategies
PDF
Data-Ed: Data Warehousing Strategies
PPTX
Management Information Systems Business Driven MIS
PDF
Analytics, Big Data and The Cloud II Conference - Kiribatu Labs
PDF
Data-Ed Webinar: Data Warehouse Strategies
PDF
M Information Systems 2nd Edition Baltzan Test Bank
PDF
M Information Systems 2nd Edition Baltzan Test Bank
PDF
M Information Systems 2nd Edition Baltzan Test Bank
PDF
Get full Test Bank for MIS 9th Edition Bidgoli free all chapters
PPTX
Analyst Alert: Data Modeling is a Required Skill
PPTX
DevTalks.ro 2019 MySQL Data Masking Talk
PDF
M Information Systems 2nd Edition Baltzan Test Bank
PPTX
Mis ch04
PDF
M Information Systems 2nd Edition Baltzan Test Bank
PPTX
Teaching organizations to fish in a data-rich future: Stories from data leaders
PPTX
Master Data Management
PPTX
WIT Career Lecture Series - CTeixeira Data Scientist
02 simon logghe_ml6
Data Done Right: Ensuring Information Integrity
M Information Systems 2nd Edition Baltzan Test Bank
Data-Ed Online Presents: Data Warehouse Strategies
Data-Ed: Data Warehousing Strategies
Management Information Systems Business Driven MIS
Analytics, Big Data and The Cloud II Conference - Kiribatu Labs
Data-Ed Webinar: Data Warehouse Strategies
M Information Systems 2nd Edition Baltzan Test Bank
M Information Systems 2nd Edition Baltzan Test Bank
M Information Systems 2nd Edition Baltzan Test Bank
Get full Test Bank for MIS 9th Edition Bidgoli free all chapters
Analyst Alert: Data Modeling is a Required Skill
DevTalks.ro 2019 MySQL Data Masking Talk
M Information Systems 2nd Edition Baltzan Test Bank
Mis ch04
M Information Systems 2nd Edition Baltzan Test Bank
Teaching organizations to fish in a data-rich future: Stories from data leaders
Master Data Management
WIT Career Lecture Series - CTeixeira Data Scientist
Ad

More from Patrick Van Renterghem (20)

PDF
Ethical AI at VDAB, presented by Vincent Buekenhout (Ethical AI Lead, VDAB) a...
PDF
Implementing error-proof, business-critical Machine Learning, presentation by...
PDF
Building Trust and Explainability into Chatbots: the Partena Ziekenfonds Busi...
PDF
AI & Ethics: The Belgian Industry Vision & Initiatives, presentation by Jelle...
PDF
Responsible AI: An Example AI Development Process with Focus on Risks and Con...
PDF
Fairness and Transparency: Algorithmic Explainability, some Legal and Ethical...
PPTX
How obedient digital twins and intelligent beings contribute to ethics and ex...
PDF
He Said, She Said: Finding and Fixing Bias in NLP (Natural Language Processin...
PDF
Business Case: Ozitem Groupe, where 80% of the company is working remotely. R...
PDF
Digital Workplace Case Study: How the Municipality of Duffel successfully swi...
PDF
Unleashing the Full Potential of People, Teams and SOLVAY, presented by Bruce...
PDF
The Building Blocks of a Digital Workplace, presented by Sam Marshall at the ...
PDF
Engie's Digital Workplace and "Connecting the company" business case, present...
PDF
Face your communication challenges when implementing a digital workplace, bas...
PDF
The first steps in Recticel's Digital Workplace program by Kenneth Meuleman (...
PDF
Presentation by Dave Geentjens at the "Successful Digital Workplace Adoption"...
PDF
Tim scottkoenverheyenpresentation
PDF
Presentation by Ivan Schotsmans (DV Community) at the Data Vault Modelling an...
PDF
Presentation by Luc Delanglez (DataLumen) at the Data Vault Modelling and Dat...
PDF
Presentation by Erik van der Hoeven (Wisdom as a Service) at the Data Vault M...
Ethical AI at VDAB, presented by Vincent Buekenhout (Ethical AI Lead, VDAB) a...
Implementing error-proof, business-critical Machine Learning, presentation by...
Building Trust and Explainability into Chatbots: the Partena Ziekenfonds Busi...
AI & Ethics: The Belgian Industry Vision & Initiatives, presentation by Jelle...
Responsible AI: An Example AI Development Process with Focus on Risks and Con...
Fairness and Transparency: Algorithmic Explainability, some Legal and Ethical...
How obedient digital twins and intelligent beings contribute to ethics and ex...
He Said, She Said: Finding and Fixing Bias in NLP (Natural Language Processin...
Business Case: Ozitem Groupe, where 80% of the company is working remotely. R...
Digital Workplace Case Study: How the Municipality of Duffel successfully swi...
Unleashing the Full Potential of People, Teams and SOLVAY, presented by Bruce...
The Building Blocks of a Digital Workplace, presented by Sam Marshall at the ...
Engie's Digital Workplace and "Connecting the company" business case, present...
Face your communication challenges when implementing a digital workplace, bas...
The first steps in Recticel's Digital Workplace program by Kenneth Meuleman (...
Presentation by Dave Geentjens at the "Successful Digital Workplace Adoption"...
Tim scottkoenverheyenpresentation
Presentation by Ivan Schotsmans (DV Community) at the Data Vault Modelling an...
Presentation by Luc Delanglez (DataLumen) at the Data Vault Modelling and Dat...
Presentation by Erik van der Hoeven (Wisdom as a Service) at the Data Vault M...

Recently uploaded (20)

PPTX
Configure Apache Mutual Authentication
PDF
NewMind AI Weekly Chronicles – August ’25 Week IV
PPTX
AI IN MARKETING- PRESENTED BY ANWAR KABIR 1st June 2025.pptx
PDF
Convolutional neural network based encoder-decoder for efficient real-time ob...
PDF
Lung cancer patients survival prediction using outlier detection and optimize...
PDF
Consumable AI The What, Why & How for Small Teams.pdf
PDF
Advancing precision in air quality forecasting through machine learning integ...
PDF
Flame analysis and combustion estimation using large language and vision assi...
PDF
Transform-Your-Supply-Chain-with-AI-Driven-Quality-Engineering.pdf
PPTX
Build Your First AI Agent with UiPath.pptx
PDF
Transform-Your-Streaming-Platform-with-AI-Driven-Quality-Engineering.pdf
PDF
Auditboard EB SOX Playbook 2023 edition.
PDF
INTERSPEECH 2025 「Recent Advances and Future Directions in Voice Conversion」
PDF
Co-training pseudo-labeling for text classification with support vector machi...
PDF
Taming the Chaos: How to Turn Unstructured Data into Decisions
PPTX
Internet of Everything -Basic concepts details
PDF
AI.gov: A Trojan Horse in the Age of Artificial Intelligence
PDF
Transform-Quality-Engineering-with-AI-A-60-Day-Blueprint-for-Digital-Success.pdf
PDF
Accessing-Finance-in-Jordan-MENA 2024 2025.pdf
PDF
The-2025-Engineering-Revolution-AI-Quality-and-DevOps-Convergence.pdf
Configure Apache Mutual Authentication
NewMind AI Weekly Chronicles – August ’25 Week IV
AI IN MARKETING- PRESENTED BY ANWAR KABIR 1st June 2025.pptx
Convolutional neural network based encoder-decoder for efficient real-time ob...
Lung cancer patients survival prediction using outlier detection and optimize...
Consumable AI The What, Why & How for Small Teams.pdf
Advancing precision in air quality forecasting through machine learning integ...
Flame analysis and combustion estimation using large language and vision assi...
Transform-Your-Supply-Chain-with-AI-Driven-Quality-Engineering.pdf
Build Your First AI Agent with UiPath.pptx
Transform-Your-Streaming-Platform-with-AI-Driven-Quality-Engineering.pdf
Auditboard EB SOX Playbook 2023 edition.
INTERSPEECH 2025 「Recent Advances and Future Directions in Voice Conversion」
Co-training pseudo-labeling for text classification with support vector machi...
Taming the Chaos: How to Turn Unstructured Data into Decisions
Internet of Everything -Basic concepts details
AI.gov: A Trojan Horse in the Age of Artificial Intelligence
Transform-Quality-Engineering-with-AI-A-60-Day-Blueprint-for-Digital-Success.pdf
Accessing-Finance-in-Jordan-MENA 2024 2025.pdf
The-2025-Engineering-Revolution-AI-Quality-and-DevOps-Convergence.pdf

Introduction to Bias in Machine Learning, presented by Matthias Feys, CTO @ ML6 at the Trustworthy & Ethical AI conference on Feb. 13th

  • 1. Copyright © 2020 ML6. All rights reserved. ML6 Confidential Information | 1Copyright © 2019 ML6. All rights reserved. ML6 Confidential Information | 1
  • 2. Copyright © 2020 ML6. All rights reserved. ML6 Confidential Information | 2 Matthias Feys
  • 3. Copyright © 2020 ML6. All rights reserved. ML6 Confidential Information | 3
  • 4. Copyright © 2020 ML6. All rights reserved. ML6 Confidential Information | 4 ■Unawareness ■Group fairness ■Individual fairness → No “best” definition of fairness
  • 5. Copyright © 2020 ML6. All rights reserved. ML6 Confidential Information | 5 Don’t include sensitive variable as a feature in the training data + simple to implement + doesn’t require knowledge of the sensitive variable - proxies of the sensitive variable are not excluded Image source: https://research.google.com/bigpicture/attacking-discrimination-in-ml/
  • 6. Copyright © 2020 ML6. All rights reserved. ML6 Confidential Information | 6 Sensitive variable divides individuals into two groups, each group should be treated “equal” + excludes potential proxy features - definition of “equal” is contradictory - prone to positive discrimination (lazy implementation) Image source: https://research.google.com/bigpicture/attacking-discrimination-in-ml/
  • 7. Copyright © 2020 ML6. All rights reserved. ML6 Confidential Information | 7 Similar individuals should be treated similarly, implemented by analysing pairs of individuals + most fine-grained fairness - ambiguous definition of similarity
  • 8. Copyright © 2020 ML6. All rights reserved. ML6 Confidential Information | 8 ■Optimizing for any definition improves the status quo ■Impossible to optimize for all, possible to calculate all → multiple tools exist to help you with this (Google What-if, IBM AI Fairness 360, ...) ■Monitoring fairness is the first step
  • 9. Copyright © 2020 ML6. All rights reserved. ML6 Confidential Information | 9 ■ sample bias ■ prejudicial bias ■ exclusion bias ■ measurement bias ■ algorithmic bias
  • 10. Copyright © 2020 ML6. All rights reserved. ML6 Confidential Information | 10 ■ sample bias ■ prejudicial bias ■ exclusion bias ■ measurement bias ■ algorithmic bias introduced by the dataset introduced by the model →
  • 11. Copyright © 2020 ML6. All rights reserved. ML6 Confidential Information | 11 ■ Reducing bias in the data (pre-processing) ■ Reducing bias in the classifier (training) ■ Reducing bias in the predictions (post-processing) training data model training inference
  • 12. Copyright © 2020 ML6. All rights reserved. ML6 Confidential Information | 12 ■ Sampling or re-weighting the data ■ Modify features or labels (obfuscate information about protected attributes) training data model training inference
  • 13. Copyright © 2020 ML6. All rights reserved. ML6 Confidential Information | 13 ■ Add a discrimination-aware regularization term to the learning objective ■Adversarial debiasing: maximize accuracy while reducing evidence of sensitive features in the predictions training data model training inference
  • 14. Copyright © 2020 ML6. All rights reserved. ML6 Confidential Information | 14 ■ Changes predictions from a classifier to make them more fair ■ Works with black box models and unknown datasets training data model training inference
  • 15. Copyright © 2020 ML6. All rights reserved. ML6 Confidential Information | 15 ■3 main types of solutions ■Trade-off between accuracy and fairness ■Typically need sensitive features to achieve fairness
  • 16. Copyright © 2020 ML6. All rights reserved. ML6 Confidential Information | 16 ■ Fairness/bias is on the map ‐ New papers ‐ New tools ‐ Work on legal context ■ “If you can measure it, you can improve it” vs. “You get what you measure” Image source: https://towardsdatascience.com/a-tutorial-on-fairness-in-machine-learning-3ff8ba1040cb