Fairness and Bias in Machine Learning

Fairness and Bias
in Machine Learning
Surya Dutta

Today, we’ll discuss:
• What does bias in machine learning look like?
• How does algorithmic bias get introduced & amplify?

Today, we’ll discuss:
• What does bias in machine learning look like?
• How does algorithmic bias get introduced & amplify?
• How can we quantify bias and fairness?
• How can we mitigate algorithmic bias?

Ferguson, Missouri 2013
Source: Sentencing Project
Source: The Verge
Source: The Next Web

Image Source: Consumer Reports
more likely to be
moderately injured
more likely to be
seriously injured
more likely
to die
71%
47%
17%
Source: Invisible Women

Predicting Recidivism
Source: “Machine Bias” by ProPublica, 2016

“In effect, Amazon’s system taught itself that male
candidates were preferable. It penalized resumes
that included the word “women’s,” as in “women’s
chess club captain.” And it downgraded graduates
of two all-women’s colleges, according to people
familiar with the matter.”
Source: Reuters 2018

Source: Automating Inequality
The screen that displays the AFST risk
score states clearly that the system
“is not intended to make investigative
or other child welfare decisions.”

Why can’t we just omit any protected
attributes from the dataset?

Latent Variables
& Proxies

Simpson’s Paradox
Latent Variables
& Proxies

Can we directly see if the inner workings
of our algorithm are biased?

Model Explainability
Source: h2o.ai
Complexity

Surrogate Models
Simpler models trained on same
inputs and predicted outputs of more
complex machine learning models
Source: h2o.ai

Surrogate Models
Simpler models trained on same
inputs and predicted outputs of more
complex machine learning models
Source: h2o.ai
Often not good enough!

What does a “perfectly unbiased”
algorithm look like?

1. An algorithm that always predicts correctly

2. An algorithm that picks predictions randomly

2. An algorithm that picks predictions randomly
3. An algorithm that makes “mistakes” equally across
privileged and unprivileged data

Can we quantitatively define fairness?

Goal: Create a metric that machine learning algorithm
can use to generate fair outcomes
Defining Fairness

Goal: Create a metric that machine learning algorithm
can use to generate fair outcomes
Definitions:
• Y is the true value (0 or 1 for binary classification)
• C is the algorithm’s predicted value
• A is the protected attribute (gender, race, etc.)
• A=1 refers to the unprivileged group, A=0 refers to privileged
Defining Fairness

Defining Fairness:
Demographic Parity
“A predictor satisfies demographic parity if the likelihood
of a positive outcome is the same, regardless of whether
the person is in the protected group or not”

Defining Fairness:
Demographic Parity
Pros: Proportional representation of groups

Defining Fairness:
Demographic Parity
Cons: Accuracy may be less in disadvantaged group

Defining Fairness:
Demographic Parity
Cons: Accuracy may be less in disadvantaged group
Greatly reduces effectiveness of predictor if true
labels have any correlation with protected attribute

Defining Fairness:
Equal Odds
“A predictor C satisfies equalized odds
with respect to a protected attribute A
and the true outcome Y if C and A are
independent conditional on Y”

A=0
C=1
C=0
Y=1 Y=0
C=1
C=0
Y=1 Y=0
A=1
Defining Fairness:
Equal Odds
In a binary classification:
• C has equal true positive rates if Y=1 for
both A=0 and A=1

A=0
C=1
C=0
Y=1 Y=0
C=1
C=0
Y=1 Y=0
A=1
Defining Fairness:
Equal Odds
In a binary classification:
• C has equal true positive rates if Y=1 for
both A=0 and A=1
• C has equal false positive rates if Y=0 for
both A=0 and A=1

# Qualified? Hired? Classification
2 Yes Yes True Positive
3 Yes No False Negative
4 No Yes False Positive
5 No No True Negative
1 Yes Yes True Positive
1 Yes No False Negative
2 No Yes False Positive
3 No No True Negative
Defining Fairness:
Equal Odds

# Qualified? Hired? Classification In-Group Rate
2 Yes Yes True Positive 2/14
3 Yes No False Negative 3/14
4 No Yes False Positive 4/14
5 No No True Negative 5/14
1 Yes Yes True Positive 1/7
1 Yes No False Negative 1/7
2 No Yes False Positive 2/7
3 No No True Negative 3/7
Defining Fairness:
Equal Odds

A=0
C=1
C=0
Y=1 Y=0
C=1
C=0
Y=1 Y=0
A=1
Defining Fairness:
Equal Odds
“Why not just accuracy?” (TP + TN)

A=0
C=1
C=0
Y=1 Y=0
C=1
C=0
Y=1 Y=0
A=1
Defining Fairness:
Equal Odds
“Why not just accuracy?” (TP + TN)
Weakness: We can “trade” the false
positive rate of one group for the false
negative rate for another group
Ex. Hiring from two groups. We can
achieve accuracy parity by exchanging
qualified applicants from privileged
group for unqualified applicants from
unprivileged group

Defining Fairness:
Equal Opportunity
• Relaxed version of Equal Odds
• Equal true positive rates for Y=1 for
both A=0 and A=1
• Useful when only care about positive
outcome
A=0
C=1
C=0
Y=1 Y=0
C=1
C=0
Y=1 Y=0
A=1

How can we actively mitigate bias
and improve fairness?

Bias Mitigation Algorithms
Preprocessing Inprocessing Postprocessing
Source: IBM AIF360

Bias Mitigation Algorithms:
Preprocessing

Preprocessing
Source: Feldman et. al 2015
Disparate Impact Remover
Modify labels in the training dataset to ensure
that the probability of a positive outcome is
equivalent for both subgroups
Less strict - ratio of probabilities is
greater than cutoff (typically 0.8)

Preprocessing
Reweighing
Weigh each observation in the training dataset
by the expected probability of the observation
ignoring the protected attribute.
(for algorithms that do not support custom
weights, sampling may be used instead)
Disparate Impact Remover
Modify labels in the training dataset to ensure
that the probability of a positive outcome is
equivalent for both subgroups
Less strict - ratio of probabilities is
greater than cutoff (typically 0.8)
Source: Kamiran, Calders 2010
Source: Feldman et. al 2015

Inprocessing

Inprocessing
Prejudice Remover
Defines prejudice index PI that increases as
correlation between outcome C and
protected attribute A increases:
Use as regularization term in loss function
– error goes up as correlation between
outcome and protected attribute goes up
Source: Kamishima et. al 2012

Inprocessing
Prejudice Remover
Defines prejudice index PI that increases as
correlation between outcome C and
protected attribute A increases:
Use as regularization term in loss function
– error goes up as correlation between
outcome and protected attribute goes up
Adversarial Debiasing
When using a neural network to train model, set up
a second adversarial network to predict protected
attribute from the predictions of the first classifier.
Total loss minimizes class prediction performance
and maximizes attribute prediction performance
Source: Kamishima et. al 2012 Source: Zhang et. al 2018

Postprocessing

Postprocessing
Equal Odds
A model’s sensitivity and specificity can be
tuned to optimize for metric like accuracy,
precision, recall, or F1 score
We choose instead to tune the model
to satisfy equal odds / equal opportunity
Source: Kamishima et. al 2012

Postprocessing
Equal Odds
A model’s sensitivity and specificity can be
tuned to optimize for metric like accuracy,
precision, recall, or F1 score
We choose instead to tune the model
to satisfy equal odds / equal opportunity
Rejection Option
Based on the fact that most bias
occurs on or near the decision
boundary of the classifier
Flip favored classification to
unprivileged group near the decision
boundary until parity is reached
Source: Kamishima et. al 2012 Source: Kamiran et. al 2012

AIF360 Demo
Model Fairness AIF360 Demo Notebooks:
https://github.com/neptune-ai/model-fairness-in-practice

Image Source: Interaction Institute for Social Change

Image Source: towardsdatascience.com

Fairness and Bias in Machine Learning

In this document

More Related Content

What's hot

Similar to Fairness and Bias in Machine Learning

Recently uploaded

Fairness and Bias in Machine Learning