He Said, She Said: Finding and Fixing Bias in NLP (Natural Language Processing, presented by Yves Peirsman, CTO at NLP Town
0 likes•212 views
The document discusses the identification and mitigation of bias in natural language processing (NLP), highlighting the significance of training data and model integration. It outlines steps for recognizing bias using explainable AI and emphasizes the importance of ensuring that training data is free from bias while employing techniques such as adversarial training and adjusting embeddings. Ultimately, the document underscores the responsibility of AI developers to minimize harmful side effects in deployed systems.
He Said, She Said: Finding and Fixing Bias in NLP (Natural Language Processing, presented by Yves Peirsman, CTO at NLP Town
- 1. Finding and Fixing Bias in Natural Language Processing Yves Peirsman
- 2. Artificial Intelligence Natural Language Processing A primer in NLP Machine translation Sentiment analysis Information retrieval Information extraction Text classification
- 3. We provide consultancy for companies that need guidance in the NLP domain We develop software and train custom NLP models for challenging or domain-specific applications.
- 4. Training data Training process Model We integrate models with workflows. NLP Town We help annotate training data. We train models for NLP applications. We provide consultancy for NLP projects.
- 5. Bias in Natural Language Processing
- 6. Bias in Natural Language Processing
- 7. A primer in NLP Training data Training process Model
- 8. A primer in NLP
- 9. Word Embeddings Word embeddings allow NLP models to generalize better.
- 10. Word Embeddings Word embeddings capture both general and linguistic knowledge.
- 11. Word Embeddings Word embeddings also encode bias: ● Man is to king as woman is to ___. ● Man is to programmer as woman is to ___. Experiment: ● Measure the similarity between occupations and ○ A set of “male” words: man, son, father, he, him, etc. ○ A set of “female” words: woman, daughter, mother, she, her, etc.
- 12. Word Embeddings
- 13. Pretrained NLP models Pretrained language models are a recent significant breakthrough in NLP: ● Language models predict masked words. ● They learn a lot about language. ● This knowledge can be reused in “downstream” tasks. This movie won her an Oscar for best actress. The keys to the house are on the table.
- 14. Pretrained NLP models ULMFit, Howard and Ruder 2018
- 15. Pretrained language models Experiment: association with a large number of positive adjectives ● One of the several recent Dutch Bert models ● Association between 240 positive adjectives and hij/zij: ○ aantrekkelijk, ambitieus, intelligent, slim, knap, nauwkeurig, nieuwsgierig, etc.
- 16. The problem with bias or
- 17. Step 1: Identify bias with explainable AI Challenge ● First we need to find out our models are biased: search for known, but also unexpected bias ● An important role for explainable AI Experiment ● A simple classifier for toxic comments ● Example: "Stupid peace of shit stop deleting my stuff asshole go die and fall in a hole go to hell!"
- 18. Step 1: Identify bias with explainable AI ● Visualize the classifier features and their weights:
- 19. Step 1: Identify bias with explainable AI
- 20. Step 1: Identify bias with explainable AI
- 21. Step 2: Fixing and avoiding bias Training data Training process Model
- 22. Training data Training process Model Ensure the training data is free of bias. Step 2: Fixing and avoiding bias
- 23. Bias in annotation Inform annotators about possible confounding factors, such as dialect. ● Example: if people are informed that a tweet contains African American English dialect, they are less likely to label it as offensive (Sap et al. 2019) Bias in text ● If you create a new corpus, ensure your texts contain as little bias as possible. ● If you use existing data, try mitigating biases through data augmentation, over- and/or undersampling, etc. Step 2: Fixing and avoiding bias
- 24. Training data Training process Model Pick a training procedure that makes the system blind to bias. Step 2: Fixing and avoiding bias
- 25. Adversarial training Train your model to shine at your task, but to fail at predicting “protected variables”, such as gender or race. ModelCV Step 2: Fixing and avoiding bias
- 26. Training data Training process Model Change the weights of the model so that the bias is reduced. Step 2: Fixing and avoiding bias
- 27. Word embeddings Transform the embeddings so that bias is removed. Pre-trained models Fine-tune on non-biased data, so that the models “forget” their bias. Step 2: Fixing and avoiding bias
- 28. None of these methods are foolproof: ● You need to be aware of the bias before you can remove it ● Often only “superficial” bias is removed, but deeper bias remains (Honen and Goldberg 2019) As AI developers, it is our responsibility to deploy our system in such a way that potentially harmful side effects are minimized. ● Effective feedback loops ● Human-in-the-loop AI Step 2: Fixing and avoiding bias