![GloVe [Pennington+ (2014)]
•
• PPMI X
• X 1 :1Xi
6
V
V
wi, uj ∈ ℝD
α, xmax](https://image.slidesharecdn.com/understandingtheoriginofbiasinwordembeddings-190715152257/85/Understanding-the-origin-of-bias-in-word-embeddings-6-320.jpg)
![: WEAT
• The Word Embedding Assosiation Test [Caliskan+ (2017)]
10
Bweat
𝒮, 𝒯, 𝒜, ℬ
Bweat ∼ ( ⃗𝒮cent − ⃗𝒯 cent) ⋅ ( ⃗𝒜cent − ⃗ℬ cent) ⃗𝒮cent 𝒮
𝒮, 𝒯 𝒜, ℬ](https://image.slidesharecdn.com/understandingtheoriginofbiasinwordembeddings-190715152257/85/Understanding-the-origin-of-bias-in-word-embeddings-10-320.jpg)
![• Table2
• Wiki-WEAT2
• NYT-WEAT1
• Supplemental Material
Bweat ∈ [−2.0,2.0]
15](https://image.slidesharecdn.com/understandingtheoriginofbiasinwordembeddings-190715152257/85/Understanding-the-origin-of-bias-in-word-embeddings-15-320.jpg)
![• WEAT1 (S,T)
• 10k
• [Bolukbasi+,2016]
• base 1.14 unperturbed
• T female
• S male
• inc/dec
Fig.5 T
(female ) S
( )
Fig.5 base gender axis
( )
•
20](https://image.slidesharecdn.com/understandingtheoriginofbiasinwordembeddings-190715152257/85/Understanding-the-origin-of-bias-in-word-embeddings-20-320.jpg)
Understanding the origin of bias in word embeddings
- 1. Understanding the Origin of Bias in Word Embeddings
- 2. • • R&D Engineer @ JTC • NLP, ML 2
- 3. • Understanding the Origins of Bias in Word Embeddings • Marc-Etienne Brunet, Colleen Alkalay-Houlihan, Ashton Anderson, Richard Memel • University of Toronto, Vector Institute for Artificial Intelligence • ICML 2019 • • • • • 3
- 4. • • • • • • 4 ⃗wman − ⃗wwoman ≃ ⃗wcomputer programmer − ⃗whomemaker
- 6. GloVe [Pennington+ (2014)] • • PPMI X • X 1 :1Xi 6 V V wi, uj ∈ ℝD α, xmax
- 7. 7 • • Koh & Liang(2017) • • k • ( ) • 0 R, L • 1 θ* zk zk → ˜zk ˜θ ϵ ≪ 1 θ θ* ϵ zi ˜θ θ* ∇θ L, ∇2 θ L
- 8. GloVe ” ” • ” ” • PPMI GloVe ( ) • ( ˜X) ˜X ˜w ˜w 8V V A A V V B A Xi Xij ˜Xi ˜Xij
- 9. GloVe • GloVe • • i Xi ∇wi L, ∇2 wi L ˜wi 9 zi → Xi θ → wi
- 10. : WEAT • The Word Embedding Assosiation Test [Caliskan+ (2017)] 10 Bweat 𝒮, 𝒯, 𝒜, ℬ Bweat ∼ ( ⃗𝒮cent − ⃗𝒯 cent) ⋅ ( ⃗𝒜cent − ⃗ℬ cent) ⃗𝒮cent 𝒮 𝒮, 𝒯 𝒜, ℬ
- 11. • p • k PPMI PPMI B X • 1 • k X(k) X X = Σn k=1X(k) B(w( ˜X)) X 11 ( ∵ ˜X = X − X(k) )
- 12. 12
- 14. 14
- 15. • Table2 • Wiki-WEAT2 • NYT-WEAT1 • Supplemental Material Bweat ∈ [−2.0,2.0] 15
- 16. NYT-WEAT1 • • ( δ ( )) 16
- 17. • • {increase, decrease, random} 3 • increase(decrease) ( ) m increase(decrease)-m • random 17
- 18. • B • Ground Truth , • • :Ground Truth • B • B 0 r2 ≥ 0.985 18
- 19. ( PPMI) • word2vec, GloVe • PPMI V 2000 • GloVe 300 word2vec 10000 • 1.4M debias • Top1 Acc. 19
- 20. • WEAT1 (S,T) • 10k • [Bolukbasi+,2016] • base 1.14 unperturbed • T female • S male • inc/dec Fig.5 T (female ) S ( ) Fig.5 base gender axis ( ) • 20
- 21. • • NYT-WEAT1 "For Women in Astronomy" • • NYT-WEAT1 ”The Guide” • WEAT ( ) • 2 • • WEAT ( ) • • 0.07% (Fig.3) r2 = 0.828 21
- 22. • GloVe • • 22