Robust Neural Machine Translation with Doubly Adversarial Inputs
0 likes•777 views
This document discusses using adversarial training methods to improve neural machine translation. Specifically, it explores training a Transformer model with both original inputs and their "doubly adversarial" corrupted versions to learn robust representations. Evaluation on two translation tasks showed this approach improved BLEU scores over the baseline Transformer model, demonstrating the effectiveness of adversarial training for more robust neural machine translation.
![•
– - 120 LDC
– - 450 WMT14
• Transformer [Vaswani+ 17]
• BPE
– - 6
– - 3 2
•
9](https://image.slidesharecdn.com/snlptranslation-190925031149/85/Robust-Neural-Machine-Translation-with-Doubly-Adversarial-Inputs-9-320.jpg)
![•
• BackTranskation [Sennrich+ 16]
11](https://image.slidesharecdn.com/snlptranslation-190925031149/85/Robust-Neural-Machine-Translation-with-Doubly-Adversarial-Inputs-11-320.jpg)
![• NMT
–
– Transformer [Vaswani+ 17]
–
•
– Teacher
forcing
14](https://image.slidesharecdn.com/snlptranslation-190925031149/85/Robust-Neural-Machine-Translation-with-Doubly-Adversarial-Inputs-14-320.jpg)
Robust Neural Machine Translation with Doubly Adversarial Inputs
- 1. Robust Neural Machine Translation with Doubly Adversarial Inputs Yong Cheng, Lu Jiang, Wolfgang Macherey ACL 2019 2019/9/28 1
- 2. • NMT 2 - he she
- 3. • • – 3 Der … geladenen … Der … vorgeladenen …<BOS> … summoned … The … continue … → <BOS> … invited … The … continue …
- 4. • xi • xi’ 4 xi gxi → xi
- 5. • xi • xi’ 5 xi gxi → xi
- 6. • • – • – • P(j) • M δ 6
- 9. • – - 120 LDC – - 450 WMT14 • Transformer [Vaswani+ 17] • BPE – - 6 – - 3 2 • 9
- 11. • • BackTranskation [Sennrich+ 16] 11
- 13. Ablation test • 13 → Teacher forcing
- 14. • NMT – – Transformer [Vaswani+ 17] – • – Teacher forcing 14