Ilya Sutskever at AI Frontiers : Progress towards the OpenAI mission
3 likes•2,107 views
OpenAI has made progress towards its mission of developing beneficial artificial general intelligence (AGI). This includes developing OpenAI Five, which reached the world's top level at the game Dota 2 through reinforcement learning techniques. OpenAI has also achieved dexterous in-hand manipulation with Dactyl and photorealistic image generation with generative adversarial networks. While challenges remain, deep learning has made rapid progress in areas like computer vision, translation and reinforcement learning. With continued growth in computing power, near-term AGI should be considered a serious possibility and risks should be proactively addressed.
Ilya Sutskever at AI Frontiers : Progress towards the OpenAI mission
- 1. Progress towards the OpenAI mission Ilya Sutskever Co-founder and Chief Scientist, OpenAI NOVEMBER 9, 2018
- 2. OpenAI’s mission OpenAI’s mission is to ensure that artificial general intelligence (AGI) — by which we mean highly autonomous systems that outperform humans at most economically valuable work — benefits all of humanity. — The OpenAI Charter
- 3. Technical progress from OpenAI
- 4. OpenAI Five
- 5. Dota
- 6. Dota is hard Partial observability 120 heroes (we integrated 18) 20,000 actions per game, massive action space Pros dedicate their lives to the game, 10K+ hrs of deliberate practice
- 7. Dota is popular Largest professional scene Annual prize pool of $40M+
- 8. Dota
- 9. Our approach Very large scale reinforcement learning millennia of practice LSTM policy = honeybee brain Self play Reward shaping
- 10. Reinforcement learning (RL) actually works! Nearly all RL experts believed that RL can’t solve tasks as hard as Dota Horizon too long
- 11. Results
- 12. Dactyl
- 13. Dexterity
- 14. Diverse objects
- 15. Strategy: Sim 2 Real
- 16. Domain randomization Train in simulation: randomize perception and physics
- 19. Core idea Novel states = reward Fix all bugs Very hard to do
- 23. Mario
- 24. Curiosity early in training
- 25. Curiosity late in training
- 27. OpenAI’s mission OpenAI’s mission is to ensure that artificial general intelligence (AGI) — by which we mean highly autonomous systems that outperform humans at most economically valuable work — benefits all of humanity. — The OpenAI Charter
- 28. Impact of AGI Generate massive wealth Potential to end poverty, achieve material abundance
- 29. Impact of AGI Generate massive wealth Potential to end poverty, achieve material abundance Generate science and technology cure disease, extend life, superhuman healthcare mitigate global warming, clean the oceans, etc massively improve education and psychological well being
- 30. Why is OpenAI’s mission relevant today? We review progress in the field over the past 6 years Our conclusion: near term AGI should be taken as a serious possibility
- 31. Algorithms
- 32. Deep Learning at the root of it all During the past 6 years, deep learning repeatedly and rapidly broke through “insurmountable” barriers A multilayered Perceptron trainable with backpropagation
- 33. Vision (2012-2016) HOG feature (2005)—image by Antonio Torrabla
- 34. Vision (2012-2016) ImageNet Classification Error (Top 5) Human 2015 (ResNet) 2016 (GoogleLeNet-v4) 0 5 10 15 20 25 30 3.13.6 5.0 2011 (XRCE) 2012 (AlexNet) 2013 (ZF) 2014 (VGG) 2014 (GoogleLeNet) 26.0 16.4 11.7 7.3 6.7
- 35. Translation (2014-2018) BLEU score on EN to FR translation on the WMT dataset
- 36. Image generation (2014-2018) GANs over the years: 2014 Goodfellow et al, 2014
- 37. Image generation (2014-2018) GANs over the years: 2015 Radford et al, 2015
- 38. Image generation (2014-2018) GANs over the years: 2017 Karras et al, 2017
- 39. Image generation (2014-2018) GANs over the years: 2018 Brock et al, 2018
- 40. Reinforcement Learning (2013-2018) DQN (2013) Mnih et al, 2013
- 41. Reinforcement Learning (2013-2018) TRPO (2015) Schulman et al, 2015
- 42. Reinforcement Learning (2013-2018) AlphaGo (2016) Silver et al, 2016
- 43. Reinforcement Learning (2013-2018) OpenAI Five (2018) Very large scale: +100,000 CPU cores, +1000 GPUs
- 44. Reinforcement Learning (2013-2018) Dactyl (2018)
- 45. Compute
- 46. Compute grows rapidly Neural networks can usefully consume all available compute Neural networks are extremely parallelizable 300,000x increase in neural net compute used for the largest neural net experiments over the past 6 years
- 47. Petaflop/s-days to train Early production convnet
- 48. Conclusion
- 49. Formidable challenges remain Unsupervised learning Robust classification Reasoning Abstraction ???
- 50. We’ve been breaking through barriers for 6 years Will this trend continue, or will it stop? And if so, when?
- 51. Means proactively thinking about risks: Machines pursuing goals misspecified by their operator Malicious humans subverting deployed systems Out-of-control economy that grows without resulting in improvements to human lives THIS TALK’S GOAL IS TO PRESENT EVIDENCE THAT: While highly uncertain, near-term AGI should be taken as a serious possibility.
- 52. Thank you