Deep Learning at Scale
1 like1,105 views
Nervana's deep learning platform provides unprecedented computing power through specialized hardware. It includes a fast deep learning framework called Neon that is 10 times faster than other frameworks on GPUs. Neon also includes pre-trained models and is under active development to improve capabilities like distributed computing and integration with other frameworks. Nervana aims to make deep learning more accessible and applicable across industries like healthcare, automotive, finance, and more.
Deep Learning at Scale
- 1. Proprietary and confidential. Do not distribute. Nervana’s Deep Learning Platform MAKING MACHINES SMARTER.™ Hanlin Tang, PhD Algorithms Engineer
- 2. Facebook DeepMask Silver et al, 2016 The Atlantic, March 2016 “The error rate has been cut by a factor of two in all the languages, more than a factor of two in many cases. That’s mostly due to deep learning and the way we have optimized it …” Alex Acero, Siri Senior Director, Apple Article in Backhannel/WIRED, Aug 2016 Deep Learning
- 3. neon deep learning framework train deployexplore nervana engine Fastest deep learning framework cloudn
- 4. • Unprecedented computing power • 10x speedup over current Maxwell GPUs (~55 TeraOps) • 32 GB High-Bandwidth Memory • Six bi-directional high-bandwidth links for 3D torus interconnect • 8 chips in a box, seamlessly scale to multiple chassis
- 6. • https://github.com/NervanaSystems/ModelZoo • Pre-trained weights and models SegNet Deep Speech 2 Skip-thought Autoencoders Deep Dream
- 7. Badrinarayanan et al., 2015
- 8. Neon (ms) Caffe (ms) Speed-up Forward 101 719 7.1x Backward 164 746 4.5x Total 265 1455 5.5x
- 9. neon v1.6 + mgpu v1.6 neon v2.0 Modular dataloader (aeon) Neural machine translation model neon v3.0 • Nervana Graph • Tensorflow inter-operability • Graph-enabled models • Distributed computing
- 11. “Training neural networks is a dark art.” Hyperparameters: •Number and type of units/layers •Convolution filter size •Weight Initialization •Optimization method •Learning Rate schedule
- 13. Command Line client Web Interface
- 14. Nervana in action Healthcare: Tumor detection Automotive: Speech interfaces Finance: Time-series search engine Positive: Negative: Agricultural Robotics Oil & Gas Positive: Negative: Proteomics: Sequence analysis Query: Results:
- 15. + n