Deep Learning with Apache MXNet
1 like114 views
This document provides an overview of Apache MXNet, an open-source library for deep learning. It discusses MXNet's capabilities such as high performance scaling across GPUs, support for mobile and IoT models, and multiple language syntax. It also demonstrates MXNet through Jupyter notebooks on MNIST data and introduces Gluon, a high-level API for MXNet. Resources for learning more about MXNet, deep learning on AWS, and the presenter's blog are provided.
Deep Learning with Apache MXNet
- 1. Deep Learning with Apache MXNet Julien Simon, AI Evangelist, EMEA @julsimon
- 2. What to expect • Apache MXNet • Demos using Jupyter notebooks • Resources • Q&A
- 3. Apache MXNet: Open Source library for Deep Learning Programmable Portable High Performance Near linear scaling across hundreds of GPUs Highly efficient models for mobile and IoT Simple syntax, multiple languages Most Open Best On AWS Optimized for Deep Learning on AWS Accepted into the Apache Incubator
- 4. CPU or GPU: your choice mod = mx.mod.Module(lenet) mod = mx.mod.Module(lenet, context=mx.gpu(0)) mod = mx.mod.Module(lenet, context=(mx.gpu(7), mx.gpu(8), mx.gpu(9)))
- 5. Ideal Inception v3 Resnet Alexnet 88% Efficiency 0 64 128 192 256 1 2 4 8 16 32 64 128 256 Multi-GPU Scaling With MXNet
- 6. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. • Storing and accessing data in multi-dimensional arrays àNDArray API • Building models (layers, weights, activation functions) à Symbol API • Serving data during training and validation à Iterators • Training and using models à Module API The Apache MXNet API
- 7. Input Output 1 1 1 1 0 1 0 0 0 3 mx.sym.Convolution(data, kernel=(5,5), num_filter=20) mx.sym.Pooling(data, pool_type="max", kernel=(2,2), stride=(2,2) lstm.lstm_unroll(num_lstm_layer, seq_len, len, num_hidden, num_embed) 4 2 2 0 4=Max 1 3 ... 4 0.2 -0.1 ... 0.7 mx.sym.FullyConnected(data, num_hidden=128) 2 mx.symbol.Embedding(data, input_dim, output_dim = k) 0.2 -0.1 ... 0.7 Queen 4 2 2 0 2=Avg Input Weights cos(w, queen) = cos(w, king) - cos(w, man) + cos(w, woman) mx.sym.Activation(data, act_type="xxxx") "relu" "tanh" "sigmoid" "softrelu" Neural Art Face Search Image Segmentation Image Caption “People Riding Bikes” Bicycle, People, Road, Sport Image Labels Image Video Speech Text “People Riding Bikes” Machine Translation “Οι άνθρωποι ιππασίας ποδήλατα” Events mx.model.FeedForward model.fit mx.sym.SoftmaxOutput
- 8. Gluon: Deep Learning gets even easier https://github.com/gluon-api/ • Available now in MXNet, soon in Microsoft Cognitive Toolkit • Developer-friendly high-level API • Dynamic networks can be modified during training • No compromise on performance • Extensive model zoo https://aws.amazon.com/blogs/aws/introducing-gluon-a-new-library-for-machine-learning-from-aws-and-microsoft/
- 10. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Deep Learning in practice AWS Deep Learning AMI MXNet Torch CTKKeras TheanoCaffe TensorFlow Amazon EC2 AnacondaIntel MKL CUDA+cuDNN Python 2+3 Caffe2 • One-click launch • Single node or distributed • CPU, GPU, FPGA • NVIDIA & Intel libraries • Anaconda Data Science Platform • Python w/ AI/ML/DL libraries
- 11. Demos https://github.com/juliensimon/dlnotebooks 1) Synthetic data set 2) Learn MNIST with a Multi-Layer Perceptron 3) Learn MNIST with the LeNet CNN 4) Predict handmade MNIST samples
- 12. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. The MNIST data set • 70,000 hand-written digits • 28x28 grayscale images
- 13. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. • MLP does well on MNIST, but not on real-life samples. • MLP flattens the 28x28 image into a 784-byte vector à the 2-D relationship between pixels is lost. • Look at the ‘5’: the top-left angle is a unique feature. No other digit exhibits this. Same thing for the intersection on the ‘8’. • CNNs such as LeNet work on 2-D images and learn to detect these unique geometric features. • The result is a more accurate and more robust network. Training models on MNIST
- 14. © 2017, Amazon Web Services, Inc. or its Affiliates. All rights reserved. Resources https://aws.amazon.com/machine-learning https://aws.amazon.com/blogs/ai https://mxnet.incubator.apache.org https://github.com/apache/incubator-mxnet https://github.com/gluon-api https://medium.com/@julsimon
- 15. Thank you! Julien Simon, AI Evangelist, EMEA @julsimon