Neural networks and google tensor flow
- 1. Neural Networks and Google TensorFlow SHANNON MCCORMICK
- 2. Outline Intro to Neural Networks Inspiration Basic explanation Practical applications Google TensorFlow What is it? Basic Usage Examples
- 3. Neural Network Inspiration Real Neurons Dendrites receive an input Based on input, axons output something to next neuron
- 4. Artificial Neural Networks Set of nodes connected by directional lines representing weights Nodes represent mathematical operations Weights learned by training
- 5. Linear Regression Example No hidden layers Inputs * Weights = Output Weights selected that minimize the error Great at modeling linear relationships
- 6. Adding Hidden Layers Add hidden layer(s) Input x Weights1 = Hidden Layer Hidden Layer * Weights2 = Output Weights selected to minimize error Can model more complex relationships
- 7. Learning weights Back Propagation Errors are back propagated through the model Determines the errors at each neuron in the network Gradient Descent Optimization method Determine how to change the weights Takes a step down gradient of the function Iterative process
- 9. Other Architectures Recurrent Neural Networks Output from first inputs fed back into network Used to predict output on future examples Text processing/prediction
- 10. Other Architectures Convolutional Neural Networks Image processing/classification
- 11. Practical Applications Pattern Recognition Image and text processing Time series prediction Stock market and weather forecasting Anomaly Detection Bank fraud Signal Detection Noise filtering
- 12. Google TensorFlow Open source machine learning library Released November 9, 2015
- 13. Features Can be used on desktop, mobile, servers Linus or Mac OS X GPU support on Linux Written in C++ with Python interface Excellent step by step tutorials and documentation Auto-differentiation Includes Tensorboard for graph visualization Active improvement and growth Google cloud (March 2016) Distributed computing support (April 2016)
- 14. Basics Data flow graph with nodes and edges Nodes: mathematical operations Edges: input/output relationship between nodes Edges carry tensors Tensors flow through the graph
- 15. Building a model “TensorFlow programs are usually structured into a construction phase, that assembles a graph, and an execution phase that uses a session to execute ops in the graph.” - TensorFlow docs Define computation graph Inputs, operations, outputs Symbolic representation of model Run session Execute graph Fetch output
- 18. More Complex Examples MNIST data set 70,000 Handwritten digits 28x28 pixels Used for benchmarking machine learning algorithms input_data.py mnist.train = 55,000 mnist.validation = 5,000 mnist.test = 10,000
- 19. Softmax / Logistic Regression
- 20. accuracy = .9213
- 23. accuracy = .9657
- 25. Additional Resources TensorFlow Tutorials Udacity Deep Learning Course Awesome TensorFlow TensorFlow Examples WildML TF Learn (Scikit Flow) Keras Standford CS224d Lecture 7 TensorBoard TensorFlow Playground
Editor's Notes
- #10: sequential processing of inputs Unreasonable effectiveness of recurrent neural networks
- #11: 2012 Image Net only 1 CovNet – winner error 16.4% vs 26.2% future Image Net challenges almost all entries used CovNets -in some ways similar to our own cortex
- #21: train has 55,000 images so 550 batches
- #24: 55,000 examples in train 55,000 / 100 = 550 550 * 15 = 8250 training cycles