AI State of Play Dec 2016 NYC
4 likes820 views
This document provides an overview of the state of artificial intelligence and deep learning. It discusses key concepts like different types of intelligence and learning, and what constitutes deep learning. It also reviews the growing AI market size and trends driving growth, including increasing data availability, advances in hardware and algorithms, and expanding applications. The document highlights progress in computer vision and natural language processing tasks. It surveys popular deep learning frameworks and datasets used for training models.
More Related Content
Similar to AI State of Play Dec 2016 NYC (20)
AI State of Play Dec 2016 NYC
- 2. Outline • Concepts • AI Market • Data • Software • Hardware • Conferences • Applications Ivy Data Science AI - State of Play v0.10 Peter Morgan Dec 2016 2© Ivy Data Science
- 5. © Ivy Data Science The Intelligence Revolution 5
- 6. Deep Learning/AI Frameworks - The Big Picture Ivy Data Science AI - State of Play v0.10 Peter Morgan Dec 2016 6 AI Frameworks Cognitive Architectures ML Frameworks Supervised, Unsupervised & Reinforcement Deep Learning Frameworks Neural Networks © Ivy Data Science
- 7. What is Intelligence? • Intelligence is an Agent’s ability to adapt to and to achieve goals within its Environment • Human vs machine intelligence – ultimately the same • “The term artificial intelligence is somewhat nonsensical. Something is either intelligent or it isn’t. Just as something either flies or it doesn’t. We don’t talk about artificial flying” - Zoubin Ghahramani, Cambridge University • Information processing, computation, physics, hardware • Exploration vs exploitation • Biological (any species) versus machine (any type) •Does substrate matter - carbon vs silicon? 7© Ivy Data Science
- 8. What is Learning? • Learning algorithms – system gets better with more data until no further improvement • Train the system – just like animals learn • Supervised, unsupervised and reinforcement learning • Physically, it is the strengthening of connections (synapses) between nodes (neurons) • Memory (short and long term) is involved • Deep learning is a step towards the goal of artificial general intelligence (AGI) • Ensemble of techniques 8© Ivy Data Science
- 9. © Ivy Data Science What is Deep Learning? 9
- 10. Deep Learning = Neural Networks • Refers to systems that learn from data • These systems are based on artificial neural networks (ANNs), which in turn are based on biological neural networks (BNN), such as the human brain • In practice such learning systems consist of data, multiple layers, nodes, weights and optimisation algorithms Ivy Data Science AI - State of Play v0.10 Peter Morgan Dec 2016 10© Ivy Data Science
- 12. Deep Learning Is Eating the World • What about the “AI winters”? 1974–80 and 1987–93, where AI companies over-promised and under-delivered https://en.wikipedia.org/wiki/AI_winter • Due to more labeled data, more compute power, better optimization algorithms, and better neural net models and architectures, deep learning has started to supersede humans when it comes to image recognition and classification • Work is being done to obtain similar levels of performance in natural language processing and understanding • According to Jeff Dean in a recent interview, Google have implemented DL in over one hundred of their products and services including search and photos • AI is enjoying a renaissance now, not simply because of the promise it holds for the future but because of the impact it is having on businesses today • Timing is everything, and the time is now! Ivy Data Science AI - State of Play v0.10 Peter Morgan Dec 2016 12© Ivy Data Science
- 15. Ivy Data Science AI - State of Play v0.10 Peter Morgan Dec 2016 15 • Hardware (compute) – Nvidia GPU, Intel (Nervana), AMD Radeon • Data available - structured and unstructured • Research activity • Conference attendance (e.g., NIPS) • Meetup groups • PhD enrolments in CS and machine learning • Performance measures - chess, Jeopardy, Go, computer vision, language processing, ... • Number of papers being published in AI/ML AI Trends 1 © Ivy Data Science
- 16. Ivy Data Science AI - State of Play v0.10 Peter Morgan Dec 2016 16 • Availability of open source deep learning frameworks, including TensorFlow, Mxnet, etc. - Number of packages - GitHub commits - Contributors etc. • For example, TF is most downloaded repo from GitHub in under a year • Fact that major corporates open sourced their AI frameworks, starting with Google (TF, etc.) • Number of AI related jobs on job boards • Salaries for AI experts AI Trends 2 © Ivy Data Science
- 17. Ivy Data Science AI - State of Play v0.10 Peter Morgan Dec 2016 17 • Backing and realignment by corporates to rebrand as AI companies - Microsoft, IBM, Amazon (Google and Facebook were already there) • For example: - IBM Watson HQ in NYC - Microsoft announcing 5000 strong AI division - Apple announcing at NIPS that it would be open sourcing its AI research - Siri, Cortana, Alexa are all NLP apps using neural nets • Number of AI products and apps • Number of AI/deep learning startups AI Trends 3 © Ivy Data Science
- 18. Ivy Data Science AI - State of Play v0.10 Peter Morgan Dec 2016 18 • Venture capital investment in AI startups • Number of press/news articles • Announcements from AI experts with 30 years experience confirming that this time is for real - there will be no more AI winters • Number of professors being hired way from academia to join AI companies , e.g. Uber and CMU, Google and Oxford, Facebook, Apple, etc. • Government level panels on the development and impact of AI on jobs, society and policy, e.g., Whitehouse and U.K. parliament • AI Safety consortium announced last month between Google, Microsoft, IBM and Amazon to track developments in AI • Recent books published by professors and engineers on AI development AI Trends 4 © Ivy Data Science
- 19. AI Related Books • Barrat, James, Our Final Invention, St. Martin's Griffin, 2014 • Bengio, Yoshua et al, Deep Learning, MIT Press, 2016 • Brynjolfsson, Erik and Andrew McAfee, The Second Machine Age, W.W. Norton & Co., 2014 • Domingos, Pedro, The Master Algorithm, Basic Books, 2015 • Ford, Martin, Rise of the Robots: Technology and the Threat of a Jobless Future, Basic Books, 2015 • Kaku, Michio, The Future of the Mind, Doubleday, 2014 • Kurzweil, Ray, The Singularity is Near, Penguin Books, 2006 • Kurzweil, Ray, How to Create a Mind, Penguin Books, 2013 • Russell and Norvig, Artificial Intelligence, A Modern Approach, Pearson, 2009 • Yampolskiy, Roman - Artificial Superintelligence, A Futuristic Approach, CRC, 2015 © Ivy Data Science 19
- 22. DL Outperforms ML 22© Ivy Data Science
- 23. Computer Vision Accuracy Ivy Data Science AI - State of Play v0.10 Peter Morgan Dec 2016 23© Ivy Data Science 23
- 24. GPU Faster than Moore’s Law Ivy Data Science AI - State of Play v0.10 Peter Morgan Dec 2016 24© Ivy Data Science 24
- 25. 25© Ivy Data Science
- 29. AIaaS - Industry Partnerships GPU Deep Learning in the cloud is accelerating enterprise AI
- 31. Where does the data come from? • Science – particle, astrophysics • Industry – oil, finance, telecom (all verticals) • Social – Facebook, LinkedIn, Twitter • Medicine – genome, neuroscience • Government – census, education, police • Sports – statistics • Environment – weather, sensors 31© Ivy Data Science
- 32. Data Sets • Raw data input into the neural network can originate from any environmental source • It can be recorded and stored in a database (e.g., text, images, audio, video), or live (incident directly from the environment) streaming data • Examples of recorded data sets include MNIST, Labeled Faces in the Wild (LFW), ImageNet, CIFAR and YouTube-8M Ivy Data Science AI - State of Play v0.10 Peter Morgan Dec 2016 32 MNIST LFW © Ivy Data Science
- 34. Deep Learning Evolution 34© Ivy Data Science
- 35. Convolutional Neural Networks • First developed in 1970’s • Widely used for image recognition and classification • Inspired by biological processes, CNN’s are a type of feed-forward ANN • The individual neurons are tiled in such a way that they respond to overlapping regions in the visual field Ivy Data Science AI - State of Play v0.10 Peter Morgan Dec 2016 35© Ivy Data Science
- 36. Recurrent Neural Networks • First developed in 1970’s • RNN’s are neural networks that are used to predict the next element in a sequence or time series • This could be, for example, words in a sentence or letters in a word • Applications include predicting or generating music, stories, news, code, financial instrument pricing, text, speech, in fact the next element in any event stream Ivy Data Science AI - State of Play v0.10 Peter Morgan Dec 2016 36© Ivy Data Science
- 37. LSTM and NTM • Long Short Term Memory (LSTM) • LSTM (Schmidhuber, 1997) is an RNN architecture that contains blocks that can remember a value for an arbitrary length of time • It solves the vanishing or exploding gradient problem when calculating back propagation • An LSTM network is universal in the sense that given enough network units it can compute anything a conventional computer can compute, provided it has the proper weight matrix • LSTM outperforms alternative RNNs and Hidden Markov Models and other sequence learning methods in numerous applications, e.g., in handwriting recognition, speech recognition and music composition • Neural Turing Machines (NTM) • NTMs are a method of extending the capabilities of recurrent neural networks by coupling them to external memory resources 37© Ivy Data Science
- 38. Deep Learning Frameworks 38 Framework = Toolkit = Package = Library © Ivy Data Science
- 39. TensorFlow • TensorFlow is the newly (Nov 2015) open sourced deep learning library from Google • It is their second generation system for the implementation and deployment of large-scale machine learning models • Written in C++ with a python interface, it is borne from research and deploying machine learning projects throughout a wide range of Google products and services • Initially TF ran only on a single node (your laptop, say), but Google have now released a version that runs on a distributed cluster • Available in the cloud on GCP • https://www.tensorflow.org/ Ivy Data Science AI - State of Play v0.10 Peter Morgan Dec 2016 39© Ivy Data Science
- 40. Cognitive Toolkit • Microsoft open source deep learning framework (Jan 25, 2016) • Version 2.0 released Oct 25, major upgrade • Renamed CNTK to Microsoft Cognitive Toolkit • Announced partnership with Nvidia and OpenAI (Elon Musk backed AI startup), Nov 16 • Languages are Python, C++ or BrainScript • Can run on Azure GPU’s • https://www.microsoft.com/en-us/research /product/cognitive-toolkit/ Ivy Data Science AI - State of Play v0.10 Peter Morgan Dec 2016 40© Ivy Data Science
- 41. Torch • First released in 2000, with over 50,000 downloads, company users include Google, Facebook, Twitter • The goal of Torch is to have maximum flexibility and speed in building scientific algorithms while making the process extremely simple • Torch is a neural network library written in Lua with a C/CUDA interface originally developed by a team from the Swiss institute EPFL • At the heart of Torch are popular neural network and optimization libraries which are simple to use, while being flexible in implementing different complex neural network topologies • http://torch.ch/ 41© Ivy Data Science
- 42. AI Hardware Ivy Data Science AI - State of Play v0.10 Peter Morgan Dec 2016 42 42© Ivy Data Science
- 43. Types of Hardware • Sensors, processors, storage, memory, network • Processors - CPU, GPU, FPGA, ASIC • GPU - Graphics Processing Units were first brought to market by Nvidia in 2007 to meet the demands of the gaming market • Massively parallel processing (MPP) • 100 x speedup compared with CPU’s • Widespread application – science, industry, government • Nvidia www.nvidia.com • Intel Xeon Phi http://www.intel.com/content/www/us/en/processors/xeon/xeon- phi-detail.html • AMD Radeon www.amd.com 43Ivy Data Science AI - State of Play v0.10 Peter Morgan Dec 2016
- 44. CPU v GPU Architecture Ivy Data Science AI - State of Play v0.10 Peter Morgan Dec 2016 44 44© Ivy Data Science
- 45. Nvidia GPU Exponentials Ivy Data Science AI - State of Play v0.10 Peter Morgan Dec 2016 45© Ivy Data Science 45
- 47. DLaaS - Cloud Services • DL/ML as a Service is offered by the major cloud providers • AWS (Amazon) http://aws.amazon.com/ • Azure (Microsoft) http://azure.microsoft.com/ • GCP (Google) https://cloud.google.com/ • Bluemix (IBM) https://www.ibm.com/cloud-computing/bluemix/ © Ivy Data Science 47
- 48. AI Conferences – Business Focussed • O’Reilly AI http://conferences.oreilly.com/artificial-intelligence/ai-ny • AI Frontiers http://www.aifrontiers.com/ • AI World http://aiworldexpo.com/program/ • AI Europe http://ai-europe.com/ • AI Summit https://theaisummit.com/london/ • Re:Work https://www.re-work.co/events/ • World of Watson https://www-01.ibm.com/software/events/wow/ © Ivy Data Science 48
- 49. AI Conferences – Research Focussed • NIPS = Neural Information Processing Systems https://nips.cc/ • IJCNN = International Joint Conference on Neural Networks http://www.ijcnn.org/ • IJCAI = International Joint Conference on Artificial Intelligence http://ijcai.org/ • ICANN = International Conference on Artificial Neural Networks http://www.icann2017.org/ • IWANN = International Work-Conference on Artificial Neural Networks http://iwann.uma.es/ • ICONIP = International Conference on Neural Information Processing http://www.iconip2017.org/papers.html • ICAART = International Conference on Agents and Artificial Intelligence http://www.icaart.org/ • ISIS = International Symposium on Advanced Intelligent Systems http://isis2017.org/ • AAAI = Association of Advancement of Artificial Intelligence http://www.aaai.org/Conferences/conferences.php • ACM = Association of Computing Machinery https://www.acm.org/conferences • AGI = Artificial General Intelligence Conference http://agi-conf.org/ • TensorCon = TensorFlow Conference https://ti.to/TensorCon/ © Ivy Data Science 49
- 50. AI Applications Ivy Data Science AI - State of Play v0.10 Peter Morgan Dec 2016 50© Ivy Data Science
- 52. AI in Healthcare - Market • $600m in 2014, expected to reach $6bn by 2021 - Frost & Sullivan • Moved from pilots and proof of concepts, to commercialisation, adoption, and utilisation • Driven by rising medical costs and increasing volumes of data • Helping physicians make more informed and accurate decisions about patient care • Evaluates and interprets medical data, offers recommendations, and makes predictions • Unlock hidden insights within the data with deep semantic understanding of the content • Fight diseases like cancer, heart stroke, autism and Parkinsons Ivy Data Science AI - State of Play v0.10 Peter Morgan Dec 2016 52© Ivy Data Science
- 53. Watson Tackles Cancer • Watson was tested on 1,000 cancer diagnoses made by human experts. In 99% of them, Watson recommended the same treatment as the oncologists • In 30% of the cases, Watson also found a treatment option the human doctors missed • Some treatments were based on research papers that the doctors had not read — more than 160,000 cancer research papers are published a year • Other treatment options surfaced in new clinical trials the oncologists had not yet seen announced on the web Ivy Data Science AI - State of Play v0.10 Peter Morgan Dec 2016 53© Ivy Data Science
- 54. Scan & X-ray Diagnostics • X-rays to detect fractures • Greater speed and accuracy • Takes workload off doctors • Tumor detection • fMRI scans & EEG’s • Anomaly detection • Eyes, kidneys, lungs, etc. • Image classification • Using convolutional neural networks (CNNs) Ivy Data Science AI - State of Play v0.10 Peter Morgan Dec 2016 54© Ivy Data Science
- 55. The team was able to: • Process 10,900 payments per second • Cut false alarm rates by 50% • Free up resources to combat true fraud PayPal is improving cybersecurity with deep learning © Ivy Data Science Fintech
- 56. Chatbots (NLP) • Chatbots, in the form of assistants and automated customer service reps, are becoming increasingly common across the industry • AI is also primed to make the massively complicated (and data-rich) world of logistics much easier for retailers • AI that can intuit what a shopper’s style is and adapt its recommendations as she or he shops • AI that can evolve a website to specific consumer needs • AI that can understand user concerns and answer complicated questions • Make shopping both easier and more personal Ivy Data Science AI - State of Play v0.10 Peter Morgan Dec 2016 56© Ivy Data Science
- 57. Where are we headed? Ivy Data Science AI - State of Play v0.10 Peter Morgan Dec 2016 57 World Economic Forum (WEF) Report, 2016: Today, we are at the beginning of a Fourth Industrial Revolution. Developments in genetics, artificial intelligence, robotics, nanotechnology, 3D printing and biotechnology, to name just a few, are all building on and amplifying one another. This will lay the foundation for a revolution more comprehensive and all-encompassing than anything we have ever seen Deepmind Mission: Solve intelligence then solve everything else © Ivy Data Science
- 59. Ivy Data Science AI - State of Play v0.10 Peter Morgan Dec 2016 59© Ivy Data Science
- 60. Training with Job Placement • Led by Experts from Harvard, Columbia, UMass, UCL • Real time paid projects in the last four weeks of bootcamp • Practical hands-on training that will prepare you for a career as a Data Scientist, Machine Learning or Deep Learning Researcher/Practitioner Ivy Data Science AI - State of Play v0.10 Peter Morgan Dec 2016 60© Ivy Data Science
- 61. Tuition Fees • Online Instructor led courses starting at $1,999 - 50hrs • Evening Bootcamp starting at $4,999 - 50hrs • Weekend Bootcamp starting at $6,999 - 100hrs • Full Immersive (12 week) Bootcamp starting at $11,999* - 320hrs *earlybird • Financing available Ivy Data Science AI with Applications v0.10 Peter Morgan Dec 2016 61© Ivy Data Science
- 62. Network of Clients Ivy Data Science AI - State of Play v0.10 Peter Morgan Dec 2016 62© Ivy Data Science
- 63. Any Questions? © Ivy Data Science 63© Ivy Data Science 63 [email protected] www.ivydatascience.com @ivydatascience.com
- 64. •www.ivydatascience.com | @ivydatascience 64Ivy Data Science AI - State of Play v0.10 Peter Morgan Dec 2016