Introduction to Deep Learning: Concepts, Architectures, and Applications
0 likes164 views
The document provides an introduction to deep learning, presented by Dr. Amr Rashed, highlighting his educational background and experience, including research interests in various fields of study. It outlines the teaching philosophy, course materials, assessments, and student feedback mechanisms, as well as a comprehensive overview of deep learning concepts, applications, architectures, and influential figures in the field. The document also includes resources and tutorials for further learning about deep learning and machine learning.
Introduction to Deep Learning: Concepts, Architectures, and Applications
- 1. Introduction to Deep Learning Presented by: Dr. Amr Rashed Ph.D in Electrical Communication Engineering
- 2. EDUCATION January 2021 Ph.D In Electrical Communications Engineering Faculty of Engineering, Mansoura University, Mansoura January 2010 M.Sc In Electrical Communications Engineering Faculty of Engineering, Mansoura University, Mansoura January 2005 B.Sc. In Electronics & Communications Engineering Faculty of Engineering, Mansoura University, Mansoura DR. AMR EZZ EL - DIN RASH ED LECTURER DEPARTMENT OF COMPUTER ENGINEERING COLLEGE OF CIT, TAIF, KSA RESEARCH INTERESTS Deep Learning, FPGA, VHDL, MATLAB, Digital image processing, Bioinformatics NLP, Encryption, Computer Vision, Digital signal processing CONTACTS 0554404723 / 01002820694 [email protected] Current location: Taif, KSA (temporary) Permanent location: New Damietta, Egypt EXPERIENCE Lecturer at College of CIT, Taif University, KSA (2018-2024) Full-stack Web Developer at Deanship of Higher Studies and Research, Taif University, Taif, KSA (2012-2018) Lecturer at MET Institute (2009-2012) Instructor (Part-time) at Jelecomm, New Horizon, ITshare, Egycet Assistant Lecturer at Faculty of Engineering, Mansoura University (2005-2009)
- 4. Teaching Philosophy In-person lectures: Essential for active participation—bring your pen and paper. Lecture reviews: Available on my YouTube Educational channel. Questions and support: Reach me during office hours, via your section's WhatsApp group, through the Blackboard discussions section, or by email. Course materials: Accessible on Blackboard and OneDrive (includes content and software). Presentations: Available on multiple platforms including SlideShare, Blackboard, ResearchGate, and cloud services like OneDrive.
- 5. Teaching Philosophy )cont.) Code resources: Accessible on GitHub, MATLAB File Exchange, and Google Colab links. Assessments: Quizzes: 2 to 3 quizzes throughout the semester. Exams: Midterm and final exams. Homework Assignments: Assigned approximately every two weeks and announced on Blackboard. Group Projects: Collaborative projects will be assigned to foster teamwork and practical application of course concepts. Student Feedback: At the end of the semester, student feedback on the course will be collected to improve future iterations.
- 6. Contact Information • Dr. Amr E. Rashed نص عىل تحتوي صورة ً تلقائيا الوصف إنشاء تم
- 7. Some of my Slidshare uploads
- 8. My YouTube channel statistics • 8706 subscribers • 809,803 views • More than 30 playlists • More than456 videos
- 13. My account on Matlab File Exchange
- 14. Agenda Human Intelligence Vs Artificial Intelligence Motivations and Deep Learning Applications Median Salary for Data Scientist Data Science and Corresponding Related Fields Top 10 Most-Popular Programming Languages to Learn in 2020 What is Python and Python Applications in Real World A Brief History Big Players :Superstar Researcher Matlab & Python Example
- 23. IBM’s deep blue beats gary Kasparov in 1997 Motivations
- 24. In 2011 IBM Watson competed in game show jeopardy and won 1 million dollars
- 25. In 2015 google deep mind defeated lisa dole an 18 times world champion
- 27. Deep Learning : Applications
- 32. ARPLAN 3D application ARULER application
- 33. Amazon go :First cashierless store
- 34. If we want to make it hard for bots, it has to be hard for human as well.
- 36. Median Salary for Data Scientist Islam Hesham , Codezilla yotube channel
- 37. Data Science and Corresponding Related Fields
- 40. •McCulloch, Warren S., and Walter Pitts. "A logical calculus of the ideas immanent in nervous activity." The bulletin of mathematical biophysics 5.4 (1943): 115-133. •Rosenblatt, Frank. "The perceptron: a probabilistic model for information storage and organization in the brain." Psychological review 65.6 (1958): 386. •Rumelhart, David E., Geoffrey E. Hinton, and Ronald J. Williams. "Learning representations by back-propagating errors." Cognitive modeling 5.3 (1988): 1. •LeCun, Yann, et al. "Backpropagation applied to handwritten zip code recognition." Neural computation 1.4 (1989): 541-551. •1993: Nvidia started… •Hinton, Geoffrey E., Simon Osindero, and Yee-Whye Teh. "A fast learning algorithm for deep belief nets." Neural computation 18.7 (2006): 1527-1554. •Raina, Rajat, Anand Madhavan, and Andrew Y. Ng. "Large-scale deep unsupervised learning using graphics processors." Proceedings of the 26th annual international conference on machine learning. ACM, 2009. •Deng, Jia, et al. "Imagenet: A large-scale hierarchical image database."Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on. IEEE, 2009. •2010: “GPUS ARE ONLY UP TO 14 TIMES FASTER THAN CPUS” SAYS INTEL –Nvidia •Glorot, Xavier, Antoine Bordes, and Yoshua Bengio. "Deep sparse rectifier neural networks." International Conference on Artificial Intelligence and Statistics. 2011. •Hinton, Geoffrey E., et al. "Improving neural networks by preventing co-adaptation of feature detectors." arXiv preprint arXiv:1207.0580 (2012). •Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. "Imagenet classification with deep convolutional neural networks." Advances in neural information processing systems. 2012. A brief history
- 44. A Brief History • 2012 was the first year that neural nets grew to prominence as Alex Krizhevsky used them to win that year’s ImageNet competition (basically, the annual Olympics of computer vision), dropping the classification error record from 26% to 15%, an astounding improvement at the time.
- 45. Big Players :Superstar Researcher Jeoferry Hilton : University of Toronto &Google Yann Lecun : New Yourk University &Facebook Andrew Ng : Stanford & Baidu Yoshua Bengio : University of Montreal Jurgen schmidhuber : Swiss AI Lab & NNAISENSE
- 46. Big Players : Companies
- 56. The Problem Space : inputs & outputs 0 Depending on the resolution and size of the image, it will see a 32 x 32 x 3 array of numbers 0 These numbers, while meaningless to us when we perform image classification, are the only inputs available to the computer. 0 The idea is that you give the computer this array of numbers and it will output numbers that describe the probability of the image being a certain class (.80 for cat, .15 for dog, .05 for bird, etc).
- 57. The Problem Space :What We Want the Computer to Do 0 To be able to differentiate between all the images it’s given and figure out the unique features that make a dog a dog or that make a cat a cat. 0 This is the process that goes on in our minds subconsciously as well. 0 When we look at a picture of a dog, we can classify it as such if the picture has identifiable features such as paws or 4 legs. 0 In a similar way, the computer is able perform image classification by looking for low level features such as edges and curves, and then building up to more abstract concepts through a series of convolutional layers.
- 58. What is Deep Learning(DL) Part or a powerful class of the machine learning field of learning representations of data. Exceptional effective at learning patterns. Utilize learning algorithms that derive meaning out of data by using hierarchy of multiple layers that mimic the neural networks of our brain. If you provide the system tons of information , it begins to understand it and respond in useful ways. Stacked “Neural Network”. Is usually indicates “Deep Neural Network”.
- 59. What is Deep Learning (DL) • Collection of simple, trainable mathematical functions. • Learning methods which have deep architecture. • It often allows end-to-end learning. • It automatically finds intermediate representation. Thus, it can be regarded as a representation learning.
- 61. NIPS(Computational Neuroscience Conference) Growth Amount of Data vs Performance
- 62. Why Deep Learning and Why Now?
- 65. The Perceptron The structural building block of deep learning
- 66. Commonalities with real brains: ● Each neuron is connected to a small subset of other neurons. ● Based on what it sees, it decides what it wants to say. ● Neurons learn to cooperate to accomplish the task. • The vental pathway in the visual cortex has multiple stages • There exist a lot of intermediate representations Deep Learning Basics :Neuron
- 67. Deep Learning Basics :Neuron An artificial neuron contains a nonlinear activation function and has several incoming and outgoing weighted connections. Neurons are trained filters and detect specific features or patterns (e.g. edge, nose) by receiving weighted input, transforming it with the activation function and passing it to the outgoing connections
- 68. Supervised Learning : Learning with a labeled training set Example : e-mail spam detector with training set of already labeled emails Unsupervised Learning :Discovering patterns in unlabeled data Example :cluster similar documents based on the text content Reinforcement learning : learning based on feedback or reward. Example :learn to play chess by wining or losing Machine Learning –Basics Learning Approach
- 78. Problem Types
- 79. Deep Neural Network : Architecture types Feed-Forward • Convolutional neural networks • De-convolutional networks Bi-Directional • Deep Boltzmann Machines • Stacked auto-encoders Sequence -Based • RNNs • LSTMs
- 80. Types of Networks used for Deep Learning Convolutional Neural Networks(Convnet,CNN) Recurrent Neural Networks(RNN) Long Short Term Memory (LSTM) networks Deep/Restricted Boltzmann Machines (RBM) Deep Q-networks Deep Belief Networks(DBN) Deep Stacking Networks
- 81. Convolution Neural Networks(CNN):Basic Idea 0 This idea was expanded upon by a fascinating experiment by Hubel and Wiesel in 1962 where they showed that some individual neuronal cells in the brain responded (or fired) only in the presence of edges of a certain orientation. 0 For example, some neurons fired when exposed to vertical edges and some when shown horizontal or diagonal edges. Hubel and Wiesel found out that all of these neurons were organized in a columnar architecture and that together, they were able to produce visual perception. 0 This idea of specialized components inside of a system having specific tasks (the neuronal cells in the visual cortex looking for specific characteristics) is one that machines use as well, and is the basis behind CNNs.
- 83. CNN Layers 1 • Convolution Layer 2 • Batch Normalization Layer 3 • RELU Layer 4 • Local Response Normalization Layer 5 • Max and AVG Pooling 6 • Dropout Layer 7 • Fully Connected Layer (FC) 8 • Output Layer (SOFT MAX ,Regression )
- 84. Convolution Neural Networks(CNN) • Convolutional neural networks learn a complex representation of visual data using vast amounts of data .they are inspired by human visual system and learn multiple layers of transformations , which are applied on top of each other to extract progressively more sophisticated representation of the input . DEFENITION • Inspired by the visual cortex and Pioneered by Yann Lecun (NYU). • CNN have multiple types of layers ,the first of which is the Convolutional layer. Notes
- 85. Test pretrained model in Matlab and Python
- 89. Dataset storage, and search platforms • https://www.kaggle.com/datasets • https://datasetsearch.research.google.com/ • https://github.com/ • https://ieee-dataport.org/ • medical-imaging-datasets • https://data.worldbank.org/ • https://data.world/ • https://data.un.org/ • https://archive.ics.uci.edu/ml/datasets/
- 90. Best Python tutorials: • Books: • Python Crash Course A Hands-On, Project-Based Introduction to Programming by E R I C M A T T H E S • Videos : • Python Beginners Tutorial – بالعربي • الحوراني حسام بايثون قناة • Mastering Python • 1- Python programming اساسيات بايثون برمجة • إم جامعة من والبرمجة الكومبيوتر علم في مقدمة آي تي
- 91. Best AI, ML, and DL tutorials: • Books and websites: • Deep Learning book by Ian Goodfellow (MIT press) • “Neural Networks and Deep Learning” by Michael Nielsen • TensorFlow • Machine Learning Mastery by Jason Brownlee • Towards data science • Data science community • http://introtodeeplearning.com(MIT) • https://www.pyimagesearch.com/ • Videos: • Machine Learning Andrew Ng Courses (coursera) • للجميع االصطناعي (الذكاءAndrew ng) with Arabic subtitle • Courses and Specializations (all AI, ML, and DL courses) • Machine Learning Course - CS 156 (Prof. Yasser abo Moustafa, Caltech university) • Introduction to data science • االصطناعي الذكاء قناة - الحوراني حسام • 01 machine learning اآللة تعليم , األول القسم : مقدمة
- 92. For more information • TensorFlow and Deep Learning without a PhD, Part 1 (Google Cloud Next '17) • TensorFlow and Deep Learning without a PhD, Part 2 (Google Cloud Next ‘17) • Welcome (Deep Learning Specialization C1W1L01) • A friendly introduction to Deep Learning and Neural Networks • Introduction to Deep Learning: Machine Learning vs. Deep Learning • Introduction to Deep Learning: What Are Convolutional Neural Networks? • How Convolutional Neural Networks work • How Deep Neural Networks Work • Deep Learning In 5 Minutes | What Is Deep Learning? | Deep Learning Explained Simply | Simplilearn • Deep Learning Crash Course for Beginners
- 93. Cont. • What is Deep Learning? | Introduction to Deep Learning | Deep Learning Tutorial | Simplilearn • MIT Introduction to Deep Learning | 6.S191(2021) • MIT 6.S191 Lecture 1: Intro to Deep Learning (2017) • Machine Learning Foundations: Ep #1 - What is ML? • The 7 steps of machine learning • But what is a neural network? | Chapter 1, Deep learning • Deep Learning State of the Art (2020) • What is Artificial Intelligence? In 5 minutes. • A friendly introduction to Convolutional Neural Networks and Image Recognition