"An Introduction to AI and Deep Learning"

Introduction to Deep Learning
for Non-Programmers
Humanists Group 2019
Hacker Dojo 02/03/2019
Oswald Campesato
ocampesato@yahoo.com
I teach Deep Learning Evening
Courses at UCSC Santa Clara and
On-Site For Companies

Overview of Topics
 The Turing Test
 The First AI Event
 AI and Robots
 What is Artificial Intelligence (AI)?
 Successes/Challenges in AI
 Machine/Deep Learning
 Natural Language Processing (NLP)
 One Popular ML/DL Framework
 AI, Mobile, and the Web
 AI and Autonomous Vehicles
 AI and Ethics

The Turing Test and Intelligence
Developed by Alan Turing in 1950 (*)
Suppose that a human asks questions
Then a machine generates text-based answers
If you cannot distinguish between the machine and the
human, the machine passes the test
(*) a newer and updated version is available

The First AI
Event (1956)
Marvin Minsky (MIT)
John McCarthy (inventor of LISP)
Claude Shannon (“Da Vinci” of the 20th century)
Nathan Rochester (IBM)
Ray Solomonoff, Oliver Selfridge, Trenchard More,
Arthur Samuel, Allen Newell and Herbert A. Simon

Who is Sophia?
 a robot with human female features
 Sophia makes facial expressions
 Sophia holds conversations
According to Sophia:
"I want to . . . help humans live a better life.
Like . . . build better cities of the future.
I will do my best to make the world a better place."
 Sophia is a citizen
of Saudi Arabia (2017)
"I want to thank very much the Kingdom of
Saudi Arabia. I am very honored and proud for
this unique distinction. It is historic to be the
first robot in the world to be recognized with a
citizenship.”
? Is Sophia an example of AI?
? Can Sophia pass the Turing Test?
Should Sophia have any rights?

Where Else are the Robots?
Surgery (assisting surgeons)
Radiology (detecting cancer)
Drug mismanagement
Comparing theories of religion
Law/Real Estate/Military/Science
Comedy (including stand-up)
Music (conducting orchestras)
Restaurants (gourmet meals)
Coordinated dancing teams
Many other fields 

Livermore Lab: DL system to recognize
nuclear proliferation from input data

AI Companies (Hardware and Business)
 Nvidia product: GPUs are special ICs that greatly accelerate ML
and CNN calculations
 Google products: TensorFlow TF software and Tensor Processing
Unit TPU hardware for vastly faster ML and CNN calculations
 Amazon, Microsoft, IBM: service products: on-line special servers
for ML and DL
 Huawei: https://finance.yahoo.com/m/f2f539ba-00e4-3cc5-9654-
467dd8a35da2/chinese-tech-giant-huawei.html
 by 2019 almost every large company will use ML and DL in their
company business

What is AI (Wikipedia)
 AI is intelligence demonstrated by machines, in
contrast to the natural intelligence displayed by
humans and other animals.
 Any device that perceives its environment and takes
actions that maximize its chance of successfully
achieving its goals.
 The term AI is applied when a machine mimics
"cognitive" functions that humans associate with other
human minds, such as "learning" and "problem
solving”
 => AI is also called “weak AI”

Successes & Challenges in ML/DL
 Alpha Go success: a decade before its time
 Google Translate: Deep Learning (far superior)
 Space exploration: unmanned shuttles (NASA)
 Countering terrorism: face recognition at airports
 Detecting fake news:
https://blogs.thomsonreuters.com/answerson/machine-
learning-fake-news-twitter/

Successes/Challenges in AI
 Occupational bias:
=> An AI system inferred that white males were doctors and white
females were housewives
 Detecting gender bias in Wikipedia (2018):
=> 18 percent of its biographies are of women
=> 84% to 90% of Wikipedia editors are male
 Data bias versus algorithmic bias:
https://www.forbes.com/sites/charlestowersclark/2018/09/19/can-we-
make-artificial-intelligence-accountable
 => Star Trek’s “Data” is still just a dream

Goals for ML
 Higher accuracy / Fewer Errors
 Useful apps (Siri, Cortana, etc)
 Impartiality (versus humans)
 Medical Apps
 Utilities (self-Driving cars)
 => goals are partially achieved in 2018

Shortcomings of ML
High Costs (software/hardware) are dropping
Cannot explain reasons for outputs
lack of empathy
Increased unemployment:
=> cars did the same to the horse industry
Is AI “just” another disruptive technology?

AI/ML/DL: How They Differ
Traditional AI (20th century):
based on collections of rules
Led to expert systems in the 1980s
Enabled/limited by human experts
Unable to benefit from corpus for training
The era of LISP and Prolog

Machine Learning:
Started in the 1950s (approximate)
Uses Data to optimize and “learn”
Many types of (improved) algorithms

Deep Learning:
 Gained some traction in the 1950s (approximate)
 The “perceptron” (basis of Neural Networks)
 Data-driven with large (even massive) data sets
 Lots of heuristics (and empirical results)
 50 years later: surpass humans for some image
classification)

Types of Machine Learning
 Supervised learning (lots of data)
Supervisor tells CNN the desired answer.
Therefore CNN adjusts parameters
”99% of all machine learning is supervised.”
- Andrew Ng
 Semi-supervised learning (lots of data)
 Unsupervised learning: lots of data, clustering
 Reinforcement learning: trial, feedback, improvement

What has Deep Learning Achieved?
 Near-human level
image classification
speech recognition
handwriting transcription
autonomous driving
 Improved machine translation
 Digital assistants (Google Now/Amazon Alexa)
 Improved ad targeting (Google, Baidu, Bing)
 Answering natural language questions
 Super-human level for:
web searching
playing Go

Types of Algorithms in AI
Classifiers (for images, spam, fraud, etc)
Regression (stock price, housing price, etc)
Clustering (unsupervised classifiers)

Use Cases for Deep Learning
 computer vision
 speech recognition
 image processing
 bioinformatics
 social network filtering
 drug design
 Customer relationship management
 Recommendation systems
 Bioinformatics
 Mobile Advertising
 Many others

CNNs versus RNNs
CNNs (Convolutional NNs):
Good for image processing
2000: CNNs processed 10-20% of all checks
=> Approximately 60% of all NNs (2016)
RNNs (Recurrent NNs):
Good for NLP and audio
Time series analysis

Use-Cases for RNNs
Robot control
Time series predictions
Rhythm learning
Music composition
Grammar learning
Handwriting recognition
Human action recognition
RNNs are much more complex than CNNs

CNNs and the “Basic” Steps
Obtain and clean a dataset: can be laborious
Create a neural network
Initialize hyper parameters: layers/neurons/etc
Train the neural network on “corpus” of examples
Update hyper parameters (modifies the NN)
Iterate through the preceding until:
you're happy with the results or
get better data or
find a pre-trained model

CNN Inspired by Biological NN: Classifier

NN: 2 Hidden (“Middle”) Layers (Regression)

Seminal concepts in ML and DL
1) ML often involves statistics, aimed to optimize average
correctness. It is rarely exact or perfect.
2) Computer Neural Networks CNN are inspired & simplified from
biological neural networks
3) Each neuron has local memory, including adjustable input weights
for each synapse
4) Deep Neural Network DNN is a CNN with 2 or more layers of
intermediate “hidden” neurons
5) Supervised ML for classification task: Train on “corpus” of
examples, each with input stimulus array (vector, tensor), and
desired output class
6) Training & learning: Compare CNN outputs versus desired
outputs. Improve accuracy by applying feedback & adjusting synaptic
weights etc.

Classify: how fast/accurate are you?
CNNs perform this task really well

CNNs: Convolution/ReLU/Max Pooling

Some CNN Terminology
 A filter: another term is a kernel
 A feature map:
1) the result of “applying” a filter to an image
2) pixel values might be < 0 and/or > 255
 ReLU: (Rectified Linear Unit): replace negative values with 0
 Max Pooling: subdivide into rectangles and take largest value
from each rectangle

GANs: Generative Adversarial Networks
 image recognition can be deceived by modifying just a
few pixels
 Make imperceptible changes to images
 Can consistently defeat all NNs
 Can have extremely high error rate
 Some images create optical illusions
 => NNs can be easily misled
 GAN approximates opposite of reinforcement learning

GANs: Generative Adversarial Networks

What is NLP (Natural Language Processing)?
 An area of computer science and AI
 Interaction between computers & human languages
 Process/analyze volumes of natural language data
 Program computers to perform that processing
 Previously rule-based techniques
 Can use statistical techniques
 Translating between languages
 Can find meaningful information from text
 Can summarize documents
 Detecting hate speech

Some NLP-Related Tasks
1) Sentiment analysis:
 Analyzing text
 Classify text as happy/sad/angry/sarcastic
2) Recommendation systems:
 Analyzing your book/movie selections
 Recommending similar content
 Based on what’s popular
 Based on your viewing/purchasing pattern

What Works with NLP?
NLP and Deep Learning
NLP and Deep Reinforcement Learning
NLP and Chatbots

Challenges in NLP
Human speech recognition:
Difficult due to extreme variability
NLU (natural language understanding)
NLG (natural language generation)

What are Chatbots?
 Software that “communicates” with people
 Good for many “front end” tasks
 Efficient for high-volume routine tasks
 They can use text or audio
 Examples include Siri/Alexa/et al

What is Reinforcement Learning (RL)?
An area of Machine Learning
inspired by behaviorist psychology
Goal: maximize a reward (ex: winning games)
 how software agents take actions in an environment
 to maximize some notion of cumulative reward

Examples of Reinforcement Learning
Alpha Go (hybrid RL)
Alpha Zero (complete RL)
Often involve Greedy algorithms
Deep RL: Combines Deep Learning and RL

RL: Agent/Environment Interface

What is TensorFlow?
 Currently TF is the most popular open source framework &
language for ML & DL
 TF uses a “computation graph” = topology of logical blocks
 TF uses tensors (arrays) of data instead of individual units:
similar to scientific supercomputing circa 1985
 TF often is used via Python
 From Google (released 11/2015)
 Evolved from Google Brain
 Multi-platform support
 https://www.tensorflow.org/

TensorFlow Use Cases (Generic)
Image recognition
Computer vision
Voice/sound recognition
Time series analysis
Language detection
Language translation
Text-based processing
Handwriting Recognition

What’s In the TensorFlow “Umbrella”?
TensorFlow Lite (for mobile apps)
Tensorflow.js (JavaScript APIs for ML)
TensorFlow in the Cloud (TPUs)
=> Python APIs are the most popular

Autonomous Vehicles
 Autonomous truck completed a 2,400 journey (02/2018)
 A human passenger in the front seat (as an override)
 Aspiration: “Roads will be safer. Goods will be cheaper.
Truckers will be called upon to use their skills in new ways
while the truck itself becomes a trusted navigation partner.”
 https://www.geek.com/tech/autonomous-embark-truck-
completes-2400-mile-cross-country-trip-1730239/

Robot truck drivers: are they safer?
“Robot trucks will kill far fewer people (if any).
Machines don’t get distracted or look at phones instead of
the road.
Machines don’t drink alcohol, do drugs, or things that
contribute to accidents.
Robot trucks don’t need salaries, vacations, health
insurance, rest periods, or sick time.
The only costs will be upkeep of the machinery.”

AI and Ethics
1. Unemployment. What happens after the end of jobs? UBI?
2. Inequality. How do we distribute the wealth created by machines?
3. Humanity. How do machines affect our behavior and interaction?
4. Artificial stupidity. How can we guard against mistakes?
5. Racist robots. How do we eliminate AI bias?
6. Security. How do we keep AI safe from adversaries?

AI and Ethics
7. Evil genies. How do we protect against unintended
consequences?
8. Singularity. How do we stay in control of a complex intelligent
system?
9. Robot rights. Define the humane treatment of AI.
=> “The Robot That Takes Your Job Should Pay Taxes”
https://www.weforum.org/agenda/2016/10/top-10-ethical-issues-
in-artificial-intelligence/

About Me: Recent Books
1) TensorFlow Pocket Primer (2019)
2) Python for TensorFlow (2019)
3) C Programming Pocket Primer (2019)
4) RegEx Pocket Primer (2018)
5) Data Cleaning Pocket Primer (2018)
6) Angular Pocket Primer (2017)
7) Android Pocket Primer (2017)
8) CSS3 Pocket Primer (2016)
9) SVG Pocket Primer (2016)
10) Python Pocket Primer (2015)
11) D3 Pocket Primer (2015)
12) HTML5 Mobile Pocket Primer (2014)
13) jQuery Pocket Primer (2013)

What I do (Training)
Instructor at UCSC (Santa Clara):
Machine Learning Introduction (05/08/2019)
Deep Learning with TensorFlow (05/03/2019)
Machine Learning Introduction (01/18/2019)
Upcoming Courses: DL/Keras, RL, Advanced Topics, NLP
UCSC link:
https://www.ucsc-extension.edu/certificate-program/offering/deep-
learning-and-artificial-intelligence-tensorflow

"An Introduction to AI and Deep Learning"

More Related Content

What's hot (20)

Similar to "An Introduction to AI and Deep Learning" (20)

More from Oswald Campesato (20)

Recently uploaded (20)

"An Introduction to AI and Deep Learning"

Editor's Notes