Types of Learning
Roadmap
1 When Can Machines Learn?
Lecture 2: Learning to Answer Yes/No
PLA A takes linear separable D and
perceptrons H to get hypothesis g
Lecture 3: Types of Learning
Learning with Different Output Space Y
Learning with Different Data Label y
n
Learning with Different Protocol f ⇒ (x
n
, y
n
)
Learning with Different Input Space X
2 Why Can Machines Learn?
3 How Can Machines Learn?
4 How Can Machines Learn Better?
Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 1/29
Types of Learning Learning with Different Output Space Y
Credit Approval Problem Revisited
age 23 years
gender female
annual salary NTD 1,000,000
year in residence 1 year
year in job 0.5 year
current debt 200,000
credit? {no(−1), yes(+1)}
unknown target function
f : X → Y
(ideal credit approval formula)
training examples
D : (x
1
, y
1
), · · · , (x
N
, y
N
)
(historical records in bank)
learning
algorithm
A
final hypothesis
g ≈ f
(‘learned’ formula to be used)
hypothesis set
H
(set of candidate formula)
Y = {−1, +1}: binary classification
Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 2/29
Types of Learning Learning with Different Output Space Y
More Binary Classification Problems
•
credit approve/disapprove
•
email spam/non-spam
•
patient sick/not sick
•
ad profitable/not profitable
•
answer correct/incorrect (KDDCup 2010)
core and important problem with
many tools as building block of other tools
Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 3/29
Types of Learning Learning with Different Output Space Y
Multiclass Classification: Coin Recognition Problem
25
5
1
Mass
Size
10
•
classify US coins (1c, 5c, 10c, 25c)
by (size, mass)
•
Y = {1c, 5c, 10c, 25c}, or
Y = {1, 2, · · · , K } (abstractly)
•
binary classification: special case
with K = 2
Other Multiclass Classification Problems
•
written digits ⇒ 0, 1, · · · , 9
•
pictures ⇒ apple, orange, strawberry
•
emails ⇒ spam, primary, social, promotion, update (Google)
many applications in practice,
especially for ‘recognition’
Hsuan-Tien Lin (NTU CSIE) Machine Learning Foundations 4/29
