svm-proyekt.pptx
- 1. Support Vector Machine Classification , Regression and Outliers detection
- 2. Introduction SVM A Support Vector Machine (SVM) is a discriminative classifier which intakes training data (supervised learning), the algorithm outputs an optimal hyperplane which categorizes new examples.
- 3. What could be drawn to classify the black dots from blue squares?
- 4. A line drawn between these data points classify the black dots and blue squares. Linearly separable data
- 5. Linear vs Nonlinear separable data
- 6. What could be drawn to classify these data points ( red dots from blue stars )? NonLinearly separable data
- 7. Here the hyperplane is a 2d plane drawn parallel to x-axis that is the separator. NonLinearly separable data
- 8. Non Linear data ( type 2 ) Raw Data Line as Hyperplane
- 9. For the previous data the line , if used as a Hyperplane ● Two black dots also fall in category of blue squares ● Data separation is not perfect ● It tolerates some outliers in the classification
- 10. This type of separator best provides the classification. But ● It is quite difficult to train a model like this . ● This is termed as Regularisation parameter.
- 11. Tuning Parameters SVM 1. Kernel 2. Regularization 3. Gamma 4. Margin
- 12. Margin Margin is the perpendicular distance between the closest data points and the Hyperplane ( on both sides ) The best optimised line ( hyperplane ) with maximum margin is termed as Margin Maximal Hyperplane. The closest points where the margin distance is calculated are considered as the support vectors.
- 14. Regularization ● Also the ‘ C ‘ parameter in Python’s SkLearn Library ● Optimises SVM classifier to avoid misclassifying the data. ● C → large ● C → small Margin of hyperplane → small Margin of hyperplane → large ● misclassification(possible) 1. C --->large , chance of overfit 2. C ---> small , chance of underfitting
- 16. Gamma ● Defines how far influences the calculation of of plausible line of separation. ● Low gamma ----->points far from plausible line are considered for calculation ● High gamma ----->points close to plausible line are considered for calculation
- 17. High Gamma Value Low Gamma Value
- 18. Kernels ● Mathematical functions for transforming data ● using some linear algebra ● Different SVM algorithms use different types of kernel functions
- 19. Various kernels available 1. Linear kernel 2. Non - linear kernel 3. Radial basis function ( RBF ) 4. Sigmoid 5. Polynomial 6. Exponential
- 20. Example : K(x, y) = <f(x), f(y)> Kernel function dot product of n- dimensional inputs
- 21. Mathematical representation x = (x1, x2, x3); y = (y1, y2, y3) f(x) = (x1x1, x1x2, x1x3, x2x1, x2x2, x2x3, x3x1, x3x2, x3x3) f(y) = (y1y1, y1y2, y1y3, y2y1, y2y2, y2y3, y3y1, y3y2, y3y3) K(x, y ) = (<x, y>)² x = (1, 2, 3) y = (4, 5, 6) f(x) = (1, 2, 3, 2, 4, 6, 3, 6, 9) f(y) = (16, 20, 24, 20, 25, 30, 24, 30, 36) <f(x), f(y)> = 16 + 40 + 72 + 40 + 100+ 180 + 72 + 180 + 324 = 1024 K(x, y ) = (4 + 10 + 18)² = 1024 ----> Kernel function
- 22. Pros : ● It works really well with clear margin of separation ● It is effective in high dimensional spaces. ● It is effective in cases where number of dimensions is greater than the number of samples. ● It uses a subset of training points in the decision function (called support vectors), so it is also memory efficient.
- 23. Cons : ● It doesn’t perform well, when we have large data set because the required training time is higher ● It also doesn’t perform very well, when the data set has more noise i.e. target classes are overlapping ● SVM doesn’t directly provide probability estimates, these are calculated using an expensive five-fold cross-validation. It is related SVC method of Python scikit-learn library.
- 24. Applications : 1. Face detection 2. Text and hypertext categorization 3. Classification of images 4. Bioinformatics 5. Handwriting recognition 6. Protein fold and remote homology detection 7. Generalized predictive control(GPC)
- 25. Thank You