Transfer Learning and Fine Tuning for Cross Domain Image Classification with Keras
Download as PPTX, PDF15 likes7,905 views
The document presents a talk by Sujit Pal from Elsevier Labs discussing the application of transfer learning and fine-tuning techniques in cross-domain image classification using Keras. It details the use of pre-trained VGG-16 models on a dataset for diabetic retinopathy detection, showcasing various accuracy metrics and leaderboard positions achieved through different methodologies. The talk emphasizes the advantages of transfer learning in leveraging existing models to classify images more efficiently compared to training from scratch.
Transfer Learning and Fine Tuning for Cross Domain Image Classification with Keras
- 1. Presented by: Sujit Pal, Elsevier Labs November 19-20 2016 Transfer Learning and Fine Tuning for Cross Domain Image Classification with Keras Demystifying Deep Learning and Artificial Intelligence Accel.AI
- 2. | 2 • Work at Elsevier Labs • Background in Search • Path into Machine Learning Started on Natural Language Processing (NLP) to enhance search. Started on Machine Learning (ML) to help with NLP tasks. • Currently working on Image Search and Classification using Deep Learning and traditional techniques. • Have applied similar ideas using Caffe pre-trained models to classify corpus of images from medical journals. About Me
- 3. | 3 • Use Deep Convolutional Neural Networks (DCNN) trained on IMAGENET to predict image classes for a completely different domain. Problem Description Photo credits: IMAGENET collage from The Morning Paper; DR Images from Kaggle Diabetic Retinopathy Detection Challlenge
- 4. | 4 • 35,126 color images of the retina. • Labels: No DR, Mild, Moderate, Severe or Proliferative DR. • Detecting DR is hard; done by trained clinicians. • DR identified by presence of lesions on retina associated with vascular abnormality caused by the disease. • Winning entry had 0.86 Kappa score (measures agreement of predictions with labels); good as human performance. • We randomly sample 1,000 images from dataset, 200 for each class. Dataset Description Photo credits: DR Images from Kaggle Diabetic Retinopathy Detection Challlenge
- 5. | 5 • Convolution is just a matrix operation. • Enhances certain features of image. • Popular approach to image feature generation. Convolutions as Feature Generators Right Sobel Bottom Sobel
- 6. | 6 DCNN Architecture • Each layer initialized with random filter weights. • Alternating layers of convolution and pooling. • Filters (depth) increase from left to right. • Multiple filters combined at each pooling layer. • Terminated by one or more fully connected layers. • Filter weights updated by back-propagation during training.
- 7. | 7 Keras Pre-trained Models • Keras - modular, minimalistic, high level Python library for building neural networks. • Runs on top of Theano and Tensorflow. • Keras Applications (Model Zoo) contains following pre-trained models: • Xception • VGG-16 • VGG-19 • ResNet50 • InceptionV3 • We will use VGG-16 for our talk.
- 8. | 8 Keras VGG-16 Model
- 9. | 9 Keras VGG-16 Model
- 10. | 10 Transfer Learning • Pre-trained model has learned to pick out features from images that are useful in distinguishing one image (class) from another. • Initial layer filters encode edges and color, while later layer filters encode texture and shape. • Cheaper to “transfer” that learning to new classification scenario than retrain a classifier from scratch. Photo Credit: Keras Blog Post “How Convolutional Networks see the world”
- 11. | 11 Transfer Learning • Remove the Fully Connected (Bottleneck layer) from pre-trained VGG16 model. • Run images from DR Dataset through this truncated network to produce (semantic) image vectors. • Use these vectors to train another classifier to predict the labels in training set. • Prediction • Image needs to be preprocessed into image vector through truncated pre-trained VGG16 model. • Prediction made with second classifier against image vector.
- 13. | 13 Transfer Learning • Train a classifier (any classifier) using the image vectors. • Accuracy: 0.36, Cohen’s Kappa: 0.51 • Position 79-80 on Public Leaderboard (Nov 9 2016).
- 14. | 14 Transfer Learning • Single layer Neural Network gives better results. • Accuracy: 0.67, Cohen’s Kappa: 0.75 • Position 25-26 on Public Leaderboard (Nov 9 2016).
- 15. | 15 Fine Tuning • Remove bottleneck (classifier) layer from pre-trained network. • Freeze all weights except the last (few) convolutional layers. • Attach our own classifier to the bottom. • Train the resulting classifier with very low learning rate. • Computationally more expensive than Transfer Learning but still cheaper than training network from scratch. • More robust model.
- 16. | 16 Fine Tuning • Accuracy: 0.62, Cohen’s Kappa: 0.74 • Position 26-27 on Public Leaderboard (Nov 9 2016)
- 17. | 17 Fine Tuning • Improvement – update weights of top classifier using learned weights from Transfer Learning classifier. • Fewer Epochs needed for convergence. • Accuracy: 0.63, Cohen’s Kappa: 0.72 • Position 32-33 on Public Leaderboard (Nov 9 2016)
- 18. | 18 • Code for this talk - https://github.com/sujitpal/fttl-with-keras • My Email Address: [email protected] Code and Contact Info
- 19. | 19 Thank you