Deep learning for Computer Vision intro
- 1. Intro to Deep Lerning for Computer Vision Nadav Carmel
- 2. Highlights ï” Common CV tasks ï” CNNâs â intro ï” Filters, maxpools, simple example ï” Normalization types ï” Inception network ï” Object detection ï” R-CNN ï” YOLO ï” Face recognition ï” One shot learning ï” Siamese net
- 3. Computer Vision problems ï” Image classification ï” Object detection ï” Face recognition ï” Segmentation ï” Style transfer ï” But there are more!! Weâll focus on these
- 4. CNNâs into
- 5. Convolutional filter concept â recap ï” Each conv-layer has 4 params: ï” Filter size (filter height = filter width) ï” stride ï” Input channels ï” Output channels (number of filters) 6 convolutional filters
- 6. Max-pool concept â recap ï” Usally has 2 params: ï” Filter size (filter height = filter width) ï” Stride ï” Operation is done per channel (thus: đ¶đđ = đ¶ đđąđĄ) ï” It is a non-learnble filter
- 8. Normalization types ï” We sometimes want the data at each layer to be normalized ï” Improves learning speed and robustness ï” There are few types of normalizations: ï” H, W: image size ï” N: batch size ï” C: cannels
- 9. Batch Norm
- 10. Inception network: âą There are many architectural questions when designing CNNâs: ï” What filter size to choose? (larger one = better spatial representation, smaller one = lower computational complexity) ï” Add maxpool or not? ï” Etc. ï” One approach of handeling these question is: letâs try everything!
- 13. Object detection
- 14. Object detection ï” Some of the most important CV tasks include: ï” Object detection = classification + localization of multiple objects ï” Detection model output: đŠ = đđ, đ đ„ , đ đŠ , đâ , đ đ€ , đ1 , đ2 , đ3
- 16. Region (sliding window) CNN ï” A reagion proposal (selective search) algorithm suggests regions for the bounding box to go over (Ross Girshick et al.) ï” These candidate boxes are resized to match the CNN input size ï” They are then fed into the convolutional neural network that produces a features vector ï” The feature vector is fed into SVM to produce the classification ï” Finally, remove boxes with the highest shared area in a process called non-max suppression
- 17. You Only Look Once - YOLO ï” Most object detection algorithms use regions to localize the object within the image, and do not look at the complete image ï” In YOLO a convolutional network uses the entire image to predicts the bounding boxes and the class probabilities for these boxes
- 18. YOLO algo description 1. Split the image into grid of cells 2. Each cell is responsible for predicting a number bounding boxes (should match the number of objects in the cell) 3. Run the model once to get all cells predictions 4. Remove boxes with the highest shared area in a process called non-max suppression
- 19. YOLO
- 20. YOLO summary ï” Main algorithm properties include: ï” Extremely fast inference â makes predictions with a single network evaluation, unlike R- CNN which requires thousands for a single image ï” Since it looks at the whole image at once, its predictions are informed by global context in the image ï” Requires bounding-box tagging for training ï” Since the model learns to predict bounding boxes (đ đ„, đ đŠ, đâ, đ đ€) from the data, it struggles to generalize to objects in new or unusual aspect ratios or configurations
- 21. Non-max suppression algorithm ï” Remove all boxes with đđ < 0.6 ï” While there are any remaining boxes: ï” Pick the box with the largest Pc - output it as prediction ï” Discard any box with IOU > 0.5 with the box from previous step
- 22. Face recognition
- 23. One shot learning ï” Say we want to have a classification system to recognize faces ï” We only have 1 or 2 images of each person ï” One aproach can be to train a CNN which maps the inputs to a (one hot) label vector, where each element corresponds to each person ï” BUT: ï” Train a neural net with only 1 or 2 imaages per class will highly overfit ï” Each new person in the âpoolâ will require a new, longer, output vector (and system retraining)
- 24. One shot learning ï” Instead of learining a multiclass classifier, we can learn a similarity function: đ đđđ1, đđđ2 ï” đŒđ: đ đđđ1, đđđ2 â„ đ â âđ đđđâ ï” đđđ đ: â âđđđđđđđđđĄâ
- 25. Siamese network ï” We want a network that predicts the similarity between 2 faces ï” We want each of the images to be encoded in a low-dimensional representation, then fed into the network ï” The most common encoding in this case is computed via a Siamese-network
- 26. Triplet loss ï” We want to train the siamese nets in such way that: ï” Differentnt images of the same person will have very similar representations ï” Images of different persons will have very different representations ï” We define a triplet loss objective: đż = đđđ„ đ đ, đ â đ đ, đ + đđđđđđ, 0
- 27. ï” Thank you!