![What makes object detection harder
than image classification?
5
http://cocodataset.org/
http://www.image-net.org/
MSCOCO
ImageNet
[20] A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural
networks. In Advances in neural information processing systems, pages 1097–1105, 2012.
[16] J. Hu, L. Shen, and G. Sun. Squeeze-and-excitation networks.
In less than five years,
the top-5 error on ImageNet 15%[20] to 2%[16]
The mAP of the best performing detector [18] COCO
[25] is only 62% – even at 50% overlap.
# of classes in COCO = 80 # of classes in Image = 1000](https://image.slidesharecdn.com/pr110-181027044208/85/PR-110-An-Analysis-of-Scale-Invariance-in-Object-Detection-SNIP-5-320.jpg)
![31
[0, 80]
[40, 160]
[120, ∞]](https://image.slidesharecdn.com/pr110-181027044208/85/PR-110-An-Analysis-of-Scale-Invariance-in-Object-Detection-SNIP-31-320.jpg)
![32
[0, 80]
[40, 160]
[120, ∞]](https://image.slidesharecdn.com/pr110-181027044208/85/PR-110-An-Analysis-of-Scale-Invariance-in-Object-Detection-SNIP-32-320.jpg)
![35
MS COCO dataset has
- Most small objects (Median 0.106)
- Large scale variation (20x)
- Large domain shift from pre-trained classification network
[0, 80]
[40, 160]
[120, ∞]](https://image.slidesharecdn.com/pr110-181027044208/85/PR-110-An-Analysis-of-Scale-Invariance-in-Object-Detection-SNIP-35-320.jpg)
PR-110: An Analysis of Scale Invariance in Object Detection – SNIP
- 1. visionNoob (Jaewon Lee) PR-110 An Analysis of Scale Invariance in Object Detection – SNIP Singh, B., & Davis, L. S. CVPR’18 1 https://arxiv.org/abs/1711.08189
- 2. 2 References for Object Detection PR-002: Deformable Convolutional Networks (2017) PR-012: Faster R-CNN : Towards Real-Time Object Detection with Region Proposal PR-016: You only look once: Unified, real-time object detection PR-023: YOLO9000: Better, Faster, Stronger PR-033: PVANet: Lightweight Deep Neural Networks for Real-time Object Detection PR-057: Mask R-CNN PR-084: MegDet: A Large Mini-Batch Object Detector (CVPR2018)
- 3. 3 MegDet: A Large Mini-Batch Object Detector(https://arxiv.org/abs/1711.07240) Path Aggregation Network for Instance Segmentation (https://arxiv.org/abs/1803.01534) Deformable ConvNets + Xception Mask RCNN + Feature Pyramid Networks(FPN) + ResNeXt Ensemble of multiple models using unlabeled data with multiple scales. (today!) An Analysis of Scale Invariance in Object Detection – SNIP MS COCO Results
- 4. What makes object detection harder than image classification? 4
- 5. What makes object detection harder than image classification? 5 http://cocodataset.org/ http://www.image-net.org/ MSCOCO ImageNet [20] A. Krizhevsky, I. Sutskever, and G. E. Hinton. Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pages 1097–1105, 2012. [16] J. Hu, L. Shen, and G. Sun. Squeeze-and-excitation networks. In less than five years, the top-5 error on ImageNet 15%[20] to 2%[16] The mAP of the best performing detector [18] COCO [25] is only 62% – even at 50% overlap. # of classes in COCO = 80 # of classes in Image = 1000
- 6. 6 Relative Scale = 𝑠𝑞𝑟𝑡(𝑎𝑟𝑒𝑎 𝑂𝑏𝑗𝑒𝑐𝑡 ) 𝑠𝑞𝑟𝑡(𝑎𝑟𝑒𝑎 𝐼𝑚𝑎𝑔𝑒 ) MS COCO dataset has - Most small objects (Median 0.106) - Large scale variation (20x) - Large domain shift from pre-trained classification network
- 7. 7 Huang, Jonathan, et al. "Speed/accuracy trade-offs for modern convolutional object detectors." IEEE CVPR. Vol. 4. 2017. Current Practices for Object Detection
- 8. Convolution Neural Networks for Classification 8Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. "Imagenet classification with deep convolutional neural networks." Advances in neural information processing systems. 2012. Spatial resolution which contain high-level semantic feature is much lower -> Make Object Detection harder
- 9. 9 (PR-002)
- 10. 10 Tutorial_ Deep Learning for Objects and Scenes
- 11. 11
- 12. 12
- 13. 13 Current Practices for Object Detection Huang, Jonathan, et al. "Speed/accuracy trade-offs for modern convolutional object detectors." IEEE CVPR. Vol. 4. 2017. High resolution models lead to significantly better mAP result on small object (note that typical resolution in coco is 640 x 480)
- 14. 14
- 15. Are CNNs robust to up-sampling? 15
- 16. 16
- 17. 17
- 18. 18
- 19. 19
- 20. 20
- 21. 21
- 22. 22
- 23. 23
- 24. 24
- 25. 25
- 27. 27
- 28. 28 Pretrained classification network : 224 x 224 Original : 640 x 480 Inference : 1400 x 2000 AP small objects
- 29. 29 Reduce variation in scale without total number of training samples!
- 30. 30
- 31. 31 [0, 80] [40, 160] [120, ∞]
- 32. 32 [0, 80] [40, 160] [120, ∞]
- 33. 33 too small small medium large too large Large data variation Large scale variation Out of Receptive field Too low spatial resolution
- 34. medium Normalize Scale 34 Large data variation Small scale variation
- 35. 35 MS COCO dataset has - Most small objects (Median 0.106) - Large scale variation (20x) - Large domain shift from pre-trained classification network [0, 80] [40, 160] [120, ∞]
- 36. 36
- 37. 37
- 38. 38
- 39. 39 Singh, Bharat, Mahyar Najibi, and Larry S. Davis. "SNIPER: Efficient Multi-Scale Training." arXiv preprint arXiv:1805.09300(2018).
- 40. Q&A 40