Deep Learning for Video: Object Tracking (UPC 2018)
5 likes•6,198 views
This document discusses deep learning approaches for video object tracking, including using fully convolutional networks to learn discriminative features for localization (conv4-3 vs conv5-3), multi-domain convolutional neural networks, correlation filter based tracking using end-to-end representation learning, recurrent neural networks for tracking, spatially supervised recurrent convolutional neural networks, and challenges in multiple object tracking. It also lists some datasets like YouTube-BB and announces upcoming deep learning courses.
![@DocXavi
Module 6
Deep Learning for Video:
Object Tracking
22nd March 2018
Xavier Giró-i-Nieto
[http://pagines.uab.cat/mcv/]](https://image.slidesharecdn.com/3-objecttracking-180326163445/85/Deep-Learning-for-Video-Object-Tracking-UPC-2018-1-320.jpg){kind=tracking}
![2
Wang, Naiyan, and Dit-Yan Yeung. "Learning a deep compact image representation for visual tracking." NIPS 2013
[Project page with code]
DNN were firstly used for feature learning to later be used by a tracker.
Object Tracking: Feature Extractor only](https://image.slidesharecdn.com/3-objecttracking-180326163445/85/Deep-Learning-for-Video-Object-Tracking-UPC-2018-2-320.jpg){kind=tracking}
![3
Wang, Lijun, Wanli Ouyang, Xiaogang Wang, and Huchuan Lu. "Visual Tracking with Fully Convolutional Networks." ICCV
2015. [code]
Object Tracking: FCNT: Localization
Despite trained for image classification, feature maps in conv5-3 enable object
localization...but are not discriminative enough to discriminate between instances of
the same class.](https://image.slidesharecdn.com/3-objecttracking-180326163445/85/Deep-Learning-for-Video-Object-Tracking-UPC-2018-3-320.jpg){kind=tracking}
![4
Wang, Lijun, Wanli Ouyang, Xiaogang Wang, and Huchuan Lu. "Visual Tracking with Fully Convolutional Networks." ICCV
2015. [code]
Object Tracking: FCNT: Localization
conv4-3 (specific) conv5-3 (general)
On the other hand, feature maps from conv4-3 are more sensitive to intra-class
appearance variation…](https://image.slidesharecdn.com/3-objecttracking-180326163445/85/Deep-Learning-for-Video-Object-Tracking-UPC-2018-4-320.jpg){kind=tracking}
![5
SNet=Specific Network (online update)
GNet=General Network (fixed)
Object Tracking: FCNT: Localization
Wang, Lijun, Wanli Ouyang, Xiaogang Wang, and Huchuan Lu. "Visual Tracking with Fully Convolutional Networks." ICCV
2015. [code]](https://image.slidesharecdn.com/3-objecttracking-180326163445/85/Deep-Learning-for-Video-Object-Tracking-UPC-2018-5-320.jpg){kind=tracking}
![9
Wang, Naiyan, and Dit-Yan Yeung. "Learning a deep compact image representation for visual tracking." NIPS 2013
[Project page with code]
Object Tracking: RNN](https://image.slidesharecdn.com/3-objecttracking-180326163445/85/Deep-Learning-for-Video-Object-Tracking-UPC-2018-9-320.jpg){kind=tracking}
![10P. Ondruska and I. Posner, “Deep Tracking: Seeing Beyond Seeing Using Recurrent Neural Networks,” AAAI 2016 [code]](https://image.slidesharecdn.com/3-objecttracking-180326163445/85/Deep-Learning-for-Video-Object-Tracking-UPC-2018-10-320.jpg){kind=tracking}
Deep Learning for Video: Object Tracking (UPC 2018)
- 1. @DocXavi Module 6 Deep Learning for Video: Object Tracking 22nd March 2018 Xavier Giró-i-Nieto [http://pagines.uab.cat/mcv/]
- 2. 2 Wang, Naiyan, and Dit-Yan Yeung. "Learning a deep compact image representation for visual tracking." NIPS 2013 [Project page with code] DNN were firstly used for feature learning to later be used by a tracker. Object Tracking: Feature Extractor only
- 3. 3 Wang, Lijun, Wanli Ouyang, Xiaogang Wang, and Huchuan Lu. "Visual Tracking with Fully Convolutional Networks." ICCV 2015. [code] Object Tracking: FCNT: Localization Despite trained for image classification, feature maps in conv5-3 enable object localization...but are not discriminative enough to discriminate between instances of the same class.
- 4. 4 Wang, Lijun, Wanli Ouyang, Xiaogang Wang, and Huchuan Lu. "Visual Tracking with Fully Convolutional Networks." ICCV 2015. [code] Object Tracking: FCNT: Localization conv4-3 (specific) conv5-3 (general) On the other hand, feature maps from conv4-3 are more sensitive to intra-class appearance variation…
- 5. 5 SNet=Specific Network (online update) GNet=General Network (fixed) Object Tracking: FCNT: Localization Wang, Lijun, Wanli Ouyang, Xiaogang Wang, and Huchuan Lu. "Visual Tracking with Fully Convolutional Networks." ICCV 2015. [code]
- 6. 6 Nam, Hyeonseob, and Bohyung Han. "Learning multi-domain convolutional neural networks for visual tracking." CVPR 2016.
- 7. 7 Nam, Hyeonseob, and Bohyung Han. "Learning multi-domain convolutional neural networks for visual tracking." CVPR 2016. Object Tracking: MDNet
- 8. 8 Valmadre, Jack, Luca Bertinetto, João F. Henriques, Andrea Vedaldi, and Philip HS Torr. "End-to-end representation learning for Correlation Filter based tracking." CVPR 2017 Object Tracking: CFNET VOT-17 Learned !!
- 9. 9 Wang, Naiyan, and Dit-Yan Yeung. "Learning a deep compact image representation for visual tracking." NIPS 2013 [Project page with code] Object Tracking: RNN
- 10. 10P. Ondruska and I. Posner, “Deep Tracking: Seeing Beyond Seeing Using Recurrent Neural Networks,” AAAI 2016 [code]
- 11. 11 Ning, Guanghan, Zhi Zhang, Chen Huang, Zhihai He, Xiaobo Ren, and Haohong Wang. "Spatially Supervised Recurrent Convolutional Neural Networks for Visual Object Tracking." ISCAS 2017. Object Tracking: ROLO
- 12. 12 Ning, Guanghan, Zhi Zhang, Chen Huang, Zhihai He, Xiaobo Ren, and Haohong Wang. "Spatially Supervised Recurrent Convolutional Neural Networks for Visual Object Tracking." ISCAS 2017.
- 13. 13 Girdhar, Rohit, Georgia Gkioxari, Lorenzo Torresani, Manohar Paluri, and Du Tran. "Detect-and-Track: Efficient Pose Estimation in Videos." CVPR 2018. Keypoint tracking
- 14. 14 Real, Esteban, Jonathon Shlens, Stefano Mazzocchi, Xin Pan, and Vincent Vanhoucke. "Youtube-boundingboxes: A large high-precision human-annotated data set for object detection in video." CVPR 2017. Object tracking: Datasets: YouTube-BB
- 15. 15 Object Tracking: Challenges Leal-Taixé, Laura, Anton Milan, Konrad Schindler, Daniel Cremers, Ian Reid, and Stefan Roth. "Tracking the trackers: an analysis of the state of the art in multiple object tracking."
- 16. ● MSc course (2017) ● BSc course (2018) 16 Deep Learning online courses by UPC: ● 1st edition (2016) ● 2nd edition (2017) ● 3rd edition (2018) ● 1st edition (2017) ● 2nd edition (2018) Next edition Autumn 2018 Next edition Winter/Spring 2019Summer School (late June 2018)