End to-end convolutional network for saliency prediction
0 likes2,087 views
The document discusses an end-to-end convolutional network, called JuntingNet, for saliency prediction, presented at the Large-Scale Scene Understanding Challenge 2015. It includes acknowledgments, related works, details on the dataset used for training, architecture, framework, and results of the saliency prediction model. The paper highlights quantitative and qualitative results from the model applied to the ISUN and Salicon datasets, along with technical specifications and training methodology.
![4
LSUN SALIENCY CHALLENGE: A Déjà vu ?
[Mohedano’14]](https://image.slidesharecdn.com/lsun2015end-to-endconvolutionalnetworkforsaliencyprediction-150706130345-lva1-app6892/85/End-to-end-convolutional-network-for-saliency-prediction-4-320.jpg)
![9
RELATED WORK: Image Classification
CaffeNet
ARCHITECTURE
[Khrizevsky’12]
DATA
[Deng’09]
FRAMEWORK
[Jia’14]](https://image.slidesharecdn.com/lsun2015end-to-endconvolutionalnetworkforsaliencyprediction-150706130345-lva1-app6892/85/End-to-end-convolutional-network-for-saliency-prediction-9-320.jpg)
![11
SALIENCY PREDICTION: JuntingNet
JuntingNet
DATA
iSun [Xu’15]
SALICON [Jiang’15]](https://image.slidesharecdn.com/lsun2015end-to-endconvolutionalnetworkforsaliencyprediction-150706130345-lva1-app6892/85/End-to-end-convolutional-network-for-saliency-prediction-11-320.jpg)
![12
SALIENCY PREDICTION: Data
TRAIN VALIDATION TEST
SALICON [Jiang’15] 10,000 5,000 5,000
iSun [Xu’15] 6,000 926 2,000
CAT2000 [Borji’15] 2,000 - 2,000
MIT300 [Judd’12] 300 . -
Large
Scale](https://image.slidesharecdn.com/lsun2015end-to-endconvolutionalnetworkforsaliencyprediction-150706130345-lva1-app6892/85/End-to-end-convolutional-network-for-saliency-prediction-12-320.jpg)
![13
SALIENCY PREDICTION: JuntingNet
JuntingNet
ARCHITECTURE
[Pan’15]
DATA
iSun [Xu’15]
SALICON [Jiang’15]](https://image.slidesharecdn.com/lsun2015end-to-endconvolutionalnetworkforsaliencyprediction-150706130345-lva1-app6892/85/End-to-end-convolutional-network-for-saliency-prediction-13-320.jpg)
![17
SALIENCY PREDICTION: JuntingNet
JuntingNet
ARCHITECTURE
[Pan’15] (soon)
DATA
iSun [Xu’15]
SALICON [Jiang’15]
FRAMEWORK
[Bergstra’10]
[Bastien’12]](https://image.slidesharecdn.com/lsun2015end-to-endconvolutionalnetworkforsaliencyprediction-150706130345-lva1-app6892/85/End-to-end-convolutional-network-for-saliency-prediction-17-320.jpg)
End to-end convolutional network for saliency prediction
- 1. End-to-end convolutional network for saliency prediction Junting Pan Xavier Giró-i-Nieto Slides online @DocXavi Large-scale Scene Understanding (LSUN) Challenge 2015 http://bit.ly/juntingnet
- 2. 2 Financial supportTechnical support Albert Gil Josep Pujal ACKNOWLEDGMENTS
- 3. 3 LSUN SALIENCY CHALLENGE: A Déjà vu ? John Markoff, “Scientists see promise in deep learning Programs”, The New York Times (Nov2012). Photo: Keith Penner
- 4. 4 LSUN SALIENCY CHALLENGE: A Déjà vu ? [Mohedano’14]
- 5. 5 LSUN SALIENCY CHALLENGE: A Déjà vu ?
- 6. 6 RELATED WORK: Deep Saliency Kümmerer, Matthias, Lucas Theis, and Matthias Bethge. "Deep Gaze I: Boosting Saliency Prediction with Feature Maps Trained on ImageNet." arXiv preprint arXiv:1411.1045 (2014).
- 7. 7 RELATED WORK: Deep Saliency Vig, Eleonora, Michael Dorr, and David Cox. "Large-scale optimization of hierarchical features for saliency prediction in natural images." Computer Vision and Pattern Recognition (CVPR), 2014 IEEE Conference on. IEEE, 2014.
- 8. 8 RELATED WORK: Fully convolutional Long, Jonathan, Evan Shelhamer, and Trevor Darrell. "Fully convolutional networks for semantic segmentation." Computer Vision and Pattern Recognition (CVPR), 2015 IEEE Conference on. IEEE, 2015.
- 9. 9 RELATED WORK: Image Classification CaffeNet ARCHITECTURE [Khrizevsky’12] DATA [Deng’09] FRAMEWORK [Jia’14]
- 11. 11 SALIENCY PREDICTION: JuntingNet JuntingNet DATA iSun [Xu’15] SALICON [Jiang’15]
- 12. 12 SALIENCY PREDICTION: Data TRAIN VALIDATION TEST SALICON [Jiang’15] 10,000 5,000 5,000 iSun [Xu’15] 6,000 926 2,000 CAT2000 [Borji’15] 2,000 - 2,000 MIT300 [Judd’12] 300 . - Large Scale
- 13. 13 SALIENCY PREDICTION: JuntingNet JuntingNet ARCHITECTURE [Pan’15] DATA iSun [Xu’15] SALICON [Jiang’15]
- 15. 15 SALIENCY PREDICTION: Architecture End to end + regression = JuntingNet
- 16. 16 SALIENCY PREDICTION: Architecture Resize 96x96 Upsample + filter 4608 = 48x48 2D map
- 17. 17 SALIENCY PREDICTION: JuntingNet JuntingNet ARCHITECTURE [Pan’15] (soon) DATA iSun [Xu’15] SALICON [Jiang’15] FRAMEWORK [Bergstra’10] [Bastien’12]
- 18. 18 SALIENCY PREDICTION: Framework Tutorial by Daniel Nouri (*) on regression for facial points for Kaggle. (*) Daniel Nouri, “Using convolution networks to detect facil points” (Dec 2014). on Lasagne
- 19. 19 SALIENCY PREDICTION: Training Data augmentation with horizontal mirroring.
- 20. 20 SALIENCY PREDICTION: Training Loss function Mean Square Error (MSE) Weight initialization Gaussian distribution Learning rate 0.03 to 0.0001 Mini batch size 128 Training time 7h (SALICON) / 3h (iSUN) Acceleration Sigmoid + nesterov momentum 0.9 Regularisation Maxout norm GPU NVidia GTX 980
- 21. 21 RESULTS: Qualitative (iSUN) JuntingNetGround TruthPixels
- 22. 22 RESULTS: Qualitative (iSUN) JuntingNetGround TruthPixels
- 24. 24 RESULTS: Qualitative (SALICON) JuntingNetGround TruthPixels
- 25. 25 RESULTS: Qualitative (SALICON) JuntingNetGround TruthPixels
- 27. 27 RESULTS: Publications by end of June http://bit.ly/juntingnet
- 28. 28 Thank you LSUN ! Thank you Boston ! http://bit.ly/juntingnetSlides online @DocXavi