Object Detection Beyond Mask R-CNN and RetinaNet II
1 like803 views
The document summarizes recent advances in AutoML for object detection. It discusses how researchers have applied neural architecture search to automatically search for optimal backbone networks, feature fusion modules, and data augmentation policies for object detection models. For backbone search, methods like DetNAS have used one-shot NAS approaches with evolutionary algorithms to efficiently search for detection backbones. NAS-FPN searched for scalable feature pyramid architectures. Auto-Augment applied NAS to learn the best image augmentation strategies for object detection. These AutoML techniques have led to state-of-the-art detection models that outperform manually designed counterparts.
Object Detection Beyond Mask R-CNN and RetinaNet II
- 1. AutoML for Object Detection Xiangyu Zhang MEGVII Research
- 2. AutoML for Object Detection • Advances in AutoML • Search for Detection Systems 2 1
- 3. AutoML for Object Detection • Advances in AutoML • Search for Detection Systems 2 1
- 4. Introduction v AutoML o A meta-approach to generate machine learning systems o Automatically search vs. manually design v AutoML for Deep Learning o Neural architecture search (NAS) o Hyper-parameters turning o Loss function o Data augmentation o Activation function o Backpropagation …
- 5. Revolution of AutoML v ImageNet 2012 - o Hand-craft feature vs. deep learning v Era of Deep Learning begins! 27 26.2 16.4 8.1 7.3 6.6 4.9 3.57 OXFORD ISI AlexNet SPPnet VGG GoogleNet PReLU ResNet 152 Classification Top-5 Error (%)
- 6. Revolution of AutoML (cont’d) v ImageNet 2017 - o Manual architecture vs. AutoML models 19.1 17.3 17.3 17.1 16.1 15.6 ResNeXt-101 SENet NASNet-A PNASNet-5 AmoebaNet-A EfficientNet Classification Top-1 Error (%) Era of AutoML?
- 7. Revolution of AutoML (cont’d) v Literature o 200+ since 2017
- 8. Revolution of AutoML (cont’d) v Literature o 200+ since 2017 v Google Trends
- 9. Recent Advances in AutoML (1) v Surpassing handcraft models o NASNet v Keynotes o RNN controller + policy gradient o Flexible search space o Proxy task needed Zoph et al. Learning Transferable Architectures for Scalable Image Recognition Zoph et al. Neural Architecture Search with Reinforcement Learning
- 10. Recent Advances in AutoML (2) v Search on the target task o MnasNet v Keynotes o Search directly on ImageNet o Platform aware search o Very costly (thousands of TPU-days) Tan et al. MnasNet: Platform-Aware Neural Architecture Search for Mobile
- 11. Recent Advances in AutoML (3) v Weight Sharing for Efficient Search & Evaluation o ENAS o One-shot methods v Keynotes o Super network o Finetuning & inference only instead of retraining o Inconsistency in super net evaluation Pham et al. Efficient Neural Architecture Search via Parameter Sharing Bender et al. Understanding and Simplifying One-Shot Architecture Search Guo et al. Single Path One-Shot Neural Architecture Search with Uniform Sampling
- 12. Recent Advances in AutoML (4) v Gradient-based methods o DARTS o SNAS, FBNet, ProxylessNAS, … v Keynotes o Joint optimization of architectures and weights o Weight sharing implied o Sometimes less flexible Liu et al. DARTS: Differentiable Architecture Search Xie et al. SNAS: Stochastic Neural Architecture Search Cai et al. ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardware Wu et al. FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search
- 13. Recent Advances in AutoML (5) v Performance Predictor o Neural Architecture Optimization o ChamNet v Keynotes o Architecture encoding o Performance prediction models o Cold start problem Luo et al. Neural Architecture Optimization Dai et al. ChamNet: Towards Efficient Network Design through Platform-Aware Model Adaptation
- 14. Recent Advances in AutoML (6) v Hardware-aware Search o Search with complexity budget o Quantization friendly o Energy-aware search … v Keynotes o Complexity-aware loss & reward o Multi-target search o Device in the loop Wu et al. Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Search V´eniat et al. Learning Time/Memory-Efficient Deep Architectures with Budgeted Super Networks Wang et al. HAQ: Hardware-Aware Automated Quantization with Mixed Precision
- 15. Recent Advances in AutoML (7) v AutoML in Model Pruning o NetAdapt o AMC o MetaPruning v Keynotes o Search for the pruned architecture o Hyper-parameters like channels, spatial size, … Yang et al. NetAdapt: Platform-Aware Neural Network Adaptation for Mobile Applications He et al. AMC: AutoML for Model Compression and Acceleration on Mobile Devices Liu et al. MetaPruning: Meta Learning for Automatic Neural Network Channel Pruning
- 16. Recent Advances in AutoML (8) v Handcraft + NAS o Human-expert guided search (IRLAS) o Boosting existing handcraft models (EfficientNet, MobileNet v3) v Keynotes o Very competitive performance o Efficient o Search space may be restricted Howard et al. Searching for MobileNetV3 Tan et al. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks Guo et al. IRLAS: Inverse Reinforcement Learning for Architecture Search
- 17. Recent Advances in AutoML (9) v Various Tasks o Object Detection o Semantic Segmentation o Super-resolution o Face Recognition … Liu et al. Auto-DeepLab: Hierarchical Neural Architecture Search for Semantic Image Segmentation Chu et al. Fast, Accurate and Lightweight Super-Resolution with Neural Architecture Search Ramachandra et al. Searching for Activation Functions Alber et al. Backprop Evolution v Not only NAS, search for everything! o Activation function o Loss function o Data augmentation o Backpropagation …
- 18. Recent Advances in AutoML (10) v Rethinking the Effectiveness of NAS o Random search o Random wire network v Keynotes o Reproducibility o Search algorithm or search space? o Baselines Li et al. Random Search and Reproducibility for Neural Architecture Search Xie et al. Exploring Randomly Wired Neural Networks for Image Recognition
- 19. Summary: Trends and Challenges v Trends o Efficient & high-performance algorithm o Flexible search space o Device-aware optimization o Multi-task / Multi-target search v Challenges o Trade-offs between efficiency, performance and flexibility o Search space matters! o Fair benchmarks o Pipeline search Efficiency FlexibilityPerformance
- 20. AutoML for Object Detection • Advances in AutoML • Search for Detection Systems 2 1
- 21. AutoML for Object Detection v Components to search o Image preprocessing o Backbone o Feature fusion o Detection head & loss function …
- 22. AutoML for Object Detection v Components to search o Image preprocessing o Backbone o Feature fusion o Detection head & loss function …
- 23. AutoML for Object Detection v Components to search o Image preprocessing o Backbone o Feature fusion o Detection head & loss function …
- 24. AutoML for Object Detection v Components to search o Image preprocessing o Backbone o Feature fusion o Detection head & loss function …
- 25. AutoML for Object Detection v Components to search o Image preprocessing o Backbone o Feature fusion o Detection head & loss function …
- 26. Search for Detection Systems Feature Fusion Augmentation DetNAS Chen et al. DetNAS: Backbone Search for Object Detection
- 27. Challenges of Backbone Search v Similar to general NAS, but … o Controller & evaluator loop o Performance evaluation is very slow v Detection backbone evaluation involves a costly pipeline o ImageNet pretraining o Finetuning on the detection dataset (e.g. COCO) o Evaluation on the validation set
- 28. Related Work: Single Path One-shot NAS v Decoupled weight training and architecture optimization v Super net training Guo et al. Single Path One-Shot Neural Architecture Search with Uniform Sampling
- 29. Pipeline v Single-pass approach o Pretrain and finetune super net only once
- 30. Search Space v Single path super net o 20 (small) or 40 (large) choice blocks o 4 candidates for each choice block o Search space size: 420 or 440
- 31. Search Algorithm v Evolutionary search o Sample & reuse the weights from super net o Very efficient
- 32. Results v High performance o Significant improvements over commonly used backbones (e.g. ResNet 50) with fewer FLOPs o Best classification backbones may be suboptimal for object detection
- 33. Results v Search cost o Super nets greatly speed up search progress!
- 34. Search for Detection Systems Backbone Feature Fusion Augmentation NAS-FPN Ghaisi et al. NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection
- 35. Feature Fusion Modules v Multi-scale feature fusion o Used in state-of-the-art detectors (e.g. SSD, FPN, SNIP, FCOS, …) v Automatic search vs. manual design
- 36. First Glance v Searched architecture o Very different from handcraft structures
- 37. Search Space v Stacking repeated FPN blocks v For each FPN block, N different merging cells v For each merging cell, 4-step generations
- 38. Search Algorithm v Controller o RNN-based controller o Search with Proximal Policy Optimization (PPO) v Candidate evaluation o Training a light-weight proxy task
- 39. Architectures During Search v Many downsamples and upsamples
- 40. Results v State-of-the-art speed/AP trade-off
- 41. Search for Detection Systems Backbone Feature Fusion Augmentation Auto-Augment for Detection Zoph et al. Learning Data Augmentation Strategies for Object Detection
- 42. Data Augmentation for Object Detection v Augmentation pool o Color distortions o Geometric transforms o Random noise (e.g. cutout, drop block, …) o Mix-up … v Search for the best augmentation configurations
- 43. Search Space Design v Mainly follows AutoAugment v Randomly sampling from K sub-policies v For each sub-policy, N image transforms v Each image transform selected from 22 operations: o Color operations o Geometric operations o Bounding box operations Cubuk et al. AutoAugment: Learning Augmentation Strategies from Data
- 44. Search Space Design (cont’d)
- 45. Search Algorithm v Very similar to NAS-FPN v Controller o RNN-based controller o Search with Proximal Policy Optimization (PPO) v Evaluation o A small proxy dataset o Short-time training
- 46. Results v Significantly outperforms previous state-of-the-arts
- 48. Future Work v More search dimensions o E.g. loss, anchor boxes, assign rules, post-processing, … v Reducing search cost v Joint optimization
- 49. Q & A