kaggle-global-wheat-detection:Kaggle全球小麦检测的第9名解决方案

# :ear_of_rice: 9th Place Solution of [Global Wheat Detection](https://www.kaggle.com/c/global-wheat-detection)  - Our team: [Miras Amir](https://www.linkedin.com/in/amirassov/), [Or Katz](https://www.linkedin.com/in/or-katz-9ba885114/), [Shlomo Kashani](https://www.linkedin.com/in/quantscientist) - [Kaggle post](https://www.kaggle.com/c/global-wheat-detection/discussion/172569) - Submission kernel: [pseudo ensemble: detectors (3 st)+universenet r10](https://www.kaggle.com/amiras/pseudo-ensemble-detectors-3-st-universenet-r10) # Solution ## Summary Our solution is based on the excellent [MMDetection framework](https://github.com/open-mmlab/mmdetection). We trained an ensemble of the following models: - [DetectoRS with the ResNet50 backbone](https://github.com/joe-siyuan-qiao/DetectoRS) - [UniverseNet+GFL with the Res2Net101 backbone](https://github.com/shinya7y/UniverseNet) To increase the score a single round of pseudo labelling was applied to each model. Additionally, for a much better generalization of our models, we used heavy augmentations.  ## Jigsaw puzzles In the original corpus provided by the organizers, the training images were cropped from an original set of larger images. Therefore, we collected and assembled the original puzzles resulting in a corpus of 1330 puzzle images. The puzzle collection algorithm we adopted was based on [this code](https://github.com/lRomul/argus-tgs-salt/blob/master/mosaic/create_mosaic.py). But we were unsuccessful in collecting the bounding boxes for puzzles. Mainly because of the existence of bounding boxes that are located on or in the vicinity the border of the image. For this reason, we generated crops for the puzzles offline in addition to training images and generated boxes for them using pseudo labelling. ## Validation approach We used MultilabelStratifiedKFold with 5 folds of [iterative stratification](https://github.com/trent-b/iterative-stratification) stratified by the number of boxes, a median of box areas and source of images. We guaranteed that there isn’t any leak between the sub-folds, so that the images of one puzzle were used only in that one particular fold. Referring to the [paper](https://arxiv.org/abs/2005.02162), one can see wheat heads from different sources. We assumed that the wheat heads of `usask_1, ethz_1` sources are very different from the test sources (`UTokyo_1, UTokyo_2, UQ_1, NAU_1`). Therefore, we did not use these sources for validation. However, our validation scores did not correlate well with the Kaggle LB. We only noticed global improvements (for example, DetectoRS is better than UniverseNet). Local improvements such as augmentation parameters, WBF parameters etc. did not correlate. We, therefore, shifted our attention to the LB scores mainly. We trained our models only on the first fold. ## Augmentations Due to the relatively small size of our training set, and another test set distribution, our approach relied heavily on data augmentation. During training, we utilized an extensive data augmentation protocol: - Various augmentations from [albumentations](https://albumentations.ai): - HorizontalFlip, ShiftScaleRotate, RandomRotate90 - RandomBrightnessContrast, HueSaturationValue, RGBShift - RandomGamma - CLAHE - Blur, MotionBlur - GaussNoise - ImageCompression - CoarseDropout - RandomBBoxesSafeCrop. Randomly select N boxes in the image and find their union. Then we cropped the image keeping this unified. - [Image colorization](https://www.kaggle.com/orkatz2/pytorch-pix-2-pix-for-image-colorization)  - [Style transfer](https://github.com/bethgelab/stylize-datasets). A random image from a small test (10 images) was used as a style.  - Mosaic augmentation. `a, b, c, d` -- randomly selected images. Then we just do the following: ``` top = np.concatenate([a, b], axis=1) bottom = np.concatenate([c, d], axis=1) result = np.concatenate([top, bottom], axis=0) ``` - Mixup augmentation. `a, b` -- randomly selected images. Then: `result = (a + b) / 2` - Multi-scale Training. In each iteration, the scale of image is randomly sampled from `[(768 + 32 * i, 768 + 32 * i) for i in range(25)]`. - All augmentations except colorization and style transfer were applied online. Examples of augmented images:  |  :-------------------------:|:-------------------------:  |  ## External data: [SPIKE dataset](https://www.kaggle.com/c/global-wheat-detection/discussion/164346) ## Models We used DetectoRS with ResNet50 and UniverseNet+GFL with Res2Net101 as main models. DetectoRS was a little bit more accurate and however much slower to train than UniverseNet: - Single DetectoRS Public LB score without pseudo labeling: 0.7592 - Single UniverseNet Public LB score without pseudo labeling: 0.7567 For DetectoRS we used: - LabelSmoothCrossEntropyLoss with parameter `0.1` - [Empirical Attention](https://github.com/open-mmlab/mmdetection/tree/master/configs/empirical_attention) ## Training pipeline In general, we used a multi-stage training pipeline:  ## Model inference We used TTA6 (Test Time Augmentation) for all our models: - Multi-scale Testing with scales `[(1408, 1408), (1536, 1536)]` - Flips: `[original, horizontal, vertical]` For TTA was used a standard MMDet algorithm with NMS that looks like this for two-stage detectors (DetectoRS):  For one-stage detectors (UniverseNet), the algorithm is similar, only without the part with RoiAlign, Head, etc. ## Pseudo labelling - Sampling positive examples. We predicted the test image and received its scores and the bounding boxes. Then we calculated `confidence = np.mean(scores > 0.75)`. If the confidence was greater than 0.6 we accepted this image and used for pseudo labelling. - Sources `[usask_1, ethz_1]` and augmentations like mosaic, mixup, colorization, style transfer weren’t used for pseudo labelling. - 1 epoch, 1 round, 1 stage. - Data: original data + pseudo test data :heavy_multiplication_x: 3 ## Ensemble We used [WBF](https://github.com/ZFTurbo/Weighted-Boxes-Fusion) for the ensemble. The distribution of DetectoRS and UniverseNet scores is different. So we applied scali