Before everything starts, I would like to say a big thank you to kaggle and the contest organizers for creating such an admirable contest.
\nMy heartfelt gratitude goes to my teammates @zephyruszx and @calvchen for their incredible support throughout the competition.
\nWe chose Nvidia Dali, which can decode dicom image files using the GPU, and generated uint16 bit png files. We did not use more complex algorithms for crop, but just cv2.connectedComponentsWithStats, which works very well and is fast.
image size: 1536*896
\nCross-validation strategy：StratifiedGroupKFold 5-Folds
negative samples strategy : 35%~50% of negative samples downsampled
\nData augmentation strategies with different levels of LIGHT and HEAVY:
\n ():\n \n alpha > , \n x.shape[] > , \n\n lam = np.random.beta(alpha, alpha)\n rand_idx = torch.randperm(x.shape[])\n\n mixed_x = lam * x + ( - lam) * x[rand_idx, :]\n yc_j, yc_k = yc, yc[rand_idx]\n\n mixed_x, yc_j, yc_k, lam\n\n ():\n data == :\n Compose([\n ToFloat(max_value=),\n RandomResizedCrop(img_size[], img_size[], scale=(, ), ratio=(, ), p=), \n HorizontalFlip(p=),\n VerticalFlip(p=),\n ShiftScaleRotate(rotate_limit=(-, ), p=),\n RandomBrightnessContrast(brightness_limit=(-,), contrast_limit=(-, ), p=),\n JpegCompression(quality_lower=, quality_upper=, p=),\n Affine(p=),\n ToTensorV2(),\n ])\n\n data == :\n Compose([\n ToFloat(max_value=),\n Resize(img_size[], img_size[]),\n ToTensorV2(),\n ])\nPseudo label: We simply used the Vindr data as an external dataset, using almost the same data processing approach.
\nBecause of our training time, we chose the smallest version of the major models to try as much as possible, and ended up using three models Ensemble, which have a number of parameters between 15M and 21M, with broadly similar model training parameters, with some tuning of the parameters, which according to our experiments have a huge and sensitive impact on the learning rate.
\nAfter Backbone, we choose GeM Pooling, p_trainable=True, and add dropout of fc layer.
Before everything starts, I would like to say a big thank you to kaggle and the contest organizers for creating such an admirable contest.
\nMy heartfelt gratitude goes to my teammates @zephyruszx and @calvchen for their incredible support throughout the competition.
\nWe chose Nvidia Dali, which can decode dicom image files using the GPU, and generated uint16 bit png files. We did not use more complex algorithms for crop, but just cv2.connectedComponentsWithStats, which works very well and is fast.
image size: 1536*896
\nCross-validation strategy：StratifiedGroupKFold 5-Folds
negative samples strategy : 35%~50% of negative samples downsampled
\nData augmentation strategies with different levels of LIGHT and HEAVY:
\n ():\n \n alpha > , \n x.shape[] > , \n\n lam = np.random.beta(alpha, alpha)\n rand_idx = torch.randperm(x.shape[])\n\n mixed_x = lam * x + ( - lam) * x[rand_idx, :]\n yc_j, yc_k = yc, yc[rand_idx]\n\n mixed_x, yc_j, yc_k, lam\n\n ():\n data == :\n Compose([\n ToFloat(max_value=),\n RandomResizedCrop(img_size[], img_size[], scale=(, ), ratio=(, ), p=), \n HorizontalFlip(p=),\n VerticalFlip(p=),\n ShiftScaleRotate(rotate_limit=(-, ), p=),\n RandomBrightnessContrast(brightness_limit=(-,), contrast_limit=(-, ), p=),\n JpegCompression(quality_lower=, quality_upper=, p=),\n Affine(p=),\n ToTensorV2(),\n ])\n\n data == :\n Compose([\n ToFloat(max_value=),\n Resize(img_size[], img_size[]),\n ToTensorV2(),\n ])\nPseudo label: We simply used the Vindr data as an external dataset, using almost the same data processing approach.
\nBecause of our training time, we chose the smallest version of the major models to try as much as possible, and ended up using three models Ensemble, which have a number of parameters between 15M and 21M, with broadly similar model training parameters, with some tuning of the parameters, which according to our experiments have a huge and sensitive impact on the learning rate.
\nAfter Backbone, we choose GeM Pooling, p_trainable=True, and add dropout of fc layer.
Great job! Could you share your trained models weights?
", "rawMarkdown": "Great job! Could you share your trained models weights?" }, { "id": 2201468, "postDate": "2023-03-29T10:33:14.667Z", "content": "Great job on providing a wonderful solution! I'm curious about the training process though. Could you please explain why only the average of the last 30 loss values is calculated? i.e. the return value np.mean(losses[-30:]) of the function train_one_epoch.
Thanks for the write up and congrats on the prize position. I hope its ok to ask…
\nDo you know how much pseudo with VinDr helped ?
\nAlso what were the Light and Heavy augmentations. Was it the mixup+aug as heavy, and just aug as light ? Our team could not get mixup to help at all.
light or heavy，both have mixup. when we set mixup rate 0.3~0.5 ，cv is better.
", "rawMarkdown": "light or heavy，both have mixup. when we set mixup rate 0.3~0.5 ，cv is better." } ] }, { "id": 2162517, "postDate": "2023-02-28T08:59:19.170Z", "content": "Hi!
\nCongratulations for your performance
\nWhat preprocessed image size / aspect ratio did you use for training?
", "rawMarkdown": "Hi!\n\nCongratulations for your performance\n\nWhat preprocessed image size / aspect ratio did you use for training?", "replies": [ { "id": 2162930, "postDate": "2023-02-28T13:32:33.420Z", "content": "(1536, 896)， I've updated it now.
", "rawMarkdown": "(1536, 896)， I've updated it now." } ] }, { "id": 2162443, "postDate": "2023-02-28T07:56:33.953Z", "content": "Well done! 👍 Did you try without the JPEG compression augmentation?
", "rawMarkdown": "Well done! 👍 Did you try without the JPEG compression augmentation?", "replies": [ { "id": 2162455, "postDate": "2023-02-28T08:03:17.547Z", "content": "Haven't experimented on the final version, but JpegCompression brings a cv boost when experimented with in earlier versions
", "rawMarkdown": "Haven't experimented on the final version, but JpegCompression brings a cv boost when experimented with in earlier versions", "votes": 2 } ] } ], "comments": [ { "id": 2215166, "author_name": "ChenxiangSun@NJU", "author_url": "", "post_date": "2023-04-09T06:11:25.920000", "content": "Great job! Could you share your trained models weights?
", "votes": 0, "replies": [] }, { "id": 2201468, "author_name": "Josie Chen", "author_url": "", "post_date": "2023-03-29T10:33:14.667000", "content": "Great job on providing a wonderful solution! I'm curious about the training process though. Could you please explain why only the average of the last 30 loss values is calculated? i.e. the return value np.mean(losses[-30:]) of the function train_one_epoch.
Thanks for the write up and congrats on the prize position. I hope its ok to ask…
\nDo you know how much pseudo with VinDr helped ?
\nAlso what were the Light and Heavy augmentations. Was it the mixup+aug as heavy, and just aug as light ? Our team could not get mixup to help at all.
light or heavy，both have mixup. when we set mixup rate 0.3~0.5 ，cv is better.
", "votes": 0, "replies": [] } ] }, { "id": 2162517, "author_name": "Paul Bacher", "author_url": "", "post_date": "2023-02-28T08:59:19.170000", "content": "Hi!
\nCongratulations for your performance
\nWhat preprocessed image size / aspect ratio did you use for training?
", "votes": 0, "replies": [ { "id": 2162930, "author_name": "BladeRunner", "author_url": "", "post_date": "2023-02-28T13:32:33.420000", "content": "(1536, 896)， I've updated it now.
", "votes": 0, "replies": [] } ] }, { "id": 2162443, "author_name": "Antti Isosalo", "author_url": "", "post_date": "2023-02-28T07:56:33.953000", "content": "Well done! 👍 Did you try without the JPEG compression augmentation?
", "votes": 0, "replies": [ { "id": 2162455, "author_name": "BladeRunner", "author_url": "", "post_date": "2023-02-28T08:03:17.547000", "content": "Haven't experimented on the final version, but JpegCompression brings a cv boost when experimented with in earlier versions
", "votes": 2, "replies": [] } ] } ], "raw_markdown_by_id": { "2162423": "Before everything starts, I would like to say a big thank you to kaggle and the contest organizers for creating such an admirable contest.\n\nMy heartfelt gratitude goes to my teammates @zephyruszx and @calvchen for their incredible support throughout the competition. \n\n\n\n## DataSet\n\nWe chose Nvidia Dali, which can decode dicom image files using the GPU, and generated uint16 bit png files. We did not use more complex algorithms for crop, but just `cv2.connectedComponentsWithStats`, which works very well and is fast.\n\nimage size: 1536*896\n\nCross-validation strategy：`StratifiedGroupKFold` 5-Folds\n\nnegative samples strategy : 35%~50% of negative samples downsampled\n\nData augmentation strategies with different levels of LIGHT and HEAVY:\n\n```python\ndef mixup_augmentation(x:torch.Tensor, yc:torch.Tensor, alpha:float = 1.0):\n \"\"\"\n Function which performs Mixup augmentation\n \"\"\"\n assert alpha > 0, \"Alpha must be greater than 0\"\n assert x.shape[0] > 1, \"Need more than 1 sample to apply mixup\"\n\n lam = np.random.beta(alpha, alpha)\n rand_idx = torch.randperm(x.shape[0])\n \n mixed_x = lam * x + (1 - lam) * x[rand_idx, :]\n yc_j, yc_k = yc, yc[rand_idx]\n\n return mixed_x, yc_j, yc_k, lam\n\ndef get_transforms_16bit(data, img_size, normalize_mean, normalize_std):\n if data == 'train':\n return Compose([\n ToFloat(max_value=65535.0),\n RandomResizedCrop(img_size[0], img_size[1], scale=(0.8, 1), ratio=(0.45, 0.55), p=1), \n HorizontalFlip(p=0.5),\n VerticalFlip(p=0.5),\n ShiftScaleRotate(rotate_limit=(-5, 5), p=0.3),\n RandomBrightnessContrast(brightness_limit=(-0.1,0.1), contrast_limit=(-0.1, 0.1), p=0.5),\n JpegCompression(quality_lower=80, quality_upper=100, p=0.3),\n Affine(p=0.3),\n ToTensorV2(),\n ])\n \n elif data == 'valid':\n return Compose([\n ToFloat(max_value=65535.0),\n Resize(img_size[0], img_size[1]),\n ToTensorV2(),\n ])\n```\n\nPseudo label: We simply used the Vindr data as an external dataset, using almost the same data processing approach.\n\n\n\n## Models\n\nBecause of our training time, we chose the smallest version of the major models to try as much as possible, and ended up using three models Ensemble, which have a number of parameters between 15M and 21M, with broadly similar model training parameters, with some tuning of the parameters, which according to our experiments have a huge and sensitive impact on the learning rate.\n\n- tf_efficientnetv2_s，lr: 1e-4\n- convnext_nano， lr: 7e-6\n- eca_nfnet_l0，lr: 3e-5\n\nAfter Backbone, we choose `GeM Pooling`, p_trainable=True, and add `dropout` of fc layer.\n\n\n\n## Training\n\n- 3 Stage Training\n - 1. Training with competition data\n - 2. Training with pseudo data\n - 3. Finetune with competition data\n- Params：\n - AdamW，weight_decay = 0.01\n - Loss：BCEWithLogitsLoss\n - scheduler：OneCycleLR\n\n\n\n# Inference\n\n- Horizontal flip tta\n- Binarization post-processing\n\n## Doesn't work or doesn't do\n\n1. RCNN/Yolo to crop\n2. Larger sizes such as 2048\n3. Focal Loss\n4. More external Data\n5. site1 and site2 Threshold\n\n\n\n## Code\n\n[RSNA_Solutions · GitHub](https://github.com/chqwer2/RSNA_Solutions/tree/main/RSNA_2023_Screening%20Mammography%20Breast%20Cancer%20Detection)\n", "2215166": "Great job! Could you share your trained models weights?", "2201468": "Great job on providing a wonderful solution! I'm curious about the training process though. Could you please explain why only the average of the last 30 loss values is calculated? i.e. the return value `np.mean(losses[-30:])` of the function `train_one_epoch`.", "2162632": "Thanks for the write up and congrats on the prize position. I hope its ok to ask...\nDo you know how much pseudo with VinDr helped ? \nAlso what were the Light and Heavy augmentations. Was it the mixup+aug as heavy, and just aug as light ? Our team could not get mixup to help at all. ", "2162517": "Hi!\n\nCongratulations for your performance\n\nWhat preprocessed image size / aspect ratio did you use for training?", "2162443": "Well done! 👍 Did you try without the JPEG compression augmentation?" } }