{ "id": 391779, "title": "3rd Place Solution (Breast level models)", "url": "/competitions/rsna-breast-cancer-detection/discussion/391779", "author_name": "Bo", "post_date": "2023-03-02T16:18:14.798000", "votes": 37, "comment_count": 9, "views": 0, "content": "

Intro

\n

@haqishen and I worked together from the start of this competition. We built and tuned breast level LSTM models. Then in the final weeks, we teamed up with @forcewithme and @kevin1742064161 , who already had a beautiful DALI based submission pipeline described here. Our breast level models also benefited a lot from the external data that they had curated. Lastly, their models are all image level (then aggregated to breast level by taking the mean probabilities), which have good diversity with our breast level models.

\n

@forcewithme describes the external data, preprocessing and image level model here. In this post, I focus on the breast level models.

\n

The model

\n

There are 54706 train images, belonging to 23826 unique breasts. 98.7% of the breasts have <= 4 images. So we build a LSTM model with seq len 4. When there are 2 or 3 images for a breast, we pad the sequence with repeat images. When there are more than 4, we truncate at 4. E.g.:

\n

[img0, img1] pad -> [img0, img1, img0, img1]
\n[img0, img1, img2] pad -> [img0, img1, img2, img0]
\n[img0, img1, …, img5] truncate -> [img0, img1, img2, img3]

\n

The model architecture is illustrated below:
\n\"\"

\n

Image size is (1280, 800)

\n

Training details

\n

We used all 4 external data sets (CBIS-DDSM, CMMD, Vindr and Mini-DDSM) as described here. The model is trained for 20 epochs. The best single model cv F1 is 0.504

\n

Augmentations:

\n
    albumentations.HorizontalFlip(p=0.5),\n    albumentations.VerticalFlip(p=0.5),\n    albumentations.RandomBrightness(limit=0.1, p=0.7),\n    albumentations.ImageCompression(quality_lower=97, quality_upper=100),\n    albumentations.ShiftScaleRotate(shift_limit=0.3, scale_limit=0.3, rotate_limit=45, border_mode=4, p=0.7),\n    albumentations.GridDistortion(num_steps=5, distort_limit=0.3),\n    albumentations.Cutout(max_h_size=int(image_size[0] * 0.5), max_w_size=int(image_size[1] * 0.5), num_holes=1, p=0.5),\n
\n

For backbone we used convnext_tiny and tf_efficientnetv2_s in the final models.

\n

Submission pipeline

\n

My teammates @forcewithme and @kevin1742064161 built a beautiful end-to-end DALI pipeline that does everything on GPU (decoding DICOM, yolox crop, CNN model inference). But we couldn't figure out how to adapt it for the LSTM models with varying number of image inputs. So we end up dumping the yolox cropped images on the disk for the LSTM models.

\n

Edit: Submission notebook: https://www.kaggle.com/code/forcewithme/final-lstm2

\n

Special thanks

\n

This competition cannot be this successful with the great sharing of how to use DALI to speed up inference by @tivfrvqhs5 , @theoviel , @christofhenkel etc. Also many thanks to my NVIDIA colleagues at DALI team who worked tirelessly behind the scenes to add support for lossless jpg decoding in a short time. I think this competition showed that DALI is a great tool that is being under utilized by the community (I'm guilty). I hope to see it become more popular on Kaggle.

", "messages": [ { "id": 2166116, "postDate": "2023-03-02T16:18:14.797Z", "content": "

Intro

\n

@haqishen and I worked together from the start of this competition. We built and tuned breast level LSTM models. Then in the final weeks, we teamed up with @forcewithme and @kevin1742064161 , who already had a beautiful DALI based submission pipeline described here. Our breast level models also benefited a lot from the external data that they had curated. Lastly, their models are all image level (then aggregated to breast level by taking the mean probabilities), which have good diversity with our breast level models.

\n

@forcewithme describes the external data, preprocessing and image level model here. In this post, I focus on the breast level models.

\n

The model

\n

There are 54706 train images, belonging to 23826 unique breasts. 98.7% of the breasts have <= 4 images. So we build a LSTM model with seq len 4. When there are 2 or 3 images for a breast, we pad the sequence with repeat images. When there are more than 4, we truncate at 4. E.g.:

\n

[img0, img1] pad -> [img0, img1, img0, img1]
\n[img0, img1, img2] pad -> [img0, img1, img2, img0]
\n[img0, img1, …, img5] truncate -> [img0, img1, img2, img3]

\n

The model architecture is illustrated below:
\n\"\"

\n

Image size is (1280, 800)

\n

Training details

\n

We used all 4 external data sets (CBIS-DDSM, CMMD, Vindr and Mini-DDSM) as described here. The model is trained for 20 epochs. The best single model cv F1 is 0.504

\n

Augmentations:

\n
    albumentations.HorizontalFlip(p=0.5),\n    albumentations.VerticalFlip(p=0.5),\n    albumentations.RandomBrightness(limit=0.1, p=0.7),\n    albumentations.ImageCompression(quality_lower=97, quality_upper=100),\n    albumentations.ShiftScaleRotate(shift_limit=0.3, scale_limit=0.3, rotate_limit=45, border_mode=4, p=0.7),\n    albumentations.GridDistortion(num_steps=5, distort_limit=0.3),\n    albumentations.Cutout(max_h_size=int(image_size[0] * 0.5), max_w_size=int(image_size[1] * 0.5), num_holes=1, p=0.5),\n
\n

For backbone we used convnext_tiny and tf_efficientnetv2_s in the final models.

\n

Submission pipeline

\n

My teammates @forcewithme and @kevin1742064161 built a beautiful end-to-end DALI pipeline that does everything on GPU (decoding DICOM, yolox crop, CNN model inference). But we couldn't figure out how to adapt it for the LSTM models with varying number of image inputs. So we end up dumping the yolox cropped images on the disk for the LSTM models.

\n

Edit: Submission notebook: https://www.kaggle.com/code/forcewithme/final-lstm2

\n

Special thanks

\n

This competition cannot be this successful with the great sharing of how to use DALI to speed up inference by @tivfrvqhs5 , @theoviel , @christofhenkel etc. Also many thanks to my NVIDIA colleagues at DALI team who worked tirelessly behind the scenes to add support for lossless jpg decoding in a short time. I think this competition showed that DALI is a great tool that is being under utilized by the community (I'm guilty). I hope to see it become more popular on Kaggle.

", "rawMarkdown": "# Intro\n\n@haqishen and I worked together from the start of this competition. We built and tuned breast level LSTM models. Then in the final weeks, we teamed up with @forcewithme and @kevin1742064161 , who already had a beautiful DALI based submission pipeline described [here](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/391059). Our breast level models also benefited a lot from the external data that they had curated. Lastly, their models are all image level (then aggregated to breast level by taking the mean probabilities), which have good diversity with our breast level models.\n\n@forcewithme describes the external data, preprocessing and image level model [here](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/391725). In this post, I focus on the breast level models.\n\n# The model\n\nThere are 54706 train images, belonging to 23826 unique breasts. 98.7% of the breasts have <= 4 images. So we build a LSTM model with seq len 4. When there are 2 or 3 images for a breast, we pad the sequence with repeat images. When there are more than 4, we truncate at 4. E.g.:\n\n[img0, img1] pad -> [img0, img1, img0, img1]\n[img0, img1, img2] pad -> [img0, img1, img2, img0]\n[img0, img1, ..., img5] truncate -> [img0, img1, img2, img3]\n\nThe model architecture is illustrated below:\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1120704%2F5f063f57735017c7eada35525d58adbc%2Frsna_plot.png?generation=1677772518056053&alt=media)\n\nImage size is (1280, 800)\n\n# Training details\n\nWe used all 4 external data sets (CBIS-DDSM, CMMD, Vindr and Mini-DDSM) as described [here](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/391725). The model is trained for 20 epochs. The best single model cv F1 is 0.504\n\nAugmentations:\n```\n albumentations.HorizontalFlip(p=0.5),\n albumentations.VerticalFlip(p=0.5),\n albumentations.RandomBrightness(limit=0.1, p=0.7),\n albumentations.ImageCompression(quality_lower=97, quality_upper=100),\n albumentations.ShiftScaleRotate(shift_limit=0.3, scale_limit=0.3, rotate_limit=45, border_mode=4, p=0.7),\n albumentations.GridDistortion(num_steps=5, distort_limit=0.3),\n albumentations.Cutout(max_h_size=int(image_size[0] * 0.5), max_w_size=int(image_size[1] * 0.5), num_holes=1, p=0.5),\n``` \n\nFor backbone we used convnext_tiny and tf_efficientnetv2_s in the final models.\n\n# Submission pipeline\nMy teammates @forcewithme and @kevin1742064161 built a beautiful end-to-end DALI pipeline that does everything on GPU (decoding DICOM, yolox crop, CNN model inference). But we couldn't figure out how to adapt it for the LSTM models with varying number of image inputs. So we end up dumping the yolox cropped images on the disk for the LSTM models.\n\nEdit: Submission notebook: https://www.kaggle.com/code/forcewithme/final-lstm2\n\n# Special thanks\nThis competition cannot be this successful with the great sharing of how to use DALI to speed up inference by @tivfrvqhs5 , @theoviel , @christofhenkel etc. Also many thanks to my NVIDIA colleagues at DALI team who worked tirelessly behind the scenes to add support for lossless jpg decoding in a short time. I think this competition showed that DALI is a great tool that is being under utilized by the community (I'm guilty). I hope to see it become more popular on Kaggle.", "votes": 37 }, { "id": 2166142, "postDate": "2023-03-02T16:41:37.853Z", "content": "

Didn't know DALI could do such thing , I will take notes for sure ! Thank you for this interesting insights of the competition and CONGRATULATIONS to you and you team for such good work, i'm impresssed.

", "rawMarkdown": "Didn't know DALI could do such thing , I will take notes for sure ! Thank you for this interesting insights of the competition and CONGRATULATIONS to you and you team for such good work, i'm impresssed.", "votes": 1, "replies": [ { "id": 2166155, "postDate": "2023-03-02T16:49:58.303Z", "content": "

Yes, it's a great tool.

", "rawMarkdown": "Yes, it's a great tool." } ] }, { "id": 2169075, "postDate": "2023-03-04T18:58:43.030Z", "content": "

Great work. Congratulations Bo and team!

", "rawMarkdown": "Great work. Congratulations Bo and team!", "votes": 2 }, { "id": 2169874, "postDate": "2023-03-05T14:15:44.317Z", "content": "

Congratulations, I just add DALI to my tool belt.

", "rawMarkdown": "Congratulations, I just add DALI to my tool belt." }, { "id": 2166169, "postDate": "2023-03-02T17:00:53.227Z", "content": "
\n

So we build a LSTM model with seq len 4. When there are 2 or 3 images for a breast, we pad the sequence with repeat images. When there are more than 4, we truncate at 4.

\n
\n

Does this mean that when, for example, MLO view was missing, you used the CC view from the same side to impute? And when truncating, you try to to preferably have CC and MLO views in the sequence? Congrats! Repeating what I already wrote to ForcewithMe, your deliverables/datasets are very comprehensive.

", "rawMarkdown": ">So we build a LSTM model with seq len 4. When there are 2 or 3 images for a breast, we pad the sequence with repeat images. When there are more than 4, we truncate at 4.\n\nDoes this mean that when, for example, MLO view was missing, you used the CC view from the same side to impute? And when truncating, you try to to preferably have CC and MLO views in the sequence? Congrats! Repeating what I already wrote to ForcewithMe, your deliverables/datasets are very comprehensive.", "replies": [ { "id": 2166170, "postDate": "2023-03-02T17:01:42.207Z", "content": "

I will also try to incorporate DALI in my research work in the future.

", "rawMarkdown": "I will also try to incorporate DALI in my research work in the future.", "votes": 1, "replies": [ { "id": 2166210, "postDate": "2023-03-02T17:23:15.203Z", "content": "

awesome, great to hear!

", "rawMarkdown": "awesome, great to hear!", "votes": 1 } ] }, { "id": 2166209, "postDate": "2023-03-02T17:23:00.250Z", "content": "
\n

Does this mean that when, for example, MLO view was missing, you used the CC view from the same side to impute?

\n
\n

Yes, the example on my plot happens to have 2 CC and 2 MLO. But if MLO was missing, then all 4 will be CC. If CC was missing, then all 4 would be MLO.

\n
\n

And when truncating, you try to to preferably have CC and MLO views in the sequence?

\n
\n

No, we just took the first 4 rows for that breast. We also tried randomly pick 4 from N, but it didn't make a difference, probably because there are so few breasts with > 4 images.

", "rawMarkdown": ">Does this mean that when, for example, MLO view was missing, you used the CC view from the same side to impute? \n\nYes, the example on my plot happens to have 2 CC and 2 MLO. But if MLO was missing, then all 4 will be CC. If CC was missing, then all 4 would be MLO.\n\n>And when truncating, you try to to preferably have CC and MLO views in the sequence? \n\nNo, we just took the first 4 rows for that breast. We also tried randomly pick 4 from N, but it didn't make a difference, probably because there are so few breasts with > 4 images.", "votes": 2, "replies": [ { "id": 2166247, "postDate": "2023-03-02T17:46:05.317Z", "content": "

Thanks. πŸ‘

\n

There are some examples found in the literature cropping the MLO view so that the pectoral muscle is left out. That would \"standardize\" the views a bit. But on the other hand, lesions which should not be taken lightly appear near the armpit and MLO view therefore has its merits in visualizing those.

", "rawMarkdown": "Thanks. πŸ‘\n\nThere are some examples found in the literature cropping the MLO view so that the pectoral muscle is left out. That would \"standardize\" the views a bit. But on the other hand, lesions which should not be taken lightly appear near the armpit and MLO view therefore has its merits in visualizing those.", "votes": 1 } ] } ] } ], "comments": [ { "id": 2166142, "author_name": "JEANMPIA", "author_url": "", "post_date": "2023-03-02T16:41:37.853000", "content": "

Didn't know DALI could do such thing , I will take notes for sure ! Thank you for this interesting insights of the competition and CONGRATULATIONS to you and you team for such good work, i'm impresssed.

", "votes": 1, "replies": [ { "id": 2166155, "author_name": "ForcewithMe", "author_url": "", "post_date": "2023-03-02T16:49:58.303000", "content": "

Yes, it's a great tool.

", "votes": 0, "replies": [] } ] }, { "id": 2169075, "author_name": "Chris Deotte", "author_url": "", "post_date": "2023-03-04T18:58:43.030000", "content": "

Great work. Congratulations Bo and team!

", "votes": 2, "replies": [] }, { "id": 2169874, "author_name": "Jose CΓ‘liz", "author_url": "", "post_date": "2023-03-05T14:15:44.317000", "content": "

Congratulations, I just add DALI to my tool belt.

", "votes": 0, "replies": [] }, { "id": 2166169, "author_name": "Antti Isosalo", "author_url": "", "post_date": "2023-03-02T17:00:53.227000", "content": "
\n

So we build a LSTM model with seq len 4. When there are 2 or 3 images for a breast, we pad the sequence with repeat images. When there are more than 4, we truncate at 4.

\n
\n

Does this mean that when, for example, MLO view was missing, you used the CC view from the same side to impute? And when truncating, you try to to preferably have CC and MLO views in the sequence? Congrats! Repeating what I already wrote to ForcewithMe, your deliverables/datasets are very comprehensive.

", "votes": 0, "replies": [ { "id": 2166170, "author_name": "Antti Isosalo", "author_url": "", "post_date": "2023-03-02T17:01:42.207000", "content": "

I will also try to incorporate DALI in my research work in the future.

", "votes": 1, "replies": [ { "id": 2166210, "author_name": "Bo", "author_url": "", "post_date": "2023-03-02T17:23:15.203000", "content": "

awesome, great to hear!

", "votes": 1, "replies": [] } ] }, { "id": 2166209, "author_name": "Bo", "author_url": "", "post_date": "2023-03-02T17:23:00.250000", "content": "
\n

Does this mean that when, for example, MLO view was missing, you used the CC view from the same side to impute?

\n
\n

Yes, the example on my plot happens to have 2 CC and 2 MLO. But if MLO was missing, then all 4 will be CC. If CC was missing, then all 4 would be MLO.

\n
\n

And when truncating, you try to to preferably have CC and MLO views in the sequence?

\n
\n

No, we just took the first 4 rows for that breast. We also tried randomly pick 4 from N, but it didn't make a difference, probably because there are so few breasts with > 4 images.

", "votes": 2, "replies": [ { "id": 2166247, "author_name": "Antti Isosalo", "author_url": "", "post_date": "2023-03-02T17:46:05.317000", "content": "

Thanks. πŸ‘

\n

There are some examples found in the literature cropping the MLO view so that the pectoral muscle is left out. That would \"standardize\" the views a bit. But on the other hand, lesions which should not be taken lightly appear near the armpit and MLO view therefore has its merits in visualizing those.

", "votes": 1, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2166116": "# Intro\n\n@haqishen and I worked together from the start of this competition. We built and tuned breast level LSTM models. Then in the final weeks, we teamed up with @forcewithme and @kevin1742064161 , who already had a beautiful DALI based submission pipeline described [here](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/391059). Our breast level models also benefited a lot from the external data that they had curated. Lastly, their models are all image level (then aggregated to breast level by taking the mean probabilities), which have good diversity with our breast level models.\n\n@forcewithme describes the external data, preprocessing and image level model [here](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/391725). In this post, I focus on the breast level models.\n\n# The model\n\nThere are 54706 train images, belonging to 23826 unique breasts. 98.7% of the breasts have <= 4 images. So we build a LSTM model with seq len 4. When there are 2 or 3 images for a breast, we pad the sequence with repeat images. When there are more than 4, we truncate at 4. E.g.:\n\n[img0, img1] pad -> [img0, img1, img0, img1]\n[img0, img1, img2] pad -> [img0, img1, img2, img0]\n[img0, img1, ..., img5] truncate -> [img0, img1, img2, img3]\n\nThe model architecture is illustrated below:\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1120704%2F5f063f57735017c7eada35525d58adbc%2Frsna_plot.png?generation=1677772518056053&alt=media)\n\nImage size is (1280, 800)\n\n# Training details\n\nWe used all 4 external data sets (CBIS-DDSM, CMMD, Vindr and Mini-DDSM) as described [here](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/391725). The model is trained for 20 epochs. The best single model cv F1 is 0.504\n\nAugmentations:\n```\n albumentations.HorizontalFlip(p=0.5),\n albumentations.VerticalFlip(p=0.5),\n albumentations.RandomBrightness(limit=0.1, p=0.7),\n albumentations.ImageCompression(quality_lower=97, quality_upper=100),\n albumentations.ShiftScaleRotate(shift_limit=0.3, scale_limit=0.3, rotate_limit=45, border_mode=4, p=0.7),\n albumentations.GridDistortion(num_steps=5, distort_limit=0.3),\n albumentations.Cutout(max_h_size=int(image_size[0] * 0.5), max_w_size=int(image_size[1] * 0.5), num_holes=1, p=0.5),\n``` \n\nFor backbone we used convnext_tiny and tf_efficientnetv2_s in the final models.\n\n# Submission pipeline\nMy teammates @forcewithme and @kevin1742064161 built a beautiful end-to-end DALI pipeline that does everything on GPU (decoding DICOM, yolox crop, CNN model inference). But we couldn't figure out how to adapt it for the LSTM models with varying number of image inputs. So we end up dumping the yolox cropped images on the disk for the LSTM models.\n\nEdit: Submission notebook: https://www.kaggle.com/code/forcewithme/final-lstm2\n\n# Special thanks\nThis competition cannot be this successful with the great sharing of how to use DALI to speed up inference by @tivfrvqhs5 , @theoviel , @christofhenkel etc. Also many thanks to my NVIDIA colleagues at DALI team who worked tirelessly behind the scenes to add support for lossless jpg decoding in a short time. I think this competition showed that DALI is a great tool that is being under utilized by the community (I'm guilty). I hope to see it become more popular on Kaggle.", "2166142": "Didn't know DALI could do such thing , I will take notes for sure ! Thank you for this interesting insights of the competition and CONGRATULATIONS to you and you team for such good work, i'm impresssed.", "2169075": "Great work. Congratulations Bo and team!", "2169874": "Congratulations, I just add DALI to my tool belt.", "2166169": ">So we build a LSTM model with seq len 4. When there are 2 or 3 images for a breast, we pad the sequence with repeat images. When there are more than 4, we truncate at 4.\n\nDoes this mean that when, for example, MLO view was missing, you used the CC view from the same side to impute? And when truncating, you try to to preferably have CC and MLO views in the sequence? Congrats! Repeating what I already wrote to ForcewithMe, your deliverables/datasets are very comprehensive." } }