![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F7285387%2F2ad9ed3ef7a2b0b9d8f8f13e1f30bc00%2FArch.png?generation=1704357989295647&alt=media\")
I would like to express my gratitude to Kaggle, the organizers, and other participants in the community. I have learned a great deal from this competition and hope that it will promote the advancement of research in MIL，Classification and outliers detection of Human tissues, and the study of women's health.
\nThe first and most crucial step in this competition is to familiarize oneself with the feature of Whole Slide Images (WSI) and Tissue Microarray (TMA) images. A simple visual examination reveals that the features of TMA and WSI are presented at vastly different scales. TMA features are at the cellular level, while WSI features are several to tens of times larger than cell clusters. Hence, from the onset of the competition, I decided to tackle WSI and TMA with two completely distinct approaches.
\n
Use arcface retrieval with the training set's TMA as templates. Output 5-class results for samples with a close cosine distance, and consider those with a far cosine distance as outliers. Samples that are uncertain in the arcface stage are left for the subsequent phase.
\nThe 6-class model uses 2-fold effv2s + 5-fold effv2l + 4-fold convnext small + 3-fold convnext large.
| Model | \nResolution | \n
|---|---|
| Effv2s | \n1280 | \n
| Effv2l | \n1280 | \n
| Convnext Small | \n1024 | \n
| Convnext Large | \n1024 | \n
Note: The arcface can tackle about 60% of the TMAs. So even the second stage is heavy, it doesn't cause timeout.
\nhealthy and dead as Other class. Instead of training with the official TMAs , which has limited number, use them for subsequent validation and retrieval. Other WSI during the training process. | Model | \nResolution | \nNumber folds | \n
|---|---|---|
| Convnext Small | \n3072 resize 768 | \n2 | \n
| Effv2s | \n3072 resize 768 | \n3 | \n
| Effv2s | \n1024 | \n4 | \n
Other pool with patches that have a high probability of 'Other'. WSI from the Other pool each epoch. 6144 resize to 1024 into my final pipeline. But the notebook crashed.This post is almost completed :
\nWSI from the Other pool each epoch for WSI. And reduce the number on 1024 resolution wo down-sample for submission. This is also proved in this paper. The path to becoming a Grandmaster has been lengthy, filled with challenges, and an experience I will cherish for life. I am incredibly thankful for the support from my teammates in past competitions, as well as the unwavering encouragement from my family and girlfriend throughout my journey. The final step to becoming a Grandmaster, achieving a solo gold medal, has been particularly solitary and tough. This was my fourth attempt at a solo gold medal. if I failed this time, it might have been three years, ten years, or perhaps never before I'd have the chance again, as I am about to graduate with my master's degree and start my career in a busy company. Fortunately, I have realized the dream I had three years ago and have now brought my student years to a close with the title of Grandmaster. Wishing everyone a Happy New Year!
", "messages": [ { "id": 2586211, "postDate": "2024-01-04T03:54:32.013Z", "content": "I would like to express my gratitude to Kaggle, the organizers, and other participants in the community. I have learned a great deal from this competition and hope that it will promote the advancement of research in MIL，Classification and outliers detection of Human tissues, and the study of women's health.
\nThe first and most crucial step in this competition is to familiarize oneself with the feature of Whole Slide Images (WSI) and Tissue Microarray (TMA) images. A simple visual examination reveals that the features of TMA and WSI are presented at vastly different scales. TMA features are at the cellular level, while WSI features are several to tens of times larger than cell clusters. Hence, from the onset of the competition, I decided to tackle WSI and TMA with two completely distinct approaches.
\n
Use arcface retrieval with the training set's TMA as templates. Output 5-class results for samples with a close cosine distance, and consider those with a far cosine distance as outliers. Samples that are uncertain in the arcface stage are left for the subsequent phase.
\nThe 6-class model uses 2-fold effv2s + 5-fold effv2l + 4-fold convnext small + 3-fold convnext large.
| Model | \nResolution | \n
|---|---|
| Effv2s | \n1280 | \n
| Effv2l | \n1280 | \n
| Convnext Small | \n1024 | \n
| Convnext Large | \n1024 | \n
Note: The arcface can tackle about 60% of the TMAs. So even the second stage is heavy, it doesn't cause timeout.
\nhealthy and dead as Other class. Instead of training with the official TMAs , which has limited number, use them for subsequent validation and retrieval. Other WSI during the training process. | Model | \nResolution | \nNumber folds | \n
|---|---|---|
| Convnext Small | \n3072 resize 768 | \n2 | \n
| Effv2s | \n3072 resize 768 | \n3 | \n
| Effv2s | \n1024 | \n4 | \n
Other pool with patches that have a high probability of 'Other'. WSI from the Other pool each epoch. 6144 resize to 1024 into my final pipeline. But the notebook crashed.This post is almost completed :
\nWSI from the Other pool each epoch for WSI. And reduce the number on 1024 resolution wo down-sample for submission. This is also proved in this paper. The path to becoming a Grandmaster has been lengthy, filled with challenges, and an experience I will cherish for life. I am incredibly thankful for the support from my teammates in past competitions, as well as the unwavering encouragement from my family and girlfriend throughout my journey. The final step to becoming a Grandmaster, achieving a solo gold medal, has been particularly solitary and tough. This was my fourth attempt at a solo gold medal. if I failed this time, it might have been three years, ten years, or perhaps never before I'd have the chance again, as I am about to graduate with my master's degree and start my career in a busy company. Fortunately, I have realized the dream I had three years ago and have now brought my student years to a close with the title of Grandmaster. Wishing everyone a Happy New Year!
", "rawMarkdown": "# Acknowledgments \n \nI would like to express my gratitude to Kaggle, the organizers, and other participants in the community. I have learned a great deal from this competition and hope that it will promote the advancement of research in MIL，Classification and outliers detection of Human tissues, and the study of women's health.\n\n# Introduction \n \nThe first and most crucial step in this competition is to familiarize oneself with the feature of Whole Slide Images (WSI) and Tissue Microarray (TMA) images. A simple visual examination reveals that **the features of TMA and WSI are presented at vastly different scales.** TMA features are at the cellular level, while WSI features are several to tens of times larger than cell clusters. Hence, from the onset of the competition, I decided to tackle WSI and TMA with two completely distinct approaches. \n\n## Key points for the global approach: \n\n1. **Address WSI and TMA separately** .\n2. **Align the magnification of TMA and WSI** to enable the reuse of TMA training code for optimizing the WSI feature extractor. \n - WSI images are 20x magnified, while TMA images are 40x. Thus, TMA images need to be downsampled by a factor of 2 to align with the physical scale of WSI, which can also be observed with a visual inspection of the training data. \n \n## Architecture of the solution\n\n\n\n## TMA Approach \n \n### Summary of Key Points: \n1. **Train with patches tiled with official masks** (most important globally). \n2. Train the model with **healthy and death patches** to predict some outliers. \n3. Employ **arcface** to retrieve some outliers and part of the 5-class classification. \n4. Heavy ensemble of 6 classification models to predict samples that arcface retrieval did not cover. \n \n### Inference: \n\n1. Use arcface retrieval with the training set's TMA as templates. Output 5-class results for samples with a close cosine distance, and consider those with a far cosine distance as outliers. Samples that are uncertain in the arcface stage are left for the subsequent phase. \n - Models: 5-fold effv2s + 5-fold convnext small with dynamic margin, subcenter=3. \n - Top1 threshold at 0.05, Other threshold at 0.2. \n \n2. The 6-class model uses 2-fold effv2s + 5-fold effv2l + 4-fold convnext small + 3-fold convnext large. \n \n| Model | Resolution |\n| --- | --- |\n| Effv2s | 1280 |\n| Effv2l | 1280 |\n| Convnext Small | 1024 |\n| Convnext Large | 1024 |\n\n\nNote: The arcface can tackle about 60% of the TMAs. So even the second stage is heavy, it doesn't cause timeout.\n\n### Training: \n\n1. Tile patches using the official segmentation masks. In addition to the official 5 categories, classify patches of `healthy` and `dead` as `Other` class. Instead of training with the official TMAs , which has limited number, use them for subsequent validation and retrieval. \n2. Use the first-phase models to generate pseudo-label patches on the remaining 300+ WSIs without mask. \n3. Inherit the weights from the first or second step, training only the last layer of the backbone and the arcface head. \n \n## WSI Approach \n \n### Summary of Key Points: \n1. **Train the feature extractor using the TMA pipeline**. \n2. **Synthesize `Other` WSI** during the training process. \n3. **Integrate different magnification scales.** \n4. Rank the DataFrame of WSI with the number of pixels, and process the WSIs with multithread can greatly speed the the WSI processing procedure.\n \n### Inference: \n\n1. Ensemble of two resolutions of feature extractor: 3072 resized to 768, and 1024 without down-sampling. After extracting features, apply DTFD-MIL. \n2. For speed up the image processing procedure, I only use the center region of each 3072 tile, so that I only need to crop patch from the WSI once. \n \n| Model | Resolution | Number folds |\n| --- | --- | ---|\n| Convnext Small | 3072 resize 768 | 2 |\n| Effv2s | 3072 resize 768 | 3 |\n| Effv2s | 1024 | 4 |\n\n\n### Training: \n\n1. Train the feature extractor using the TMA pipeline. \n2. Extract features using the feature extractor. \n3. Predict the probability of 'Other' on all patches using the feature extractor, and **create an `Other pool`** with patches that have a high probability of 'Other'. \n4. During the training of DTFD-MIL, dynamically synthesize some `WSI` from the `Other pool` each epoch. \n - Training DTFD-MIL serves as a validation for whether the TMA pipeline models truly learned useful features. If we use the pretrained weights from ImageNet, DTFD-MIL requires up to 200 epochs to converge. In contrast, using models trained on TMA, the MIL Head may take as little as 1 epoch and at most 20 epochs to converge.\n\n## Something I don't have time to try but I think may work\n1. Arcface for WSI. \n2. Large transformer pretrained on Large set of Slides. In fact I tried PLIP at the early stage of this competition, But I didn't dig deeper.\n3. Better retrieve strategy for arcface.\n4. More resolution for WSI . I tried to add `6144 resize to 1024` into my final pipeline. But the notebook crashed.\n5. Ensemble of MIL head\n6. External data.\n\n## Todo: \n\nThis post is ~~not completed . Here are todo lists today~~ almost completed :\n1. ~~Model details~~\n2. ~~Citation of some methods in the post.~~\n3. ~~Discussion on MIL~~\n4. ~~Some figures about the pipeline and EDA\n~~\nIf possible, I may add more ablation study in two weeks.\n\n## Discussion and Citation\n1. [DTFD-MIL](https://arxiv.org/abs/2203.12081) is a strong and robust baseline for MIL, which follows ABMIL. \n2. I think MIL methods are not sensitive to the position and the number of patches, which is observed in my experiment. That's why I randomly synthesized some `WSI` from the `Other pool` each epoch for WSI. And reduce the number on `1024 resolution wo down-sample` for submission. This is also proved in [this paper](https://ieeexplore.ieee.org/document/10219719/authors#authors). \n3. The reason why I ensemble multiple resolution for WSI: if you check the [clarification on WHO](https://www.pathologyoutlines.com/topic/ovarytumorwhoclassif.html). You will find that Pathologists distinguish the subtypes of Ovarian cancer on different magnification. So I believe ensemble multiple resolution for classifying a WSI should be very important. \n4. About Larger transformer-based model: I believe the competition needs model with better generalization. And larger models are often more robust. At the early stage of this competition, I takes some time to test PLIP offline, but the CV is not good. Also, since I decide to solo this time, So I have to spend my limited time on the direction I am most confident about, solving this task in a more traditional and solid way. \n\n## Closing Thoughts: My Journey to Grandmaster \n \nThe path to becoming a Grandmaster has been lengthy, filled with challenges, and an experience I will cherish for life. I am incredibly thankful for the support from my teammates in past competitions, as well as the unwavering encouragement from my family and girlfriend throughout my journey. The final step to becoming a Grandmaster, achieving a solo gold medal, has been particularly solitary and tough. This was my fourth attempt at a solo gold medal. if I failed this time, it might have been three years, ten years, or perhaps never before I'd have the chance again, as I am about to graduate with my master's degree and start my career in a busy company. Fortunately, I have realized the dream I had three years ago and have now brought my student years to a close with the title of Grandmaster. Wishing everyone a Happy New Year!", "votes": 49 }, { "id": 2586391, "postDate": "2024-01-04T07:07:08.817Z", "content": "congratulation for GM!!
", "rawMarkdown": "congratulation for GM!!", "votes": 3 }, { "id": 2591621, "postDate": "2024-01-08T05:42:03.237Z", "content": "Congratulations!! I really love your graph of the Architecture of the solution. So clear and informative!!
\nMay I ask how you came up with the idea of \"rough detection on outliers\" with Arcface on medical images? (Based on your description, the Arcface is a cheap way to handle most of the TMAs, but how did you come up with it the first time? )
\nAnd you fine-tuned the Arcface with the only 25 TMAs the dataset provided?
\nSorry for bothering you with newbie questions🙋♀️
Hi @lizzylee1111111111 ,
\nMay I ask how you came up with the idea of \"rough detection on outliers\" with Arcface on medical images? . Arcface is usually used in Image Retrieval task. To detect outliers, tackling with image retrieval instead of image classification is better.
I fine-tuned the arcface with the patch cropped on the masks provided by the host.
Congratulations on becoming a new kaggle GM! 祝贺来自土木老哥😆
", "rawMarkdown": "Congratulations on becoming a new kaggle GM! 祝贺来自土木老哥😆", "votes": 1 }, { "id": 2587109, "postDate": "2024-01-04T15:11:31.650Z", "content": "Congrats on becoming a GM, and thanks for the write-up! Some interesting ideas here.
", "rawMarkdown": "Congrats on becoming a GM, and thanks for the write-up! Some interesting ideas here.", "votes": 1 }, { "id": 2586965, "postDate": "2024-01-04T13:52:36.760Z", "content": "Thanks for the solution! It is really good!
", "rawMarkdown": "Thanks for the solution! It is really good!", "votes": 1 }, { "id": 2586722, "postDate": "2024-01-04T11:13:16.690Z", "content": "Congrats for becoming Grandmaster🥳, I just wondering if the code is available for this solution?
", "rawMarkdown": "Congrats for becoming Grandmaster🥳, I just wondering if the code is available for this solution?", "votes": 1, "replies": [ { "id": 2586865, "postDate": "2024-01-04T12:53:56.563Z", "content": "Thank you! There are nothing special in my code. I may make it available after I review them. But you remind me an important key points which I forget to write in my post. That is, Rank the DataFrame of WSI with the number of pixels first, and process the WSIs in the order with multi-thread can greatly speed the the WSI processing procedure.
", "rawMarkdown": "Thank you! There are nothing special in my code. I may make it available after I review them. But you remind me an important key points which I forget to write in my post. That is, **Rank the DataFrame of WSI with the number of pixels first, and process the WSIs in the order with multi-thread can greatly speed the the WSI processing procedure.**\n", "votes": 1, "replies": [ { "id": 2586909, "postDate": "2024-01-04T13:28:33.390Z", "content": "Thanks for your sharing！
", "rawMarkdown": "Thanks for your sharing！" } ] } ] }, { "id": 2586520, "postDate": "2024-01-04T09:15:56.323Z", "content": "Great top solution. I learned a lot, thanks for the detailed method and congratulations becoming GM!
", "rawMarkdown": "Great top solution. I learned a lot, thanks for the detailed method and congratulations becoming GM!", "votes": 1 }, { "id": 2586280, "postDate": "2024-01-04T05:29:56.587Z", "content": "Congratulations on winning your first solo gold and becoming a grandmaster!🎉🎉🎉
", "rawMarkdown": "Congratulations on winning your first solo gold and becoming a grandmaster!🎉🎉🎉", "votes": 1, "replies": [ { "id": 2586308, "postDate": "2024-01-04T05:56:11.157Z", "content": "@kevin1742064161 Thank you! You are an extraordinary teammate. Hope to have more cooperation with you in the future. Also, wish you to reach a grandmaster soon.
", "rawMarkdown": "@kevin1742064161 Thank you! You are an extraordinary teammate. Hope to have more cooperation with you in the future. Also, wish you to reach a grandmaster soon." } ] }, { "id": 2586274, "postDate": "2024-01-04T05:22:17.133Z", "content": "Congratulations on becoming GM. I also became a Master through this competition and will work hard for GM.
", "rawMarkdown": "Congratulations on becoming GM. I also became a Master through this competition and will work hard for GM.", "votes": 1, "replies": [ { "id": 2586310, "postDate": "2024-01-04T05:57:45.453Z", "content": "Congratulations! Wish you become a gm as you wish in the future.
", "rawMarkdown": "Congratulations! Wish you become a gm as you wish in the future. " } ] }, { "id": 2586273, "postDate": "2024-01-04T05:20:04.443Z", "content": "Congrats for becoming GM!
", "rawMarkdown": "Congrats for becoming GM!", "votes": 1, "replies": [ { "id": 2586276, "postDate": "2024-01-04T05:24:49.917Z", "content": "Thank you LTY, working with you in previous competition is enjoyable.
", "rawMarkdown": "Thank you LTY, working with you in previous competition is enjoyable. " } ] }, { "id": 2586237, "postDate": "2024-01-04T04:27:24.703Z", "content": "Happy New Year!
", "rawMarkdown": "Happy New Year!", "votes": 1 }, { "id": 2586220, "postDate": "2024-01-04T04:00:10.760Z", "content": "Congratulation on achieving the 8th position in this competition. Thanks for sharing the details of your solution.
", "rawMarkdown": "Congratulation on achieving the 8th position in this competition. Thanks for sharing the details of your solution.\n", "votes": 1, "replies": [ { "id": 2586221, "postDate": "2024-01-04T04:00:38.120Z", "content": "Thank you!
", "rawMarkdown": "Thank you! " } ] }, { "id": 2586495, "postDate": "2024-01-04T08:56:12.940Z", "content": "Great work, I have noticed that all your models use very large resolutions, so was it trained on Kaggle or outside? and if on Kaggle how do you fit to limited P100 memory?
", "rawMarkdown": "Great work, I have noticed that all your models use very large resolutions, so was it trained on Kaggle or outside? and if on Kaggle how do you fit to limited P100 memory?", "replies": [ { "id": 2586833, "postDate": "2024-01-04T12:31:47.060Z", "content": "I only use the GPU on Kaggle for inference debugging. I finish all my training on 1 A100 .
", "rawMarkdown": "I only use the GPU on Kaggle for inference debugging. I finish all my training on 1 A100 .", "votes": 1, "replies": [ { "id": 2586846, "postDate": "2024-01-04T12:39:30.873Z", "content": "I though so :) and what was the batch size on A100?
", "rawMarkdown": "I though so :) and what was the batch size on A100?" }, { "id": 2586857, "postDate": "2024-01-04T12:43:54.590Z", "content": "depends on the backbone and the resolution. For all my backbones and all the resolution, BS is from 16 to 64.
", "rawMarkdown": "depends on the backbone and the resolution. For all my backbones and all the resolution, BS is from 16 to 64." } ] } ] }, { "id": 2586468, "postDate": "2024-01-04T08:43:39.003Z", "content": "\nThe architecture of my methods is updated.
", "rawMarkdown": "~~I have uploaded a figure of the architecture of my methods. However, I can't present it in this page. Maybe you have to click [https://i.imgs.ovh/2024/01/04/B2va2.png](this url) to see.~~\n\nThe architecture of my methods is updated. " }, { "id": 2586443, "postDate": "2024-01-04T08:09:40.520Z", "content": "congrat on solo gold and GM title!
\na question regarding inference, when you say \"Models: 5-fold effv2s\", does it mean you used 5 models trained on 5 fold of data? or it is a single model?
", "rawMarkdown": "congrat on solo gold and GM title!\n\na question regarding inference, when you say \"Models: 5-fold effv2s\", does it mean you used 5 models trained on 5 fold of data? or it is a single model?", "replies": [ { "id": 2586445, "postDate": "2024-01-04T08:14:04.337Z", "content": "Yes, it's a 5-folds cross validation.
", "rawMarkdown": "Yes, it's a 5-folds cross validation." } ] }, { "id": 2586359, "postDate": "2024-01-04T06:37:46.160Z", "content": "Congratulations！
", "rawMarkdown": "Congratulations！" }, { "id": 2586428, "postDate": "2024-01-04T07:49:37.780Z", "rawMarkdown": "", "votes": 1, "isDeleted": true } ], "comments": [ { "id": 2586391, "author_name": "", "author_url": "", "post_date": "2024-01-04T07:07:08.817000", "content": "congratulation for GM!!
", "votes": 3, "replies": [] }, { "id": 2591621, "author_name": "L-Lee", "author_url": "", "post_date": "2024-01-08T05:42:03.237000", "content": "Congratulations!! I really love your graph of the Architecture of the solution. So clear and informative!!
\nMay I ask how you came up with the idea of \"rough detection on outliers\" with Arcface on medical images? (Based on your description, the Arcface is a cheap way to handle most of the TMAs, but how did you come up with it the first time? )
\nAnd you fine-tuned the Arcface with the only 25 TMAs the dataset provided?
\nSorry for bothering you with newbie questions🙋♀️
Hi @lizzylee1111111111 ,
\nMay I ask how you came up with the idea of \"rough detection on outliers\" with Arcface on medical images? . Arcface is usually used in Image Retrieval task. To detect outliers, tackling with image retrieval instead of image classification is better.
I fine-tuned the arcface with the patch cropped on the masks provided by the host.
Congratulations on becoming a new kaggle GM! 祝贺来自土木老哥😆
", "votes": 1, "replies": [] }, { "id": 2587109, "author_name": "Bartley", "author_url": "", "post_date": "2024-01-04T15:11:31.650000", "content": "Congrats on becoming a GM, and thanks for the write-up! Some interesting ideas here.
", "votes": 1, "replies": [] }, { "id": 2586965, "author_name": "MrSimple", "author_url": "", "post_date": "2024-01-04T13:52:36.760000", "content": "Thanks for the solution! It is really good!
", "votes": 1, "replies": [] }, { "id": 2586722, "author_name": "Seeing Times", "author_url": "", "post_date": "2024-01-04T11:13:16.690000", "content": "Congrats for becoming Grandmaster🥳, I just wondering if the code is available for this solution?
", "votes": 1, "replies": [ { "id": 2586865, "author_name": "ForcewithMe", "author_url": "", "post_date": "2024-01-04T12:53:56.563000", "content": "Thank you! There are nothing special in my code. I may make it available after I review them. But you remind me an important key points which I forget to write in my post. That is, Rank the DataFrame of WSI with the number of pixels first, and process the WSIs in the order with multi-thread can greatly speed the the WSI processing procedure.
", "votes": 1, "replies": [ { "id": 2586909, "author_name": "Seeing Times", "author_url": "", "post_date": "2024-01-04T13:28:33.390000", "content": "Thanks for your sharing！
", "votes": 0, "replies": [] } ] } ] }, { "id": 2586520, "author_name": "olivepicker", "author_url": "", "post_date": "2024-01-04T09:15:56.323000", "content": "Great top solution. I learned a lot, thanks for the detailed method and congratulations becoming GM!
", "votes": 1, "replies": [] }, { "id": 2586280, "author_name": "MOONMOON", "author_url": "", "post_date": "2024-01-04T05:29:56.587000", "content": "Congratulations on winning your first solo gold and becoming a grandmaster!🎉🎉🎉
", "votes": 1, "replies": [ { "id": 2586308, "author_name": "ForcewithMe", "author_url": "", "post_date": "2024-01-04T05:56:11.157000", "content": "@kevin1742064161 Thank you! You are an extraordinary teammate. Hope to have more cooperation with you in the future. Also, wish you to reach a grandmaster soon.
", "votes": 0, "replies": [] } ] }, { "id": 2586274, "author_name": "devchopin", "author_url": "", "post_date": "2024-01-04T05:22:17.133000", "content": "Congratulations on becoming GM. I also became a Master through this competition and will work hard for GM.
", "votes": 1, "replies": [ { "id": 2586310, "author_name": "ForcewithMe", "author_url": "", "post_date": "2024-01-04T05:57:45.453000", "content": "Congratulations! Wish you become a gm as you wish in the future.
", "votes": 0, "replies": [] } ] }, { "id": 2586273, "author_name": "lty", "author_url": "", "post_date": "2024-01-04T05:20:04.443000", "content": "Congrats for becoming GM!
", "votes": 1, "replies": [ { "id": 2586276, "author_name": "ForcewithMe", "author_url": "", "post_date": "2024-01-04T05:24:49.917000", "content": "Thank you LTY, working with you in previous competition is enjoyable.
", "votes": 0, "replies": [] } ] }, { "id": 2586237, "author_name": "shigengtian", "author_url": "", "post_date": "2024-01-04T04:27:24.703000", "content": "Happy New Year!
", "votes": 1, "replies": [] }, { "id": 2586220, "author_name": "C R Suthikshn Kumar", "author_url": "", "post_date": "2024-01-04T04:00:10.760000", "content": "Congratulation on achieving the 8th position in this competition. Thanks for sharing the details of your solution.
", "votes": 1, "replies": [ { "id": 2586221, "author_name": "ForcewithMe", "author_url": "", "post_date": "2024-01-04T04:00:38.120000", "content": "Thank you!
", "votes": 0, "replies": [] } ] }, { "id": 2586495, "author_name": "Jirka", "author_url": "", "post_date": "2024-01-04T08:56:12.940000", "content": "Great work, I have noticed that all your models use very large resolutions, so was it trained on Kaggle or outside? and if on Kaggle how do you fit to limited P100 memory?
", "votes": 0, "replies": [ { "id": 2586833, "author_name": "ForcewithMe", "author_url": "", "post_date": "2024-01-04T12:31:47.060000", "content": "I only use the GPU on Kaggle for inference debugging. I finish all my training on 1 A100 .
", "votes": 1, "replies": [ { "id": 2586846, "author_name": "Jirka", "author_url": "", "post_date": "2024-01-04T12:39:30.873000", "content": "I though so :) and what was the batch size on A100?
", "votes": 0, "replies": [] }, { "id": 2586857, "author_name": "ForcewithMe", "author_url": "", "post_date": "2024-01-04T12:43:54.590000", "content": "depends on the backbone and the resolution. For all my backbones and all the resolution, BS is from 16 to 64.
", "votes": 0, "replies": [] } ] } ] }, { "id": 2586468, "author_name": "ForcewithMe", "author_url": "", "post_date": "2024-01-04T08:43:39.003000", "content": "\nThe architecture of my methods is updated.
", "votes": 0, "replies": [] }, { "id": 2586443, "author_name": "Ali", "author_url": "", "post_date": "2024-01-04T08:09:40.520000", "content": "congrat on solo gold and GM title!
\na question regarding inference, when you say \"Models: 5-fold effv2s\", does it mean you used 5 models trained on 5 fold of data? or it is a single model?
", "votes": 0, "replies": [ { "id": 2586445, "author_name": "ForcewithMe", "author_url": "", "post_date": "2024-01-04T08:14:04.337000", "content": "Yes, it's a 5-folds cross validation.
", "votes": 0, "replies": [] } ] }, { "id": 2586359, "author_name": "Haoran Zhao", "author_url": "", "post_date": "2024-01-04T06:37:46.160000", "content": "Congratulations！
", "votes": 0, "replies": [] }, { "id": 2586428, "author_name": "", "author_url": "", "post_date": "2024-01-04T07:49:37.780000", "content": "", "votes": 1, "replies": [] } ], "raw_markdown_by_id": { "2586211": "# Acknowledgments \n \nI would like to express my gratitude to Kaggle, the organizers, and other participants in the community. I have learned a great deal from this competition and hope that it will promote the advancement of research in MIL，Classification and outliers detection of Human tissues, and the study of women's health.\n\n# Introduction \n \nThe first and most crucial step in this competition is to familiarize oneself with the feature of Whole Slide Images (WSI) and Tissue Microarray (TMA) images. A simple visual examination reveals that **the features of TMA and WSI are presented at vastly different scales.** TMA features are at the cellular level, while WSI features are several to tens of times larger than cell clusters. Hence, from the onset of the competition, I decided to tackle WSI and TMA with two completely distinct approaches. \n\n## Key points for the global approach: \n\n1. **Address WSI and TMA separately** .\n2. **Align the magnification of TMA and WSI** to enable the reuse of TMA training code for optimizing the WSI feature extractor. \n - WSI images are 20x magnified, while TMA images are 40x. Thus, TMA images need to be downsampled by a factor of 2 to align with the physical scale of WSI, which can also be observed with a visual inspection of the training data. \n \n## Architecture of the solution\n\n\n\n## TMA Approach \n \n### Summary of Key Points: \n1. **Train with patches tiled with official masks** (most important globally). \n2. Train the model with **healthy and death patches** to predict some outliers. \n3. Employ **arcface** to retrieve some outliers and part of the 5-class classification. \n4. Heavy ensemble of 6 classification models to predict samples that arcface retrieval did not cover. \n \n### Inference: \n\n1. Use arcface retrieval with the training set's TMA as templates. Output 5-class results for samples with a close cosine distance, and consider those with a far cosine distance as outliers. Samples that are uncertain in the arcface stage are left for the subsequent phase. \n - Models: 5-fold effv2s + 5-fold convnext small with dynamic margin, subcenter=3. \n - Top1 threshold at 0.05, Other threshold at 0.2. \n \n2. The 6-class model uses 2-fold effv2s + 5-fold effv2l + 4-fold convnext small + 3-fold convnext large. \n \n| Model | Resolution |\n| --- | --- |\n| Effv2s | 1280 |\n| Effv2l | 1280 |\n| Convnext Small | 1024 |\n| Convnext Large | 1024 |\n\n\nNote: The arcface can tackle about 60% of the TMAs. So even the second stage is heavy, it doesn't cause timeout.\n\n### Training: \n\n1. Tile patches using the official segmentation masks. In addition to the official 5 categories, classify patches of `healthy` and `dead` as `Other` class. Instead of training with the official TMAs , which has limited number, use them for subsequent validation and retrieval. \n2. Use the first-phase models to generate pseudo-label patches on the remaining 300+ WSIs without mask. \n3. Inherit the weights from the first or second step, training only the last layer of the backbone and the arcface head. \n \n## WSI Approach \n \n### Summary of Key Points: \n1. **Train the feature extractor using the TMA pipeline**. \n2. **Synthesize `Other` WSI** during the training process. \n3. **Integrate different magnification scales.** \n4. Rank the DataFrame of WSI with the number of pixels, and process the WSIs with multithread can greatly speed the the WSI processing procedure.\n \n### Inference: \n\n1. Ensemble of two resolutions of feature extractor: 3072 resized to 768, and 1024 without down-sampling. After extracting features, apply DTFD-MIL. \n2. For speed up the image processing procedure, I only use the center region of each 3072 tile, so that I only need to crop patch from the WSI once. \n \n| Model | Resolution | Number folds |\n| --- | --- | ---|\n| Convnext Small | 3072 resize 768 | 2 |\n| Effv2s | 3072 resize 768 | 3 |\n| Effv2s | 1024 | 4 |\n\n\n### Training: \n\n1. Train the feature extractor using the TMA pipeline. \n2. Extract features using the feature extractor. \n3. Predict the probability of 'Other' on all patches using the feature extractor, and **create an `Other pool`** with patches that have a high probability of 'Other'. \n4. During the training of DTFD-MIL, dynamically synthesize some `WSI` from the `Other pool` each epoch. \n - Training DTFD-MIL serves as a validation for whether the TMA pipeline models truly learned useful features. If we use the pretrained weights from ImageNet, DTFD-MIL requires up to 200 epochs to converge. In contrast, using models trained on TMA, the MIL Head may take as little as 1 epoch and at most 20 epochs to converge.\n\n## Something I don't have time to try but I think may work\n1. Arcface for WSI. \n2. Large transformer pretrained on Large set of Slides. In fact I tried PLIP at the early stage of this competition, But I didn't dig deeper.\n3. Better retrieve strategy for arcface.\n4. More resolution for WSI . I tried to add `6144 resize to 1024` into my final pipeline. But the notebook crashed.\n5. Ensemble of MIL head\n6. External data.\n\n## Todo: \n\nThis post is ~~not completed . Here are todo lists today~~ almost completed :\n1. ~~Model details~~\n2. ~~Citation of some methods in the post.~~\n3. ~~Discussion on MIL~~\n4. ~~Some figures about the pipeline and EDA\n~~\nIf possible, I may add more ablation study in two weeks.\n\n## Discussion and Citation\n1. [DTFD-MIL](https://arxiv.org/abs/2203.12081) is a strong and robust baseline for MIL, which follows ABMIL. \n2. I think MIL methods are not sensitive to the position and the number of patches, which is observed in my experiment. That's why I randomly synthesized some `WSI` from the `Other pool` each epoch for WSI. And reduce the number on `1024 resolution wo down-sample` for submission. This is also proved in [this paper](https://ieeexplore.ieee.org/document/10219719/authors#authors). \n3. The reason why I ensemble multiple resolution for WSI: if you check the [clarification on WHO](https://www.pathologyoutlines.com/topic/ovarytumorwhoclassif.html). You will find that Pathologists distinguish the subtypes of Ovarian cancer on different magnification. So I believe ensemble multiple resolution for classifying a WSI should be very important. \n4. About Larger transformer-based model: I believe the competition needs model with better generalization. And larger models are often more robust. At the early stage of this competition, I takes some time to test PLIP offline, but the CV is not good. Also, since I decide to solo this time, So I have to spend my limited time on the direction I am most confident about, solving this task in a more traditional and solid way. \n\n## Closing Thoughts: My Journey to Grandmaster \n \nThe path to becoming a Grandmaster has been lengthy, filled with challenges, and an experience I will cherish for life. I am incredibly thankful for the support from my teammates in past competitions, as well as the unwavering encouragement from my family and girlfriend throughout my journey. The final step to becoming a Grandmaster, achieving a solo gold medal, has been particularly solitary and tough. This was my fourth attempt at a solo gold medal. if I failed this time, it might have been three years, ten years, or perhaps never before I'd have the chance again, as I am about to graduate with my master's degree and start my career in a busy company. Fortunately, I have realized the dream I had three years ago and have now brought my student years to a close with the title of Grandmaster. Wishing everyone a Happy New Year!", "2586391": "congratulation for GM!!", "2591621": "Congratulations!! I really love your graph of the Architecture of the solution. So clear and informative!!\nMay I ask how you came up with the idea of \"rough detection on outliers\" with Arcface on medical images? (Based on your description, the Arcface is a cheap way to handle most of the TMAs, but how did you come up with it the first time? )\nAnd you fine-tuned the Arcface with the only 25 TMAs the dataset provided?\nSorry for bothering you with newbie questions🙋♀️", "2587165": "Congratulations on becoming a new kaggle GM! 祝贺来自土木老哥😆", "2587109": "Congrats on becoming a GM, and thanks for the write-up! Some interesting ideas here.", "2586965": "Thanks for the solution! It is really good!", "2586722": "Congrats for becoming Grandmaster🥳, I just wondering if the code is available for this solution?", "2586520": "Great top solution. I learned a lot, thanks for the detailed method and congratulations becoming GM!", "2586280": "Congratulations on winning your first solo gold and becoming a grandmaster!🎉🎉🎉", "2586274": "Congratulations on becoming GM. I also became a Master through this competition and will work hard for GM.", "2586273": "Congrats for becoming GM!", "2586237": "Happy New Year!", "2586220": "Congratulation on achieving the 8th position in this competition. Thanks for sharing the details of your solution.\n", "2586495": "Great work, I have noticed that all your models use very large resolutions, so was it trained on Kaggle or outside? and if on Kaggle how do you fit to limited P100 memory?", "2586468": "~~I have uploaded a figure of the architecture of my methods. However, I can't present it in this page. Maybe you have to click [https://i.imgs.ovh/2024/01/04/B2va2.png](this url) to see.~~\n\nThe architecture of my methods is updated. ", "2586443": "congrat on solo gold and GM title!\n\na question regarding inference, when you say \"Models: 5-fold effv2s\", does it mean you used 5 models trained on 5 fold of data? or it is a single model?", "2586359": "Congratulations！", "2586428": "" } }