{ "id": 376251, "title": "New loss for imbalanced medical image classification", "url": "/competitions/rsna-breast-cancer-detection/discussion/376251", "author_name": "阳光开朗大男孩", "post_date": "2023-01-05T13:48:44.978000", "votes": 5, "comment_count": 8, "views": 0, "content": "

Link here, maybe we can have a try, good luck!

", "messages": [ { "id": 2087284, "postDate": "2023-01-05T13:48:44.980Z", "content": "

Link here, maybe we can have a try, good luck!

", "rawMarkdown": "Link [here](https://arxiv.org/abs/2212.12741v1), maybe we can have a try, good luck!", "votes": 5 }, { "id": 2087491, "postDate": "2023-01-05T16:22:32.697Z", "content": "

Thank you for providing this paper. I think that “problem” is in DS :) I have done many experiments training (using different losses, sampling strategies, sophisticated augumentations, different models even custom one) and still see images where model is unable to see cancer (even they are labeled positive). I am not able to cross 0.80 AUC_ROC (my best one was 0.83 -> LB: single model 0.39) on valid fold.

", "rawMarkdown": "Thank you for providing this paper. I think that “problem” is in DS :) I have done many experiments training (using different losses, sampling strategies, sophisticated augumentations, different models even custom one) and still see images where model is unable to see cancer (even they are labeled positive). I am not able to cross 0.80 AUC_ROC (my best one was 0.83 -> LB: single model 0.39) on valid fold.", "replies": [ { "id": 2088008, "postDate": "2023-01-06T01:44:36.967Z", "content": "

So, any idea on DS? Add external DS, or try higher resolution? BTW, in my test, ROI (Resize to 1024_512 in transformation) doesn't seem more useful than the original image (1024_1024)…desperate😂

", "rawMarkdown": "So, any idea on DS? Add external DS, or try higher resolution? BTW, in my test, ROI (Resize to 1024_512 in transformation) doesn't seem more useful than the original image (1024_1024)...desperate😂", "replies": [ { "id": 2090495, "postDate": "2023-01-07T11:37:41.170Z", "content": "

In my experiments:

\n", "rawMarkdown": "In my experiments:\n- ROI is always better (currently I validate only locally - submission is only done for single model to check some hypotesis)\n- external DS will help for sure (most of external datasets break competition rules - are not for commercial use)\n- higher resolution helps - but not so much \n- I can see that some images are easly classified but other no (I did some analysis with data and ... it appeared that correct classified images are easy ceases - I can see cancer as well)\n- models easly overfit (I am able to get model with low training loss but the lower it is the AUC_ROC is lower as well - model do not learn cancer) \n", "votes": 1 }, { "id": 2090496, "postDate": "2023-01-07T11:39:00.940Z", "content": "

I made code for this loss and now testing it. It can help but … I think that this is no main solution to push model to see cancer.

", "rawMarkdown": "I made code for this loss and now testing it. It can help but ... I think that this is no main solution to push model to see cancer." }, { "id": 2090509, "postDate": "2023-01-07T11:58:19.017Z", "content": "

Nice work! I'll also keep trying on your advice. My eval pf1 for each fold and local cv has stuck at around 0.23-0.26 for long, which gives me 0.38-0.4 LB, no idea how to get a breakthrough😂(having tried different arch, more augs, different num_fold)

", "rawMarkdown": "Nice work! I'll also keep trying on your advice. My eval pf1 for each fold and local cv has stuck at around 0.23-0.26 for long, which gives me 0.38-0.4 LB, no idea how to get a breakthrough😂(having tried different arch, more augs, different num_fold)", "votes": 1 }, { "id": 2090912, "postDate": "2023-01-07T19:21:33.193Z", "content": "

I think many people are in the same situation. Keep trying. We have 2 months. We can learn a lot.
\nFor example … today I managed to cross AUC_ROC over 0.845 … (one model for one fold). Why? I made more demanding augumentation for model.

", "rawMarkdown": "I think many people are in the same situation. Keep trying. We have 2 months. We can learn a lot. \nFor example ... today I managed to cross AUC_ROC over 0.845 ... (one model for one fold). Why? I made more demanding augumentation for model." } ] } ] }, { "id": 2087324, "postDate": "2023-01-05T14:28:01.643Z", "content": "

Your link to the paper didn't work. I guess this is the correct link,
\nhttps://arxiv.org/abs/2212.12741v1

", "rawMarkdown": "Your link to the paper didn't work. I guess this is the correct link,\nhttps://arxiv.org/abs/2212.12741v1", "replies": [ { "id": 2087334, "postDate": "2023-01-05T14:33:14.050Z", "content": "

yep, forget to delete 'url'😂, thx!

", "rawMarkdown": "yep, forget to delete 'url'😂, thx!" } ] } ], "comments": [ { "id": 2087491, "author_name": "Remek Kinas", "author_url": "", "post_date": "2023-01-05T16:22:32.697000", "content": "

Thank you for providing this paper. I think that “problem” is in DS :) I have done many experiments training (using different losses, sampling strategies, sophisticated augumentations, different models even custom one) and still see images where model is unable to see cancer (even they are labeled positive). I am not able to cross 0.80 AUC_ROC (my best one was 0.83 -> LB: single model 0.39) on valid fold.

", "votes": 0, "replies": [ { "id": 2088008, "author_name": "阳光开朗大男孩", "author_url": "", "post_date": "2023-01-06T01:44:36.967000", "content": "

So, any idea on DS? Add external DS, or try higher resolution? BTW, in my test, ROI (Resize to 1024_512 in transformation) doesn't seem more useful than the original image (1024_1024)…desperate😂

", "votes": 0, "replies": [ { "id": 2090495, "author_name": "Remek Kinas", "author_url": "", "post_date": "2023-01-07T11:37:41.170000", "content": "

In my experiments:

\n", "votes": 1, "replies": [] }, { "id": 2090496, "author_name": "Remek Kinas", "author_url": "", "post_date": "2023-01-07T11:39:00.940000", "content": "

I made code for this loss and now testing it. It can help but … I think that this is no main solution to push model to see cancer.

", "votes": 0, "replies": [] }, { "id": 2090509, "author_name": "阳光开朗大男孩", "author_url": "", "post_date": "2023-01-07T11:58:19.017000", "content": "

Nice work! I'll also keep trying on your advice. My eval pf1 for each fold and local cv has stuck at around 0.23-0.26 for long, which gives me 0.38-0.4 LB, no idea how to get a breakthrough😂(having tried different arch, more augs, different num_fold)

", "votes": 1, "replies": [] }, { "id": 2090912, "author_name": "Remek Kinas", "author_url": "", "post_date": "2023-01-07T19:21:33.193000", "content": "

I think many people are in the same situation. Keep trying. We have 2 months. We can learn a lot.
\nFor example … today I managed to cross AUC_ROC over 0.845 … (one model for one fold). Why? I made more demanding augumentation for model.

", "votes": 0, "replies": [] } ] } ] }, { "id": 2087324, "author_name": "Rasoul Mojtahedzadeh", "author_url": "", "post_date": "2023-01-05T14:28:01.643000", "content": "

Your link to the paper didn't work. I guess this is the correct link,
\nhttps://arxiv.org/abs/2212.12741v1

", "votes": 0, "replies": [ { "id": 2087334, "author_name": "阳光开朗大男孩", "author_url": "", "post_date": "2023-01-05T14:33:14.050000", "content": "

yep, forget to delete 'url'😂, thx!

", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "2087284": "Link [here](https://arxiv.org/abs/2212.12741v1), maybe we can have a try, good luck!", "2087491": "Thank you for providing this paper. I think that “problem” is in DS :) I have done many experiments training (using different losses, sampling strategies, sophisticated augumentations, different models even custom one) and still see images where model is unable to see cancer (even they are labeled positive). I am not able to cross 0.80 AUC_ROC (my best one was 0.83 -> LB: single model 0.39) on valid fold.", "2087324": "Your link to the paper didn't work. I guess this is the correct link,\nhttps://arxiv.org/abs/2212.12741v1" } }