Dear every kagglers,
\nI am new to join formal competition.
\nI tried to finetune a ResNet50 model with tensorflow, but I can not get any good performance on my training pipeline.
\nCan someone discuss with me and point out where I went wrong? (I don't think this pipeline only can get LB0.04, it should be a little bit higher if I fix some strange bugs.)
\nPlease check the code below:
\nhttps://www.kaggle.com/dog14230pp/rsna1-resnet50
I really wanna to know where I made mistake then I can improve my ability on CV!
\nAlso, here are something I will try to do in the future (maybe this weekend):
\nReally thanks for your kindly help! Hope you can have a nice year!
", "messages": [ { "id": 2134918, "postDate": "2023-02-08T10:40:27.923Z", "content": "Hi @dog14230pp,
\nI believe the issue is with the way you preprocess the images. Here are a few things you can try to help your model learn, as a score of 0.04 suggests that it's not learning at all right now:
dicomsdl and you'll need to write it yourself.filter function in TensorFlow Datasets.Even though you may be able to achieve better results with a different model than ResNet50, focus on solving the preprocessing issue first. By fixing the preprocessing, you should be able to see improvements, and then you can consider trying other models later.)
", "rawMarkdown": "Hi @dog14230pp,\nI believe the issue is with the way you preprocess the images. Here are a few things you can try to help your model learn, as a score of 0.04 suggests that it's not learning at all right now:\n\n- The image size of 224 pixels may be **too small** to detect the contrasts within the breast tissue. Try using images with a minimum size of 512 pixels as a baseline (1024 is even better). Additionally, the dataset you're using still contains a lot of background information, which reduces the amount of relevant information even further.\n- Consider using **cropped images** (smallest amount of background as possible) with a different aspect ratio. I personally use a 1:2 aspect ratio and it works well (512x256, 1024x512 or 1456x728).\n- **Apply windowing** to all images. This is an important step that you did not use. Windowing allows you to view the images in the same way a radiologist would. Check my [preprocessing notebook](https://www.kaggle.com/code/paulbacher/custom-preprocessor-rsna-breast-cancer) for guidance on how to do this step, as the function is not implemented in `dicomsdl` and you'll need to write it yourself.\n- Regarding the class imbalance: While using class weights is a good idea, you can also **undersample** the negative examples using the `filter` function in TensorFlow Datasets.\n\nEven though you may be able to achieve better results with a different model than ResNet50, focus on solving the preprocessing issue first. By fixing the preprocessing, you should be able to see improvements, and then you can consider trying other models later.)", "votes": 5, "replies": [ { "id": 2136782, "postDate": "2023-02-09T14:38:43.590Z", "content": "Hi, really thanks for your help!
\nThere are many interesting things that I can try, especially for windowing.
\nI will try these things this weekend, then update the results here.
\nReally thanks for your kindly reply! They are useful to me!
Dear every kagglers,
\nI am new to join formal competition.
\nI tried to finetune a ResNet50 model with tensorflow, but I can not get any good performance on my training pipeline.
\nCan someone discuss with me and point out where I went wrong? (I don't think this pipeline only can get LB0.04, it should be a little bit higher if I fix some strange bugs.)
\nPlease check the code below:
\nhttps://www.kaggle.com/dog14230pp/rsna1-resnet50
I really wanna to know where I made mistake then I can improve my ability on CV!
\nAlso, here are something I will try to do in the future (maybe this weekend):
\nReally thanks for your kindly help! Hope you can have a nice year!
", "rawMarkdown": "Dear every kagglers,\nI am new to join formal competition. \nI tried to finetune a ResNet50 model with tensorflow, but I can not get any good performance on my training pipeline.\nCan someone discuss with me and point out where I went wrong? (I don't think this pipeline only can get LB0.04, it should be a little bit higher if I fix some strange bugs.)\nPlease check the code below:\nhttps://www.kaggle.com/dog14230pp/rsna1-resnet50\n\nI really wanna to know where I made mistake then I can improve my ability on CV!\n\nAlso, here are something I will try to do in the future (maybe this weekend):\n* ROI (But I don't know it is important or not.)\n* Change base model\n\nReally thanks for your kindly help! Hope you can have a nice year!", "votes": 6 }, { "id": 2135338, "postDate": "2023-02-08T15:12:26.153Z", "content": "cool。。。。。。
", "rawMarkdown": "cool。。。。。。", "votes": -1, "replies": [ { "id": 2136767, "postDate": "2023-02-09T14:34:09.853Z", "content": "Do you have any interesting thing to share? XD
", "rawMarkdown": "Do you have any interesting thing to share? XD", "votes": -1 } ] }, { "id": 2132310, "postDate": "2023-02-06T18:31:05.220Z", "content": "I also have around that score on my CV! I tried changing base model and ROI, but it didn't make a big change. Hope someones response will help
", "rawMarkdown": "I also have around that score on my CV! I tried changing base model and ROI, but it didn't make a big change. Hope someones response will help", "replies": [ { "id": 2133667, "postDate": "2023-02-07T14:46:57.640Z", "content": "Hi, so your experience told you that ROI is not really important at all?
", "rawMarkdown": "Hi, so your experience told you that ROI is not really important at all?", "votes": -1, "replies": [ { "id": 2133941, "postDate": "2023-02-07T17:14:47.393Z", "content": "Don't know. Papers suggest doing ROI, but I guess my model just can't learn to distinguish positive examples and negative examples. And so I'm stuck
", "rawMarkdown": "Don't know. Papers suggest doing ROI, but I guess my model just can't learn to distinguish positive examples and negative examples. And so I'm stuck " }, { "id": 2136792, "postDate": "2023-02-09T14:40:43.253Z", "content": "Hi, I think that we can refer to Paul Bacher's suggestions. Let's try and see whether ROI is important or not!
", "rawMarkdown": "Hi, I think that we can refer to Paul Bacher's suggestions. Let's try and see whether ROI is important or not!", "votes": -1 }, { "id": 2137392, "postDate": "2023-02-10T00:07:24.867Z", "content": "I did the preprocessing as suggested, but this was the result :/
", "rawMarkdown": "I did the preprocessing as suggested, but this was the result :/" } ] } ] }, { "id": 2132135, "postDate": "2023-02-06T16:06:17.690Z", "content": "Do you check your local CV score?
\nTo check the gap between your local CV and LB to make sure that no any bug in your inference stage.
\nAlso, you can try to \"optimize\" you pf1 score by setting a threshold in this discussion.
Hi, thanks for your suggestion.
\nI haven't tried CV, I will add this in my todo list.
\nAnd I will try to apply to optimize pf1 score! This is really interesting!
Hi @dog14230pp,
\nI believe the issue is with the way you preprocess the images. Here are a few things you can try to help your model learn, as a score of 0.04 suggests that it's not learning at all right now:
dicomsdl and you'll need to write it yourself.filter function in TensorFlow Datasets.Even though you may be able to achieve better results with a different model than ResNet50, focus on solving the preprocessing issue first. By fixing the preprocessing, you should be able to see improvements, and then you can consider trying other models later.)
", "votes": 5, "replies": [ { "id": 2136782, "author_name": "Hao-Lun Sun", "author_url": "", "post_date": "2023-02-09T14:38:43.590000", "content": "Hi, really thanks for your help!
\nThere are many interesting things that I can try, especially for windowing.
\nI will try these things this weekend, then update the results here.
\nReally thanks for your kindly reply! They are useful to me!
cool。。。。。。
", "votes": -1, "replies": [ { "id": 2136767, "author_name": "Hao-Lun Sun", "author_url": "", "post_date": "2023-02-09T14:34:09.853000", "content": "Do you have any interesting thing to share? XD
", "votes": -1, "replies": [] } ] }, { "id": 2132310, "author_name": "Illia Rohalskyi", "author_url": "", "post_date": "2023-02-06T18:31:05.220000", "content": "I also have around that score on my CV! I tried changing base model and ROI, but it didn't make a big change. Hope someones response will help
", "votes": 0, "replies": [ { "id": 2133667, "author_name": "Hao-Lun Sun", "author_url": "", "post_date": "2023-02-07T14:46:57.640000", "content": "Hi, so your experience told you that ROI is not really important at all?
", "votes": -1, "replies": [ { "id": 2133941, "author_name": "Illia Rohalskyi", "author_url": "", "post_date": "2023-02-07T17:14:47.393000", "content": "Don't know. Papers suggest doing ROI, but I guess my model just can't learn to distinguish positive examples and negative examples. And so I'm stuck
", "votes": 0, "replies": [] }, { "id": 2136792, "author_name": "Hao-Lun Sun", "author_url": "", "post_date": "2023-02-09T14:40:43.253000", "content": "Hi, I think that we can refer to Paul Bacher's suggestions. Let's try and see whether ROI is important or not!
", "votes": -1, "replies": [] }, { "id": 2137392, "author_name": "Illia Rohalskyi", "author_url": "", "post_date": "2023-02-10T00:07:24.867000", "content": "I did the preprocessing as suggested, but this was the result :/
", "votes": 0, "replies": [] } ] } ] }, { "id": 2132135, "author_name": "JimmyLiao", "author_url": "", "post_date": "2023-02-06T16:06:17.690000", "content": "Do you check your local CV score?
\nTo check the gap between your local CV and LB to make sure that no any bug in your inference stage.
\nAlso, you can try to \"optimize\" you pf1 score by setting a threshold in this discussion.
Hi, thanks for your suggestion.
\nI haven't tried CV, I will add this in my todo list.
\nAnd I will try to apply to optimize pf1 score! This is really interesting!