![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2626211%2F9adfa6782e56688e5b0b4a45dd620794%2F999.PNG?generation=1696398858857774&alt=media\")
Hi to all!
\nI don't understand what I'm missing. I use this code as a base from https://www.kaggle.com/code/awsaf49/rsna-atd-cnn-tpu-infer (with major modifications). Now I'm testing spleen classification, but it seems that tf.keras.losses.CategoricalCrossentropy(label_smoothing=0.05) instead of classifying objects just dances around their frequency:
Here, the frequencies differ slightly from the real ones due to the weighting of the classes. I tried Huge loss but it only made my result worse. At the same time, the AUC at this fold is stable at 0.92-0.93. This model performs slightly better than the better public model 0.65, but essentially
\nmodel trained using tf.keras.losses.CategoricalCrossentropy does not actually classify the spleen, it only returns the average frequencies of the classes.
\nWhat could be the reason? The model cannot distinguish features that separate ct, or did I miss something in the implementation? Considering that 95% of participants could not cross the 0.65 barrier, this problem is not only mine. Important I transfer the whole ct to the model as one object. For the spleen I form a slice. Thank you all for your comments!
Hi to all!
\nI don't understand what I'm missing. I use this code as a base from https://www.kaggle.com/code/awsaf49/rsna-atd-cnn-tpu-infer (with major modifications). Now I'm testing spleen classification, but it seems that tf.keras.losses.CategoricalCrossentropy(label_smoothing=0.05) instead of classifying objects just dances around their frequency:
Here, the frequencies differ slightly from the real ones due to the weighting of the classes. I tried Huge loss but it only made my result worse. At the same time, the AUC at this fold is stable at 0.92-0.93. This model performs slightly better than the better public model 0.65, but essentially
\nmodel trained using tf.keras.losses.CategoricalCrossentropy does not actually classify the spleen, it only returns the average frequencies of the classes.
\nWhat could be the reason? The model cannot distinguish features that separate ct, or did I miss something in the implementation? Considering that 95% of participants could not cross the 0.65 barrier, this problem is not only mine. Important I transfer the whole ct to the model as one object. For the spleen I form a slice. Thank you all for your comments!
I think that the problem that we are facing might not be related to the choice of model or loss function, rather it is with the data that we are feeding this model. There is a very high probability that the model is not able to learn anything meaningful from the data, and it just learns the frequency of the predictions.
", "rawMarkdown": "I think that the problem that we are facing might not be related to the choice of model or loss function, rather it is with the data that we are feeding this model. There is a very high probability that the model is not able to learn anything meaningful from the data, and it just learns the frequency of the predictions.", "votes": 3, "replies": [ { "id": 2472010, "postDate": "2023-10-06T21:03:17.143Z", "content": "Agree, in my experience full size image doesn't works. To many noise for a weak label. You should realize how to cease area of interests for an every organ. For example it will work if you try to classify images inside bboxes objects like spleen kidney liver.
", "rawMarkdown": "Agree, in my experience full size image doesn't works. To many noise for a weak label. You should realize how to cease area of interests for an every organ. For example it will work if you try to classify images inside bboxes objects like spleen kidney liver. ", "votes": 3 } ] }, { "id": 2467290, "postDate": "2023-10-04T12:19:38.887Z", "content": "I think the worst part is most of the submissions were just weighted scaled mean values of particular target category.
", "rawMarkdown": "I think the worst part is most of the submissions were just weighted scaled mean values of particular target category.", "votes": 3, "replies": [ { "id": 2467296, "postDate": "2023-10-04T12:24:22.120Z", "content": "That's right, these models don't solve the real problem, simple statistics work better.
", "rawMarkdown": "That's right, these models don't solve the real problem, simple statistics work better.", "votes": 2, "replies": [ { "id": 2467370, "postDate": "2023-10-04T13:47:19.380Z", "content": "literally today my youtube homepage has this video : https://youtube.com/shorts/h21Hc8O6nhY?feature=shared
", "rawMarkdown": "literally today my youtube homepage has this video : https://youtube.com/shorts/h21Hc8O6nhY?feature=shared", "votes": 2 } ] } ] }, { "id": 2472473, "postDate": "2023-10-07T11:03:41.753Z", "content": "I don't know if you noticed but some people instantly broke that barrier you just mentioned (including me). The reason is they have experience of dealing with similar data. If it was early stages, I would share my approach but it's too late to share anything now.
", "rawMarkdown": "I don't know if you noticed but some people instantly broke that barrier you just mentioned (including me). The reason is they have experience of dealing with similar data. If it was early stages, I would share my approach but it's too late to share anything now. ", "votes": 4 }, { "id": 2472305, "postDate": "2023-10-07T07:57:52.490Z", "content": "Hi @aikhmelnytskyy, I think it is better if you don't mind about
\n\n\nConsidering that 95% of participants could not cross the 0.65 barrier, this problem is not only mine.
\n
I am also one of these 95% you have mentioned.
\nI assume most of the people who are not crossing 0.65 barrier are the ones who make the simple modifications to the scales.
\nAs I mentioned in one of my notebooks (focusing on this scale modification), there will be big changes to the leaderboard for those who are focusing on the scales (Like myself).
\nThat is also why I am working on vision techniques (but it is tough to break that 0.65 as well xD)
\nI will share my solution with you once I find one!
I think that the problem that we are facing might not be related to the choice of model or loss function, rather it is with the data that we are feeding this model. There is a very high probability that the model is not able to learn anything meaningful from the data, and it just learns the frequency of the predictions.
", "votes": 3, "replies": [ { "id": 2472010, "author_name": "Aleksandr Lavrikov", "author_url": "", "post_date": "2023-10-06T21:03:17.143000", "content": "Agree, in my experience full size image doesn't works. To many noise for a weak label. You should realize how to cease area of interests for an every organ. For example it will work if you try to classify images inside bboxes objects like spleen kidney liver.
", "votes": 3, "replies": [] } ] }, { "id": 2467290, "author_name": "naidu_tmp", "author_url": "", "post_date": "2023-10-04T12:19:38.887000", "content": "I think the worst part is most of the submissions were just weighted scaled mean values of particular target category.
", "votes": 3, "replies": [ { "id": 2467296, "author_name": "Andrij", "author_url": "", "post_date": "2023-10-04T12:24:22.120000", "content": "That's right, these models don't solve the real problem, simple statistics work better.
", "votes": 2, "replies": [ { "id": 2467370, "author_name": "naidu_tmp", "author_url": "", "post_date": "2023-10-04T13:47:19.380000", "content": "literally today my youtube homepage has this video : https://youtube.com/shorts/h21Hc8O6nhY?feature=shared
", "votes": 2, "replies": [] } ] } ] }, { "id": 2472473, "author_name": "Gunes Evitan", "author_url": "", "post_date": "2023-10-07T11:03:41.753000", "content": "I don't know if you noticed but some people instantly broke that barrier you just mentioned (including me). The reason is they have experience of dealing with similar data. If it was early stages, I would share my approach but it's too late to share anything now.
", "votes": 4, "replies": [] }, { "id": 2472305, "author_name": "Jason Heesang Lee", "author_url": "", "post_date": "2023-10-07T07:57:52.490000", "content": "Hi @aikhmelnytskyy, I think it is better if you don't mind about
\n\n\nConsidering that 95% of participants could not cross the 0.65 barrier, this problem is not only mine.
\n
I am also one of these 95% you have mentioned.
\nI assume most of the people who are not crossing 0.65 barrier are the ones who make the simple modifications to the scales.
\nAs I mentioned in one of my notebooks (focusing on this scale modification), there will be big changes to the leaderboard for those who are focusing on the scales (Like myself).
\nThat is also why I am working on vision techniques (but it is tough to break that 0.65 as well xD)
\nI will share my solution with you once I find one!