{ "id": 354451, "title": "patient_overall estimates", "url": "/competitions/rsna-2022-cervical-spine-fracture-detection/discussion/354451", "author_name": "Samuel Cortinhas", "post_date": "2022-09-22T10:30:51.450000", "votes": 33, "comment_count": 2, "views": 0, "content": "

I wanted to open a discussion on how to best deal with predicting the most important component of the competition loss, patient_overall.

\n

A big advantage of 3D models is that you can make this a training label and let the model figure out the best predictions. I think most people are moving towards 2D models though as they are performing better, which makes this not possible anymore.

\n
\n

@vslaykovsky showed us how to start; by assuming independence of the vertebrae C1, …, C7.

\n

\n

This works surprisingly (at least to me) well, although I find that it tends to slightly overestimate the probability of patient_overall. We can see this by applying a 'squishing' function, which either pushes probabilities closer to the tails 0, 1 or to the middle 0.5, by varying the parameter beta.

\n

\n

\n

And if you plot beta against a local CV, you get a curve looking like this. (note: this is a different 2D model to Vladimir's)

\n

\n

This shows you that you can get a slight improvement by squishing your probabilities closer to 0.5 (i.e. making them less extreme)

\n
\n

This had me thinking of another way to estimate patient_overall. The exact formula including dependence is quite messy, it looks like this

\n

\n

I tried some different approximations, e.g. truncating this series to only include interactions of up to 2 sets and estimating conditional probabilities from the training set. Long story short it doesn't work because you end up with lots of negative probabilities and it is just a bad approximation.

\n
\n

So my question is has anybody else thought about this? Is there a way to improve on the Vladimir's formula by not assuming independence?

", "messages": [ { "id": 1950433, "postDate": "2022-09-22T10:30:51.450Z", "content": "

I wanted to open a discussion on how to best deal with predicting the most important component of the competition loss, patient_overall.

\n

A big advantage of 3D models is that you can make this a training label and let the model figure out the best predictions. I think most people are moving towards 2D models though as they are performing better, which makes this not possible anymore.

\n
\n

@vslaykovsky showed us how to start; by assuming independence of the vertebrae C1, …, C7.

\n

\n

This works surprisingly (at least to me) well, although I find that it tends to slightly overestimate the probability of patient_overall. We can see this by applying a 'squishing' function, which either pushes probabilities closer to the tails 0, 1 or to the middle 0.5, by varying the parameter beta.

\n

\n

\n

And if you plot beta against a local CV, you get a curve looking like this. (note: this is a different 2D model to Vladimir's)

\n

\n

This shows you that you can get a slight improvement by squishing your probabilities closer to 0.5 (i.e. making them less extreme)

\n
\n

This had me thinking of another way to estimate patient_overall. The exact formula including dependence is quite messy, it looks like this

\n

\n

I tried some different approximations, e.g. truncating this series to only include interactions of up to 2 sets and estimating conditional probabilities from the training set. Long story short it doesn't work because you end up with lots of negative probabilities and it is just a bad approximation.

\n
\n

So my question is has anybody else thought about this? Is there a way to improve on the Vladimir's formula by not assuming independence?

", "rawMarkdown": "I wanted to open a discussion on how to best deal with predicting the most important component of the competition loss, **patient_overall**.\n\nA big advantage of **3D models** is that you can make this a training label and let the model figure out the best predictions. I think most people are moving towards **2D models** though as they are performing better, which makes this not possible anymore.\n\n
\n\n@vslaykovsky showed us how to start; by **assuming independence** of the vertebrae C1, ..., C7. \n\n\n\nThis works surprisingly (at least to me) well, although I find that it tends to **slightly overestimate** the probability of patient_overall. We can see this by applying a 'squishing' function, which either pushes probabilities closer to the tails 0, 1 or to the middle 0.5, by varying the parameter beta.\n\n\n\n\n\nAnd if you plot beta against a local CV, you get a curve looking like this. (note: this is a different 2D model to Vladimir's)\n\n\n\nThis shows you that you can get a slight improvement by squishing your probabilities closer to 0.5 (i.e. making them less extreme)\n\n
\n\nThis had me thinking of another way to estimate patient_overall. The exact formula including dependence is quite messy, it looks like this \n\n\n\nI tried some different approximations, e.g. truncating this series to only include interactions of up to 2 sets and estimating conditional probabilities from the training set. Long story short it doesn't work because you end up with lots of negative probabilities and it is just a bad approximation.\n\n
\n\nSo my question is has anybody else thought about this? Is there a way to improve on the Vladimir's formula by not assuming independence? ", "votes": 33 }, { "id": 1958427, "postDate": "2022-09-27T12:41:57.507Z", "content": "

I think 2D based models are effective because there is an implicit effect of increasing the number of samples.

\n

I'm working on a hybrid approcha to use CNNs trained using @vslaykovsky 's approach as a feature extractor to pre-compute features(1280 for each image) and train a Transformer-based model to aggregate information from the 2D features of all the slices when predicting fractures for each patient.

", "rawMarkdown": "I think 2D based models are effective because there is an implicit effect of increasing the number of samples. \n\nI'm working on a hybrid approcha to use CNNs trained using @vslaykovsky 's approach as a feature extractor to pre-compute features(1280 for each image) and train a Transformer-based model to aggregate information from the 2D features of all the slices when predicting fractures for each patient. ", "votes": 2 }, { "id": 1989602, "postDate": "2022-10-16T05:05:44.430Z", "content": "

Interesting topic, I like probabilities. I think 3D models are not so bad as they do not use pretrained models. They have to learn from scratch. But 3D models do not see all vertebrae and could be better to recognize one or two vertebrae but not 7. It depends on the volume shape that must not be a cube. Recognize the vertebra is not only useful for patient_overall, but for all the predictions.

", "rawMarkdown": "Interesting topic, I like probabilities. I think 3D models are not so bad as they do not use pretrained models. They have to learn from scratch. But 3D models do not see all vertebrae and could be better to recognize one or two vertebrae but not 7. It depends on the volume shape that must not be a cube. Recognize the vertebra is not only useful for patient_overall, but for all the predictions." } ], "comments": [ { "id": 1958427, "author_name": "SieunPark", "author_url": "", "post_date": "2022-09-27T12:41:57.507000", "content": "

I think 2D based models are effective because there is an implicit effect of increasing the number of samples.

\n

I'm working on a hybrid approcha to use CNNs trained using @vslaykovsky 's approach as a feature extractor to pre-compute features(1280 for each image) and train a Transformer-based model to aggregate information from the 2D features of all the slices when predicting fractures for each patient.

", "votes": 2, "replies": [] }, { "id": 1989602, "author_name": "Pierre Tisseur", "author_url": "", "post_date": "2022-10-16T05:05:44.430000", "content": "

Interesting topic, I like probabilities. I think 3D models are not so bad as they do not use pretrained models. They have to learn from scratch. But 3D models do not see all vertebrae and could be better to recognize one or two vertebrae but not 7. It depends on the volume shape that must not be a cube. Recognize the vertebra is not only useful for patient_overall, but for all the predictions.

", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1950433": "I wanted to open a discussion on how to best deal with predicting the most important component of the competition loss, **patient_overall**.\n\nA big advantage of **3D models** is that you can make this a training label and let the model figure out the best predictions. I think most people are moving towards **2D models** though as they are performing better, which makes this not possible anymore.\n\n
\n\n@vslaykovsky showed us how to start; by **assuming independence** of the vertebrae C1, ..., C7. \n\n\n\nThis works surprisingly (at least to me) well, although I find that it tends to **slightly overestimate** the probability of patient_overall. We can see this by applying a 'squishing' function, which either pushes probabilities closer to the tails 0, 1 or to the middle 0.5, by varying the parameter beta.\n\n\n\n\n\nAnd if you plot beta against a local CV, you get a curve looking like this. (note: this is a different 2D model to Vladimir's)\n\n\n\nThis shows you that you can get a slight improvement by squishing your probabilities closer to 0.5 (i.e. making them less extreme)\n\n
\n\nThis had me thinking of another way to estimate patient_overall. The exact formula including dependence is quite messy, it looks like this \n\n\n\nI tried some different approximations, e.g. truncating this series to only include interactions of up to 2 sets and estimating conditional probabilities from the training set. Long story short it doesn't work because you end up with lots of negative probabilities and it is just a bad approximation.\n\n
\n\nSo my question is has anybody else thought about this? Is there a way to improve on the Vladimir's formula by not assuming independence? ", "1958427": "I think 2D based models are effective because there is an implicit effect of increasing the number of samples. \n\nI'm working on a hybrid approcha to use CNNs trained using @vslaykovsky 's approach as a feature extractor to pre-compute features(1280 for each image) and train a Transformer-based model to aggregate information from the 2D features of all the slices when predicting fractures for each patient. ", "1989602": "Interesting topic, I like probabilities. I think 3D models are not so bad as they do not use pretrained models. They have to learn from scratch. But 3D models do not see all vertebrae and could be better to recognize one or two vertebrae but not 7. It depends on the volume shape that must not be a cube. Recognize the vertebra is not only useful for patient_overall, but for all the predictions." } }