{ "id": 347670, "title": "CV - LB Thread [Best Single Model Scores]", "url": "/competitions/rsna-2022-cervical-spine-fracture-detection/discussion/347670", "author_name": "Nischay Dhankhar", "post_date": "2022-08-25T04:16:54.256000", "votes": 30, "comment_count": 29, "views": 0, "content": "

Creating one common thread for sharing your best single model scores.

\n

Update 0:
\nCV - 0.52
\nLB - N.A

\n

Update 1:
\nCV - 0.465
\nLB - N.A

", "messages": [ { "id": 1912956, "postDate": "2022-08-25T04:16:54.257Z", "content": "

Creating one common thread for sharing your best single model scores.

\n

Update 0:
\nCV - 0.52
\nLB - N.A

\n

Update 1:
\nCV - 0.465
\nLB - N.A

", "rawMarkdown": "Creating one common thread for sharing your best single model scores.\n\nUpdate 0:\nCV - 0.52\nLB - N.A\n\nUpdate 1:\nCV - 0.465\nLB - N.A", "votes": 30 }, { "id": 1913717, "postDate": "2022-08-25T13:14:10.847Z", "content": "

5fold resnext50_32x4d - 512x512 LB: 0.28, CV was not calculated for this model [POST] [RUNTIME: 7-8 HOURS]
\n5fold resnext50_32x4d - 512x512 LB: 0.31, CV Below [FULL DATA] [RUNTIME: 2.5 HOURS]
\n5fold resnext50_32x4d - 512x512 LB: 0.37 CV: CV was lower than but close to .37 [PART DATA] [RUNTIME: 2.5 HOURS]

\n

CV: [0.4234, 0.3247, 0.3488, 0.3317, 0.3029] (CV was only done on part data because easy and its not the point to finetune so I only need a rough estimate)
\nLB: 0.31

", "rawMarkdown": "5fold resnext50_32x4d - 512x512 LB: 0.28, CV was not calculated for this model [POST] [RUNTIME: 7-8 HOURS]\n5fold resnext50_32x4d - 512x512 LB: 0.31, CV Below [FULL DATA] [RUNTIME: 2.5 HOURS]\n5fold resnext50_32x4d - 512x512 LB: 0.37 CV: CV was lower than but close to .37 [PART DATA] [RUNTIME: 2.5 HOURS]\n\nCV: [0.4234, 0.3247, 0.3488, 0.3317, 0.3029] (CV was only done on part data because easy and its not the point to finetune so I only need a rough estimate)\nLB: 0.31", "votes": 15, "replies": [ { "id": 1914005, "postDate": "2022-08-25T16:35:31.503Z", "content": "

Given that both cases are the same model and image size, can I assume the difference is in the pre-processing part and training parameters?

", "rawMarkdown": "Given that both cases are the same model and image size, can I assume the difference is in the pre-processing part and training parameters?" }, { "id": 1914008, "postDate": "2022-08-25T16:39:46.267Z", "content": "

Difference is that the first one is better post-processing and full data training instead of last one which is part data training… No changes in parameters or pre-processing

", "rawMarkdown": "Difference is that the first one is better post-processing and full data training instead of last one which is part data training... No changes in parameters or pre-processing", "votes": 2 }, { "id": 1949765, "postDate": "2022-09-21T21:47:36.503Z", "content": "

\"Selection-311\"

\n

if you estimate only one slice per input, you will need to shift over all slices.
\nhowever, if you estimate a few slices per input, you can have a larger stride when shifting.
\nmaybe this will have a speedup

", "rawMarkdown": "\"Selection-311\"\n\nif you estimate only one slice per input, you will need to shift over all slices.\nhowever, if you estimate a few slices per input, you can have a larger stride when shifting.\nmaybe this will have a speedup" }, { "id": 1950046, "postDate": "2022-09-22T05:49:10.977Z", "content": "

Lol, already doing it, I already have pretty fast notebooks, I misreported the time for [POST], it was just about 5 hours only and currently I have no efficient things in there, so I can do that if I need to at any point

", "rawMarkdown": "Lol, already doing it, I already have pretty fast notebooks, I misreported the time for [POST], it was just about 5 hours only and currently I have no efficient things in there, so I can do that if I need to at any point" }, { "id": 1950262, "postDate": "2022-09-22T07:53:05.707Z", "content": "

If you're willing to share, I'd be really interested to know whether once you have some reference slices for each vertebrae if you are predicting fractures for each vertebrae independently and combining them; or passing all of them to a 3d model? I'm guessing from your run times that it is the former? ;)

", "rawMarkdown": "If you're willing to share, I'd be really interested to know whether once you have some reference slices for each vertebrae if you are predicting fractures for each vertebrae independently and combining them; or passing all of them to a 3d model? I'm guessing from your run times that it is the former? ;) " }, { "id": 1950314, "postDate": "2022-09-22T08:22:30.550Z", "content": "

It is prediction in a vertebrae at a time (the former one) right now…

", "rawMarkdown": "It is prediction in a vertebrae at a time (the former one) right now... ", "votes": 1 }, { "id": 1950643, "postDate": "2022-09-22T13:04:24.413Z", "content": "

Interesting, thank you very much for the response. I am also moving to this approach and it seems many others are too. Good luck !

", "rawMarkdown": "Interesting, thank you very much for the response. I am also moving to this approach and it seems many others are too. Good luck !" }, { "id": 1950693, "postDate": "2022-09-22T13:20:38.040Z", "content": "

We are doing this approach too but I don't get how it's possible to use only few slices per vertebrae since the fracture can be only in certain slices and not all of them. Could someone please elaborate more what do you mean by reference slices?

", "rawMarkdown": "We are doing this approach too but I don't get how it's possible to use only few slices per vertebrae since the fracture can be only in certain slices and not all of them. Could someone please elaborate more what do you mean by reference slices? " }, { "id": 1950768, "postDate": "2022-09-22T14:21:21.980Z", "content": "

I have no idea what \"reference slides\" means either, but I guess he is talking about taking slides (slices/images) from a particular bone at a time..

", "rawMarkdown": "I have no idea what \"reference slides\" means either, but I guess he is talking about taking slides (slices/images) from a particular bone at a time.." }, { "id": 1950820, "postDate": "2022-09-22T14:48:19.270Z", "content": "

Yes sorry for the confusion. By reference I meant that having identified which slices contain which vertebrae, we use only a subset of the slices, and assume if a fracture is present then it will be visible in the subset and this is sufficient for detection.

\n

I'm probably making a poor assumption here, and was doing it to try and speed up the training/inference process (since each vertebrae can span many slices).

", "rawMarkdown": "Yes sorry for the confusion. By reference I meant that having identified which slices contain which vertebrae, we use only a subset of the slices, and assume if a fracture is present then it will be visible in the subset and this is sufficient for detection.\n\nI'm probably making a poor assumption here, and was doing it to try and speed up the training/inference process (since each vertebrae can span many slices)." }, { "id": 1950829, "postDate": "2022-09-22T14:53:39.720Z", "content": "

I can understand your assumption now, however I in my understanding it's not a good idea bc like I said a vertebrae can have few slices with fracture present. But it seems other folks have found a way to use only a few slices in a meaningful way.

", "rawMarkdown": "I can understand your assumption now, however I in my understanding it's not a good idea bc like I said a vertebrae can have few slices with fracture present. But it seems other folks have found a way to use only a few slices in a meaningful way.", "votes": 1 }, { "id": 1950875, "postDate": "2022-09-22T15:32:55.193Z", "content": "

That is absolutely correct (I have concrete proof for this understanding of yours..), except for the fact that I am not currently in \"other folks who have found a way to use only a few slices in a meaningful way\"

", "rawMarkdown": "That is absolutely correct (I have concrete proof for this understanding of yours..), except for the fact that I am not currently in \"other folks who have found a way to use only a few slices in a meaningful way\"", "votes": 1 }, { "id": 1951025, "postDate": "2022-09-22T18:01:58.027Z", "content": "

Ah thanks for your input both, I think I need to explore using more slices because I have not had much luck just using a few per vertebrae. I'm thinking of trying the following:

\n\n

Then train CNN on all vertebrae examples. But I need to decide how to handle variable channel depth and also how to hopefully make use of pre-trained networks.

\n

What do you think? :)

", "rawMarkdown": "Ah thanks for your input both, I think I need to explore using more slices because I have not had much luck just using a few per vertebrae. I'm thinking of trying the following:\n\n- For each study:\n - For each slice pixel data:\n - Predict presence of vertebrae c1, c2, ..., c7.\n - For vertebrae c1, c2, ..., c7:\n - Stack pixel data from all slice with vertebrae detected p > 0.5 along channel dimension\n - Label 0/1 depending on fracture position\n\nThen train CNN on all vertebrae examples. But I need to decide how to handle variable channel depth and also how to hopefully make use of pre-trained networks. \n\nWhat do you think? :)", "votes": 1 }, { "id": 1951031, "postDate": "2022-09-22T18:05:32.863Z", "content": "

I am sorry, I won't able to comment particularly on that with what I think.. all I can say is Good Luck!

", "rawMarkdown": "I am sorry, I won't able to comment particularly on that with what I think.. all I can say is Good Luck!", "votes": 1 }, { "id": 1951038, "postDate": "2022-09-22T18:10:52.177Z", "content": "

Of course no problem, thanks again for your comments here and in other threads and good luck to you too, hope you ship the win!

", "rawMarkdown": "Of course no problem, thanks again for your comments here and in other threads and good luck to you too, hope you ship the win!" }, { "id": 1994781, "postDate": "2022-10-19T08:25:22.107Z", "content": "

Hi! I wonder why set the image size to 512. The transformation used to pretrain resnext in ImageNet is 256, or did I make a mistake?
\nThanks so much for your time!

", "rawMarkdown": "Hi! I wonder why set the image size to 512. The transformation used to pretrain resnext in ImageNet is 256, or did I make a mistake?\nThanks so much for your time!" } ] }, { "id": 1913711, "postDate": "2022-08-25T13:11:37.217Z", "content": "

transformer CV/LB=0.46/0.46
\ndeberta CV/LB=0.459/0.46
\ndeberta long epoch CV/LB=0.458/0.45 (best)

", "rawMarkdown": "transformer CV/LB=0.46/0.46 \ndeberta CV/LB=0.459/0.46\ndeberta long epoch CV/LB=0.458/0.45 (best)", "votes": 8 }, { "id": 1913589, "postDate": "2022-08-25T11:43:44.277Z", "content": "

3D-DenseNet

\n

CV 0.556 LB: 0.55
\nCV 0.534 LB 0.60……

", "rawMarkdown": "3D-DenseNet\n\nCV 0.556 LB: 0.55\nCV 0.534 LB 0.60......", "votes": 5, "replies": [ { "id": 1913629, "postDate": "2022-08-25T12:21:17.817Z", "content": "

Thanks for sharing your scores @yosukeyama I was expecting a better cv-lb correlation from the competition given high number of samples.

", "rawMarkdown": "Thanks for sharing your scores @yosukeyama I was expecting a better cv-lb correlation from the competition given high number of samples. ", "votes": 2 } ] }, { "id": 1913636, "postDate": "2022-08-25T12:26:42.400Z", "content": "

backbone - resnset34 - CV - 0.48, LB - 0.47

", "rawMarkdown": "`backbone - resnset34` - CV - `0.48`, LB - `0.47` ", "votes": 6 }, { "id": 1969707, "postDate": "2022-10-03T16:32:57.770Z", "content": "

5fold tf_efficientnetv2_s 384x384 CV:0.467 LB: 0.43

", "rawMarkdown": "5fold tf_efficientnetv2_s 384x384 CV:0.467 LB: 0.43", "votes": 1 }, { "id": 1974301, "postDate": "2022-10-06T07:08:34.530Z", "content": "

tf_efficientnetv2_s 384x384
\nholdout 10% Val score : 0.4625 LB : 0.44

", "rawMarkdown": "tf_efficientnetv2_s 384x384\nholdout 10% Val score : 0.4625 LB : 0.44", "votes": 2, "replies": [ { "id": 1999804, "postDate": "2022-10-22T18:03:30.673Z", "content": "

train 1 epoch?

", "rawMarkdown": "train 1 epoch?" }, { "id": 2000994, "postDate": "2022-10-23T17:53:13.013Z", "content": "

Yes, validating 3-4 times in between the epoch.

", "rawMarkdown": "Yes, validating 3-4 times in between the epoch." }, { "id": 2001663, "postDate": "2022-10-24T08:07:47.027Z", "content": "

oh..
\nis there augmentation??
\ni tried only resize and normalize augmentation and got cv 0.6~

", "rawMarkdown": "oh..\nis there augmentation??\ni tried only resize and normalize augmentation and got cv 0.6~" }, { "id": 2001834, "postDate": "2022-10-24T10:29:03.687Z", "content": "

There is. random shift, rotate, saturation and brightness. Can experiment with flips as well.

", "rawMarkdown": "There is. random shift, rotate, saturation and brightness. Can experiment with flips as well." } ] }, { "id": 1990327, "postDate": "2022-10-16T13:15:38.327Z", "content": "

EfficientNext - v2L

\n

Cv 0.398 lb 0.42

", "rawMarkdown": "EfficientNext - v2L\n\nCv 0.398 lb 0.42" }, { "id": 1977059, "postDate": "2022-10-07T18:21:47.907Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 1913717, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2022-08-25T13:14:10.847000", "content": "

5fold resnext50_32x4d - 512x512 LB: 0.28, CV was not calculated for this model [POST] [RUNTIME: 7-8 HOURS]
\n5fold resnext50_32x4d - 512x512 LB: 0.31, CV Below [FULL DATA] [RUNTIME: 2.5 HOURS]
\n5fold resnext50_32x4d - 512x512 LB: 0.37 CV: CV was lower than but close to .37 [PART DATA] [RUNTIME: 2.5 HOURS]

\n

CV: [0.4234, 0.3247, 0.3488, 0.3317, 0.3029] (CV was only done on part data because easy and its not the point to finetune so I only need a rough estimate)
\nLB: 0.31

", "votes": 15, "replies": [ { "id": 1914005, "author_name": "IgorMuniz", "author_url": "", "post_date": "2022-08-25T16:35:31.503000", "content": "

Given that both cases are the same model and image size, can I assume the difference is in the pre-processing part and training parameters?

", "votes": 0, "replies": [] }, { "id": 1914008, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2022-08-25T16:39:46.267000", "content": "

Difference is that the first one is better post-processing and full data training instead of last one which is part data training… No changes in parameters or pre-processing

", "votes": 2, "replies": [] }, { "id": 1949765, "author_name": "hengck23", "author_url": "", "post_date": "2022-09-21T21:47:36.503000", "content": "

\"Selection-311\"

\n

if you estimate only one slice per input, you will need to shift over all slices.
\nhowever, if you estimate a few slices per input, you can have a larger stride when shifting.
\nmaybe this will have a speedup

", "votes": 0, "replies": [] }, { "id": 1950046, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2022-09-22T05:49:10.977000", "content": "

Lol, already doing it, I already have pretty fast notebooks, I misreported the time for [POST], it was just about 5 hours only and currently I have no efficient things in there, so I can do that if I need to at any point

", "votes": 0, "replies": [] }, { "id": 1950262, "author_name": "Tom Aindow", "author_url": "", "post_date": "2022-09-22T07:53:05.707000", "content": "

If you're willing to share, I'd be really interested to know whether once you have some reference slices for each vertebrae if you are predicting fractures for each vertebrae independently and combining them; or passing all of them to a 3d model? I'm guessing from your run times that it is the former? ;)

", "votes": 0, "replies": [] }, { "id": 1950314, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2022-09-22T08:22:30.550000", "content": "

It is prediction in a vertebrae at a time (the former one) right now…

", "votes": 1, "replies": [] }, { "id": 1950643, "author_name": "Tom Aindow", "author_url": "", "post_date": "2022-09-22T13:04:24.413000", "content": "

Interesting, thank you very much for the response. I am also moving to this approach and it seems many others are too. Good luck !

", "votes": 0, "replies": [] }, { "id": 1950693, "author_name": "IgorMuniz", "author_url": "", "post_date": "2022-09-22T13:20:38.040000", "content": "

We are doing this approach too but I don't get how it's possible to use only few slices per vertebrae since the fracture can be only in certain slices and not all of them. Could someone please elaborate more what do you mean by reference slices?

", "votes": 0, "replies": [] }, { "id": 1950768, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2022-09-22T14:21:21.980000", "content": "

I have no idea what \"reference slides\" means either, but I guess he is talking about taking slides (slices/images) from a particular bone at a time..

", "votes": 0, "replies": [] }, { "id": 1950820, "author_name": "Tom Aindow", "author_url": "", "post_date": "2022-09-22T14:48:19.270000", "content": "

Yes sorry for the confusion. By reference I meant that having identified which slices contain which vertebrae, we use only a subset of the slices, and assume if a fracture is present then it will be visible in the subset and this is sufficient for detection.

\n

I'm probably making a poor assumption here, and was doing it to try and speed up the training/inference process (since each vertebrae can span many slices).

", "votes": 0, "replies": [] }, { "id": 1950829, "author_name": "IgorMuniz", "author_url": "", "post_date": "2022-09-22T14:53:39.720000", "content": "

I can understand your assumption now, however I in my understanding it's not a good idea bc like I said a vertebrae can have few slices with fracture present. But it seems other folks have found a way to use only a few slices in a meaningful way.

", "votes": 1, "replies": [] }, { "id": 1950875, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2022-09-22T15:32:55.193000", "content": "

That is absolutely correct (I have concrete proof for this understanding of yours..), except for the fact that I am not currently in \"other folks who have found a way to use only a few slices in a meaningful way\"

", "votes": 1, "replies": [] }, { "id": 1951025, "author_name": "Tom Aindow", "author_url": "", "post_date": "2022-09-22T18:01:58.027000", "content": "

Ah thanks for your input both, I think I need to explore using more slices because I have not had much luck just using a few per vertebrae. I'm thinking of trying the following:

\n\n

Then train CNN on all vertebrae examples. But I need to decide how to handle variable channel depth and also how to hopefully make use of pre-trained networks.

\n

What do you think? :)

", "votes": 1, "replies": [] }, { "id": 1951031, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2022-09-22T18:05:32.863000", "content": "

I am sorry, I won't able to comment particularly on that with what I think.. all I can say is Good Luck!

", "votes": 1, "replies": [] }, { "id": 1951038, "author_name": "Tom Aindow", "author_url": "", "post_date": "2022-09-22T18:10:52.177000", "content": "

Of course no problem, thanks again for your comments here and in other threads and good luck to you too, hope you ship the win!

", "votes": 0, "replies": [] }, { "id": 1994781, "author_name": "Ytao Sun", "author_url": "", "post_date": "2022-10-19T08:25:22.107000", "content": "

Hi! I wonder why set the image size to 512. The transformation used to pretrain resnext in ImageNet is 256, or did I make a mistake?
\nThanks so much for your time!

", "votes": 0, "replies": [] } ] }, { "id": 1913711, "author_name": "patriot", "author_url": "", "post_date": "2022-08-25T13:11:37.217000", "content": "

transformer CV/LB=0.46/0.46
\ndeberta CV/LB=0.459/0.46
\ndeberta long epoch CV/LB=0.458/0.45 (best)

", "votes": 8, "replies": [] }, { "id": 1913589, "author_name": "YYama", "author_url": "", "post_date": "2022-08-25T11:43:44.277000", "content": "

3D-DenseNet

\n

CV 0.556 LB: 0.55
\nCV 0.534 LB 0.60……

", "votes": 5, "replies": [ { "id": 1913629, "author_name": "Nischay Dhankhar", "author_url": "", "post_date": "2022-08-25T12:21:17.817000", "content": "

Thanks for sharing your scores @yosukeyama I was expecting a better cv-lb correlation from the competition given high number of samples.

", "votes": 2, "replies": [] } ] }, { "id": 1913636, "author_name": "DrHB", "author_url": "", "post_date": "2022-08-25T12:26:42.400000", "content": "

backbone - resnset34 - CV - 0.48, LB - 0.47

", "votes": 6, "replies": [] }, { "id": 1969707, "author_name": "IgorMuniz", "author_url": "", "post_date": "2022-10-03T16:32:57.770000", "content": "

5fold tf_efficientnetv2_s 384x384 CV:0.467 LB: 0.43

", "votes": 1, "replies": [] }, { "id": 1974301, "author_name": "Yerram Varun", "author_url": "", "post_date": "2022-10-06T07:08:34.530000", "content": "

tf_efficientnetv2_s 384x384
\nholdout 10% Val score : 0.4625 LB : 0.44

", "votes": 2, "replies": [ { "id": 1999804, "author_name": "ParkSom", "author_url": "", "post_date": "2022-10-22T18:03:30.673000", "content": "

train 1 epoch?

", "votes": 0, "replies": [] }, { "id": 2000994, "author_name": "Yerram Varun", "author_url": "", "post_date": "2022-10-23T17:53:13.013000", "content": "

Yes, validating 3-4 times in between the epoch.

", "votes": 0, "replies": [] }, { "id": 2001663, "author_name": "ParkSom", "author_url": "", "post_date": "2022-10-24T08:07:47.027000", "content": "

oh..
\nis there augmentation??
\ni tried only resize and normalize augmentation and got cv 0.6~

", "votes": 0, "replies": [] }, { "id": 2001834, "author_name": "Yerram Varun", "author_url": "", "post_date": "2022-10-24T10:29:03.687000", "content": "

There is. random shift, rotate, saturation and brightness. Can experiment with flips as well.

", "votes": 0, "replies": [] } ] }, { "id": 1990327, "author_name": "jcerpent", "author_url": "", "post_date": "2022-10-16T13:15:38.327000", "content": "

EfficientNext - v2L

\n

Cv 0.398 lb 0.42

", "votes": 0, "replies": [] }, { "id": 1977059, "author_name": "", "author_url": "", "post_date": "2022-10-07T18:21:47.907000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1912956": "Creating one common thread for sharing your best single model scores.\n\nUpdate 0:\nCV - 0.52\nLB - N.A\n\nUpdate 1:\nCV - 0.465\nLB - N.A", "1913717": "5fold resnext50_32x4d - 512x512 LB: 0.28, CV was not calculated for this model [POST] [RUNTIME: 7-8 HOURS]\n5fold resnext50_32x4d - 512x512 LB: 0.31, CV Below [FULL DATA] [RUNTIME: 2.5 HOURS]\n5fold resnext50_32x4d - 512x512 LB: 0.37 CV: CV was lower than but close to .37 [PART DATA] [RUNTIME: 2.5 HOURS]\n\nCV: [0.4234, 0.3247, 0.3488, 0.3317, 0.3029] (CV was only done on part data because easy and its not the point to finetune so I only need a rough estimate)\nLB: 0.31", "1913711": "transformer CV/LB=0.46/0.46 \ndeberta CV/LB=0.459/0.46\ndeberta long epoch CV/LB=0.458/0.45 (best)", "1913589": "3D-DenseNet\n\nCV 0.556 LB: 0.55\nCV 0.534 LB 0.60......", "1913636": "`backbone - resnset34` - CV - `0.48`, LB - `0.47` ", "1969707": "5fold tf_efficientnetv2_s 384x384 CV:0.467 LB: 0.43", "1974301": "tf_efficientnetv2_s 384x384\nholdout 10% Val score : 0.4625 LB : 0.44", "1990327": "EfficientNext - v2L\n\nCv 0.398 lb 0.42", "1977059": "" } }