{ "id": 117228, "title": "2nd Place Solution - Sequential model", "url": "/competitions/rsna-intracranial-hemorrhage-detection/discussion/117228", "author_name": "Darragh", "post_date": "2019-11-14T02:23:38.016000", "votes": 117, "comment_count": 24, "views": 0, "content": "

Code & Val/LB scores : https://github.com/darraghdog/rsna
\nCongrats all winners, looking forward to go through your solutions. Big shout out to competition hosts RSNA, kaggle community, pytorch community, albumentations and FB's work on resnext -very cool how they trained this. \nWe were very sad not to get a top3 in Recursion competition, now we are very happy 😄

\n\n

\"\"

\n\n

Overview\nIn general we just have a single image classifier, data split on 5 folds, we only trained on 3 of them, and then extracted pre-logit layer from the classifier and fed into an LSTM.\nClassifier trained on 5 epochs each fold, 480 images with below pre-processing. Each epoch, each fold, we extract embedding layer (use TTA and avg embeddings) train a separate LSTM for 12 epochs on each of those - so 15 LSTMs (3 fold image models X 5 epochs), and average the predictions. \nWas a bit concerned the preprocessing filter may lose information, so trained the above again without the preprocessing filter and it did worse; but averaging both pipelines did ever so slightly better. The pipeline from first paragraph above would, for all intensive purposes be just as good as final solution, but as we needed to fix docu pre-stage 2 the two pipelines are in github and final solution.

\n\n

Preprocessing:\n- Used Appian’s windowing from dicom images. Linky\n- Cut any black space. There were then headrest or machine artifacts in the image making the head much smaller than it could be - see visual above. These were generally thin lines, so used scipy.ndimage minimum_filter to try to wipe those thin lines. Linky\n- Albumentations as mentioned in visual above.

\n\n

Image classifier\n- Resnext101 - did not spend a whole lot of time here as it ran so long. But tested SeResenext and Efficitentnetv0 and they did not work as well. \n- Extract pre logit layer (GAP layer) at inference time Linky

\n\n

Create Sequences\n- Extract metadata from dicoms : Linky \n- Sequence images on Patient, Study and Series - most sequences were between 24 and 60 images in length. Linky

\n\n

LSTM\n- Feed in the embeddings in sequence on above key - Patient, Study and Series - also concat on the deltas between current and previous/next embeddings (current-previous embedding and current-next embedding) to give the model knowledge of changes around the image. Linky \n- LSTM architecture lifted from the winners of first stage toxic competition. This is a beast - only improvements came from making the hiddens layers larger. Oh, we added on the embeddings to the lstm output and this helped a bit also. Linky \n- For sequences of different length, padded them to same length, made a dummy embedding of zeros, and then through the results of this away before calculating loss and saving the predictions.

\n\n

What did not help...
\nToo long to do justice... mixup on image, mixup on embedding, augmentations on sequences (partial sequences, reversed sequences), 1d convolutions for sequences (although SeuTao got it working)

\n\n

Given more time
\nMake the classifier and the lstm model single end-to-end model. \nTrain all on stage2 data, we only got to train two folds of the image model on stage-2 data.

", "messages": [ { "id": 672594, "postDate": "2019-11-14T02:23:38.017Z", "content": "

Code & Val/LB scores : https://github.com/darraghdog/rsna
\nCongrats all winners, looking forward to go through your solutions. Big shout out to competition hosts RSNA, kaggle community, pytorch community, albumentations and FB's work on resnext -very cool how they trained this. \nWe were very sad not to get a top3 in Recursion competition, now we are very happy 😄

\n\n

\"\"

\n\n

Overview\nIn general we just have a single image classifier, data split on 5 folds, we only trained on 3 of them, and then extracted pre-logit layer from the classifier and fed into an LSTM.\nClassifier trained on 5 epochs each fold, 480 images with below pre-processing. Each epoch, each fold, we extract embedding layer (use TTA and avg embeddings) train a separate LSTM for 12 epochs on each of those - so 15 LSTMs (3 fold image models X 5 epochs), and average the predictions. \nWas a bit concerned the preprocessing filter may lose information, so trained the above again without the preprocessing filter and it did worse; but averaging both pipelines did ever so slightly better. The pipeline from first paragraph above would, for all intensive purposes be just as good as final solution, but as we needed to fix docu pre-stage 2 the two pipelines are in github and final solution.

\n\n

Preprocessing:\n- Used Appian’s windowing from dicom images. Linky\n- Cut any black space. There were then headrest or machine artifacts in the image making the head much smaller than it could be - see visual above. These were generally thin lines, so used scipy.ndimage minimum_filter to try to wipe those thin lines. Linky\n- Albumentations as mentioned in visual above.

\n\n

Image classifier\n- Resnext101 - did not spend a whole lot of time here as it ran so long. But tested SeResenext and Efficitentnetv0 and they did not work as well. \n- Extract pre logit layer (GAP layer) at inference time Linky

\n\n

Create Sequences\n- Extract metadata from dicoms : Linky \n- Sequence images on Patient, Study and Series - most sequences were between 24 and 60 images in length. Linky

\n\n

LSTM\n- Feed in the embeddings in sequence on above key - Patient, Study and Series - also concat on the deltas between current and previous/next embeddings (current-previous embedding and current-next embedding) to give the model knowledge of changes around the image. Linky \n- LSTM architecture lifted from the winners of first stage toxic competition. This is a beast - only improvements came from making the hiddens layers larger. Oh, we added on the embeddings to the lstm output and this helped a bit also. Linky \n- For sequences of different length, padded them to same length, made a dummy embedding of zeros, and then through the results of this away before calculating loss and saving the predictions.

\n\n

What did not help...
\nToo long to do justice... mixup on image, mixup on embedding, augmentations on sequences (partial sequences, reversed sequences), 1d convolutions for sequences (although SeuTao got it working)

\n\n

Given more time
\nMake the classifier and the lstm model single end-to-end model. \nTrain all on stage2 data, we only got to train two folds of the image model on stage-2 data.

", "rawMarkdown": "Code & Val/LB scores : https://github.com/darraghdog/rsna \nCongrats all winners, looking forward to go through your solutions. Big shout out to competition hosts RSNA, kaggle community, pytorch community, albumentations and FB's work on resnext -very cool how they trained this. \nWe were very sad not to get a top3 in Recursion competition, now we are very happy 😄 \n \n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F213493%2Fc76202951ffb4afcc5d56acb579ac552%2Frsna_nobrainer.png?generation=1573698154700507&alt=media)\n \n**Overview**\nIn general we just have a single image classifier, data split on 5 folds, we only trained on 3 of them, and then extracted pre-logit layer from the classifier and fed into an LSTM.\nClassifier trained on 5 epochs each fold, 480 images with below pre-processing. Each epoch, each fold, we extract embedding layer (use TTA and avg embeddings) train a separate LSTM for 12 epochs on each of those - so 15 LSTMs (3 fold image models X 5 epochs), and average the predictions. \nWas a bit concerned the preprocessing filter may lose information, so trained the above again without the preprocessing filter and it did worse; but averaging both pipelines did ever so slightly better. The pipeline from first paragraph above would, for all intensive purposes be just as good as final solution, but as we needed to fix docu pre-stage 2 the two pipelines are in github and final solution. \n\n**Preprocessing:**\n- Used Appian’s windowing from dicom images. [Linky](https://github.com/darraghdog/rsna/blob/master/eda/window_v1_test.py#L66)\n- Cut any black space. There were then headrest or machine artifacts in the image making the head much smaller than it could be - see visual above. These were generally thin lines, so used scipy.ndimage minimum_filter to try to wipe those thin lines. [Linky](https://github.com/darraghdog/rsna/blob/a97018a7b7ec920425189c7e37c1128dd9cb0158/scripts/resnext101v12/trainorig.py#L159)\n- Albumentations as mentioned in visual above. \n\n**Image classifier**\n- Resnext101 - did not spend a whole lot of time here as it ran so long. But tested SeResenext and Efficitentnetv0 and they did not work as well. \n- Extract pre logit layer (GAP layer) at inference time [Linky](https://github.com/darraghdog/rsna/blob/a97018a7b7ec920425189c7e37c1128dd9cb0158/scripts/resnext101v12/trainorig.py#L387) \n\n**Create Sequences**\n- Extract metadata from dicoms : [Linky](https://github.com/darraghdog/rsna/blob/master/eda/meta_eda_v1.py) \n- Sequence images on Patient, Study and Series - most sequences were between 24 and 60 images in length. [Linky](https://github.com/darraghdog/rsna/blob/a97018a7b7ec920425189c7e37c1128dd9cb0158/scripts/resnext101v12/trainlstmdeltasum.py#L200) \n\n**LSTM**\n- Feed in the embeddings in sequence on above key - Patient, Study and Series - also concat on the deltas between current and previous/next embeddings (`current-previous embedding` and `current-next embedding`) to give the model knowledge of changes around the image. [Linky](https://github.com/darraghdog/rsna/blob/a97018a7b7ec920425189c7e37c1128dd9cb0158/scripts/resnext101v12/trainlstmdeltasum.py#L133) \n- LSTM architecture lifted from the winners of first stage toxic competition. This is a beast - only improvements came from making the hiddens layers larger. Oh, we added on the embeddings to the lstm output and this helped a bit also. [Linky](https://github.com/darraghdog/rsna/blob/a97018a7b7ec920425189c7e37c1128dd9cb0158/scripts/resnext101v12/trainlstmdeltasum.py#L352) \n- For sequences of different length, padded them to same length, made a dummy embedding of zeros, and then through the results of this away before calculating loss and saving the predictions. \n\n**What did not help...** \nToo long to do justice... mixup on image, mixup on embedding, augmentations on sequences (partial sequences, reversed sequences), 1d convolutions for sequences (although SeuTao got it working)\n\n**Given more time** \nMake the classifier and the lstm model single end-to-end model. \nTrain all on stage2 data, we only got to train two folds of the image model on stage-2 data.\n", "votes": 117 }, { "id": 672610, "postDate": "2019-11-14T02:36:34.323Z", "content": "

Very nice! did you pad the sequences to fix the input size? For example, some studies consisted of 20 slices others 40 slices, etc. How did you deal with variable amount of slices per study?

", "rawMarkdown": "Very nice! did you pad the sequences to fix the input size? For example, some studies consisted of 20 slices others 40 slices, etc. How did you deal with variable amount of slices per study?", "votes": 5, "replies": [ { "id": 672613, "postDate": "2019-11-14T02:42:23.880Z", "content": "

Yes, and used a mask for padded, then removed padding in loss calculation, and for saving results.

", "rawMarkdown": "Yes, and used a mask for padded, then removed padding in loss calculation, and for saving results. ", "votes": 8 } ] }, { "id": 735322, "postDate": "2020-02-02T21:55:05.450Z", "content": "

This amazing! It's really a great work!

", "rawMarkdown": "This amazing! It's really a great work! ", "votes": 1 }, { "id": 682100, "postDate": "2019-11-26T23:20:57.077Z", "content": "

Great work!

", "rawMarkdown": "Great work!", "votes": 1 }, { "id": 678855, "postDate": "2019-11-22T01:20:10.170Z", "content": "

Simply amazing @darraghdog . Since there was windowing dependencies, you went for resnext lstm??. Just curious to know , if this works even for normal image classification competitions as well when there is not much of spatial temporal dependency..

", "rawMarkdown": "Simply amazing @darraghdog . Since there was windowing dependencies, you went for resnext lstm??. Just curious to know , if this works even for normal image classification competitions as well when there is not much of spatial temporal dependency.. ", "votes": 1, "replies": [ { "id": 679265, "postDate": "2019-11-22T14:10:30.500Z", "content": "

Hi Manoj, the lstm was less about windowing in each individual image and more about what folks call the z-axis in the data... the CT scan moves over the head and takes a number of images in sequence. We put this sequence of images (or the GAP layer from it) to the LSTM, so that let the LSTM simulate the scan moving over the head... hope I understood the question right.

", "rawMarkdown": "Hi Manoj, the lstm was less about windowing in each individual image and more about what folks call the `z-axis` in the data... the CT scan moves over the head and takes a number of images in sequence. We put this sequence of images (or the GAP layer from it) to the LSTM, so that let the LSTM simulate the scan moving over the head... hope I understood the question right. ", "votes": 2 } ] }, { "id": 673431, "postDate": "2019-11-15T00:44:54.643Z", "content": "

Congrats and thank you for sharing. Using Bi-LSTM for image classification is interesting to me!

", "rawMarkdown": "Congrats and thank you for sharing. Using Bi-LSTM for image classification is interesting to me!", "votes": 1 }, { "id": 672716, "postDate": "2019-11-14T04:49:50.323Z", "content": "

Congratulations\nThanks for Sharing your Valuable Insights & Approach @darraghdog

", "rawMarkdown": "Congratulations\nThanks for Sharing your Valuable Insights & Approach @darraghdog ", "votes": 1 }, { "id": 672674, "postDate": "2019-11-14T04:02:49.020Z", "content": "

Congrats!

", "rawMarkdown": "Congrats!", "votes": 1 }, { "id": 672609, "postDate": "2019-11-14T02:35:13.237Z", "content": "

Congrats!

", "rawMarkdown": "Congrats! ", "votes": 1 }, { "id": 672706, "postDate": "2019-11-14T04:36:14.067Z", "content": "

Congrats! It is much different from mine, so looks very cool! Thank you for your sharing.

", "rawMarkdown": " Congrats! It is much different from mine, so looks very cool! Thank you for your sharing.", "votes": 2 }, { "id": 1046269, "postDate": "2020-10-11T14:10:28.650Z", "content": "

Hi @darraghdog ,
\nCongratulations on the win.
\nCan you please help me understand when you extract the embeddings do you use pretrained weights to get the embeddings or do you use your weights that has been trained for 5 epochs??

\n

Any information regarding this will be really helpful.
\nThank you

", "rawMarkdown": "Hi @darraghdog , \nCongratulations on the win.\nCan you please help me understand when you extract the embeddings do you use pretrained weights to get the embeddings or do you use your weights that has been trained for 5 epochs??\n\nAny information regarding this will be really helpful.\nThank you" }, { "id": 749351, "postDate": "2020-02-18T16:10:08.210Z", "content": "

Can you explain more on the 2048 embedding? How do you convert the logit output to multiple vectors?

", "rawMarkdown": "Can you explain more on the 2048 embedding? How do you convert the logit output to multiple vectors?", "replies": [ { "id": 797135, "postDate": "2020-04-04T08:27:46.083Z", "content": "

A single brain hemorrhage scan is made up of multiple images, as the scanner moves over the brain. There is one vector per image, and these vectors are sequenced according to time that image was taken within the scan.

", "rawMarkdown": "A single brain hemorrhage scan is made up of multiple images, as the scanner moves over the brain. There is one vector per image, and these vectors are sequenced according to time that image was taken within the scan. " } ] }, { "id": 673309, "postDate": "2019-11-14T19:55:29.467Z", "content": "

Amazing work darragh!!

", "rawMarkdown": "Amazing work darragh!!" }, { "id": 673020, "postDate": "2019-11-14T12:06:23.910Z", "content": "

Congratulations @darraghdog and thanks for sharing your solution code.

", "rawMarkdown": "Congratulations @darraghdog and thanks for sharing your solution code." }, { "id": 672949, "postDate": "2019-11-14T10:23:38.967Z", "content": "

@darraghdog congrats and thanks for sharing.

", "rawMarkdown": "@darraghdog congrats and thanks for sharing." }, { "id": 672925, "postDate": "2019-11-14T09:42:14.717Z", "content": "

Congrats! The solution is very interesting and insightful. Thanks for sharing.

", "rawMarkdown": "Congrats! The solution is very interesting and insightful. Thanks for sharing." }, { "id": 672889, "postDate": "2019-11-14T08:52:57.493Z", "content": "

Congratulations!

", "rawMarkdown": "**Congratulations!**" }, { "id": 672852, "postDate": "2019-11-14T08:24:08.720Z", "content": "

Thank you for sharing your solution and code\nCongratulations @darraghdog !!

", "rawMarkdown": "Thank you for sharing your solution and code\nCongratulations @darraghdog !!" }, { "id": 678142, "postDate": "2019-11-21T03:00:04.097Z", "content": "

cool, thanks for sharing

", "rawMarkdown": "cool, thanks for sharing", "votes": 1 }, { "id": 672608, "postDate": "2019-11-14T02:34:08.267Z", "content": "

thanks

", "rawMarkdown": "thanks", "votes": 1 }, { "id": 673210, "postDate": "2019-11-14T17:10:11.777Z", "content": "

Great work - thank you for sharing!

", "rawMarkdown": "Great work - thank you for sharing!" }, { "id": 672855, "postDate": "2019-11-14T08:26:42.050Z", "content": "

Congrats! Thank you for your sharing.

", "rawMarkdown": "Congrats! Thank you for your sharing." } ], "comments": [ { "id": 672610, "author_name": "Tim Yee", "author_url": "", "post_date": "2019-11-14T02:36:34.323000", "content": "

Very nice! did you pad the sequences to fix the input size? For example, some studies consisted of 20 slices others 40 slices, etc. How did you deal with variable amount of slices per study?

", "votes": 5, "replies": [ { "id": 672613, "author_name": "Darragh", "author_url": "", "post_date": "2019-11-14T02:42:23.880000", "content": "

Yes, and used a mask for padded, then removed padding in loss calculation, and for saving results.

", "votes": 8, "replies": [] } ] }, { "id": 735322, "author_name": "Vítor Gama Lemos", "author_url": "", "post_date": "2020-02-02T21:55:05.450000", "content": "

This amazing! It's really a great work!

", "votes": 1, "replies": [] }, { "id": 682100, "author_name": "Andy Crowe", "author_url": "", "post_date": "2019-11-26T23:20:57.077000", "content": "

Great work!

", "votes": 1, "replies": [] }, { "id": 678855, "author_name": "Manoj Prabhakar", "author_url": "", "post_date": "2019-11-22T01:20:10.170000", "content": "

Simply amazing @darraghdog . Since there was windowing dependencies, you went for resnext lstm??. Just curious to know , if this works even for normal image classification competitions as well when there is not much of spatial temporal dependency..

", "votes": 1, "replies": [ { "id": 679265, "author_name": "Darragh", "author_url": "", "post_date": "2019-11-22T14:10:30.500000", "content": "

Hi Manoj, the lstm was less about windowing in each individual image and more about what folks call the z-axis in the data... the CT scan moves over the head and takes a number of images in sequence. We put this sequence of images (or the GAP layer from it) to the LSTM, so that let the LSTM simulate the scan moving over the head... hope I understood the question right.

", "votes": 2, "replies": [] } ] }, { "id": 673431, "author_name": "corochann", "author_url": "", "post_date": "2019-11-15T00:44:54.643000", "content": "

Congrats and thank you for sharing. Using Bi-LSTM for image classification is interesting to me!

", "votes": 1, "replies": [] }, { "id": 672716, "author_name": "Ailurophile", "author_url": "", "post_date": "2019-11-14T04:49:50.323000", "content": "

Congratulations\nThanks for Sharing your Valuable Insights & Approach @darraghdog

", "votes": 1, "replies": [] }, { "id": 672674, "author_name": "Aleksandr Larko", "author_url": "", "post_date": "2019-11-14T04:02:49.020000", "content": "

Congrats!

", "votes": 1, "replies": [] }, { "id": 672609, "author_name": "SeuTao", "author_url": "", "post_date": "2019-11-14T02:35:13.237000", "content": "

Congrats!

", "votes": 1, "replies": [] }, { "id": 672706, "author_name": "takuoko", "author_url": "", "post_date": "2019-11-14T04:36:14.067000", "content": "

Congrats! It is much different from mine, so looks very cool! Thank you for your sharing.

", "votes": 2, "replies": [] }, { "id": 1046269, "author_name": "Nitin Datta", "author_url": "", "post_date": "2020-10-11T14:10:28.650000", "content": "

Hi @darraghdog ,
\nCongratulations on the win.
\nCan you please help me understand when you extract the embeddings do you use pretrained weights to get the embeddings or do you use your weights that has been trained for 5 epochs??

\n

Any information regarding this will be really helpful.
\nThank you

", "votes": 0, "replies": [] }, { "id": 749351, "author_name": "Jacky Ko", "author_url": "", "post_date": "2020-02-18T16:10:08.210000", "content": "

Can you explain more on the 2048 embedding? How do you convert the logit output to multiple vectors?

", "votes": 0, "replies": [ { "id": 797135, "author_name": "Darragh", "author_url": "", "post_date": "2020-04-04T08:27:46.083000", "content": "

A single brain hemorrhage scan is made up of multiple images, as the scanner moves over the brain. There is one vector per image, and these vectors are sequenced according to time that image was taken within the scan.

", "votes": 0, "replies": [] } ] }, { "id": 673309, "author_name": "Daithí O'Manacháin", "author_url": "", "post_date": "2019-11-14T19:55:29.467000", "content": "

Amazing work darragh!!

", "votes": 0, "replies": [] }, { "id": 673020, "author_name": "Deep Chatterjee", "author_url": "", "post_date": "2019-11-14T12:06:23.910000", "content": "

Congratulations @darraghdog and thanks for sharing your solution code.

", "votes": 0, "replies": [] }, { "id": 672949, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2019-11-14T10:23:38.967000", "content": "

@darraghdog congrats and thanks for sharing.

", "votes": 0, "replies": [] }, { "id": 672925, "author_name": "Mihail Burduja", "author_url": "", "post_date": "2019-11-14T09:42:14.717000", "content": "

Congrats! The solution is very interesting and insightful. Thanks for sharing.

", "votes": 0, "replies": [] }, { "id": 672889, "author_name": "LongYin/杰少", "author_url": "", "post_date": "2019-11-14T08:52:57.493000", "content": "

Congratulations!

", "votes": 0, "replies": [] }, { "id": 672852, "author_name": "KhanhVD", "author_url": "", "post_date": "2019-11-14T08:24:08.720000", "content": "

Thank you for sharing your solution and code\nCongratulations @darraghdog !!

", "votes": 0, "replies": [] }, { "id": 678142, "author_name": "chew wx", "author_url": "", "post_date": "2019-11-21T03:00:04.097000", "content": "

cool, thanks for sharing

", "votes": 1, "replies": [] }, { "id": 672608, "author_name": "cksugarme", "author_url": "", "post_date": "2019-11-14T02:34:08.267000", "content": "

thanks

", "votes": 1, "replies": [] }, { "id": 673210, "author_name": "Μαριος Μιχαηλιδης KazAnova", "author_url": "", "post_date": "2019-11-14T17:10:11.777000", "content": "

Great work - thank you for sharing!

", "votes": 0, "replies": [] }, { "id": 672855, "author_name": "hqhz1817", "author_url": "", "post_date": "2019-11-14T08:26:42.050000", "content": "

Congrats! Thank you for your sharing.

", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "672594": "Code & Val/LB scores : https://github.com/darraghdog/rsna \nCongrats all winners, looking forward to go through your solutions. Big shout out to competition hosts RSNA, kaggle community, pytorch community, albumentations and FB's work on resnext -very cool how they trained this. \nWe were very sad not to get a top3 in Recursion competition, now we are very happy 😄 \n \n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F213493%2Fc76202951ffb4afcc5d56acb579ac552%2Frsna_nobrainer.png?generation=1573698154700507&alt=media)\n \n**Overview**\nIn general we just have a single image classifier, data split on 5 folds, we only trained on 3 of them, and then extracted pre-logit layer from the classifier and fed into an LSTM.\nClassifier trained on 5 epochs each fold, 480 images with below pre-processing. Each epoch, each fold, we extract embedding layer (use TTA and avg embeddings) train a separate LSTM for 12 epochs on each of those - so 15 LSTMs (3 fold image models X 5 epochs), and average the predictions. \nWas a bit concerned the preprocessing filter may lose information, so trained the above again without the preprocessing filter and it did worse; but averaging both pipelines did ever so slightly better. The pipeline from first paragraph above would, for all intensive purposes be just as good as final solution, but as we needed to fix docu pre-stage 2 the two pipelines are in github and final solution. \n\n**Preprocessing:**\n- Used Appian’s windowing from dicom images. [Linky](https://github.com/darraghdog/rsna/blob/master/eda/window_v1_test.py#L66)\n- Cut any black space. There were then headrest or machine artifacts in the image making the head much smaller than it could be - see visual above. These were generally thin lines, so used scipy.ndimage minimum_filter to try to wipe those thin lines. [Linky](https://github.com/darraghdog/rsna/blob/a97018a7b7ec920425189c7e37c1128dd9cb0158/scripts/resnext101v12/trainorig.py#L159)\n- Albumentations as mentioned in visual above. \n\n**Image classifier**\n- Resnext101 - did not spend a whole lot of time here as it ran so long. But tested SeResenext and Efficitentnetv0 and they did not work as well. \n- Extract pre logit layer (GAP layer) at inference time [Linky](https://github.com/darraghdog/rsna/blob/a97018a7b7ec920425189c7e37c1128dd9cb0158/scripts/resnext101v12/trainorig.py#L387) \n\n**Create Sequences**\n- Extract metadata from dicoms : [Linky](https://github.com/darraghdog/rsna/blob/master/eda/meta_eda_v1.py) \n- Sequence images on Patient, Study and Series - most sequences were between 24 and 60 images in length. [Linky](https://github.com/darraghdog/rsna/blob/a97018a7b7ec920425189c7e37c1128dd9cb0158/scripts/resnext101v12/trainlstmdeltasum.py#L200) \n\n**LSTM**\n- Feed in the embeddings in sequence on above key - Patient, Study and Series - also concat on the deltas between current and previous/next embeddings (`current-previous embedding` and `current-next embedding`) to give the model knowledge of changes around the image. [Linky](https://github.com/darraghdog/rsna/blob/a97018a7b7ec920425189c7e37c1128dd9cb0158/scripts/resnext101v12/trainlstmdeltasum.py#L133) \n- LSTM architecture lifted from the winners of first stage toxic competition. This is a beast - only improvements came from making the hiddens layers larger. Oh, we added on the embeddings to the lstm output and this helped a bit also. [Linky](https://github.com/darraghdog/rsna/blob/a97018a7b7ec920425189c7e37c1128dd9cb0158/scripts/resnext101v12/trainlstmdeltasum.py#L352) \n- For sequences of different length, padded them to same length, made a dummy embedding of zeros, and then through the results of this away before calculating loss and saving the predictions. \n\n**What did not help...** \nToo long to do justice... mixup on image, mixup on embedding, augmentations on sequences (partial sequences, reversed sequences), 1d convolutions for sequences (although SeuTao got it working)\n\n**Given more time** \nMake the classifier and the lstm model single end-to-end model. \nTrain all on stage2 data, we only got to train two folds of the image model on stage-2 data.\n", "672610": "Very nice! did you pad the sequences to fix the input size? For example, some studies consisted of 20 slices others 40 slices, etc. How did you deal with variable amount of slices per study?", "735322": "This amazing! It's really a great work! ", "682100": "Great work!", "678855": "Simply amazing @darraghdog . Since there was windowing dependencies, you went for resnext lstm??. Just curious to know , if this works even for normal image classification competitions as well when there is not much of spatial temporal dependency.. ", "673431": "Congrats and thank you for sharing. Using Bi-LSTM for image classification is interesting to me!", "672716": "Congratulations\nThanks for Sharing your Valuable Insights & Approach @darraghdog ", "672674": "Congrats!", "672609": "Congrats! ", "672706": " Congrats! It is much different from mine, so looks very cool! Thank you for your sharing.", "1046269": "Hi @darraghdog , \nCongratulations on the win.\nCan you please help me understand when you extract the embeddings do you use pretrained weights to get the embeddings or do you use your weights that has been trained for 5 epochs??\n\nAny information regarding this will be really helpful.\nThank you", "749351": "Can you explain more on the 2048 embedding? How do you convert the logit output to multiple vectors?", "673309": "Amazing work darragh!!", "673020": "Congratulations @darraghdog and thanks for sharing your solution code.", "672949": "@darraghdog congrats and thanks for sharing.", "672925": "Congrats! The solution is very interesting and insightful. Thanks for sharing.", "672889": "**Congratulations!**", "672852": "Thank you for sharing your solution and code\nCongratulations @darraghdog !!", "678142": "cool, thanks for sharing", "672608": "thanks", "673210": "Great work - thank you for sharing!", "672855": "Congrats! Thank you for your sharing." } }