{ "id": 183176, "title": "Starting Point: Last Year's RSNA Challenge", "url": "/competitions/rsna-str-pulmonary-embolism-detection/discussion/183176", "author_name": "Ian Pan", "post_date": "2020-09-15T20:53:56.232000", "votes": 57, "comment_count": 15, "views": 0, "content": "

Last year's RSNA challenge was focused on detecting brain bleeds in head CTs. It's the most similar Kaggle challenge to this current one.

\n

I would encourage all of you to read the top solutions from last year's challenge:
\nhttps://www.kaggle.com/c/rsna-intracranial-hemorrhage-detection/discussion/117242

\n

Some key points: many competitors stacked continuous slices into a multi-channel image - while not fully 3D, this still helped the model access additional spatial information from neighboring slices. Top solutions also used sequence modeling (CNN into LSTM) to improve performance.

\n

This challenge is more complex, with the addition of exam-level labels in addition to image-level labels.

", "messages": [ { "id": 1012010, "postDate": "2020-09-15T20:53:56.233Z", "content": "

Last year's RSNA challenge was focused on detecting brain bleeds in head CTs. It's the most similar Kaggle challenge to this current one.

\n

I would encourage all of you to read the top solutions from last year's challenge:
\nhttps://www.kaggle.com/c/rsna-intracranial-hemorrhage-detection/discussion/117242

\n

Some key points: many competitors stacked continuous slices into a multi-channel image - while not fully 3D, this still helped the model access additional spatial information from neighboring slices. Top solutions also used sequence modeling (CNN into LSTM) to improve performance.

\n

This challenge is more complex, with the addition of exam-level labels in addition to image-level labels.

", "rawMarkdown": "Last year's RSNA challenge was focused on detecting brain bleeds in head CTs. It's the most similar Kaggle challenge to this current one. \n\nI would encourage all of you to read the top solutions from last year's challenge:\nhttps://www.kaggle.com/c/rsna-intracranial-hemorrhage-detection/discussion/117242\n\nSome key points: many competitors stacked continuous slices into a multi-channel image - while not fully 3D, this still helped the model access additional spatial information from neighboring slices. Top solutions also used sequence modeling (CNN into LSTM) to improve performance.\n\nThis challenge is more complex, with the addition of exam-level labels in addition to image-level labels. ", "votes": 56 }, { "id": 1013608, "postDate": "2020-09-16T20:03:55.790Z", "content": "

I'm trying to understand the rationale behind the sequence modeling in the winning submission. I understand that the input to Sequence Model 1 is just the concatenated outputs from all the CNN Models. But why does sequence modeling in this case work better over straight dense layers? Is it the bidirectionality of the RNN layers that allow for the model to make more complex correlations between what the CNN Models are seeing?

", "rawMarkdown": "I'm trying to understand the rationale behind the sequence modeling in the winning submission. I understand that the input to Sequence Model 1 is just the concatenated outputs from all the CNN Models. But why does sequence modeling in this case work better over straight dense layers? Is it the bidirectionality of the RNN layers that allow for the model to make more complex correlations between what the CNN Models are seeing?", "votes": 1, "replies": [ { "id": 1013793, "postDate": "2020-09-17T01:44:49.297Z", "content": "

There were 24-40 CT slice images per study if I remember correctly. Neighbouring slices tend to have the same hemorrhage types because these brain slices are physically very close. RNN can model that inter-slice relationships and help enhancing predictions from CNN which only relies on indivisual slices.

", "rawMarkdown": "There were 24-40 CT slice images per study if I remember correctly. Neighbouring slices tend to have the same hemorrhage types because these brain slices are physically very close. RNN can model that inter-slice relationships and help enhancing predictions from CNN which only relies on indivisual slices.", "votes": 10 }, { "id": 1013838, "postDate": "2020-09-17T02:51:02.613Z", "content": "

I see. Interesting application of RNNs. Thanks!

", "rawMarkdown": "I see. Interesting application of RNNs. Thanks!", "votes": 1 } ] }, { "id": 1012114, "postDate": "2020-09-15T23:44:58.840Z", "content": "

@vaillant is any specific reason none of the top solutions are used tensorflow. All top solutions are used torch. Is tpu is not good option for this competition ?

", "rawMarkdown": "@vaillant is any specific reason none of the top solutions are used tensorflow. All top solutions are used torch. Is tpu is not good option for this competition ?", "votes": 1, "replies": [ { "id": 1012269, "postDate": "2020-09-16T02:19:05.283Z", "content": "

I think TPU will be used in this competition. I think the main issue isn't tensorflow vs pytorch, it's how to create a pipeline that can reasonably read/write the incredibly large dataset. With a TPU, the hope is that it can read faster because data is stored in GCS buckets. Most of the time, your data will be sitting in a harddisk waiting to be read.

", "rawMarkdown": "I think TPU will be used in this competition. I think the main issue isn't tensorflow vs pytorch, it's how to create a pipeline that can reasonably read/write the incredibly large dataset. With a TPU, the hope is that it can read faster because data is stored in GCS buckets. Most of the time, your data will be sitting in a harddisk waiting to be read.", "votes": 3 }, { "id": 1023378, "postDate": "2020-09-23T07:12:42.383Z", "content": "

Hi @seshurajup
\nWhy do you say that none of the top solution are TF, but PyTorch?
\nCan you share your source of this inference?

", "rawMarkdown": "Hi @seshurajup \nWhy do you say that none of the top solution are TF, but PyTorch?\nCan you share your source of this inference?", "votes": -3 }, { "id": 1053260, "postDate": "2020-10-18T19:15:31.827Z", "content": "

how much time is reasonable? I got it to read the whole dataset in about 1h. is that good enough?

", "rawMarkdown": "how much time is reasonable? I got it to read the whole dataset in about 1h. is that good enough?" } ] }, { "id": 1049786, "postDate": "2020-10-14T19:13:38.443Z", "content": "

Which augmentation do you think is better over the 2-D version of images?

", "rawMarkdown": "Which augmentation do you think is better over the 2-D version of images?" }, { "id": 1028883, "postDate": "2020-09-27T09:36:56.660Z", "content": "

Gona follow your guidance in this competition, haha

", "rawMarkdown": "Gona follow your guidance in this competition, haha" }, { "id": 1026157, "postDate": "2020-09-25T05:49:56.373Z", "content": "

How could i train such complex models with kaggle notebook !!!☹️

", "rawMarkdown": "How could i train such complex models with kaggle notebook !!!☹️", "replies": [ { "id": 1026646, "postDate": "2020-09-25T13:19:04.703Z", "content": "

@kingofdaydream The models from last year weren't that complex. They are divided into two steps:

\n
    \n
  1. first you train a base model (any model pretrained on imagenet will do)
    2. \n
  2. you use the input to the classification layer as a feature vector (i.e. you delete the classification layer an use the output of the previous layer as the model's output).
    3. \n
  3. using this model you extract features for all the images
    4. \n
  4. you train the 2nd stage model (CNN, LSTM, etc.) on the feature vectors
    5. \n
", "rawMarkdown": "@kingofdaydream The models from last year weren't that complex. They are divided into two steps:\n1. first you train a base model (any model pretrained on imagenet will do)\n2. you use the input to the classification layer as a feature vector (i.e. you delete the classification layer an use the output of the previous layer as the model's output).\n3. using this model you extract features for all the images\n4. you train the 2nd stage model (CNN, LSTM, etc.) on the feature vectors", "votes": 18 }, { "id": 1026836, "postDate": "2020-09-25T15:29:04.113Z", "content": "

Nice! Thank you very much.

", "rawMarkdown": "Nice! Thank you very much." }, { "id": 1058694, "postDate": "2020-10-24T05:49:00.827Z", "content": "

Hi @yuval6967

\n

have you tried LSTM in the current competition as well?
\nChecking to see if that helped vs GRUs

\n

Thanks!

", "rawMarkdown": "Hi @yuval6967\n\nhave you tried LSTM in the current competition as well?\nChecking to see if that helped vs GRUs\n\nThanks!" }, { "id": 1058751, "postDate": "2020-10-24T07:44:28.307Z", "content": "

No I haven't.

", "rawMarkdown": "No I haven't.", "votes": 1 }, { "id": 1058795, "postDate": "2020-10-24T09:33:01.417Z", "rawMarkdown": "", "isDeleted": true } ] } ], "comments": [ { "id": 1013608, "author_name": "Joseph Tan", "author_url": "", "post_date": "2020-09-16T20:03:55.790000", "content": "

I'm trying to understand the rationale behind the sequence modeling in the winning submission. I understand that the input to Sequence Model 1 is just the concatenated outputs from all the CNN Models. But why does sequence modeling in this case work better over straight dense layers? Is it the bidirectionality of the RNN layers that allow for the model to make more complex correlations between what the CNN Models are seeing?

", "votes": 1, "replies": [ { "id": 1013793, "author_name": "Appian", "author_url": "", "post_date": "2020-09-17T01:44:49.297000", "content": "

There were 24-40 CT slice images per study if I remember correctly. Neighbouring slices tend to have the same hemorrhage types because these brain slices are physically very close. RNN can model that inter-slice relationships and help enhancing predictions from CNN which only relies on indivisual slices.

", "votes": 10, "replies": [] }, { "id": 1013838, "author_name": "Joseph Tan", "author_url": "", "post_date": "2020-09-17T02:51:02.613000", "content": "

I see. Interesting application of RNNs. Thanks!

", "votes": 1, "replies": [] } ] }, { "id": 1012114, "author_name": "SeshuRaju 🧘‍♂️", "author_url": "", "post_date": "2020-09-15T23:44:58.840000", "content": "

@vaillant is any specific reason none of the top solutions are used tensorflow. All top solutions are used torch. Is tpu is not good option for this competition ?

", "votes": 1, "replies": [ { "id": 1012269, "author_name": "Tim Yee", "author_url": "", "post_date": "2020-09-16T02:19:05.283000", "content": "

I think TPU will be used in this competition. I think the main issue isn't tensorflow vs pytorch, it's how to create a pipeline that can reasonably read/write the incredibly large dataset. With a TPU, the hope is that it can read faster because data is stored in GCS buckets. Most of the time, your data will be sitting in a harddisk waiting to be read.

", "votes": 3, "replies": [] }, { "id": 1023378, "author_name": "imnishantg", "author_url": "", "post_date": "2020-09-23T07:12:42.383000", "content": "

Hi @seshurajup
\nWhy do you say that none of the top solution are TF, but PyTorch?
\nCan you share your source of this inference?

", "votes": -3, "replies": [] }, { "id": 1053260, "author_name": "DarkCube", "author_url": "", "post_date": "2020-10-18T19:15:31.827000", "content": "

how much time is reasonable? I got it to read the whole dataset in about 1h. is that good enough?

", "votes": 0, "replies": [] } ] }, { "id": 1049786, "author_name": "Baran Hashemi", "author_url": "", "post_date": "2020-10-14T19:13:38.443000", "content": "

Which augmentation do you think is better over the 2-D version of images?

", "votes": 0, "replies": [] }, { "id": 1028883, "author_name": "豆柴金鯱", "author_url": "", "post_date": "2020-09-27T09:36:56.660000", "content": "

Gona follow your guidance in this competition, haha

", "votes": 0, "replies": [] }, { "id": 1026157, "author_name": "KingOfDayDream", "author_url": "", "post_date": "2020-09-25T05:49:56.373000", "content": "

How could i train such complex models with kaggle notebook !!!☹️

", "votes": 0, "replies": [ { "id": 1026646, "author_name": "yuval reina", "author_url": "", "post_date": "2020-09-25T13:19:04.703000", "content": "

@kingofdaydream The models from last year weren't that complex. They are divided into two steps:

\n
    \n
  1. first you train a base model (any model pretrained on imagenet will do)
    2. \n
  2. you use the input to the classification layer as a feature vector (i.e. you delete the classification layer an use the output of the previous layer as the model's output).
    3. \n
  3. using this model you extract features for all the images
    4. \n
  4. you train the 2nd stage model (CNN, LSTM, etc.) on the feature vectors
    5. \n
", "votes": 18, "replies": [] }, { "id": 1026836, "author_name": "KingOfDayDream", "author_url": "", "post_date": "2020-09-25T15:29:04.113000", "content": "

Nice! Thank you very much.

", "votes": 0, "replies": [] }, { "id": 1058694, "author_name": "Kamal Das", "author_url": "", "post_date": "2020-10-24T05:49:00.827000", "content": "

Hi @yuval6967

\n

have you tried LSTM in the current competition as well?
\nChecking to see if that helped vs GRUs

\n

Thanks!

", "votes": 0, "replies": [] }, { "id": 1058751, "author_name": "yuval reina", "author_url": "", "post_date": "2020-10-24T07:44:28.307000", "content": "

No I haven't.

", "votes": 1, "replies": [] }, { "id": 1058795, "author_name": "", "author_url": "", "post_date": "2020-10-24T09:33:01.417000", "content": "", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "1012010": "Last year's RSNA challenge was focused on detecting brain bleeds in head CTs. It's the most similar Kaggle challenge to this current one. \n\nI would encourage all of you to read the top solutions from last year's challenge:\nhttps://www.kaggle.com/c/rsna-intracranial-hemorrhage-detection/discussion/117242\n\nSome key points: many competitors stacked continuous slices into a multi-channel image - while not fully 3D, this still helped the model access additional spatial information from neighboring slices. Top solutions also used sequence modeling (CNN into LSTM) to improve performance.\n\nThis challenge is more complex, with the addition of exam-level labels in addition to image-level labels. ", "1013608": "I'm trying to understand the rationale behind the sequence modeling in the winning submission. I understand that the input to Sequence Model 1 is just the concatenated outputs from all the CNN Models. But why does sequence modeling in this case work better over straight dense layers? Is it the bidirectionality of the RNN layers that allow for the model to make more complex correlations between what the CNN Models are seeing?", "1012114": "@vaillant is any specific reason none of the top solutions are used tensorflow. All top solutions are used torch. Is tpu is not good option for this competition ?", "1049786": "Which augmentation do you think is better over the 2-D version of images?", "1028883": "Gona follow your guidance in this competition, haha", "1026157": "How could i train such complex models with kaggle notebook !!!☹️" } }