{
  "id": 119079,
  "title": "13th place solution with code",
  "url": "/competitions/rsna-intracranial-hemorrhage-detection/discussion/119079",
  "author_name": "nan",
  "post_date": "2019-11-26T12:55:29.618000",
  "votes": 8,
  "comment_count": 1,
  "views": 0,
  "content": "<p>Thanks Kaggle and RSNA for hosting such an interesting competition.\nThanks the whole team @andy2709 @moewie94 @lego1st @nguyenbadung for the great collaboration.</p>\n\n<p>The code is publicly available at <a href=\"https://github.com/dattran2346/rsna-2019\">https://github.com/dattran2346/rsna-2019</a></p>\n\n<h2>1. Preprocessing</h2>\n\n<h3>Windowing</h3>\n\n<ul>\n<li>We use various combination of  brain, subdural, bony, and default window and stack them to create a normal 3-channel image.</li>\n</ul>\n\n<p><code>\nct_windows = {\n    'brain': {'L': 40, 'W': 80},\n    'subdural': {'L': 75, 'W': 215},\n    'bony': {'L': 600, 'W': 2800},\n    'default': { # from metadata }\n}\n</code></p>\n\n<h3>Window setting optimization</h3>\n\n<ul>\n<li>The idea is to use a 1x1 convolution and sigmoid activation to learn relevant windows, the weight is initialized to be the default brain, subdural and bony window. @andy2709 tried this method and noticed that the final learned window is very closed to the default window. </li>\n</ul>\n\n<h3>Data split</h3>\n\n<ul>\n<li>We splited the dataset by both patient id and study id. I, @andy2709 and @lego1st trained the models by patient split, while @nguyenbadung and @moewie94 trained by study split.</li>\n</ul>\n\n<h2>2. Model</h2>\n\n<h3>2D Model</h3>\n\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3342872%2F1f61df62cbecc1d1777aeeb6069c19ae%2F2dmodel.png?generation=1574771438363883&amp;alt=media\" alt=\"\"></p>\n\n<p>We applied 2 stage training here:\n- In the 1st stage, just a normal CNN training,  backbones are EfficientNetB2-B5, SEResNeXt50, SEResNeXt101.\n- In the 2nd, we use 5 consecutive slices’ outputs and applied a simple CNN to predict the center slice.</p>\n\n<h3>3D Model</h3>\n\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3342872%2F45cd1a578fa954caba1551680f907b69%2F3dmodels.png?generation=1574771686608008&amp;alt=media\" alt=\"\"></p>\n\n<ul>\n<li>We use a normal backbone as a decoder and a bi-directional LSTM with a FC layer as the decoder, the model was trained end-to-end.</li>\n<li>For each study, we select 10 random slices (contiguous and not) in order and put though the network during training. For inference, all slices are considered.</li>\n</ul>\n\n<h2>3. Stacking</h2>\n\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3342872%2F3e4aba9020f00b35b66a03f103057fc0%2Fstacking.png?generation=1574771786432271&amp;alt=media\" alt=\"\"></p>\n\n<ul>\n<li>Concatenate prediction from all model (split by both study-id and patient-id), build simple cnn model:</li>\n</ul>\n\n<p><code>\nmodel = nn.Sequential(\n          nn.Linear(input_dim, 1024),\n          nn.ReLU(),\n          nn.Dropout(0.5),\n          nn.Linear(1024, 1024),\n          nn.ReLU(),\n          nn.Dropout(0.5),\n          nn.Linear(1024, 6),)\n</code></p>\n\n<ul>\n<li>Average prediction from 2 types of model:  study id split and patient id split.</li>\n</ul>\n\n<h2>4. Summary</h2>\n\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3342872%2F9d5b6d677fa3626ac13f6299e2b6d094%2FScreenshot%20from%202019-11-26%2019-46-08.png?generation=1574772402149756&amp;alt=media\" alt=\"\"></p>",
  "messages": [
    {
      "id": 681714,
      "postDate": "2019-11-26T12:55:29.620Z",
      "content": "<p>Thanks Kaggle and RSNA for hosting such an interesting competition.\nThanks the whole team @andy2709 @moewie94 @lego1st @nguyenbadung for the great collaboration.</p>\n\n<p>The code is publicly available at <a href=\"https://github.com/dattran2346/rsna-2019\">https://github.com/dattran2346/rsna-2019</a></p>\n\n<h2>1. Preprocessing</h2>\n\n<h3>Windowing</h3>\n\n<ul>\n<li>We use various combination of  brain, subdural, bony, and default window and stack them to create a normal 3-channel image.</li>\n</ul>\n\n<p><code>\nct_windows = {\n    'brain': {'L': 40, 'W': 80},\n    'subdural': {'L': 75, 'W': 215},\n    'bony': {'L': 600, 'W': 2800},\n    'default': { # from metadata }\n}\n</code></p>\n\n<h3>Window setting optimization</h3>\n\n<ul>\n<li>The idea is to use a 1x1 convolution and sigmoid activation to learn relevant windows, the weight is initialized to be the default brain, subdural and bony window. @andy2709 tried this method and noticed that the final learned window is very closed to the default window. </li>\n</ul>\n\n<h3>Data split</h3>\n\n<ul>\n<li>We splited the dataset by both patient id and study id. I, @andy2709 and @lego1st trained the models by patient split, while @nguyenbadung and @moewie94 trained by study split.</li>\n</ul>\n\n<h2>2. Model</h2>\n\n<h3>2D Model</h3>\n\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3342872%2F1f61df62cbecc1d1777aeeb6069c19ae%2F2dmodel.png?generation=1574771438363883&amp;alt=media\" alt=\"\"></p>\n\n<p>We applied 2 stage training here:\n- In the 1st stage, just a normal CNN training,  backbones are EfficientNetB2-B5, SEResNeXt50, SEResNeXt101.\n- In the 2nd, we use 5 consecutive slices’ outputs and applied a simple CNN to predict the center slice.</p>\n\n<h3>3D Model</h3>\n\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3342872%2F45cd1a578fa954caba1551680f907b69%2F3dmodels.png?generation=1574771686608008&amp;alt=media\" alt=\"\"></p>\n\n<ul>\n<li>We use a normal backbone as a decoder and a bi-directional LSTM with a FC layer as the decoder, the model was trained end-to-end.</li>\n<li>For each study, we select 10 random slices (contiguous and not) in order and put though the network during training. For inference, all slices are considered.</li>\n</ul>\n\n<h2>3. Stacking</h2>\n\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3342872%2F3e4aba9020f00b35b66a03f103057fc0%2Fstacking.png?generation=1574771786432271&amp;alt=media\" alt=\"\"></p>\n\n<ul>\n<li>Concatenate prediction from all model (split by both study-id and patient-id), build simple cnn model:</li>\n</ul>\n\n<p><code>\nmodel = nn.Sequential(\n          nn.Linear(input_dim, 1024),\n          nn.ReLU(),\n          nn.Dropout(0.5),\n          nn.Linear(1024, 1024),\n          nn.ReLU(),\n          nn.Dropout(0.5),\n          nn.Linear(1024, 6),)\n</code></p>\n\n<ul>\n<li>Average prediction from 2 types of model:  study id split and patient id split.</li>\n</ul>\n\n<h2>4. Summary</h2>\n\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3342872%2F9d5b6d677fa3626ac13f6299e2b6d094%2FScreenshot%20from%202019-11-26%2019-46-08.png?generation=1574772402149756&amp;alt=media\" alt=\"\"></p>",
      "rawMarkdown": "Thanks Kaggle and RSNA for hosting such an interesting competition.\nThanks the whole team @andy2709 @moewie94 @lego1st @nguyenbadung for the great collaboration.\n\nThe code is publicly available at https://github.com/dattran2346/rsna-2019\n\n## 1. Preprocessing\n\n### Windowing\n\n- We use various combination of  brain, subdural, bony, and default window and stack them to create a normal 3-channel image.\n\n```\nct_windows = {\n    'brain': {'L': 40, 'W': 80},\n    'subdural': {'L': 75, 'W': 215},\n    'bony': {'L': 600, 'W': 2800},\n    'default': { # from metadata }\n}\n```\n\n### Window setting optimization\n- The idea is to use a 1x1 convolution and sigmoid activation to learn relevant windows, the weight is initialized to be the default brain, subdural and bony window. @andy2709 tried this method and noticed that the final learned window is very closed to the default window. \n\n### Data split\n- We splited the dataset by both patient id and study id. I, @andy2709 and @lego1st trained the models by patient split, while @nguyenbadung and @moewie94 trained by study split.\n\n## 2. Model\n### 2D Model\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3342872%2F1f61df62cbecc1d1777aeeb6069c19ae%2F2dmodel.png?generation=1574771438363883&amp;alt=media)\n\nWe applied 2 stage training here:\n- In the 1st stage, just a normal CNN training,  backbones are EfficientNetB2-B5, SEResNeXt50, SEResNeXt101.\n- In the 2nd, we use 5 consecutive slices’ outputs and applied a simple CNN to predict the center slice.\n\n### 3D Model\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3342872%2F45cd1a578fa954caba1551680f907b69%2F3dmodels.png?generation=1574771686608008&amp;alt=media)\n\n- We use a normal backbone as a decoder and a bi-directional LSTM with a FC layer as the decoder, the model was trained end-to-end.\n- For each study, we select 10 random slices (contiguous and not) in order and put though the network during training. For inference, all slices are considered.\n\n## 3. Stacking\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3342872%2F3e4aba9020f00b35b66a03f103057fc0%2Fstacking.png?generation=1574771786432271&amp;alt=media)\n\n- Concatenate prediction from all model (split by both study-id and patient-id), build simple cnn model:\n\n```\nmodel = nn.Sequential(\n          nn.Linear(input_dim, 1024),\n          nn.ReLU(),\n          nn.Dropout(0.5),\n          nn.Linear(1024, 1024),\n          nn.ReLU(),\n          nn.Dropout(0.5),\n          nn.Linear(1024, 6),)\n```\n\n- Average prediction from 2 types of model:  study id split and patient id split.\n\n## 4. Summary\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3342872%2F9d5b6d677fa3626ac13f6299e2b6d094%2FScreenshot%20from%202019-11-26%2019-46-08.png?generation=1574772402149756&amp;alt=media)\n",
      "votes": 8
    },
    {
      "id": 682211,
      "postDate": "2019-11-27T05:05:17.753Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 682211,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-11-27T05:05:17.753000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "681714": "Thanks Kaggle and RSNA for hosting such an interesting competition.\nThanks the whole team @andy2709 @moewie94 @lego1st @nguyenbadung for the great collaboration.\n\nThe code is publicly available at https://github.com/dattran2346/rsna-2019\n\n## 1. Preprocessing\n\n### Windowing\n\n- We use various combination of  brain, subdural, bony, and default window and stack them to create a normal 3-channel image.\n\n```\nct_windows = {\n    'brain': {'L': 40, 'W': 80},\n    'subdural': {'L': 75, 'W': 215},\n    'bony': {'L': 600, 'W': 2800},\n    'default': { # from metadata }\n}\n```\n\n### Window setting optimization\n- The idea is to use a 1x1 convolution and sigmoid activation to learn relevant windows, the weight is initialized to be the default brain, subdural and bony window. @andy2709 tried this method and noticed that the final learned window is very closed to the default window. \n\n### Data split\n- We splited the dataset by both patient id and study id. I, @andy2709 and @lego1st trained the models by patient split, while @nguyenbadung and @moewie94 trained by study split.\n\n## 2. Model\n### 2D Model\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3342872%2F1f61df62cbecc1d1777aeeb6069c19ae%2F2dmodel.png?generation=1574771438363883&amp;alt=media)\n\nWe applied 2 stage training here:\n- In the 1st stage, just a normal CNN training,  backbones are EfficientNetB2-B5, SEResNeXt50, SEResNeXt101.\n- In the 2nd, we use 5 consecutive slices’ outputs and applied a simple CNN to predict the center slice.\n\n### 3D Model\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3342872%2F45cd1a578fa954caba1551680f907b69%2F3dmodels.png?generation=1574771686608008&amp;alt=media)\n\n- We use a normal backbone as a decoder and a bi-directional LSTM with a FC layer as the decoder, the model was trained end-to-end.\n- For each study, we select 10 random slices (contiguous and not) in order and put though the network during training. For inference, all slices are considered.\n\n## 3. Stacking\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3342872%2F3e4aba9020f00b35b66a03f103057fc0%2Fstacking.png?generation=1574771786432271&amp;alt=media)\n\n- Concatenate prediction from all model (split by both study-id and patient-id), build simple cnn model:\n\n```\nmodel = nn.Sequential(\n          nn.Linear(input_dim, 1024),\n          nn.ReLU(),\n          nn.Dropout(0.5),\n          nn.Linear(1024, 1024),\n          nn.ReLU(),\n          nn.Dropout(0.5),\n          nn.Linear(1024, 6),)\n```\n\n- Average prediction from 2 types of model:  study id split and patient id split.\n\n## 4. Summary\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3342872%2F9d5b6d677fa3626ac13f6299e2b6d094%2FScreenshot%20from%202019-11-26%2019-46-08.png?generation=1574772402149756&amp;alt=media)\n",
    "682211": ""
  }
}