{
  "id": 118873,
  "title": "10th place solution (+ github code)",
  "url": "/competitions/rsna-intracranial-hemorrhage-detection/discussion/118873",
  "author_name": "shimacos",
  "post_date": "2019-11-25T05:10:32.164000",
  "votes": 8,
  "comment_count": 1,
  "views": 0,
  "content": "<p>Congratulations to all participants and the winners ! \nAnd, I became KaggleMaster on this competition ! \nWe will go to Japanese BBQ (Yakiniku) by prize money of this competition, haha.</p>\n\n<p>Following is a summary of our solutions.</p>\n\n<p>Code : <a href=\"https://github.com/shimacos37/kaggle_rsna_2019_10th_solution\">https://github.com/shimacos37/kaggle_rsna_2019_10th_solution</a>\nWe mostly used <a href=\"/appian\">@appian</a> code. Thank you very much <a href=\"/appian\">@appian</a> !!</p>\n\n<h1>Pipeline</h1>\n\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1227363%2F393c14ac48f011e5cab030c1493eb47e%2FRSNA_pipeline%20(4\" alt=\"\">.png?generation=1574649511734185&amp;alt=media)</p>\n\n<h1>Summary</h1>\n\n<h2>Stage 1</h2>\n\n<h3>Preprocess</h3>\n\n<ul>\n<li>As most of people did, we applied three window (brain, blood/subdural, bone).</li>\n<li>Delete some noisy image (image which has small brain area).</li>\n<li>PatientID based 5-fold split.</li>\n</ul>\n\n<h3>Train</h3>\n\n<ul>\n<li>We trained simply changed backbone in <a href=\"/appian\">@appian</a> code and applied some ideas.</li>\n<li>We usually used 512x512 img_size and applied simple augmentations (flip, resize, etc...)</li>\n<li>Finally, We constructed eleven models. Consequently, I think that it doesn't need to construct so many models...</li>\n</ul>\n\nSimple CNN models\n\n<ol>\n<li>SeResNext-50</li>\n<li>SeResNext-50 (Resize 410x410)</li>\n<li>SeResNext-101 (Mixup used)</li>\n<li>Efficientnetb3</li>\n<li>InceptionV4</li>\n<li>InceptionResNetV2</li>\n<li>Xception</li>\n</ol>\n\nSome Ideas\n\n<ul>\n<li><p>We predicted label without 'any' and 'any' by other label probability (1 - (1-p_1)*(1-p_2)...)</p>\n\n<ul><li>This is not so high score, but should be have some contribution when stacking.</li></ul></li>\n<li><p>We used adjacent images for input, and predict center label. Please see following figure.</p></li>\n</ul>\n\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1227363%2Ff3dc21d1aa487195426175d6aa5f4401%2FUntitled%20Diagram%20(3\" alt=\"\">.png?generation=1574650665165165&amp;alt=media)</p>\n\n<ul>\n<li>We applied label smoothing by moving average or interpolation of the sandwiched label area.\n<ul><li>Because, we noticed the the boundary of label tends to have high log_loss by our EDA.</li></ul></li>\n</ul>\n\n<h2>Stage 2</h2>\n\n<h3>Preprocess</h3>\n\n<ul>\n<li>First, we predicted the probabilities of labels per an image.</li>\n<li>Second, we sorted the probabilities by Position2 per StudyInstanceUID.</li>\n<li>We extracted below features.\n<ul><li>Aggregate feature (min, max, mean, std), pred-pred_mean, pred / pred_mean, etc</li>\n<li>Moving average feature (3, 5, 7, 9 adjacent prediction), pred - moving_average_pred, pred / moving_average_pred, etc</li></ul></li>\n</ul>\n\n<h3>Stacking</h3>\n\n<ul>\n<li>We simply trained LightGBM and MLP by above features.</li>\n<li>And we constructed CNN stacking model like below figure.</li>\n</ul>\n\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1227363%2Fce8d6f1abe7fcd102dcdea7a1b93fb72%2Fcnn_stacking%20(3\" alt=\"\">.png?generation=1574658076805462&amp;alt=media)</p>\n\n<ul>\n<li>We treated above features as images.\n<ul><li>height : features from different models</li>\n<li>width : feature dimension</li>\n<li>channel : adjacent features sorted by Position2</li></ul></li>\n</ul>\n\n<h2>Stage 3</h2>\n\n<h3>Preprocess</h3>\n\n<ul>\n<li>We used the same method of Stage 2.</li>\n</ul>\n\n<h3>Stacking</h3>\n\n<ul>\n<li>We simply trained LightGBM</li>\n<li>We clipped prediction values by [1e-6  1 - 1e-6] and made submissions.</li>\n</ul>",
  "messages": [
    {
      "id": 680701,
      "postDate": "2019-11-25T05:10:32.163Z",
      "content": "<p>Congratulations to all participants and the winners ! \nAnd, I became KaggleMaster on this competition ! \nWe will go to Japanese BBQ (Yakiniku) by prize money of this competition, haha.</p>\n\n<p>Following is a summary of our solutions.</p>\n\n<p>Code : <a href=\"https://github.com/shimacos37/kaggle_rsna_2019_10th_solution\">https://github.com/shimacos37/kaggle_rsna_2019_10th_solution</a>\nWe mostly used <a href=\"/appian\">@appian</a> code. Thank you very much <a href=\"/appian\">@appian</a> !!</p>\n\n<h1>Pipeline</h1>\n\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1227363%2F393c14ac48f011e5cab030c1493eb47e%2FRSNA_pipeline%20(4\" alt=\"\">.png?generation=1574649511734185&amp;alt=media)</p>\n\n<h1>Summary</h1>\n\n<h2>Stage 1</h2>\n\n<h3>Preprocess</h3>\n\n<ul>\n<li>As most of people did, we applied three window (brain, blood/subdural, bone).</li>\n<li>Delete some noisy image (image which has small brain area).</li>\n<li>PatientID based 5-fold split.</li>\n</ul>\n\n<h3>Train</h3>\n\n<ul>\n<li>We trained simply changed backbone in <a href=\"/appian\">@appian</a> code and applied some ideas.</li>\n<li>We usually used 512x512 img_size and applied simple augmentations (flip, resize, etc...)</li>\n<li>Finally, We constructed eleven models. Consequently, I think that it doesn't need to construct so many models...</li>\n</ul>\n\nSimple CNN models\n\n<ol>\n<li>SeResNext-50</li>\n<li>SeResNext-50 (Resize 410x410)</li>\n<li>SeResNext-101 (Mixup used)</li>\n<li>Efficientnetb3</li>\n<li>InceptionV4</li>\n<li>InceptionResNetV2</li>\n<li>Xception</li>\n</ol>\n\nSome Ideas\n\n<ul>\n<li><p>We predicted label without 'any' and 'any' by other label probability (1 - (1-p_1)*(1-p_2)...)</p>\n\n<ul><li>This is not so high score, but should be have some contribution when stacking.</li></ul></li>\n<li><p>We used adjacent images for input, and predict center label. Please see following figure.</p></li>\n</ul>\n\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1227363%2Ff3dc21d1aa487195426175d6aa5f4401%2FUntitled%20Diagram%20(3\" alt=\"\">.png?generation=1574650665165165&amp;alt=media)</p>\n\n<ul>\n<li>We applied label smoothing by moving average or interpolation of the sandwiched label area.\n<ul><li>Because, we noticed the the boundary of label tends to have high log_loss by our EDA.</li></ul></li>\n</ul>\n\n<h2>Stage 2</h2>\n\n<h3>Preprocess</h3>\n\n<ul>\n<li>First, we predicted the probabilities of labels per an image.</li>\n<li>Second, we sorted the probabilities by Position2 per StudyInstanceUID.</li>\n<li>We extracted below features.\n<ul><li>Aggregate feature (min, max, mean, std), pred-pred_mean, pred / pred_mean, etc</li>\n<li>Moving average feature (3, 5, 7, 9 adjacent prediction), pred - moving_average_pred, pred / moving_average_pred, etc</li></ul></li>\n</ul>\n\n<h3>Stacking</h3>\n\n<ul>\n<li>We simply trained LightGBM and MLP by above features.</li>\n<li>And we constructed CNN stacking model like below figure.</li>\n</ul>\n\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1227363%2Fce8d6f1abe7fcd102dcdea7a1b93fb72%2Fcnn_stacking%20(3\" alt=\"\">.png?generation=1574658076805462&amp;alt=media)</p>\n\n<ul>\n<li>We treated above features as images.\n<ul><li>height : features from different models</li>\n<li>width : feature dimension</li>\n<li>channel : adjacent features sorted by Position2</li></ul></li>\n</ul>\n\n<h2>Stage 3</h2>\n\n<h3>Preprocess</h3>\n\n<ul>\n<li>We used the same method of Stage 2.</li>\n</ul>\n\n<h3>Stacking</h3>\n\n<ul>\n<li>We simply trained LightGBM</li>\n<li>We clipped prediction values by [1e-6  1 - 1e-6] and made submissions.</li>\n</ul>",
      "rawMarkdown": "Congratulations to all participants and the winners ! \nAnd, I became KaggleMaster on this competition ! \nWe will go to Japanese BBQ (Yakiniku) by prize money of this competition, haha.\n\nFollowing is a summary of our solutions.\n\nCode : https://github.com/shimacos37/kaggle_rsna_2019_10th_solution\nWe mostly used @appian code. Thank you very much @appian !!\n\n#  Pipeline\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1227363%2F393c14ac48f011e5cab030c1493eb47e%2FRSNA_pipeline%20(4).png?generation=1574649511734185&amp;alt=media)\n\n#  Summary\n\n## Stage 1\n### Preprocess\n\n- As most of people did, we applied three window (brain, blood/subdural, bone).\n- Delete some noisy image (image which has small brain area).\n- PatientID based 5-fold split.\n\n### Train\n\n- We trained simply changed backbone in @appian code and applied some ideas.\n- We usually used 512x512 img_size and applied simple augmentations (flip, resize, etc...)\n- Finally, We constructed eleven models. Consequently, I think that it doesn't need to construct so many models...\n\n#### Simple CNN models\n\n1. SeResNext-50\n1. SeResNext-50 (Resize 410x410)\n1. SeResNext-101 (Mixup used)\n1. Efficientnetb3\n1. InceptionV4\n1. InceptionResNetV2\n1. Xception\n\n#### Some Ideas\n\n- We predicted label without 'any' and 'any' by other label probability (1 - (1-p_1)*(1-p_2)...)\n  - This is not so high score, but should be have some contribution when stacking.\n\n- We used adjacent images for input, and predict center label. Please see following figure.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1227363%2Ff3dc21d1aa487195426175d6aa5f4401%2FUntitled%20Diagram%20(3).png?generation=1574650665165165&amp;alt=media)\n\n- We applied label smoothing by moving average or interpolation of the sandwiched label area.\n  - Because, we noticed the the boundary of label tends to have high log_loss by our EDA.\n\n## Stage 2 \n\n### Preprocess\n\n- First, we predicted the probabilities of labels per an image.\n- Second, we sorted the probabilities by Position2 per StudyInstanceUID.\n- We extracted below features.\n  - Aggregate feature (min, max, mean, std), pred-pred_mean, pred / pred_mean, etc\n  - Moving average feature (3, 5, 7, 9 adjacent prediction), pred - moving_average_pred, pred / moving_average_pred, etc\n\n### Stacking\n\n- We simply trained LightGBM and MLP by above features.\n- And we constructed CNN stacking model like below figure.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1227363%2Fce8d6f1abe7fcd102dcdea7a1b93fb72%2Fcnn_stacking%20(3).png?generation=1574658076805462&amp;alt=media)\n\n- We treated above features as images.\n  - height : features from different models\n  - width : feature dimension\n  - channel : adjacent features sorted by Position2\n\n## Stage 3\n\n### Preprocess\n\n- We used the same method of Stage 2.\n\n### Stacking\n\n- We simply trained LightGBM\n- We clipped prediction values by [1e-6  1 - 1e-6] and made submissions.",
      "votes": 8
    },
    {
      "id": 680723,
      "postDate": "2019-11-25T06:05:36.237Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 680723,
      "author_name": "",
      "author_url": "",
      "post_date": "2019-11-25T06:05:36.237000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "680701": "Congratulations to all participants and the winners ! \nAnd, I became KaggleMaster on this competition ! \nWe will go to Japanese BBQ (Yakiniku) by prize money of this competition, haha.\n\nFollowing is a summary of our solutions.\n\nCode : https://github.com/shimacos37/kaggle_rsna_2019_10th_solution\nWe mostly used @appian code. Thank you very much @appian !!\n\n#  Pipeline\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1227363%2F393c14ac48f011e5cab030c1493eb47e%2FRSNA_pipeline%20(4).png?generation=1574649511734185&amp;alt=media)\n\n#  Summary\n\n## Stage 1\n### Preprocess\n\n- As most of people did, we applied three window (brain, blood/subdural, bone).\n- Delete some noisy image (image which has small brain area).\n- PatientID based 5-fold split.\n\n### Train\n\n- We trained simply changed backbone in @appian code and applied some ideas.\n- We usually used 512x512 img_size and applied simple augmentations (flip, resize, etc...)\n- Finally, We constructed eleven models. Consequently, I think that it doesn't need to construct so many models...\n\n#### Simple CNN models\n\n1. SeResNext-50\n1. SeResNext-50 (Resize 410x410)\n1. SeResNext-101 (Mixup used)\n1. Efficientnetb3\n1. InceptionV4\n1. InceptionResNetV2\n1. Xception\n\n#### Some Ideas\n\n- We predicted label without 'any' and 'any' by other label probability (1 - (1-p_1)*(1-p_2)...)\n  - This is not so high score, but should be have some contribution when stacking.\n\n- We used adjacent images for input, and predict center label. Please see following figure.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1227363%2Ff3dc21d1aa487195426175d6aa5f4401%2FUntitled%20Diagram%20(3).png?generation=1574650665165165&amp;alt=media)\n\n- We applied label smoothing by moving average or interpolation of the sandwiched label area.\n  - Because, we noticed the the boundary of label tends to have high log_loss by our EDA.\n\n## Stage 2 \n\n### Preprocess\n\n- First, we predicted the probabilities of labels per an image.\n- Second, we sorted the probabilities by Position2 per StudyInstanceUID.\n- We extracted below features.\n  - Aggregate feature (min, max, mean, std), pred-pred_mean, pred / pred_mean, etc\n  - Moving average feature (3, 5, 7, 9 adjacent prediction), pred - moving_average_pred, pred / moving_average_pred, etc\n\n### Stacking\n\n- We simply trained LightGBM and MLP by above features.\n- And we constructed CNN stacking model like below figure.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1227363%2Fce8d6f1abe7fcd102dcdea7a1b93fb72%2Fcnn_stacking%20(3).png?generation=1574658076805462&amp;alt=media)\n\n- We treated above features as images.\n  - height : features from different models\n  - width : feature dimension\n  - channel : adjacent features sorted by Position2\n\n## Stage 3\n\n### Preprocess\n\n- We used the same method of Stage 2.\n\n### Stacking\n\n- We simply trained LightGBM\n- We clipped prediction values by [1e-6  1 - 1e-6] and made submissions.",
    "680723": ""
  }
}