{
  "id": 169163,
  "title": "16th place solution",
  "url": "/competitions/prostate-cancer-grade-assessment/discussion/169163",
  "author_name": "YuryBolkonsky",
  "post_date": "2020-07-23T05:26:16.792000",
  "votes": 15,
  "comment_count": 4,
  "views": 0,
  "content": "<p><strong>Thank you very much to organizers, participants and my teammates</strong> <a href=\"/ryunosukeishizaki\">@ryunosukeishizaki</a> <a href=\"/rinnqd\">@rinnqd</a> for such competition.</p>\n\n<p>We can name our solution as <strong>\"zero public LB to hero private LB\"</strong>.  In public we could even get a bronze and in private we are in top 20 teams. It's not a lucky submission because we have a lot of them and success points are real.</p>\n\n<ol>\n<li><p><strong>Removing noise</strong> (marks, duplicates) based on <a href=\"https://www.kaggle.com/c/prostate-cancer-grade-assessment/discussion/151323\">this Zac Dannelly</a> and <a href=\"https://www.kaggle.com/iamleonie/panda-eda-visualizations-suspicious-data\">this Leonie</a>, and also my own manual clean up</p></li>\n<li><p>Training efficientnet-b0, b2, b4 and mixnet-xl on <strong>different tiles sizes</strong> 36x256x256 (level 1) =&gt; 49x256x256 (level 1) =&gt; 64x256x256 (level 1) without regularization and with high (dropout 0.4)</p></li>\n<li><p><strong>Combining cleaning dataset training and raw data</strong></p></li>\n<li><p>Blending based on local <strong>CV weights</strong> w = w / np.sum(w)</p></li>\n</ol>\n\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1192776%2Fe5ab54359ade13256e9c62dbfdf5a80f%2F.PNG?generation=1595481733380627&amp;alt=media\" alt=\"\"></p>\n\n<p>5.<strong>Trust your local CV</strong> and train stable models!</p>",
  "messages": [
    {
      "id": 940721,
      "postDate": "2020-07-23T05:26:16.793Z",
      "content": "<p><strong>Thank you very much to organizers, participants and my teammates</strong> <a href=\"/ryunosukeishizaki\">@ryunosukeishizaki</a> <a href=\"/rinnqd\">@rinnqd</a> for such competition.</p>\n\n<p>We can name our solution as <strong>\"zero public LB to hero private LB\"</strong>.  In public we could even get a bronze and in private we are in top 20 teams. It's not a lucky submission because we have a lot of them and success points are real.</p>\n\n<ol>\n<li><p><strong>Removing noise</strong> (marks, duplicates) based on <a href=\"https://www.kaggle.com/c/prostate-cancer-grade-assessment/discussion/151323\">this Zac Dannelly</a> and <a href=\"https://www.kaggle.com/iamleonie/panda-eda-visualizations-suspicious-data\">this Leonie</a>, and also my own manual clean up</p></li>\n<li><p>Training efficientnet-b0, b2, b4 and mixnet-xl on <strong>different tiles sizes</strong> 36x256x256 (level 1) =&gt; 49x256x256 (level 1) =&gt; 64x256x256 (level 1) without regularization and with high (dropout 0.4)</p></li>\n<li><p><strong>Combining cleaning dataset training and raw data</strong></p></li>\n<li><p>Blending based on local <strong>CV weights</strong> w = w / np.sum(w)</p></li>\n</ol>\n\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1192776%2Fe5ab54359ade13256e9c62dbfdf5a80f%2F.PNG?generation=1595481733380627&amp;alt=media\" alt=\"\"></p>\n\n<p>5.<strong>Trust your local CV</strong> and train stable models!</p>",
      "rawMarkdown": "**Thank you very much to organizers, participants and my teammates** @ryunosukeishizaki @rinnqd for such competition.\n\nWe can name our solution as **\"zero public LB to hero private LB\"**.  In public we could even get a bronze and in private we are in top 20 teams. It's not a lucky submission because we have a lot of them and success points are real.\n\n\n1. **Removing noise** (marks, duplicates) based on [this Zac Dannelly](https://www.kaggle.com/c/prostate-cancer-grade-assessment/discussion/151323) and [this Leonie](https://www.kaggle.com/iamleonie/panda-eda-visualizations-suspicious-data), and also my own manual clean up\n\n2. Training efficientnet-b0, b2, b4 and mixnet-xl on **different tiles sizes** 36x256x256 (level 1) =&gt; 49x256x256 (level 1) =&gt; 64x256x256 (level 1) without regularization and with high (dropout 0.4)\n\n3. **Combining cleaning dataset training and raw data**\n\n4. Blending based on local **CV weights** w = w / np.sum(w)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1192776%2Fe5ab54359ade13256e9c62dbfdf5a80f%2F.PNG?generation=1595481733380627&amp;alt=media)\n\n5.**Trust your local CV** and train stable models!",
      "votes": 14
    },
    {
      "id": 941686,
      "postDate": "2020-07-23T10:57:18.223Z",
      "content": "<p>Congratulations! 👍 </p>\n\n<p>Also glad to see a vary familiar view of the excellence graduate diploma and the main building of our University (at your avatar) in the top 20 listing 😊 </p>\n\n<p>Would be happy to team up in the future competitions if you want.</p>",
      "rawMarkdown": "Congratulations! 👍 \n\nAlso glad to see a vary familiar view of the excellence graduate diploma and the main building of our University (at your avatar) in the top 20 listing 😊 \n\nWould be happy to team up in the future competitions if you want.\n",
      "replies": [
        {
          "id": 941868,
          "postDate": "2020-07-23T13:12:37.247Z",
          "content": "<p>Sure</p>",
          "rawMarkdown": "Sure"
        }
      ]
    },
    {
      "id": 940760,
      "postDate": "2020-07-23T05:31:51.673Z",
      "content": "<p>for us \"vovnet was the  winner \"achieved 0.92\" </p>\n\n<p>Training efficientnet-b0, b2, b4 and mixnet-xl on different tiles sizes 36x256x256 (level 1) =&gt; 48x256x256 (level 1) =&gt; 64x256x256 (level 1) without regularization and with high (dropout 0.4)</p>\n\n<p>did you train all those models with batch size 2?</p>",
      "rawMarkdown": "for us \"vovnet was the  winner \"achieved 0.92\" \n\nTraining efficientnet-b0, b2, b4 and mixnet-xl on different tiles sizes 36x256x256 (level 1) =&gt; 48x256x256 (level 1) =&gt; 64x256x256 (level 1) without regularization and with high (dropout 0.4)\n\n\ndid you train all those models with batch size 2?\n",
      "replies": [
        {
          "id": 940796,
          "postDate": "2020-07-23T05:35:11.223Z",
          "content": "<p>For one card with mixed precision it was at least 3 bs (64x256x256) for b4 + 2-8 accumulations steps ~ virtual batch was around 24 always</p>",
          "rawMarkdown": "For one card with mixed precision it was at least 3 bs (64x256x256) for b4 + 2-8 accumulations steps ~ virtual batch was around 24 always",
          "votes": 2
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 941686,
      "author_name": "Dmitry A. Grechka",
      "author_url": "",
      "post_date": "2020-07-23T10:57:18.223000",
      "content": "<p>Congratulations! 👍 </p>\n\n<p>Also glad to see a vary familiar view of the excellence graduate diploma and the main building of our University (at your avatar) in the top 20 listing 😊 </p>\n\n<p>Would be happy to team up in the future competitions if you want.</p>",
      "votes": 0,
      "replies": [
        {
          "id": 941868,
          "author_name": "YuryBolkonsky",
          "author_url": "",
          "post_date": "2020-07-23T13:12:37.247000",
          "content": "<p>Sure</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 940760,
      "author_name": "Mobassir",
      "author_url": "",
      "post_date": "2020-07-23T05:31:51.673000",
      "content": "<p>for us \"vovnet was the  winner \"achieved 0.92\" </p>\n\n<p>Training efficientnet-b0, b2, b4 and mixnet-xl on different tiles sizes 36x256x256 (level 1) =&gt; 48x256x256 (level 1) =&gt; 64x256x256 (level 1) without regularization and with high (dropout 0.4)</p>\n\n<p>did you train all those models with batch size 2?</p>",
      "votes": 0,
      "replies": [
        {
          "id": 940796,
          "author_name": "YuryBolkonsky",
          "author_url": "",
          "post_date": "2020-07-23T05:35:11.223000",
          "content": "<p>For one card with mixed precision it was at least 3 bs (64x256x256) for b4 + 2-8 accumulations steps ~ virtual batch was around 24 always</p>",
          "votes": 2,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "940721": "**Thank you very much to organizers, participants and my teammates** @ryunosukeishizaki @rinnqd for such competition.\n\nWe can name our solution as **\"zero public LB to hero private LB\"**.  In public we could even get a bronze and in private we are in top 20 teams. It's not a lucky submission because we have a lot of them and success points are real.\n\n\n1. **Removing noise** (marks, duplicates) based on [this Zac Dannelly](https://www.kaggle.com/c/prostate-cancer-grade-assessment/discussion/151323) and [this Leonie](https://www.kaggle.com/iamleonie/panda-eda-visualizations-suspicious-data), and also my own manual clean up\n\n2. Training efficientnet-b0, b2, b4 and mixnet-xl on **different tiles sizes** 36x256x256 (level 1) =&gt; 49x256x256 (level 1) =&gt; 64x256x256 (level 1) without regularization and with high (dropout 0.4)\n\n3. **Combining cleaning dataset training and raw data**\n\n4. Blending based on local **CV weights** w = w / np.sum(w)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1192776%2Fe5ab54359ade13256e9c62dbfdf5a80f%2F.PNG?generation=1595481733380627&amp;alt=media)\n\n5.**Trust your local CV** and train stable models!",
    "941686": "Congratulations! 👍 \n\nAlso glad to see a vary familiar view of the excellence graduate diploma and the main building of our University (at your avatar) in the top 20 listing 😊 \n\nWould be happy to team up in the future competitions if you want.\n",
    "940760": "for us \"vovnet was the  winner \"achieved 0.92\" \n\nTraining efficientnet-b0, b2, b4 and mixnet-xl on different tiles sizes 36x256x256 (level 1) =&gt; 48x256x256 (level 1) =&gt; 64x256x256 (level 1) without regularization and with high (dropout 0.4)\n\n\ndid you train all those models with batch size 2?\n"
  }
}