{
  "id": 336420,
  "title": "How not to waste your only submission a day",
  "url": "/competitions/mayo-clinic-strip-ai/discussion/336420",
  "author_name": "Nikita Kuzmenkov",
  "post_date": "2022-07-11T04:21:35.731000",
  "votes": 28,
  "comment_count": 4,
  "views": 0,
  "content": "<p>This is probably one of the most painful things about this competition right now</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5933676%2F82c49260f29f41a7e39952e1162c5634%2FScreenshot%202022-07-11%20at%2006.58.43.png?generation=1657511962000920&amp;alt=media\" alt=\"\"></p>\n<p>If you haven't yet seen it, according to this competition rules you have only 1 submission a day. If you don't want to waste it so silly as I did, here's a couple ideas:</p>\n<ul>\n<li><strong>Wrap your prediction function/code block in try/except loop.</strong> Return 0.5/0.5 in the except - this is the best you can do. If you're concerned about whether you hit the exception all the time, add a counter and reraise exception if you hit it, say, 10 times (out of 280 total):</li>\n</ul>\n<pre><code>exception counter = 0\n\ntry:\n    predict = model(image)\nexcept Exception:\n    predict = 0.5\n    exception counter += 1\n\n    if exception_counter &gt;= 10:\n        raise Exception\n</code></pre>\n<ul>\n<li><strong>Don't forget about the edge cases.</strong> My ex-boss used to say: <em>\"Where there's big data, there's garbage\"</em>. Make sure your code doesn't crash in case it gets a very big image as well as a very small image. Try feed your model with 50,000x50,000 <code>numpy</code> zeros as well as 256x256 zeros. Think about other edge cases.</li>\n<li><strong>Run inference on the train data or other data before making submission.</strong> Sure, that would waste at least 3 to 4 hours of your GPU quota, but it can save you a lot of time in case nothing else worked. There's 754 images in the train set and 396 images in the supplementary set, and this data is very diverse. If your model is able to run on the either set without getting an OOM, or taking more than 9 hours, or crashing on an edge case, there's a good chance it can successfully pass the submission.</li>\n</ul>\n<p>Hope you'll find it helpful. Feel free to come up with your ideas on avoiding submission errors.</p>\n<p>Good luck!</p>",
  "messages": [
    {
      "id": 1851168,
      "postDate": "2022-07-11T04:21:35.730Z",
      "content": "<p>This is probably one of the most painful things about this competition right now</p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5933676%2F82c49260f29f41a7e39952e1162c5634%2FScreenshot%202022-07-11%20at%2006.58.43.png?generation=1657511962000920&amp;alt=media\" alt=\"\"></p>\n<p>If you haven't yet seen it, according to this competition rules you have only 1 submission a day. If you don't want to waste it so silly as I did, here's a couple ideas:</p>\n<ul>\n<li><strong>Wrap your prediction function/code block in try/except loop.</strong> Return 0.5/0.5 in the except - this is the best you can do. If you're concerned about whether you hit the exception all the time, add a counter and reraise exception if you hit it, say, 10 times (out of 280 total):</li>\n</ul>\n<pre><code>exception counter = 0\n\ntry:\n    predict = model(image)\nexcept Exception:\n    predict = 0.5\n    exception counter += 1\n\n    if exception_counter &gt;= 10:\n        raise Exception\n</code></pre>\n<ul>\n<li><strong>Don't forget about the edge cases.</strong> My ex-boss used to say: <em>\"Where there's big data, there's garbage\"</em>. Make sure your code doesn't crash in case it gets a very big image as well as a very small image. Try feed your model with 50,000x50,000 <code>numpy</code> zeros as well as 256x256 zeros. Think about other edge cases.</li>\n<li><strong>Run inference on the train data or other data before making submission.</strong> Sure, that would waste at least 3 to 4 hours of your GPU quota, but it can save you a lot of time in case nothing else worked. There's 754 images in the train set and 396 images in the supplementary set, and this data is very diverse. If your model is able to run on the either set without getting an OOM, or taking more than 9 hours, or crashing on an edge case, there's a good chance it can successfully pass the submission.</li>\n</ul>\n<p>Hope you'll find it helpful. Feel free to come up with your ideas on avoiding submission errors.</p>\n<p>Good luck!</p>",
      "rawMarkdown": "This is probably one of the most painful things about this competition right now\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5933676%2F82c49260f29f41a7e39952e1162c5634%2FScreenshot%202022-07-11%20at%2006.58.43.png?generation=1657511962000920&alt=media)\n\nIf you haven't yet seen it, according to this competition rules you have only 1 submission a day. If you don't want to waste it so silly as I did, here's a couple ideas:\n\n* **Wrap your prediction function/code block in try/except loop.** Return 0.5/0.5 in the except - this is the best you can do. If you're concerned about whether you hit the exception all the time, add a counter and reraise exception if you hit it, say, 10 times (out of 280 total):\n\n```python\nexception counter = 0\n\ntry:\n    predict = model(image)\nexcept Exception:\n    predict = 0.5\n    exception counter += 1\n\n    if exception_counter >= 10:\n        raise Exception\n```\n\n* **Don't forget about the edge cases.** My ex-boss used to say: *\"Where there's big data, there's garbage\"*. Make sure your code doesn't crash in case it gets a very big image as well as a very small image. Try feed your model with 50,000x50,000 `numpy` zeros as well as 256x256 zeros. Think about other edge cases.\n* **Run inference on the train data or other data before making submission.** Sure, that would waste at least 3 to 4 hours of your GPU quota, but it can save you a lot of time in case nothing else worked. There's 754 images in the train set and 396 images in the supplementary set, and this data is very diverse. If your model is able to run on the either set without getting an OOM, or taking more than 9 hours, or crashing on an edge case, there's a good chance it can successfully pass the submission.\n\nHope you'll find it helpful. Feel free to come up with your ideas on avoiding submission errors.\n\nGood luck!",
      "votes": 27
    },
    {
      "id": 1851421,
      "postDate": "2022-07-11T08:26:48.530Z",
      "content": "<p>This is likely to be an issue towards the end of the competition where each submission directly impacts the final leaderboard rank</p>",
      "rawMarkdown": "This is likely to be an issue towards the end of the competition where each submission directly impacts the final leaderboard rank",
      "votes": 1
    },
    {
      "id": 1852086,
      "postDate": "2022-07-11T18:42:54Z",
      "content": "<p>This is great advice! I hadn’t thought about some of this. Thanks!!</p>",
      "rawMarkdown": "This is great advice! I hadn’t thought about some of this. Thanks!!",
      "votes": 2
    },
    {
      "id": 1893110,
      "postDate": "2022-08-10T14:35:49.867Z",
      "content": "<p>Good one <a href=\"https://www.kaggle.com/nickuzmenkov\" target=\"_blank\">@nickuzmenkov</a> </p>",
      "rawMarkdown": "Good one @nickuzmenkov "
    },
    {
      "id": 1853539,
      "postDate": "2022-07-13T00:13:51.417Z",
      "content": "<p>I should have done this..<br>\nI am wasting day after day of submissions for about a week..</p>",
      "rawMarkdown": "I should have done this..\nI am wasting day after day of submissions for about a week..\n"
    }
  ],
  "comments": [
    {
      "id": 1851421,
      "author_name": "Ravi Ramakrishnan",
      "author_url": "",
      "post_date": "2022-07-11T08:26:48.530000",
      "content": "<p>This is likely to be an issue towards the end of the competition where each submission directly impacts the final leaderboard rank</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1852086,
      "author_name": "Darien Schettler",
      "author_url": "",
      "post_date": "2022-07-11T18:42:54",
      "content": "<p>This is great advice! I hadn’t thought about some of this. Thanks!!</p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 1893110,
      "author_name": "Pankaj Kumar",
      "author_url": "",
      "post_date": "2022-08-10T14:35:49.867000",
      "content": "<p>Good one <a href=\"https://www.kaggle.com/nickuzmenkov\" target=\"_blank\">@nickuzmenkov</a> </p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1853539,
      "author_name": "The Devastator",
      "author_url": "",
      "post_date": "2022-07-13T00:13:51.417000",
      "content": "<p>I should have done this..<br>\nI am wasting day after day of submissions for about a week..</p>",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1851168": "This is probably one of the most painful things about this competition right now\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5933676%2F82c49260f29f41a7e39952e1162c5634%2FScreenshot%202022-07-11%20at%2006.58.43.png?generation=1657511962000920&alt=media)\n\nIf you haven't yet seen it, according to this competition rules you have only 1 submission a day. If you don't want to waste it so silly as I did, here's a couple ideas:\n\n* **Wrap your prediction function/code block in try/except loop.** Return 0.5/0.5 in the except - this is the best you can do. If you're concerned about whether you hit the exception all the time, add a counter and reraise exception if you hit it, say, 10 times (out of 280 total):\n\n```python\nexception counter = 0\n\ntry:\n    predict = model(image)\nexcept Exception:\n    predict = 0.5\n    exception counter += 1\n\n    if exception_counter >= 10:\n        raise Exception\n```\n\n* **Don't forget about the edge cases.** My ex-boss used to say: *\"Where there's big data, there's garbage\"*. Make sure your code doesn't crash in case it gets a very big image as well as a very small image. Try feed your model with 50,000x50,000 `numpy` zeros as well as 256x256 zeros. Think about other edge cases.\n* **Run inference on the train data or other data before making submission.** Sure, that would waste at least 3 to 4 hours of your GPU quota, but it can save you a lot of time in case nothing else worked. There's 754 images in the train set and 396 images in the supplementary set, and this data is very diverse. If your model is able to run on the either set without getting an OOM, or taking more than 9 hours, or crashing on an edge case, there's a good chance it can successfully pass the submission.\n\nHope you'll find it helpful. Feel free to come up with your ideas on avoiding submission errors.\n\nGood luck!",
    "1851421": "This is likely to be an issue towards the end of the competition where each submission directly impacts the final leaderboard rank",
    "1852086": "This is great advice! I hadn’t thought about some of this. Thanks!!",
    "1893110": "Good one @nickuzmenkov ",
    "1853539": "I should have done this..\nI am wasting day after day of submissions for about a week..\n"
  }
}