{
  "id": 347046,
  "title": "Query on submission",
  "url": "/competitions/mayo-clinic-strip-ai/discussion/347046",
  "author_name": "VISHWANATHAN R",
  "post_date": "2022-08-22T15:51:20.185000",
  "votes": 2,
  "comment_count": 5,
  "views": 0,
  "content": "<p>Hi,<br>\nFor the public leaderboard submission, is our notebook run on <strong>all hidden test images</strong>, but <strong>evaluated on only a subset</strong> of the test images<br>\nOR<br>\nis our notebook <strong>run and evaluated on only a subset</strong> of the test images?</p>",
  "messages": [
    {
      "id": 1909784,
      "postDate": "2022-08-22T23:34:22.060Z",
      "content": "<p>:) Thanks for carrying the torch. I'm after the same confirmation.</p>\n<p>Although, I have submitted a notebook that just loads images, rescales them, deletes the image, and enter a 0.5 prediction for inference to submission.csv. This process took over an hour for images allowed up to 2Gb in size. I guess this is not possible if only 7% (or about 20 images) were being run for inference. I guess the whole data is being inferred, but LB score uses 7% of that inference.</p>\n<p>But we need confirmation.</p>",
      "rawMarkdown": ":) Thanks for carrying the torch. I'm after the same confirmation.\n\nAlthough, I have submitted a notebook that just loads images, rescales them, deletes the image, and enter a 0.5 prediction for inference to submission.csv. This process took over an hour for images allowed up to 2Gb in size. I guess this is not possible if only 7% (or about 20 images) were being run for inference. I guess the whole data is being inferred, but LB score uses 7% of that inference.\n\nBut we need confirmation.",
      "votes": 2
    },
    {
      "id": 1909422,
      "postDate": "2022-08-22T15:51:20.187Z",
      "content": "<p>Hi,<br>\nFor the public leaderboard submission, is our notebook run on <strong>all hidden test images</strong>, but <strong>evaluated on only a subset</strong> of the test images<br>\nOR<br>\nis our notebook <strong>run and evaluated on only a subset</strong> of the test images?</p>",
      "rawMarkdown": "Hi,\nFor the public leaderboard submission, is our notebook run on **all hidden test images**, but **evaluated on only a subset** of the test images\nOR\nis our notebook **run and evaluated on only a subset** of the test images?",
      "votes": 2
    },
    {
      "id": 1912322,
      "postDate": "2022-08-24T16:37:57.607Z",
      "content": "<p>This leaderboard is calculated with approximately 7% of the test data. The final results will be based on the other 93%, so the final standings may be different.</p>\n<p>part of test data hidden.</p>",
      "rawMarkdown": "This leaderboard is calculated with approximately 7% of the test data. The final results will be based on the other 93%, so the final standings may be different.\n\npart of test data hidden.",
      "replies": [
        {
          "id": 1912332,
          "postDate": "2022-08-24T16:42:50.350Z",
          "content": "<p><a href=\"https://www.kaggle.com/zhehaoliang\" target=\"_blank\">@zhehaoliang</a> thanks for responding, but is it possible to confirm if only 7% data is used for inference, and not just LB score at this point? The confusion is in terms of notebook run times. If 7% data is all that our submission sees, then it <strong>must</strong> run within 40 minutes now to be able to run rest of the 93% data within 9 hour notebook limit. That's what I find difficult to comprehend, I don't think it is simple enough to have the current submission run successfully within 40 mins. </p>\n<p>That's the confirmation I am looking for. That would mean I have to scale back on sophisticated processing and make it faster at the cost of accuracy, which is very bad for this dataset :)</p>",
          "rawMarkdown": "@zhehaoliang thanks for responding, but is it possible to confirm if only 7% data is used for inference, and not just LB score at this point? The confusion is in terms of notebook run times. If 7% data is all that our submission sees, then it **must** run within 40 minutes now to be able to run rest of the 93% data within 9 hour notebook limit. That's what I find difficult to comprehend, I don't think it is simple enough to have the current submission run successfully within 40 mins. \n\nThat's the confirmation I am looking for. That would mean I have to scale back on sophisticated processing and make it faster at the cost of accuracy, which is very bad for this dataset :)",
          "votes": 1
        },
        {
          "id": 1912354,
          "postDate": "2022-08-24T16:59:40.993Z",
          "content": "<p>Hi. I am afraid I dont know.</p>",
          "rawMarkdown": "Hi. I am afraid I dont know.\n",
          "votes": 2
        },
        {
          "id": 1912364,
          "postDate": "2022-08-24T17:08:21.477Z",
          "content": "<p>Hope we find out in time  :)</p>",
          "rawMarkdown": "Hope we find out in time  :)",
          "votes": 1
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 1909784,
      "author_name": "tdiceman",
      "author_url": "",
      "post_date": "2022-08-22T23:34:22.060000",
      "content": "<p>:) Thanks for carrying the torch. I'm after the same confirmation.</p>\n<p>Although, I have submitted a notebook that just loads images, rescales them, deletes the image, and enter a 0.5 prediction for inference to submission.csv. This process took over an hour for images allowed up to 2Gb in size. I guess this is not possible if only 7% (or about 20 images) were being run for inference. I guess the whole data is being inferred, but LB score uses 7% of that inference.</p>\n<p>But we need confirmation.</p>",
      "votes": 2,
      "replies": []
    },
    {
      "id": 1912322,
      "author_name": "zhehao liang",
      "author_url": "",
      "post_date": "2022-08-24T16:37:57.607000",
      "content": "<p>This leaderboard is calculated with approximately 7% of the test data. The final results will be based on the other 93%, so the final standings may be different.</p>\n<p>part of test data hidden.</p>",
      "votes": 0,
      "replies": [
        {
          "id": 1912332,
          "author_name": "tdiceman",
          "author_url": "",
          "post_date": "2022-08-24T16:42:50.350000",
          "content": "<p><a href=\"https://www.kaggle.com/zhehaoliang\" target=\"_blank\">@zhehaoliang</a> thanks for responding, but is it possible to confirm if only 7% data is used for inference, and not just LB score at this point? The confusion is in terms of notebook run times. If 7% data is all that our submission sees, then it <strong>must</strong> run within 40 minutes now to be able to run rest of the 93% data within 9 hour notebook limit. That's what I find difficult to comprehend, I don't think it is simple enough to have the current submission run successfully within 40 mins. </p>\n<p>That's the confirmation I am looking for. That would mean I have to scale back on sophisticated processing and make it faster at the cost of accuracy, which is very bad for this dataset :)</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1912354,
          "author_name": "zhehao liang",
          "author_url": "",
          "post_date": "2022-08-24T16:59:40.993000",
          "content": "<p>Hi. I am afraid I dont know.</p>",
          "votes": 2,
          "replies": []
        },
        {
          "id": 1912364,
          "author_name": "tdiceman",
          "author_url": "",
          "post_date": "2022-08-24T17:08:21.477000",
          "content": "<p>Hope we find out in time  :)</p>",
          "votes": 1,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "1909784": ":) Thanks for carrying the torch. I'm after the same confirmation.\n\nAlthough, I have submitted a notebook that just loads images, rescales them, deletes the image, and enter a 0.5 prediction for inference to submission.csv. This process took over an hour for images allowed up to 2Gb in size. I guess this is not possible if only 7% (or about 20 images) were being run for inference. I guess the whole data is being inferred, but LB score uses 7% of that inference.\n\nBut we need confirmation.",
    "1909422": "Hi,\nFor the public leaderboard submission, is our notebook run on **all hidden test images**, but **evaluated on only a subset** of the test images\nOR\nis our notebook **run and evaluated on only a subset** of the test images?",
    "1912322": "This leaderboard is calculated with approximately 7% of the test data. The final results will be based on the other 93%, so the final standings may be different.\n\npart of test data hidden."
  }
}