{
  "id": 606980,
  "title": "Question about potential timeout on the private leaderboard",
  "url": "/competitions/rsna-intracranial-aneurysm-detection/discussion/606980",
  "author_name": "oshiko0450",
  "post_date": "2025-09-11T09:58:21.852000",
  "votes": 2,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Hi everyone,<br>\nI have a quick question about the scoring runtime.<br>\nThe competition details state:<br>\n• The public leaderboard is calculated with approximately 32% of the test data.<br>\n• The final results will be based on the other 68%.<br>\n• The GPU Notebook runtime limit is 12 hours.<br>\nFor example, let's say a submission takes about 7-8 hours to run and get a public score.<br>\nGiven that the private test set (68%) is more than twice as large as the public test set (32%), does this mean there's a high risk that this code would time out during the final scoring for the private leaderboard?<br>\nThanks for any clarification!</p>",
  "messages": [
    {
      "id": 3287325,
      "postDate": "2025-09-11T11:33:06.450Z",
      "content": "<p>Public and private set are evaluated simultaneously. You don't need to worry about time-out. If your submission successfully, then your submission is fine in private. </p>",
      "rawMarkdown": "Public and private set are evaluated simultaneously. You don't need to worry about time-out. If your submission successfully, then your submission is fine in private. ",
      "votes": 1,
      "replies": [
        {
          "id": 3287328,
          "postDate": "2025-09-11T11:50:26.557Z",
          "content": "<p>Got it, thank you for clarifying! So that means our code makes predictions on the entire test set at once, and the public leaderboard score is just calculated from 32% of those results. That makes sense.</p>",
          "rawMarkdown": "Got it, thank you for clarifying! So that means our code makes predictions on the entire test set at once, and the public leaderboard score is just calculated from 32% of those results. That makes sense."
        }
      ]
    },
    {
      "id": 3287301,
      "postDate": "2025-09-11T09:58:21.853Z",
      "content": "<p>Hi everyone,<br>\nI have a quick question about the scoring runtime.<br>\nThe competition details state:<br>\n• The public leaderboard is calculated with approximately 32% of the test data.<br>\n• The final results will be based on the other 68%.<br>\n• The GPU Notebook runtime limit is 12 hours.<br>\nFor example, let's say a submission takes about 7-8 hours to run and get a public score.<br>\nGiven that the private test set (68%) is more than twice as large as the public test set (32%), does this mean there's a high risk that this code would time out during the final scoring for the private leaderboard?<br>\nThanks for any clarification!</p>",
      "rawMarkdown": "\nHi everyone,\n\nI have a quick question about the scoring runtime.\nThe competition details state:\n• The public leaderboard is calculated with approximately 32% of the test data.\n• The final results will be based on the other 68%.\n• The GPU Notebook runtime limit is 12 hours.\n\n\nFor example, let's say a submission takes about 7-8 hours to run and get a public score.\nGiven that the private test set (68%) is more than twice as large as the public test set (32%), does this mean there's a high risk that this code would time out during the final scoring for the private leaderboard?\nThanks for any clarification!",
      "votes": 2
    }
  ],
  "comments": [
    {
      "id": 3287325,
      "author_name": "Tom",
      "author_url": "",
      "post_date": "2025-09-11T11:33:06.450000",
      "content": "<p>Public and private set are evaluated simultaneously. You don't need to worry about time-out. If your submission successfully, then your submission is fine in private. </p>",
      "votes": 1,
      "replies": [
        {
          "id": 3287328,
          "author_name": "oshiko0450",
          "author_url": "",
          "post_date": "2025-09-11T11:50:26.557000",
          "content": "<p>Got it, thank you for clarifying! So that means our code makes predictions on the entire test set at once, and the public leaderboard score is just calculated from 32% of those results. That makes sense.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "3287325": "Public and private set are evaluated simultaneously. You don't need to worry about time-out. If your submission successfully, then your submission is fine in private. ",
    "3287301": "\nHi everyone,\n\nI have a quick question about the scoring runtime.\nThe competition details state:\n• The public leaderboard is calculated with approximately 32% of the test data.\n• The final results will be based on the other 68%.\n• The GPU Notebook runtime limit is 12 hours.\n\n\nFor example, let's say a submission takes about 7-8 hours to run and get a public score.\nGiven that the private test set (68%) is more than twice as large as the public test set (32%), does this mean there's a high risk that this code would time out during the final scoring for the private leaderboard?\nThanks for any clarification!"
  }
}