{
  "topic": {
    "id": 563314,
    "title": "Requests for the Forecasting Phase",
    "authorName": "Camaro",
    "commentCount": 13,
    "votes": 18,
    "postDate": "2025-02-16T12:57:32.923000"
  },
  "comments": [
    {
      "id": 3130527,
      "authorName": "Sohier Dane",
      "votes": 5,
      "postDate": "2025-02-21T18:23:57.590000",
      "content": "<p>We can provide everyone with a consistent inference order for the private set (it's not consistent for the public set), but that's the only feasible proposal. 1-3 need to be managed by your own error handling. As for #4, we have to allow some margin for error based on the (unknowable) viability of the git issues that will actually be available to us in the future.</p>"
    },
    {
      "id": 3131191,
      "authorName": "Camaro",
      "votes": 1,
      "postDate": "2025-02-22T14:57:48.533000",
      "content": "<p>OK, thanks for your answer.<br>\nI'm trying to implement #1-3, but I'm struggling especially with #3.<br>\nIf you could show a demo notebook, I would really appreciate it!</p>"
    },
    {
      "id": 3126148,
      "authorName": "ISAKA Tsuyoshi",
      "votes": 1,
      "postDate": "2025-02-17T03:58:52.420000",
      "content": "<p>Hi Camaro,</p>\n<p>Thank you for your suggestions.</p>\n<p>Regarding points 1, 2, and 3 (skipping instances instead of immediate disqualification), while I understand the concern, implementing such changes at this stage might be technically challenging given the limited time remaining. There is also a risk of introducing new environment-related bugs.</p>\n<p>For point 4, I am also interested in getting clarification on the finalized number of instances.</p>\n<p>As for point 5, my understanding was that the inference order is already fixed during private evaluation(not public evaluation), but I agree that ensuring all submissions follow the same order would be the fairest approach.</p>"
    },
    {
      "id": 3126193,
      "authorName": "Tong Hui Kang",
      "votes": 2,
      "postDate": "2025-02-17T05:32:13.997000",
      "content": "<p>Would like to know the finalized number of instances. I am okay with something like - you have <code>6n + 60</code> minutes to solve <code>n</code> problems.</p>\n<p>Was the inference order for private evaluation fixed for AIMO1?</p>"
    },
    {
      "id": 3126295,
      "authorName": "ISAKA Tsuyoshi",
      "votes": 0,
      "postDate": "2025-02-17T08:02:37.143000",
      "content": "<p>Hi Tong Hui Kang,</p>\n<p>As far as I know, the organizers have not explicitly stated whether the inference order for AIMO1 was fixed during private evaluation. <br>\nIt would be great if the organizers could provide clarity on this topic, as well as confirm the finalized number of instances.</p>\n<p><strong>P.S.</strong> I encountered an error in one of my final submissions for AIMO1 (<a href=\"https://www.kaggle.com/competitions/ai-mathematical-olympiad-prize/discussion/516936\" target=\"_blank\">ref</a>).</p>"
    },
    {
      "id": 3128259,
      "authorName": "CPMP",
      "votes": 1,
      "postDate": "2025-02-19T12:35:45.223000",
      "content": "<p>Why not implementing this yourself?</p>\n<ul>\n<li>Instead of immediate disqualification for exceeding the total processing time limit (24 hours), skip the remaining instances.</li>\n</ul>"
    },
    {
      "id": 3128267,
      "authorName": "Camaro",
      "votes": 2,
      "postDate": "2025-02-19T12:43:52.777000",
      "content": "<p>Any way to distinguish between a public test and a future re-run?</p>"
    },
    {
      "id": 3128317,
      "authorName": "Pavel Orlov",
      "votes": 4,
      "postDate": "2025-02-19T13:17:20.373000",
      "content": "<p>1) today &gt; Final Submission Deadline<br>\n2) <a href=\"https://www.kaggle.com/code/sohier/konwinski-prize-demo-submission\" target=\"_blank\">instance_count</a>  &gt; 71</p>"
    },
    {
      "id": 3128387,
      "authorName": "CPMP",
      "votes": 0,
      "postDate": "2025-02-19T14:09:12.307000",
      "content": "<p>I'd put instances &gt; 100 to be safe, but yes, that's the way to do it iMHO.</p>"
    },
    {
      "id": 3128423,
      "authorName": "Camaro",
      "votes": 2,
      "postDate": "2025-02-19T14:33:25.643000",
      "content": "<p>Thanks, I'll give it a try!<br>\nMaybe I'm just too worried about disqualification after a future re-run since my submission keeps failing randomly.</p>"
    },
    {
      "id": 3128461,
      "authorName": "Pavel Orlov",
      "votes": 1,
      "postDate": "2025-02-19T15:25:22.473000",
      "content": "<p>And what is the error - \"Notebook Inference Server Never Started\"?</p>"
    },
    {
      "id": 3130697,
      "authorName": "Jeronimo",
      "votes": 0,
      "postDate": "2025-02-21T23:21:38.960000",
      "content": "<p>How did that go ? Still worried about possible disqualification ?</p>"
    },
    {
      "id": 3134828,
      "authorName": "Sohier Dane",
      "votes": 3,
      "postDate": "2025-02-26T18:08:59.670000",
      "content": "<p><a href=\"https://www.kaggle.com/sapr3s\" target=\"_blank\">@sapr3s</a> I would strongly recommend only considering the date. We remain committed to the target of 150-200 issues in the final test set, but life happens. It would be a real shame to see otherwise valid submissions fail if, for example, someone on the issue sourcing team gets sick and we aren't able to hit the target.</p>"
    }
  ],
  "index": {
    "id": "563314",
    "title": "Requests for the Forecasting Phase",
    "authorName": "",
    "commentCount": "13",
    "votes": "18",
    "postDate": "2025-02-16 12:57:32.923000"
  },
  "competition": "konwinski-prize"
}