![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F25984450%2Fa87b9f887658508adb6b17956fe1ef62%2F23333ScreenShot_2026-04-01_120954_888.png?generation=1775016755708840&alt=media\")
We're receiving questions from teams who see a high Private Leaderboard score in old notebooks that were not selected for re-run.
\nPlease note: any Private LB scores before competition close (March 25, 2026) are not directly comparable to the current private leaderboard scores!
\nDuring the competition (before March 25, 2026), the Private Leaderboard comprised a quite distinct set of hidden targets that were actually a bit easier than the final leaderboard (after March 25, 2026).
\nSo high Private LB scores in prior notebooks should be ignored.
\nLater in 2026, we'll reopen the competition for scoring and you can re-score prior notebooks with the final Private Leaderboard.
", "messages": [ { "id": 3432726, "postDate": "2026-03-31T21:57:27.170Z", "content": "@rhijudas, I have never seen this kind of discrepancies in scoring in my many years participating in Kaggle competitions. What are you guys thinking that doing this will attract some of us to ever participate in a competition any one involved in this organizes? I sure won't and have lost my appetite for participating in the April 13th CASP17 dry run as earlier planned. Good luck.
", "rawMarkdown": "@rhijudas, I have never seen this kind of discrepancies in scoring in my many years participating in Kaggle competitions. What are you guys thinking that doing this will attract some of us to ever participate in a competition any one involved in this organizes? I sure won't and have lost my appetite for participating in the April 13th CASP17 dry run as earlier planned. Good luck.", "votes": 9, "replies": [ { "id": 3432816, "postDate": "2026-03-31T23:58:15.140Z", "content": "I posted my full thoughts on the rerun debacle here (https://www.kaggle.com/competitions/stanford-rna-3d-folding-2/discussion/686651). It looks like a couple of people already downvoted that post but I don't care. There you go.
", "rawMarkdown": "I posted my full thoughts on the rerun debacle here (https://www.kaggle.com/competitions/stanford-rna-3d-folding-2/discussion/686651). It looks like a couple of people already downvoted that post but I don't care. There you go.", "votes": 5, "replies": [ { "id": 3432831, "postDate": "2026-04-01T00:08:27.900Z", "content": "follow you
", "rawMarkdown": "follow you" } ] } ] }, { "id": 3432716, "postDate": "2026-03-31T21:43:47.937Z", "content": "@rhijudas Thanks for adding further clarification. But the public-to-private variance still seems unusually large.
\nFor example:\ndo-jo: public 1408 / 0.263 → private 13 / 0.466\nLizaSheina: public 1425 / 0.244 → private 42 / 0.436
\nEven accounting for the changed final dataset, these score shifts still seem quite large. Could you add further context on this?
", "rawMarkdown": "@rhijudas Thanks for adding further clarification. But the public-to-private variance still seems unusually large.\n\nFor example:\ndo-jo: public 1408 / 0.263 → private 13 / 0.466\nLizaSheina: public 1425 / 0.244 → private 42 / 0.436\n\nEven accounting for the changed final dataset, these score shifts still seem quite large. Could you add further context on this? ", "votes": 5 }, { "id": 3432815, "postDate": "2026-03-31T23:58:14.543Z", "content": "unbelievable competition
", "rawMarkdown": "unbelievable competition", "votes": 4 }, { "id": 3432744, "postDate": "2026-03-31T22:29:44.543Z", "content": "i am sorry mr from your clarification explanation,is possible to rerank final private leaderboard in the phase late submission,thank you
", "rawMarkdown": "i am sorry mr from your clarification explanation,is possible to rerank final private leaderboard in the phase late submission,thank you", "votes": 1 }, { "id": 3432978, "postDate": "2026-04-01T04:12:37.463Z", "content": "On the consistency between public and private leaderboards
\n
Thanks for creating this. This distribution is so weird, there are quite high number of candidates who weren't in medal range got moved to it with this new dataset. ![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F7985021%2Fc81e63b40aed8129e27efd18022e2ebb%2FPasted%20Graphic.png?generation=1775017438829549&alt=media\")
I think the bottom left rectangular-like plotted areas are due to some high-scoring notebooks being shared, which have some randomness in their output.
\nThe public leaderboard score for rank 100 is 0.417.
\nThe public leaderboard score for rank 1000 is 0.388.
\nThe rank difference is 900, but the score difference is 0.029.
The public leaderboard score for rank 1000 is 0.388.
\nThe public leaderboard score for rank 1100 is 0.347.
\nThe rank difference is 100, but the score difference is 0.041.
The former area is very dense, and even small score differences cause significant changes in the rankings.
", "rawMarkdown": "I think the bottom left rectangular-like plotted areas are due to some high-scoring notebooks being shared, which have some randomness in their output.\n\nThe public leaderboard score for rank 100 is 0.417. \nThe public leaderboard score for rank 1000 is 0.388. \nThe rank difference is 900, but the score difference is 0.029. \n\nThe public leaderboard score for rank 1000 is 0.388. \nThe public leaderboard score for rank 1100 is 0.347. \nThe rank difference is 100, but the score difference is 0.041. \n\nThe former area is very dense, and even small score differences cause significant changes in the rankings.", "votes": 3 } ] }, { "id": 3432712, "postDate": "2026-03-31T21:39:14.627Z", "content": "A key transparency issue is that it was not made sufficiently explicit that the public scores visible before the deadline might change after the rerun. In practice, the public scores did change after rerun, yet participants were not clearly warned in advance that they could ultimately be confronted with rerun-based results rather than the public scores they had previously seen.
", "rawMarkdown": "A key transparency issue is that it was not made sufficiently explicit that the public scores visible before the deadline might change after the rerun. In practice, the public scores did change after rerun, yet participants were not clearly warned in advance that they could ultimately be confronted with rerun-based results rather than the public scores they had previously seen.", "votes": 2, "replies": [ { "id": 3433060, "postDate": "2026-04-01T06:09:36.890Z", "content": "Is this include a violation in kaggle policy or not,and if this competition have a violation,is possible to resubmit and rerunk submission or not,thank you?
", "rawMarkdown": "Is this include a violation in kaggle policy or not,and if this competition have a violation,is possible to resubmit and rerunk submission or not,thank you?", "votes": 1 } ] }, { "id": 3432652, "postDate": "2026-03-31T20:30:18.953Z", "content": "So that low score is our final highest score？Our higher score comes from eazier test set, but somehow, host only updated the highest one in all notebooks for hard test set. That is mine understand, is it right?
", "rawMarkdown": "So that low score is our final highest score？Our higher score comes from eazier test set, but somehow, host only updated the highest one in all notebooks for hard test set. That is mine understand, is it right?", "votes": 2 }, { "id": 3432628, "postDate": "2026-03-31T20:13:02.843Z", "content": "We're receiving questions from teams who see a high Private Leaderboard score in old notebooks that were not selected for re-run.
\nPlease note: any Private LB scores before competition close (March 25, 2026) are not directly comparable to the current private leaderboard scores!
\nDuring the competition (before March 25, 2026), the Private Leaderboard comprised a quite distinct set of hidden targets that were actually a bit easier than the final leaderboard (after March 25, 2026).
\nSo high Private LB scores in prior notebooks should be ignored.
\nLater in 2026, we'll reopen the competition for scoring and you can re-score prior notebooks with the final Private Leaderboard.
", "rawMarkdown": "We're receiving questions from teams who see a high Private Leaderboard score in old notebooks that were not selected for re-run.\n\nPlease note: any Private LB scores before competition close (March 25, 2026) are *not* directly comparable to the current private leaderboard scores!\n\nDuring the competition (before March 25, 2026), the Private Leaderboard comprised a quite distinct set of hidden targets that were actually a bit easier than the final leaderboard (after March 25, 2026). \n\n**So high Private LB scores in prior notebooks should be ignored.** \n\nLater in 2026, we'll reopen the competition for scoring and you can re-score prior notebooks with the final Private Leaderboard.", "votes": -9 }, { "id": 3434460, "postDate": "2026-04-03T00:30:35.530Z", "content": "@rhijudas I submitted a notebook before the deadline, but it finished running after the deadline. It uses a different approach than the other notebooks and achieved a 0.59 on the private dataset. Was it evaluated on the easier dataset?
\nAlso, when I compare it to the Protenix + TBM notebook (which scored higher on the public dataset), it still significantly outperforms it on this easier dataset. Is there a property of this dataset that explains this behavior?\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F7588330%2Fff34579d4749cb3a616e90ce7b5bbcfb%2FNewRun.png?generation=1775176309054253&alt=media\")
Here is the score that i got from TBM + protenix ![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F7588330%2F77640371229e9b1e1056753901113e54%2FOldRun.png?generation=1775176388383459&alt=media\")
Focus this discussion friend?
", "rawMarkdown": "Focus this discussion friend?" }, { "id": 3433067, "postDate": "2026-04-01T06:28:03.247Z", "content": "Later in 2026, we'll reopen the competition for scoring and you can re-score prior notebooks with the final Private Leaderboard?Is this enough policy or not from violance in Transparency rule kaggle policy? thank you
", "rawMarkdown": "Later in 2026, we'll reopen the competition for scoring and you can re-score prior notebooks with the final Private Leaderboard?Is this enough policy or not from violance in Transparency rule kaggle policy? thank you" }, { "id": 3432990, "postDate": "2026-04-01T04:37:54.663Z", "content": "I am sorry mr,i need a explanation rule from this competition is in rule release final private leaderboard at the 4 April,thank you
", "rawMarkdown": "I am sorry mr,i need a explanation rule from this competition is in rule release final private leaderboard at the 4 April,thank you" }, { "id": 3432988, "postDate": "2026-04-01T04:36:06.823Z", "content": "I am sorry mr,from your explanation is this possible to fix private leaderboard rank after open late submission from your explanation and is if possible to fix,thank you?
", "rawMarkdown": "I am sorry mr,from your explanation is this possible to fix private leaderboard rank after open late submission from your explanation and is if possible to fix,thank you?" }, { "id": 3432902, "postDate": "2026-04-01T02:12:47.313Z", "content": "You mean the former private score should be the \"public score\", so you should let us rechoose the notebook instead of doing this.
", "rawMarkdown": "You mean the former private score should be the \"public score\", so you should let us rechoose the notebook instead of doing this.", "replies": [ { "id": 3432945, "postDate": "2026-04-01T03:18:38.297Z", "content": "Exactly, my highest pvt score is .585, they didn't even select the notebook with the highest public score, what a joke
\nI had expected the notebook with the highest public score, to be selected \nI have seen people who selected a notebook, that not run and some other notebook be selected, what a joke is this
", "rawMarkdown": "Exactly, my highest pvt score is .585, they didn't even select the notebook with the highest public score, what a joke\n\nI had expected the notebook with the highest public score, to be selected \nI have seen people who selected a notebook, that not run and some other notebook be selected, what a joke is this ", "replies": [ { "id": 3432950, "postDate": "2026-04-01T03:23:55.243Z", "content": "Mine was 0.613. Funny thing is, I was at 50th-ish before the competition re-run, and then I am here, 1000th-ish. Feels like a waste.
", "rawMarkdown": "Mine was 0.613. Funny thing is, I was at 50th-ish before the competition re-run, and then I am here, 1000th-ish. Feels like a waste." }, { "id": 3432954, "postDate": "2026-04-01T03:29:23.750Z", "content": "Oh that sucks man, I was in medal range, and now I fell by 600 ranks approx, genuinely felt like a betrayal, changing the dataset for the private return is a scammy move imho, if they did, should have ran all my notebooks against it and chosen the best public score one or something, not just taking the last 2 submissions, even if they did this, I would have felt it to be fair
", "rawMarkdown": "Oh that sucks man, I was in medal range, and now I fell by 600 ranks approx, genuinely felt like a betrayal, changing the dataset for the private return is a scammy move imho, if they did, should have ran all my notebooks against it and chosen the best public score one or something, not just taking the last 2 submissions, even if they did this, I would have felt it to be fair " }, { "id": 3432959, "postDate": "2026-04-01T03:35:56.193Z", "content": "I agree. Most of the high rankings have also been shaken down, which is neither fair nor scientific.
", "rawMarkdown": "I agree. Most of the high rankings have also been shaken down, which is neither fair nor scientific." }, { "id": 3432963, "postDate": "2026-04-01T03:43:51.767Z", "content": "We can't do anything now, just hope they come to their senses and fix the mess they have created.
", "rawMarkdown": "We can't do anything now, just hope they come to their senses and fix the mess they have created." }, { "id": 3433693, "postDate": "2026-04-02T00:23:28.463Z", "content": "@udaybhatia, I doubt it since they are still answering questions only selectively. We all can see what is happening, right:-)
", "rawMarkdown": "@udaybhatia, I doubt it since they are still answering questions only selectively. We all can see what is happening, right:-)" }, { "id": 3433701, "postDate": "2026-04-02T00:46:30.767Z", "content": "and they deleted a lot comments, no hope
", "rawMarkdown": "and they deleted a lot comments, no hope " } ] } ] }, { "id": 3432638, "postDate": "2026-03-31T20:22:35.890Z", "content": "Hi, I received a Private Leaderboard score of 0.527 on a notebook that was submitted just after the deadline (I was trying to get a submission done right before the deadline but unfortunately not). Does this mean that would of gotten the same score had I selected that one for scoring?
", "rawMarkdown": "Hi, I received a Private Leaderboard score of 0.527 on a notebook that was submitted just after the deadline (I was trying to get a submission done right before the deadline but unfortunately not). Does this mean that would of gotten the same score had I selected that one for scoring?", "replies": [ { "id": 3432646, "postDate": "2026-03-31T20:27:23.707Z", "content": "Thanks for the question. The answer is no: if the notebook was submitted before the deadline but did not complete, it was scored with the old Private leaderboard and was not eligible for re-scoring with the final Private leaderboard.
", "rawMarkdown": "Thanks for the question. The answer is no: if the notebook was submitted before the deadline but did not complete, it was scored with the old Private leaderboard and was not eligible for re-scoring with the final Private leaderboard." }, { "id": 3432647, "postDate": "2026-03-31T20:27:42.880Z", "content": "For reference, this is it![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F31669346%2F5fb53a64db8d07de54dac535f29df72d%2Fimage_2026-03-31_162734051.png?generation=1774988853894477&alt=media\")
It was still in the submission process at the deadline, so it was a late submission.
", "rawMarkdown": "It was still in the submission process at the deadline, so it was a late submission." } ] } ] }, { "id": 3432753, "postDate": "2026-03-31T22:43:54.397Z", "rawMarkdown": "", "isDeleted": true }, { "id": 3432651, "postDate": "2026-03-31T20:29:54.073Z", "rawMarkdown": "", "votes": 2, "isDeleted": true }, { "id": 3432637, "postDate": "2026-03-31T20:21:13.227Z", "rawMarkdown": "", "isDeleted": true, "replies": [ { "id": 3432640, "postDate": "2026-03-31T20:23:25.967Z", "rawMarkdown": "", "isDeleted": true }, { "id": 3432694, "postDate": "2026-03-31T21:11:56.493Z", "content": "is any can answer this question?
", "rawMarkdown": "is any can answer this question?", "replies": [ { "id": 3433083, "postDate": "2026-04-01T07:15:05.077Z", "content": "what was the question, that has been deleted lol
", "rawMarkdown": "what was the question, that has been deleted lol" } ] }, { "id": 3432695, "postDate": "2026-03-31T21:14:48.477Z", "content": "Is any host can explain this question,thank you?
", "rawMarkdown": "Is any host can explain this question,thank you?" }, { "id": 3432810, "postDate": "2026-03-31T23:50:20.307Z", "content": "@rhijudas why this？
", "rawMarkdown": "@rhijudas why this？" } ] }, { "id": 3432633, "postDate": "2026-03-31T20:17:46.730Z", "rawMarkdown": "", "votes": 2, "isDeleted": true, "replies": [ { "id": 3432642, "postDate": "2026-03-31T20:26:04.063Z", "rawMarkdown": "", "votes": 1, "isDeleted": true } ] } ], "comments": [ { "id": 3432726, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2026-03-31T21:57:27.170000", "content": "@rhijudas, I have never seen this kind of discrepancies in scoring in my many years participating in Kaggle competitions. What are you guys thinking that doing this will attract some of us to ever participate in a competition any one involved in this organizes? I sure won't and have lost my appetite for participating in the April 13th CASP17 dry run as earlier planned. Good luck.
", "votes": 9, "replies": [ { "id": 3432816, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2026-03-31T23:58:15.140000", "content": "I posted my full thoughts on the rerun debacle here (https://www.kaggle.com/competitions/stanford-rna-3d-folding-2/discussion/686651). It looks like a couple of people already downvoted that post but I don't care. There you go.
", "votes": 5, "replies": [ { "id": 3432831, "author_name": "grifth", "author_url": "", "post_date": "2026-04-01T00:08:27.900000", "content": "follow you
", "votes": 0, "replies": [] } ] } ] }, { "id": 3432716, "author_name": "", "author_url": "", "post_date": "2026-03-31T21:43:47.937000", "content": "@rhijudas Thanks for adding further clarification. But the public-to-private variance still seems unusually large.
\nFor example:\ndo-jo: public 1408 / 0.263 → private 13 / 0.466\nLizaSheina: public 1425 / 0.244 → private 42 / 0.436
\nEven accounting for the changed final dataset, these score shifts still seem quite large. Could you add further context on this?
", "votes": 5, "replies": [] }, { "id": 3432815, "author_name": "bianfu", "author_url": "", "post_date": "2026-03-31T23:58:14.543000", "content": "unbelievable competition
", "votes": 4, "replies": [] }, { "id": 3432744, "author_name": "Hellboy", "author_url": "", "post_date": "2026-03-31T22:29:44.543000", "content": "i am sorry mr from your clarification explanation,is possible to rerank final private leaderboard in the phase late submission,thank you
", "votes": 1, "replies": [] }, { "id": 3432978, "author_name": "Aleck", "author_url": "", "post_date": "2026-04-01T04:12:37.463000", "content": "On the consistency between public and private leaderboards
\n
Thanks for creating this. This distribution is so weird, there are quite high number of candidates who weren't in medal range got moved to it with this new dataset.
I think the bottom left rectangular-like plotted areas are due to some high-scoring notebooks being shared, which have some randomness in their output.
\nThe public leaderboard score for rank 100 is 0.417.
\nThe public leaderboard score for rank 1000 is 0.388.
\nThe rank difference is 900, but the score difference is 0.029.
The public leaderboard score for rank 1000 is 0.388.
\nThe public leaderboard score for rank 1100 is 0.347.
\nThe rank difference is 100, but the score difference is 0.041.
The former area is very dense, and even small score differences cause significant changes in the rankings.
", "votes": 3, "replies": [] } ] }, { "id": 3432712, "author_name": "ömer kiraz", "author_url": "", "post_date": "2026-03-31T21:39:14.627000", "content": "A key transparency issue is that it was not made sufficiently explicit that the public scores visible before the deadline might change after the rerun. In practice, the public scores did change after rerun, yet participants were not clearly warned in advance that they could ultimately be confronted with rerun-based results rather than the public scores they had previously seen.
", "votes": 2, "replies": [ { "id": 3433060, "author_name": "Hellboy", "author_url": "", "post_date": "2026-04-01T06:09:36.890000", "content": "Is this include a violation in kaggle policy or not,and if this competition have a violation,is possible to resubmit and rerunk submission or not,thank you?
", "votes": 1, "replies": [] } ] }, { "id": 3432652, "author_name": "Zejun_", "author_url": "", "post_date": "2026-03-31T20:30:18.953000", "content": "So that low score is our final highest score？Our higher score comes from eazier test set, but somehow, host only updated the highest one in all notebooks for hard test set. That is mine understand, is it right?
", "votes": 2, "replies": [] }, { "id": 3434460, "author_name": "soltani abdellatif", "author_url": "", "post_date": "2026-04-03T00:30:35.530000", "content": "@rhijudas I submitted a notebook before the deadline, but it finished running after the deadline. It uses a different approach than the other notebooks and achieved a 0.59 on the private dataset. Was it evaluated on the easier dataset?
\nAlso, when I compare it to the Protenix + TBM notebook (which scored higher on the public dataset), it still significantly outperforms it on this easier dataset. Is there a property of this dataset that explains this behavior?\n
Here is the score that i got from TBM + protenix
Focus this discussion friend?
", "votes": 0, "replies": [] }, { "id": 3433067, "author_name": "Hellboy", "author_url": "", "post_date": "2026-04-01T06:28:03.247000", "content": "Later in 2026, we'll reopen the competition for scoring and you can re-score prior notebooks with the final Private Leaderboard?Is this enough policy or not from violance in Transparency rule kaggle policy? thank you
", "votes": 0, "replies": [] }, { "id": 3432990, "author_name": "Hellboy", "author_url": "", "post_date": "2026-04-01T04:37:54.663000", "content": "I am sorry mr,i need a explanation rule from this competition is in rule release final private leaderboard at the 4 April,thank you
", "votes": 0, "replies": [] }, { "id": 3432988, "author_name": "Hellboy", "author_url": "", "post_date": "2026-04-01T04:36:06.823000", "content": "I am sorry mr,from your explanation is this possible to fix private leaderboard rank after open late submission from your explanation and is if possible to fix,thank you?
", "votes": 0, "replies": [] }, { "id": 3432902, "author_name": "sam_the_rice_cake", "author_url": "", "post_date": "2026-04-01T02:12:47.313000", "content": "You mean the former private score should be the \"public score\", so you should let us rechoose the notebook instead of doing this.
", "votes": 0, "replies": [ { "id": 3432945, "author_name": "Uday Bhatia", "author_url": "", "post_date": "2026-04-01T03:18:38.297000", "content": "Exactly, my highest pvt score is .585, they didn't even select the notebook with the highest public score, what a joke
\nI had expected the notebook with the highest public score, to be selected \nI have seen people who selected a notebook, that not run and some other notebook be selected, what a joke is this
", "votes": 0, "replies": [ { "id": 3432950, "author_name": "sam_the_rice_cake", "author_url": "", "post_date": "2026-04-01T03:23:55.243000", "content": "Mine was 0.613. Funny thing is, I was at 50th-ish before the competition re-run, and then I am here, 1000th-ish. Feels like a waste.
", "votes": 0, "replies": [] }, { "id": 3432954, "author_name": "Uday Bhatia", "author_url": "", "post_date": "2026-04-01T03:29:23.750000", "content": "Oh that sucks man, I was in medal range, and now I fell by 600 ranks approx, genuinely felt like a betrayal, changing the dataset for the private return is a scammy move imho, if they did, should have ran all my notebooks against it and chosen the best public score one or something, not just taking the last 2 submissions, even if they did this, I would have felt it to be fair
", "votes": 0, "replies": [] }, { "id": 3432959, "author_name": "sam_the_rice_cake", "author_url": "", "post_date": "2026-04-01T03:35:56.193000", "content": "I agree. Most of the high rankings have also been shaken down, which is neither fair nor scientific.
", "votes": 0, "replies": [] }, { "id": 3432963, "author_name": "Uday Bhatia", "author_url": "", "post_date": "2026-04-01T03:43:51.767000", "content": "We can't do anything now, just hope they come to their senses and fix the mess they have created.
", "votes": 0, "replies": [] }, { "id": 3433693, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2026-04-02T00:23:28.463000", "content": "@udaybhatia, I doubt it since they are still answering questions only selectively. We all can see what is happening, right:-)
", "votes": 0, "replies": [] }, { "id": 3433701, "author_name": "sam_the_rice_cake", "author_url": "", "post_date": "2026-04-02T00:46:30.767000", "content": "and they deleted a lot comments, no hope
", "votes": 0, "replies": [] } ] } ] }, { "id": 3432638, "author_name": "Andrew Lee", "author_url": "", "post_date": "2026-03-31T20:22:35.890000", "content": "Hi, I received a Private Leaderboard score of 0.527 on a notebook that was submitted just after the deadline (I was trying to get a submission done right before the deadline but unfortunately not). Does this mean that would of gotten the same score had I selected that one for scoring?
", "votes": 0, "replies": [ { "id": 3432646, "author_name": "Rhiju Das", "author_url": "", "post_date": "2026-03-31T20:27:23.707000", "content": "Thanks for the question. The answer is no: if the notebook was submitted before the deadline but did not complete, it was scored with the old Private leaderboard and was not eligible for re-scoring with the final Private leaderboard.
", "votes": 0, "replies": [] }, { "id": 3432647, "author_name": "Andrew Lee", "author_url": "", "post_date": "2026-03-31T20:27:42.880000", "content": "For reference, this is it
It was still in the submission process at the deadline, so it was a late submission.
", "votes": 0, "replies": [] } ] } ] }, { "id": 3432753, "author_name": "", "author_url": "", "post_date": "2026-03-31T22:43:54.397000", "content": "", "votes": 0, "replies": [] }, { "id": 3432651, "author_name": "", "author_url": "", "post_date": "2026-03-31T20:29:54.073000", "content": "", "votes": 2, "replies": [] }, { "id": 3432637, "author_name": "", "author_url": "", "post_date": "2026-03-31T20:21:13.227000", "content": "", "votes": 0, "replies": [ { "id": 3432640, "author_name": "", "author_url": "", "post_date": "2026-03-31T20:23:25.967000", "content": "", "votes": 0, "replies": [] }, { "id": 3432694, "author_name": "Hellboy", "author_url": "", "post_date": "2026-03-31T21:11:56.493000", "content": "is any can answer this question?
", "votes": 0, "replies": [ { "id": 3433083, "author_name": "sam_the_rice_cake", "author_url": "", "post_date": "2026-04-01T07:15:05.077000", "content": "what was the question, that has been deleted lol
", "votes": 0, "replies": [] } ] }, { "id": 3432695, "author_name": "Hellboy", "author_url": "", "post_date": "2026-03-31T21:14:48.477000", "content": "Is any host can explain this question,thank you?
", "votes": 0, "replies": [] }, { "id": 3432810, "author_name": "hongyu Guo", "author_url": "", "post_date": "2026-03-31T23:50:20.307000", "content": "@rhijudas why this？
", "votes": 0, "replies": [] } ] }, { "id": 3432633, "author_name": "", "author_url": "", "post_date": "2026-03-31T20:17:46.730000", "content": "", "votes": 2, "replies": [ { "id": 3432642, "author_name": "", "author_url": "", "post_date": "2026-03-31T20:26:04.063000", "content": "", "votes": 1, "replies": [] } ] } ], "raw_markdown_by_id": { "3432726": "@rhijudas, I have never seen this kind of discrepancies in scoring in my many years participating in Kaggle competitions. What are you guys thinking that doing this will attract some of us to ever participate in a competition any one involved in this organizes? I sure won't and have lost my appetite for participating in the April 13th CASP17 dry run as earlier planned. Good luck.", "3432716": "@rhijudas Thanks for adding further clarification. But the public-to-private variance still seems unusually large.\n\nFor example:\ndo-jo: public 1408 / 0.263 → private 13 / 0.466\nLizaSheina: public 1425 / 0.244 → private 42 / 0.436\n\nEven accounting for the changed final dataset, these score shifts still seem quite large. Could you add further context on this? ", "3432815": "unbelievable competition", "3432744": "i am sorry mr from your clarification explanation,is possible to rerank final private leaderboard in the phase late submission,thank you", "3432978": "On the consistency between public and private leaderboards\n\n", "3432712": "A key transparency issue is that it was not made sufficiently explicit that the public scores visible before the deadline might change after the rerun. In practice, the public scores did change after rerun, yet participants were not clearly warned in advance that they could ultimately be confronted with rerun-based results rather than the public scores they had previously seen.", "3432652": "So that low score is our final highest score？Our higher score comes from eazier test set, but somehow, host only updated the highest one in all notebooks for hard test set. That is mine understand, is it right?", "3432628": "We're receiving questions from teams who see a high Private Leaderboard score in old notebooks that were not selected for re-run.\n\nPlease note: any Private LB scores before competition close (March 25, 2026) are *not* directly comparable to the current private leaderboard scores!\n\nDuring the competition (before March 25, 2026), the Private Leaderboard comprised a quite distinct set of hidden targets that were actually a bit easier than the final leaderboard (after March 25, 2026). \n\n**So high Private LB scores in prior notebooks should be ignored.** \n\nLater in 2026, we'll reopen the competition for scoring and you can re-score prior notebooks with the final Private Leaderboard.", "3434460": "@rhijudas I submitted a notebook before the deadline, but it finished running after the deadline. It uses a different approach than the other notebooks and achieved a 0.59 on the private dataset. Was it evaluated on the easier dataset?\n\nAlso, when I compare it to the Protenix + TBM notebook (which scored higher on the public dataset), it still significantly outperforms it on this easier dataset. Is there a property of this dataset that explains this behavior?\n", "3433071": "Focus this discussion friend?", "3433067": "Later in 2026, we'll reopen the competition for scoring and you can re-score prior notebooks with the final Private Leaderboard?Is this enough policy or not from violance in Transparency rule kaggle policy? thank you", "3432990": "I am sorry mr,i need a explanation rule from this competition is in rule release final private leaderboard at the 4 April,thank you", "3432988": "I am sorry mr,from your explanation is this possible to fix private leaderboard rank after open late submission from your explanation and is if possible to fix,thank you?", "3432902": "You mean the former private score should be the \"public score\", so you should let us rechoose the notebook instead of doing this.", "3432638": "Hi, I received a Private Leaderboard score of 0.527 on a notebook that was submitted just after the deadline (I was trying to get a submission done right before the deadline but unfortunately not). Does this mean that would of gotten the same score had I selected that one for scoring?", "3432753": "", "3432651": "", "3432637": "", "3432633": "" } }