![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F954117%2F02809c0bd4a64820fbece1d6f6a0f6fc%2FScreenshot%202025-07-22%20at%2014.33.10.png?generation=1753162499176641&alt=media\")
Do you suddenly get much worse scores? No worries, it's not your bug but the test data has changed. The training data has changed too.
\nDataset Regeneration Notice: https://www.kaggle.com/competitions/ariel-data-challenge-2025/discussion/590210
\nMy Public LB score changed 0.303 → 0.200 for exactly same code.
The Local CV has changed, too: 0.2880 → 0.1528.
The leaderboard scores stay the same, . For those higher in the leaderboard, do you get similar change of ~0.1, too?
\n", "messages": [ { "id": 3251117, "postDate": "2025-07-20T01:13:34.560Z", "content": "Do you suddenly get much worse scores? No worries, it's not your bug but the test data has changed. The training data has changed too.
\nDataset Regeneration Notice: https://www.kaggle.com/competitions/ariel-data-challenge-2025/discussion/590210
\nMy Public LB score changed 0.303 → 0.200 for exactly same code.
The Local CV has changed, too: 0.2880 → 0.1528.
The leaderboard scores stay the same, . For those higher in the leaderboard, do you get similar change of ~0.1, too?
\n", "rawMarkdown": "Do you suddenly get much worse scores? No worries, it's not your bug but the test data has changed. The training data has changed too.\n\nDataset Regeneration Notice: https://www.kaggle.com/competitions/ariel-data-challenge-2025/discussion/590210\n\nMy Public LB score changed `0.303 → 0.200` for exactly same code.\n\nThe Local CV has changed, too: `0.2880 → 0.1528`.\n\nThe leaderboard scores stay the same, ~~that is, only new submissions are calculated with the new data~~. For those higher in the leaderboard, do you get similar change of ~0.1, too?\n\n~~If so, I think all the scores should be recalculated, or old scores deleted.~~\n\n", "votes": 10 }, { "id": 3252142, "postDate": "2025-07-22T05:37:49.573Z", "content": "I can see both old and new scores in the Submissions tab. New ones have messange starting from \"Notebook rerun.\" I think the Leaderboard score is the highest among both new and old scores, now.
\n
Lucky you. I can't see any rerun scores.
", "rawMarkdown": "Lucky you. I can't see any rerun scores." }, { "id": 3252342, "postDate": "2025-07-22T13:52:07.357Z", "content": "Staff still hasn't fixed this.
", "rawMarkdown": "Staff still hasn't fixed this." }, { "id": 3252346, "postDate": "2025-07-22T14:00:43.987Z", "content": "check code i have like you but not same my code, why i say that because old code for me on test in data take different numbers on lw_…. and sigama…. but on rerun take same number on all targets and sigma
\nso i think that rerun it's not really but only for show not for new evaluation not really, so i suggest delete all evaluation and everyone submit again with byself.
\ni want you check your pred test if you code don't use already same number to all targets and sigma !!!
Probable cause is that they don't show failed reruns, see https://www.kaggle.com/competitions/ariel-data-challenge-2025/discussion/590339#3252205
", "rawMarkdown": "Probable cause is that they don't show failed reruns, see https://www.kaggle.com/competitions/ariel-data-challenge-2025/discussion/590339#3252205" } ] }, { "id": 3252424, "postDate": "2025-07-22T17:41:59.360Z", "content": "I also see old and new scores (automatic reruns of all old submissions) in my submissions tab, but the leaderboard reflects my highest score, which is an old score (not a rerun). I assume at some point they need to delete all the old scores and then the leaderboard will change.
", "rawMarkdown": "I also see old and new scores (automatic reruns of all old submissions) in my submissions tab, but the leaderboard reflects my highest score, which is an old score (not a rerun). I assume at some point they need to delete all the old scores and then the leaderboard will change.", "votes": 2 }, { "id": 3252692, "postDate": "2025-07-23T07:59:28.300Z", "content": "The score recalculation is still in progress and in confusion. The same rerun become error now.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F954117%2F5bdf546f735947ab1f3b08f4d7f46c8e%2FScreenshot%202025-07-23%20at%2016.57.03.png?generation=1753257457401214&alt=media\")
Please note that the updated scores for older submissions are now available.
", "rawMarkdown": "Please note that the updated scores for older submissions are now available.", "votes": 1, "replies": [ { "id": 3251834, "postDate": "2025-07-21T16:33:55.340Z", "content": "great thanks!
\nWill the old scores be set as invalid?
On my submission page I still see the olds scores for my old submissions. And my LB score is an old score of an old submission. what you claim doesn't see to be correct at this time.
", "rawMarkdown": "On my submission page I still see the olds scores for my old submissions. And my LB score is an old score of an old submission. what you claim doesn't see to be correct at this time.", "votes": 1, "replies": [ { "id": 3251843, "postDate": "2025-07-21T16:49:02.633Z", "content": "I think it's because the old scores are still valid. Although they have been recalculate
", "rawMarkdown": "I think it's because the old scores are still valid. Although they have been recalculate" }, { "id": 3251953, "postDate": "2025-07-21T19:21:39.593Z", "content": "I find it extremely unlikely that scores don't change with the new hidden test set.
", "rawMarkdown": "I find it extremely unlikely that scores don't change with the new hidden test set.", "votes": 1 }, { "id": 3252205, "postDate": "2025-07-22T07:49:38.020Z", "content": "Same. Probably the old code just failed with the new data.
", "rawMarkdown": "Same. Probably the old code just failed with the new data.", "votes": 1 } ] } ] }, { "id": 3251591, "postDate": "2025-07-21T07:08:18.023Z", "content": "my current lb score is 0.398
\nwhile an almost identical code with fixes for extracting ingress and egress : 0.365
", "rawMarkdown": "my current lb score is 0.398\n\nwhile an almost identical code with fixes for extracting ingress and egress : 0.365\n", "votes": 1, "replies": [ { "id": 3251596, "postDate": "2025-07-21T07:14:15.877Z", "content": "mine is 0.433 → 0.384
", "rawMarkdown": "mine is 0.433 → 0.384", "votes": 2 } ] }, { "id": 3251444, "postDate": "2025-07-20T19:17:04.803Z", "content": "My Public LB score changed 0.349 → 0.363 for exactly same code.
\nThe Local CV has changed, too: 0.300 → 0.236
\n🤷🤷
Thanks! New data are not necessarily difficult, but larger gap 🤔?
", "rawMarkdown": "Thanks! New data are not necessarily difficult, but larger gap 🤔?" } ] }, { "id": 3251279, "postDate": "2025-07-20T10:43:06.473Z", "content": "In addition, my observation is similar, and my cv and lb both dropped by about 0.07 to 0.05.
", "rawMarkdown": "In addition, my observation is similar, and my cv and lb both dropped by about 0.07 to 0.05.", "votes": 1 }, { "id": 3251199, "postDate": "2025-07-20T07:16:48.097Z", "content": "I agree
\nI think they will recalculate the score later.
\n", "rawMarkdown": "I agree\nI think they will recalculate the score later.\n\n> @horikitasaku I can confirm that there have been successful submissions using the updated dataset. However, I'm going to hold off on rerunning older submissions on the new dataset until I've had a chance to look into some of the new submission errors you and other competitors are seeing in more detail. I can confirm that some of them are issues that reasonably occur with new data (such as out of memory or GPU memory) but I want to make sure there's nothing else going on.\n\n", "votes": 1, "replies": [ { "id": 3251566, "postDate": "2025-07-21T05:36:44.197Z", "content": "@horikitasaku I can confirm that there have been successful submissions using the updated dataset. However, I'm going to hold off on rerunning older submissions on the new dataset until I've had a chance to look into some of the new submission errors you and other competitors are seeing in more detail. I can confirm that some of them are issues that reasonably occur with new data (such as out of memory or GPU memory) but I want to make sure there's nothing else going on.
\n
Right, I was not reading the official message…
\n\n", "rawMarkdown": "Right, I was not reading the official message…\n\n> This will mean the leaderboard will reset, which of course will be inconvenient for participants, and for that we would like to apologise. We'll be re-running inference for existing submissions soon, though it will likely take a few days to get the results. You can continue to make submissions in the meantime" } ] }, { "id": 3251928, "postDate": "2025-07-21T18:55:50Z", "content": "This will mean the leaderboard will reset, which of course will be inconvenient for participants, and for that we would like to apologise. We'll be re-running inference for existing submissions soon, though it will likely take a few days to get the results. You can continue to make submissions in the meantime
\n
Kaggle comps regenerating data mid-way always feel like a sneaky plot twist. Time to retrain on the new split and pray the private LB doesn’t get nuked next…
\nAnyone else noticing patterns in what’s less affected? e.g., certain models or features?
", "rawMarkdown": "Kaggle comps regenerating data mid-way always feel like a sneaky plot twist. Time to retrain on the new split and pray the private LB doesn’t get nuked next…\n\nAnyone else noticing patterns in what’s less affected? e.g., certain models or features?", "votes": -1, "replies": [ { "id": 3251965, "postDate": "2025-07-21T19:52:22.787Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 3251957, "postDate": "2025-07-21T19:28:22.653Z", "content": "I saw today that all my Old Codes Have Been Operated, any one too
", "rawMarkdown": "I saw today that all my Old Codes Have Been Operated, any one too", "replies": [ { "id": 3251964, "postDate": "2025-07-21T19:51:31.407Z", "content": "My old scores are the same as before. Yours changed?
", "rawMarkdown": "My old scores are the same as before. Yours changed?", "replies": [ { "id": 3251992, "postDate": "2025-07-21T20:56:09.960Z", "content": "Yes every scores changed, I saw more 15 submitted today. I say about on my submissions not on LB public
\nMake sure on your submissions not on LB
What is a submission on on LB? I am confused by what you say.
", "rawMarkdown": "What is a submission on on LB? I am confused by what you say." }, { "id": 3252012, "postDate": "2025-07-21T22:13:39.470Z", "content": "I am sorry for my language
\n Look at your submissions . You must show a new submited , but you did not do that (old codes return run and take new score)
\n If you can not see it, I think you should wait for some time
None of the scores of th esubmisions I made before the dataste change hav ebeen modified. Thsi si wrong.
\nHave the scores of your old submissions changed?
", "rawMarkdown": "None of the scores of th esubmisions I made before the dataste change hav ebeen modified. Thsi si wrong.\n\nHave the scores of your old submissions changed?" }, { "id": 3252254, "postDate": "2025-07-22T10:07:12.333Z", "content": "yes, all changed
", "rawMarkdown": "yes, all changed ", "votes": 1 } ] } ] } ], "comments": [ { "id": 3252142, "author_name": "🐢 Jun Koda", "author_url": "", "post_date": "2025-07-22T05:37:49.573000", "content": "I can see both old and new scores in the Submissions tab. New ones have messange starting from \"Notebook rerun.\" I think the Leaderboard score is the highest among both new and old scores, now.
\n
Lucky you. I can't see any rerun scores.
", "votes": 0, "replies": [] }, { "id": 3252342, "author_name": "Horikita Saku", "author_url": "", "post_date": "2025-07-22T13:52:07.357000", "content": "Staff still hasn't fixed this.
", "votes": 0, "replies": [] }, { "id": 3252346, "author_name": "Michael Kamal", "author_url": "", "post_date": "2025-07-22T14:00:43.987000", "content": "check code i have like you but not same my code, why i say that because old code for me on test in data take different numbers on lw_…. and sigama…. but on rerun take same number on all targets and sigma
\nso i think that rerun it's not really but only for show not for new evaluation not really, so i suggest delete all evaluation and everyone submit again with byself.
\ni want you check your pred test if you code don't use already same number to all targets and sigma !!!
Probable cause is that they don't show failed reruns, see https://www.kaggle.com/competitions/ariel-data-challenge-2025/discussion/590339#3252205
", "votes": 0, "replies": [] } ] }, { "id": 3252424, "author_name": "SolverWorld", "author_url": "", "post_date": "2025-07-22T17:41:59.360000", "content": "I also see old and new scores (automatic reruns of all old submissions) in my submissions tab, but the leaderboard reflects my highest score, which is an old score (not a rerun). I assume at some point they need to delete all the old scores and then the leaderboard will change.
", "votes": 2, "replies": [] }, { "id": 3252692, "author_name": "🐢 Jun Koda", "author_url": "", "post_date": "2025-07-23T07:59:28.300000", "content": "The score recalculation is still in progress and in confusion. The same rerun become error now.
\n
Please note that the updated scores for older submissions are now available.
", "votes": 1, "replies": [ { "id": 3251834, "author_name": "Horikita Saku", "author_url": "", "post_date": "2025-07-21T16:33:55.340000", "content": "great thanks!
\nWill the old scores be set as invalid?
On my submission page I still see the olds scores for my old submissions. And my LB score is an old score of an old submission. what you claim doesn't see to be correct at this time.
", "votes": 1, "replies": [ { "id": 3251843, "author_name": "Horikita Saku", "author_url": "", "post_date": "2025-07-21T16:49:02.633000", "content": "I think it's because the old scores are still valid. Although they have been recalculate
", "votes": 0, "replies": [] }, { "id": 3251953, "author_name": "CPMP", "author_url": "", "post_date": "2025-07-21T19:21:39.593000", "content": "I find it extremely unlikely that scores don't change with the new hidden test set.
", "votes": 1, "replies": [] }, { "id": 3252205, "author_name": "DennisSakva", "author_url": "", "post_date": "2025-07-22T07:49:38.020000", "content": "Same. Probably the old code just failed with the new data.
", "votes": 1, "replies": [] } ] } ] }, { "id": 3251591, "author_name": "Timmy Juicehouse", "author_url": "", "post_date": "2025-07-21T07:08:18.023000", "content": "my current lb score is 0.398
\nwhile an almost identical code with fixes for extracting ingress and egress : 0.365
", "votes": 1, "replies": [ { "id": 3251596, "author_name": "Horikita Saku", "author_url": "", "post_date": "2025-07-21T07:14:15.877000", "content": "mine is 0.433 → 0.384
", "votes": 2, "replies": [] } ] }, { "id": 3251444, "author_name": "Michael Kamal", "author_url": "", "post_date": "2025-07-20T19:17:04.803000", "content": "My Public LB score changed 0.349 → 0.363 for exactly same code.
\nThe Local CV has changed, too: 0.300 → 0.236
\n🤷🤷
Thanks! New data are not necessarily difficult, but larger gap 🤔?
", "votes": 0, "replies": [] } ] }, { "id": 3251279, "author_name": "Horikita Saku", "author_url": "", "post_date": "2025-07-20T10:43:06.473000", "content": "In addition, my observation is similar, and my cv and lb both dropped by about 0.07 to 0.05.
", "votes": 1, "replies": [] }, { "id": 3251199, "author_name": "Horikita Saku", "author_url": "", "post_date": "2025-07-20T07:16:48.097000", "content": "I agree
\nI think they will recalculate the score later.
\n", "votes": 1, "replies": [ { "id": 3251566, "author_name": "🐢 Jun Koda", "author_url": "", "post_date": "2025-07-21T05:36:44.197000", "content": "@horikitasaku I can confirm that there have been successful submissions using the updated dataset. However, I'm going to hold off on rerunning older submissions on the new dataset until I've had a chance to look into some of the new submission errors you and other competitors are seeing in more detail. I can confirm that some of them are issues that reasonably occur with new data (such as out of memory or GPU memory) but I want to make sure there's nothing else going on.
\n
Right, I was not reading the official message…
\n\n", "votes": 0, "replies": [] } ] }, { "id": 3251928, "author_name": "Tamuno opubo Dappa", "author_url": "", "post_date": "2025-07-21T18:55:50", "content": "This will mean the leaderboard will reset, which of course will be inconvenient for participants, and for that we would like to apologise. We'll be re-running inference for existing submissions soon, though it will likely take a few days to get the results. You can continue to make submissions in the meantime
\n
Kaggle comps regenerating data mid-way always feel like a sneaky plot twist. Time to retrain on the new split and pray the private LB doesn’t get nuked next…
\nAnyone else noticing patterns in what’s less affected? e.g., certain models or features?
", "votes": -1, "replies": [ { "id": 3251965, "author_name": "", "author_url": "", "post_date": "2025-07-21T19:52:22.787000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 3251957, "author_name": "Michael Kamal", "author_url": "", "post_date": "2025-07-21T19:28:22.653000", "content": "I saw today that all my Old Codes Have Been Operated, any one too
", "votes": 0, "replies": [ { "id": 3251964, "author_name": "CPMP", "author_url": "", "post_date": "2025-07-21T19:51:31.407000", "content": "My old scores are the same as before. Yours changed?
", "votes": 0, "replies": [ { "id": 3251992, "author_name": "Michael Kamal", "author_url": "", "post_date": "2025-07-21T20:56:09.960000", "content": "Yes every scores changed, I saw more 15 submitted today. I say about on my submissions not on LB public
\nMake sure on your submissions not on LB
What is a submission on on LB? I am confused by what you say.
", "votes": 0, "replies": [] }, { "id": 3252012, "author_name": "Michael Kamal", "author_url": "", "post_date": "2025-07-21T22:13:39.470000", "content": "I am sorry for my language
\n Look at your submissions . You must show a new submited , but you did not do that (old codes return run and take new score)
\n If you can not see it, I think you should wait for some time
None of the scores of th esubmisions I made before the dataste change hav ebeen modified. Thsi si wrong.
\nHave the scores of your old submissions changed?
", "votes": 0, "replies": [] }, { "id": 3252254, "author_name": "Michael Kamal", "author_url": "", "post_date": "2025-07-22T10:07:12.333000", "content": "yes, all changed
", "votes": 1, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "3251117": "Do you suddenly get much worse scores? No worries, it's not your bug but the test data has changed. The training data has changed too.\n\nDataset Regeneration Notice: https://www.kaggle.com/competitions/ariel-data-challenge-2025/discussion/590210\n\nMy Public LB score changed `0.303 → 0.200` for exactly same code.\n\nThe Local CV has changed, too: `0.2880 → 0.1528`.\n\nThe leaderboard scores stay the same, ~~that is, only new submissions are calculated with the new data~~. For those higher in the leaderboard, do you get similar change of ~0.1, too?\n\n~~If so, I think all the scores should be recalculated, or old scores deleted.~~\n\n", "3252142": "I can see **both** old and new scores in the Submissions tab. New ones have messange starting from \"Notebook rerun.\" I think the Leaderboard score is the highest among both new and old scores, now.\n\n\n", "3251831": "Please note that the updated scores for older submissions are now available.", "3251591": "my current lb score is 0.398\n\nwhile an almost identical code with fixes for extracting ingress and egress : 0.365\n", "3251444": "My Public LB score changed 0.349 → 0.363 for exactly same code.\n\nThe Local CV has changed, too: 0.300 → 0.236\n🤷🤷", "3251279": "In addition, my observation is similar, and my cv and lb both dropped by about 0.07 to 0.05.", "3251199": "I agree\nI think they will recalculate the score later.\n\n> @horikitasaku I can confirm that there have been successful submissions using the updated dataset. However, I'm going to hold off on rerunning older submissions on the new dataset until I've had a chance to look into some of the new submission errors you and other competitors are seeing in more detail. I can confirm that some of them are issues that reasonably occur with new data (such as out of memory or GPU memory) but I want to make sure there's nothing else going on.\n\n", "3251928": "Kaggle comps regenerating data mid-way always feel like a sneaky plot twist. Time to retrain on the new split and pray the private LB doesn’t get nuked next…\n\nAnyone else noticing patterns in what’s less affected? e.g., certain models or features?", "3251957": "I saw today that all my Old Codes Have Been Operated, any one too" } }