\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1132983%2F2b37a5c93864b969173303de102ca5f3%2Fcomparison.png?generation=1752861285328781&alt=media\")
Dear Participants,
\nWe need to apologize for an error we made in our dataset generation. While investigating this question, we discovered that our data generation process failed to properly account for planetary orbital periods.This introduced physically implausible transit durations.
\nHere is a side by side comparison of an original and patched light curve:
\n
While we could easily patch this by replacing the period with default values, doing so would violate Kepler's 3rd law—the fundamental relationship between orbital period and semi-major axis. This would compromise the scientific integrity of the dataset and fail to maintain the high standards we strive to achieve for the challenge (not to mention, hurt our pride as physicists!)
\nWe had two options: apply a quick patch or regenerate the dataset entirely. We chose regeneration because maintaining high scientific standards is not only important to us, but we also deeply respect the time and effort Kagglers invest in participating in this Data Challenge. You deserve a scientifically rigorous dataset to work with.
\nThis will mean the leaderboard will reset, which of course will be inconvenient for participants, and for that we would like to apologise. We'll be re-running inference for existing submissions soon, though it will likely take a few days to get the results. You can continue to make submissions in the meantime.
\nThis error does not affect the competition's core objective of measuring transit depth, but it falls short of the standards we strive to maintain. We sincerely apologize for this oversight and any inconvenience it may cause.
\nWe're excited to see your innovative solutions on the corrected dataset!
\nBest regards,
\nThe Ariel Data Challenge Team
\nPS - Due to the size of the datasets, it may be another few hours before the updates are actually available for download. I'll post progress updates here.
\nEdit 1 (14:40 PST July 18): You can now download the updated train set. The hidden test set update is still uploading.
\nEdit 2 (21:20 PST July 18): The hidden test set and solution file have been updated. I've kicked off a fresh update of the competition dataset zip creation to make it feasible to download the train set.
\nEdit 3: The train set can now be downloaded as a single zip.
\nEdit 4: Updated scores for older submissions are now live on the leaderboard.
\nEdit 5: Due to an administrative error on my part, rerun submissions were entered both as new submissions and updates of the old submissions. The duplicate copies have been invalidated. They'll clutter some lists of submissions but can be safely ignored. Apologies for the confusion and to all teams that were temporarily blocked from submitting.
\nEdit 6: Some submissions had been showing their old scores. This turned out to happen if the submission failed during the rerun. We've now errored out the affected submissions, of which there are a few hundred, and are looking into the root cause.
\nEdit 7: I believe the leaderboard is in a good state now. Apologies for the confusion, that update turned out to be much more painful than it needed to be. I've proposed some upgrades to our internal tooling to prevent similar issues in the future.
", "messages": [ { "id": 3250578, "postDate": "2025-07-18T17:54:49.533Z", "content": "Dear Participants,
\nWe need to apologize for an error we made in our dataset generation. While investigating this question, we discovered that our data generation process failed to properly account for planetary orbital periods.This introduced physically implausible transit durations.
\nHere is a side by side comparison of an original and patched light curve:
\n
While we could easily patch this by replacing the period with default values, doing so would violate Kepler's 3rd law—the fundamental relationship between orbital period and semi-major axis. This would compromise the scientific integrity of the dataset and fail to maintain the high standards we strive to achieve for the challenge (not to mention, hurt our pride as physicists!)
\nWe had two options: apply a quick patch or regenerate the dataset entirely. We chose regeneration because maintaining high scientific standards is not only important to us, but we also deeply respect the time and effort Kagglers invest in participating in this Data Challenge. You deserve a scientifically rigorous dataset to work with.
\nThis will mean the leaderboard will reset, which of course will be inconvenient for participants, and for that we would like to apologise. We'll be re-running inference for existing submissions soon, though it will likely take a few days to get the results. You can continue to make submissions in the meantime.
\nThis error does not affect the competition's core objective of measuring transit depth, but it falls short of the standards we strive to maintain. We sincerely apologize for this oversight and any inconvenience it may cause.
\nWe're excited to see your innovative solutions on the corrected dataset!
\nBest regards,
\nThe Ariel Data Challenge Team
\nPS - Due to the size of the datasets, it may be another few hours before the updates are actually available for download. I'll post progress updates here.
\nEdit 1 (14:40 PST July 18): You can now download the updated train set. The hidden test set update is still uploading.
\nEdit 2 (21:20 PST July 18): The hidden test set and solution file have been updated. I've kicked off a fresh update of the competition dataset zip creation to make it feasible to download the train set.
\nEdit 3: The train set can now be downloaded as a single zip.
\nEdit 4: Updated scores for older submissions are now live on the leaderboard.
\nEdit 5: Due to an administrative error on my part, rerun submissions were entered both as new submissions and updates of the old submissions. The duplicate copies have been invalidated. They'll clutter some lists of submissions but can be safely ignored. Apologies for the confusion and to all teams that were temporarily blocked from submitting.
\nEdit 6: Some submissions had been showing their old scores. This turned out to happen if the submission failed during the rerun. We've now errored out the affected submissions, of which there are a few hundred, and are looking into the root cause.
\nEdit 7: I believe the leaderboard is in a good state now. Apologies for the confusion, that update turned out to be much more painful than it needed to be. I've proposed some upgrades to our internal tooling to prevent similar issues in the future.
", "rawMarkdown": "Dear Participants,\n\nWe need to apologize for an error we made in our dataset generation. While investigating this question, we discovered that our data generation process failed to properly account for planetary orbital periods.This introduced physically implausible transit durations. \n \nHere is a side by side comparison of an original and patched light curve:\n\n\nWhile we could easily patch this by replacing the period with default values, doing so would violate Kepler's 3rd law—the fundamental relationship between orbital period and semi-major axis. This would compromise the scientific integrity of the dataset and fail to maintain the high standards we strive to achieve for the challenge (not to mention, hurt our pride as physicists!)\n\nWe had two options: apply a quick patch or regenerate the dataset entirely. We chose regeneration because maintaining high scientific standards is not only important to us, but we also deeply respect the time and effort Kagglers invest in participating in this Data Challenge. You deserve a scientifically rigorous dataset to work with.\n \nThis will mean the leaderboard will reset, which of course will be inconvenient for participants, and for that we would like to apologise. We'll be re-running inference for existing submissions soon, though it will likely take a few days to get the results. You can continue to make submissions in the meantime.\n\nThis error does not affect the competition's core objective of measuring transit depth, but it falls short of the standards we strive to maintain. We sincerely apologize for this oversight and any inconvenience it may cause.\n\nWe're excited to see your innovative solutions on the corrected dataset!\n\nBest regards,\n\nThe Ariel Data Challenge Team\n\nPS - Due to the size of the datasets, it may be another few hours before the updates are actually available for download. I'll post progress updates here.\n\nEdit 1 (14:40 PST July 18): You can now download the updated train set. The hidden test set update is still uploading.\n\nEdit 2 (21:20 PST July 18): The hidden test set and solution file have been updated. I've kicked off a fresh update of the competition dataset zip creation to make it feasible to download the train set.\n\nEdit 3: The train set can now be downloaded as a single zip.\n\nEdit 4: Updated scores for older submissions are now live on the leaderboard.\n\nEdit 5: Due to an administrative error on my part, rerun submissions were entered both as new submissions and updates of the old submissions. The duplicate copies have been invalidated. They'll clutter some lists of submissions but can be safely ignored. Apologies for the confusion and to all teams that were temporarily blocked from submitting.\n\nEdit 6: Some submissions had been showing their old scores. This turned out to happen if the submission failed during the rerun. We've now errored out the affected submissions, of which there are a few hundred, and are looking into the root cause.\n \nEdit 7: I believe the leaderboard is in a good state now. Apologies for the confusion, that update turned out to be much more painful than it needed to be. I've proposed some upgrades to our internal tooling to prevent similar issues in the future.", "votes": 15 }, { "id": 3251675, "postDate": "2025-07-21T11:35:02.323Z", "content": "I just got 0.36 after fixing the phase detection.
\nI had 0.364 before dataset update but the code is a bit different, hard to compare.
\nTL;DR phase detection is where most problems are probably.
", "rawMarkdown": "I just got 0.36 after fixing the phase detection.\n\nI had 0.364 before dataset update but the code is a bit different, hard to compare.\n\nTL;DR phase detection is where most problems are probably.", "votes": 3, "replies": [ { "id": 3251681, "postDate": "2025-07-21T11:49:58.680Z", "content": "i guess after the score recalculation, my Submissions page will turn to a lot of red💀
", "rawMarkdown": "i guess after the score recalculation, my Submissions page will turn to a lot of red💀", "votes": 1, "replies": [ { "id": 3251704, "postDate": "2025-07-21T12:35:26.347Z", "content": "Yesterday I had two failed subs and one with score 0.
", "rawMarkdown": "Yesterday I had two failed subs and one with score 0.", "votes": 1 } ] } ] }, { "id": 3250692, "postDate": "2025-07-18T22:54:39.723Z", "content": "\n\nEdit 1 (14:40 PST July 18): You can now download the updated train set. The hidden test set update is still uploading.
\n
@sohier This looks great but this command fails:
\n\n\nkaggle competitions download -c ariel-data-challenge-2025
\n
It fails even if I use an old version of the cli as recommended somewhere (I don't remember where).
\nError message is:
\n\n\n404 - Not Found - no gcs url
\n
Manual download from the Kaggle web site does not work either. It says that this page doesn't exist: https://www.kaggle.com/competitions/ariel-data-challenge-2025/download-all
", "rawMarkdown": "> Edit 1 (14:40 PST July 18): You can now download the updated train set. The hidden test set update is still uploading.\n\n@sohier This looks great but this command fails:\n\n> kaggle competitions download -c ariel-data-challenge-2025\n\nIt fails even if I use an old version of the cli as recommended somewhere (I don't remember where).\n\nError message is:\n\n> 404 - Not Found - no gcs url\n\nManual download from the Kaggle web site does not work either. It says that this page doesn't exist: https://www.kaggle.com/competitions/ariel-data-challenge-2025/download-all", "votes": 4, "replies": [ { "id": 3250697, "postDate": "2025-07-18T23:28:10.970Z", "content": "Thanks for flagging the problem. This indicates something failed during the creation of the zip file. I've just kicked off a fresh run of the process but it will likely take several hours to complete.
", "rawMarkdown": "Thanks for flagging the problem. This indicates something failed during the creation of the zip file. I've just kicked off a fresh run of the process but it will likely take several hours to complete.", "replies": [ { "id": 3250709, "postDate": "2025-07-19T00:49:27.213Z", "content": "Thanks. Please check that the data can be downloaded when it is complete. We are at risk of losing a full week end here :D
", "rawMarkdown": "Thanks. Please check that the data can be downloaded when it is complete. We are at risk of losing a full week end here :D", "votes": 1 }, { "id": 3250860, "postDate": "2025-07-19T10:08:59.070Z", "content": "Currently downloading, was getting 404 earlier too.
", "rawMarkdown": "Currently downloading, was getting 404 earlier too." }, { "id": 3251023, "postDate": "2025-07-19T17:55:49.173Z", "content": "The train set can now be downloaded as a single zip.
", "rawMarkdown": "The train set can now be downloaded as a single zip." } ] } ] }, { "id": 3251717, "postDate": "2025-07-21T12:59:05.633Z", "content": "A big thank you to the Ariel Data Challenge team and Kaggle staff for the detailed explanation and transparent handling of the dataset issue.
\nIt’s clear that choosing to regenerate the dataset rather than applying a shortcut patch was the right move from a scientific integrity standpoint. Kepler’s 3rd law isn’t something you can just \"fudge\" without consequence—especially in a challenge built around modeling astrophysical realism. As someone working through the light curves, I really appreciate the care that went into preserving the fidelity of orbital dynamics in the dataset.
", "rawMarkdown": "A big thank you to the Ariel Data Challenge team and Kaggle staff for the detailed explanation and transparent handling of the dataset issue.\n\nIt’s clear that choosing to regenerate the dataset rather than applying a shortcut patch was the right move from a scientific integrity standpoint. Kepler’s 3rd law isn’t something you can just \"fudge\" without consequence—especially in a challenge built around modeling astrophysical realism. As someone working through the light curves, I really appreciate the care that went into preserving the fidelity of orbital dynamics in the dataset.", "votes": 1 }, { "id": 3250903, "postDate": "2025-07-19T12:47:44.197Z", "content": "My code ran successfully on the old test dataset. I had hoped to improve my score today, but all three submissions resulted in errors. I spent a long time debugging yet couldn't figure out what was wrong—until I saw this announcement.
", "rawMarkdown": "My code ran successfully on the old test dataset. I had hoped to improve my score today, but all three submissions resulted in errors. I spent a long time debugging yet couldn't figure out what was wrong—until I saw this announcement.", "votes": 1, "replies": [ { "id": 3250904, "postDate": "2025-07-19T12:49:40.653Z", "content": "yeah
\ntotally same here
\nI had used all the submissions before I found this notice. And my new submissions are all error.
\n😭
very sorry ! I hope it will be resolved soon!
", "rawMarkdown": "very sorry ! I hope it will be resolved soon!", "votes": 2 }, { "id": 3251600, "postDate": "2025-07-21T07:19:23.170Z", "content": "It's okay. I don't have a deep background in astrophysics myself, but I believe we should establish more robust testing to handle complex and real-world observational scenarios.
", "rawMarkdown": "It's okay. I don't have a deep background in astrophysics myself, but I believe we should establish more robust testing to handle complex and real-world observational scenarios.", "votes": 1 } ] }, { "id": 3251051, "postDate": "2025-07-19T20:21:39.820Z", "content": "There are several samples in the train where the transit has already started at the beginning of the data. I know this broke my code which couldn't detect the ingress since there isn't one.
", "rawMarkdown": "There are several samples in the train where the transit has already started at the beginning of the data. I know this broke my code which couldn't detect the ingress since there isn't one. ", "votes": 1, "replies": [ { "id": 3251090, "postDate": "2025-07-19T23:30:19.360Z", "content": "That makes sense, thanks! I didn't see errors indicative of systemic problems so I'm going to begin rerunning older submissions. The actual results won't be available until early next week.
", "rawMarkdown": "That makes sense, thanks! I didn't see errors indicative of systemic problems so I'm going to begin rerunning older submissions. The actual results won't be available until early next week." } ] } ] }, { "id": 3250972, "postDate": "2025-07-19T15:51:54.337Z", "content": "Hi everyone -
\nHost here - We know how frustrating it must be to lose work you’ve put time and effort into due to our data refresh issues. We completely understand your disappointment.
\nWe take full responsibility for this situation. We had planned for the data refresh to happen seamlessly so you could submit to the new dataset immediately, but technical difficulties prevented this from working as intended.
\nWe are working with Kaggle to see what can be done.
\nIn hindsight, we should have given advance warning about the data refresh, and we sincerely apologise for the inconvenience caused by our mistake.
", "rawMarkdown": "Hi everyone - \n\nHost here - We know how frustrating it must be to lose work you’ve put time and effort into due to our data refresh issues. We completely understand your disappointment.\n\nWe take full responsibility for this situation. We had planned for the data refresh to happen seamlessly so you could submit to the new dataset immediately, but technical difficulties prevented this from working as intended.\n\nWe are working with Kaggle to see what can be done. \n\nIn hindsight, we should have given advance warning about the data refresh, and we sincerely apologise for the inconvenience caused by our mistake.\n", "votes": 2, "replies": [ { "id": 3251006, "postDate": "2025-07-19T17:06:27.723Z", "content": "So, hasn't the current test set been updated yet?
", "rawMarkdown": "So, hasn't the current test set been updated yet?", "replies": [ { "id": 3251027, "postDate": "2025-07-19T18:08:01.597Z", "content": "@horikitasaku the test set has been updated.
", "rawMarkdown": "@horikitasaku the test set has been updated.", "votes": 1 } ] } ] }, { "id": 3250948, "postDate": "2025-07-19T14:50:50.583Z", "content": "Is the original dataset useful additional training data, albeit with incorrect transit periods? Or did other parameters change so much that it is completely out-of-band?
", "rawMarkdown": "Is the original dataset useful additional training data, albeit with incorrect transit periods? Or did other parameters change so much that it is completely out-of-band?", "votes": 2, "replies": [ { "id": 3250970, "postDate": "2025-07-19T15:49:33.857Z", "content": "I think the nature of the problem didnt change between the two datasets - since the objective is to measure the transit depth accurately, changing the transit period will only affect the duration (width of the transit), as shown in the figure above.
", "rawMarkdown": "I think the nature of the problem didnt change between the two datasets - since the objective is to measure the transit depth accurately, changing the transit period will only affect the duration (width of the transit), as shown in the figure above. ", "votes": 2 } ] }, { "id": 3252504, "postDate": "2025-07-22T22:02:11.983Z", "content": "An interesting case happened. Yesterday the programs were recalculated, in one the result became smaller. All the programs were glowing green, except for my new program, it did not go with the new data. This evening, all the programs that were recalculated yesterday suddenly turned red, but the result remained. How can this be explained?
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F7279405%2F47768e7d2f4a7a9e1bd0871a5754cd2b%2F1.png?generation=1753221438853862&alt=media\")
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F7279405%2F6f461c8677729319e6c05cece39d7772%2F2.png?generation=1753221451366587&alt=media\")
In the following message.
\nHowever there are still some old scores remain valid and i think some scores on lb have not changed. So I'm not sure if there is no problem.
\n", "rawMarkdown": "In the following message.\nHowever there are still some old scores remain valid and i think some scores on lb have not changed. So I'm not sure if there is no problem.\n> Due to an administrative error on my part, rerun submissions were entered both as new submissions and updates of the old submissions. The duplicate copies have been invalidated. They'll clutter some lists of submissions but can be safely ignored. Apologies for the confusion and to all teams that were temporarily blocked from submitting." } ] }, { "id": 3252493, "postDate": "2025-07-22T21:35:20.620Z", "content": "Due to an administrative error on my part, rerun submissions were entered both as new submissions and updates of the old submissions. The duplicate copies have been invalidated. They'll clutter some lists of submissions but can be safely ignored. Apologies for the confusion and to all teams that were temporarily blocked from submitting.
\n
Due to an administrative error on my part, rerun submissions were entered both as new submissions and updates of the old submissions. The duplicate copies have been invalidated. They'll clutter some lists of submissions but can be safely ignored. Apologies for the confusion and to all teams that were temporarily blocked from submitting.
", "rawMarkdown": "Due to an administrative error on my part, rerun submissions were entered both as new submissions and updates of the old submissions. The duplicate copies have been invalidated. They'll clutter some lists of submissions but can be safely ignored. Apologies for the confusion and to all teams that were temporarily blocked from submitting.", "replies": [ { "id": 3252515, "postDate": "2025-07-22T23:29:36.073Z", "rawMarkdown": "", "isDeleted": true }, { "id": 3252517, "postDate": "2025-07-22T23:35:34.277Z", "content": "But there are still old scores being used?
\nI noticed that some of my old scores are still valid, and some scores on lb have not changed.
\nhttps://www.kaggle.com/competitions/ariel-data-challenge-2025/discussion/590773
@sohier The leaderboard still shows many scores obtained with the old dataset. Can you remove all the old scores? It is rather simple IMHO: remove all subs that were submitted before the dataset update.
", "rawMarkdown": "@sohier The leaderboard still shows many scores obtained with the old dataset. Can you remove all the old scores? It is rather simple IMHO: remove all subs that were submitted before the dataset update.", "votes": 2 }, { "id": 3253545, "postDate": "2025-07-24T22:12:54.337Z", "content": "I believe the leaderboard is in a good state now. Apologies for the confusion, that update turned out to be much more painful than it needed to be. I've proposed some upgrades to our internal tooling to prevent similar issues in the future.
", "rawMarkdown": "I believe the leaderboard is in a good state now. Apologies for the confusion, that update turned out to be much more painful than it needed to be. I've proposed some upgrades to our internal tooling to prevent similar issues in the future.", "votes": 2, "replies": [ { "id": 3253590, "postDate": "2025-07-25T00:56:19.113Z", "content": "Great thanks to your effort
", "rawMarkdown": "Great thanks to your effort", "votes": 1 } ] } ] }, { "id": 3251814, "postDate": "2025-07-21T15:53:07.637Z", "content": "Updated scores for older submissions are now live on the leaderboard.
", "rawMarkdown": "Updated scores for older submissions are now live on the leaderboard.", "replies": [ { "id": 3251826, "postDate": "2025-07-21T16:17:56.153Z", "content": "@sohier The old scores are still on leaderboard, and I know that one of my notebooks which definitely error on new data still says it was successful with previous score from old data.
", "rawMarkdown": "@sohier The old scores are still on leaderboard, and I know that one of my notebooks which definitely error on new data still says it was successful with previous score from old data. ", "votes": 1, "replies": [ { "id": 3251827, "postDate": "2025-07-21T16:19:09.077Z", "rawMarkdown": "", "isDeleted": true }, { "id": 3252520, "postDate": "2025-07-22T23:41:54.360Z", "content": "@devinanzelmo thanks for reporting. Those notebooks should now being showing the correct errors.
", "rawMarkdown": "@devinanzelmo thanks for reporting. Those notebooks should now being showing the correct errors." } ] }, { "id": 3251828, "postDate": "2025-07-21T16:19:20.377Z", "content": "Great thanks! what about the old scores?
\nwill you delete them or just make them unvalid
Are they? LB is the same as yesterday basically. It looks like old scores are shown.
", "rawMarkdown": "Are they? LB is the same as yesterday basically. It looks like old scores are shown." }, { "id": 3251845, "postDate": "2025-07-21T16:54:38.383Z", "content": "Moreover, it seems that re-running the test was also regarded as my submission, which led to an increase in the number of my submissions.
", "rawMarkdown": "Moreover, it seems that re-running the test was also regarded as my submission, which led to an increase in the number of my submissions." }, { "id": 3252038, "postDate": "2025-07-21T23:24:40.820Z", "content": "But I feel like nothing has changed？
", "rawMarkdown": "But I feel like nothing has changed？" }, { "id": 3252653, "postDate": "2025-07-23T06:21:39.213Z", "content": "seems they only have been updated (invalidated) today?
", "rawMarkdown": "seems they only have been updated (invalidated) today?" } ] }, { "id": 3251598, "postDate": "2025-07-21T07:15:29.990Z", "content": "my current lb score is 0.398
\nwhile an almost identical code with fixes for extracting ingress and egress : 0.365
\nI'm not sure if there's still an issue with extracting ingress and egress. I'm still checking it through EDA.
", "rawMarkdown": "my current lb score is 0.398\n\nwhile an almost identical code with fixes for extracting ingress and egress : 0.365\n\nI'm not sure if there's still an issue with extracting ingress and egress. I'm still checking it through EDA." }, { "id": 3251372, "postDate": "2025-07-20T15:43:39.477Z", "content": "Has the current test set been updated?
", "rawMarkdown": "Has the current test set been updated?", "replies": [ { "id": 3251378, "postDate": "2025-07-20T16:00:02.923Z", "content": "yes
\nit has been updated
Thank you. No wonder I just found that the same code has different results.
", "rawMarkdown": "Thank you. No wonder I just found that the same code has different results.", "replies": [ { "id": 3250870, "postDate": "2025-07-19T10:54:32.807Z", "content": "In addition, regardless of what has been submitted, the new test set will be used in the current new submission, right?
", "rawMarkdown": "In addition, regardless of what has been submitted, the new test set will be used in the current new submission, right?", "replies": [ { "id": 3250924, "postDate": "2025-07-19T13:42:35.197Z", "content": "I can run normally on the new training set, but failed on the test set …
", "rawMarkdown": "I can run normally on the new training set, but failed on the test set ..." }, { "id": 3251029, "postDate": "2025-07-19T18:12:36.647Z", "content": "@horikitasaku I can confirm that there have been successful submissions using the updated dataset. However, I'm going to hold off on rerunning older submissions on the new dataset until I've had a chance to look into some of the new submission errors you and other competitors are seeing in more detail. I can confirm that some of them are issues that reasonably occur with new data (such as out of memory or GPU memory) but I want to make sure there's nothing else going on.
", "rawMarkdown": "@horikitasaku I can confirm that there have been successful submissions using the updated dataset. However, I'm going to hold off on rerunning older submissions on the new dataset until I've had a chance to look into some of the new submission errors you and other competitors are seeing in more detail. I can confirm that some of them are issues that reasonably occur with new data (such as out of memory or GPU memory) but I want to make sure there's nothing else going on. ", "votes": 1 }, { "id": 3251054, "postDate": "2025-07-19T20:33:28.053Z", "content": "@sohier There are several examples in the train where data starts at the end of ingress(or there is no ingress). Code that ran fine on old data now errors. I am sure this is happening to a lot of other solutions as well. Not sure if this is an issue or not, we can adapt code to handle it pretty easily.
", "rawMarkdown": "@sohier There are several examples in the train where data starts at the end of ingress(or there is no ingress). Code that ran fine on old data now errors. I am sure this is happening to a lot of other solutions as well. Not sure if this is an issue or not, we can adapt code to handle it pretty easily. ", "votes": 3 }, { "id": 3251187, "postDate": "2025-07-20T06:22:57.193Z", "content": "@sohier - I am able to resubmit old submission without error and with lower score. Not yet tried training with new dataset.
", "rawMarkdown": "@sohier - I am able to resubmit old submission without error and with lower score. Not yet tried training with new dataset." }, { "id": 3251271, "postDate": "2025-07-20T09:46:59.650Z", "content": "thanks guys
\nI repaired the detection of ingress and egress and successfully submitted it.
\n@sohier @devinanzelmo @viji1609
I just got 0.36 after fixing the phase detection.
\nI had 0.364 before dataset update but the code is a bit different, hard to compare.
\nTL;DR phase detection is where most problems are probably.
", "votes": 3, "replies": [ { "id": 3251681, "author_name": "Horikita Saku", "author_url": "", "post_date": "2025-07-21T11:49:58.680000", "content": "i guess after the score recalculation, my Submissions page will turn to a lot of red💀
", "votes": 1, "replies": [ { "id": 3251704, "author_name": "CPMP", "author_url": "", "post_date": "2025-07-21T12:35:26.347000", "content": "Yesterday I had two failed subs and one with score 0.
", "votes": 1, "replies": [] } ] } ] }, { "id": 3250692, "author_name": "CPMP", "author_url": "", "post_date": "2025-07-18T22:54:39.723000", "content": "\n\nEdit 1 (14:40 PST July 18): You can now download the updated train set. The hidden test set update is still uploading.
\n
@sohier This looks great but this command fails:
\n\n\nkaggle competitions download -c ariel-data-challenge-2025
\n
It fails even if I use an old version of the cli as recommended somewhere (I don't remember where).
\nError message is:
\n\n\n404 - Not Found - no gcs url
\n
Manual download from the Kaggle web site does not work either. It says that this page doesn't exist: https://www.kaggle.com/competitions/ariel-data-challenge-2025/download-all
", "votes": 4, "replies": [ { "id": 3250697, "author_name": "Sohier Dane", "author_url": "", "post_date": "2025-07-18T23:28:10.970000", "content": "Thanks for flagging the problem. This indicates something failed during the creation of the zip file. I've just kicked off a fresh run of the process but it will likely take several hours to complete.
", "votes": 0, "replies": [ { "id": 3250709, "author_name": "CPMP", "author_url": "", "post_date": "2025-07-19T00:49:27.213000", "content": "Thanks. Please check that the data can be downloaded when it is complete. We are at risk of losing a full week end here :D
", "votes": 1, "replies": [] }, { "id": 3250860, "author_name": "sroger", "author_url": "", "post_date": "2025-07-19T10:08:59.070000", "content": "Currently downloading, was getting 404 earlier too.
", "votes": 0, "replies": [] }, { "id": 3251023, "author_name": "Sohier Dane", "author_url": "", "post_date": "2025-07-19T17:55:49.173000", "content": "The train set can now be downloaded as a single zip.
", "votes": 0, "replies": [] } ] } ] }, { "id": 3251717, "author_name": "Erigitola Wahab Olatunji", "author_url": "", "post_date": "2025-07-21T12:59:05.633000", "content": "A big thank you to the Ariel Data Challenge team and Kaggle staff for the detailed explanation and transparent handling of the dataset issue.
\nIt’s clear that choosing to regenerate the dataset rather than applying a shortcut patch was the right move from a scientific integrity standpoint. Kepler’s 3rd law isn’t something you can just \"fudge\" without consequence—especially in a challenge built around modeling astrophysical realism. As someone working through the light curves, I really appreciate the care that went into preserving the fidelity of orbital dynamics in the dataset.
", "votes": 1, "replies": [] }, { "id": 3250903, "author_name": "Timmy Juicehouse", "author_url": "", "post_date": "2025-07-19T12:47:44.197000", "content": "My code ran successfully on the old test dataset. I had hoped to improve my score today, but all three submissions resulted in errors. I spent a long time debugging yet couldn't figure out what was wrong—until I saw this announcement.
", "votes": 1, "replies": [ { "id": 3250904, "author_name": "Horikita Saku", "author_url": "", "post_date": "2025-07-19T12:49:40.653000", "content": "yeah
\ntotally same here
\nI had used all the submissions before I found this notice. And my new submissions are all error.
\n😭
very sorry ! I hope it will be resolved soon!
", "votes": 2, "replies": [] }, { "id": 3251600, "author_name": "Timmy Juicehouse", "author_url": "", "post_date": "2025-07-21T07:19:23.170000", "content": "It's okay. I don't have a deep background in astrophysics myself, but I believe we should establish more robust testing to handle complex and real-world observational scenarios.
", "votes": 1, "replies": [] } ] }, { "id": 3251051, "author_name": "Devin Anzelmo", "author_url": "", "post_date": "2025-07-19T20:21:39.820000", "content": "There are several samples in the train where the transit has already started at the beginning of the data. I know this broke my code which couldn't detect the ingress since there isn't one.
", "votes": 1, "replies": [ { "id": 3251090, "author_name": "Sohier Dane", "author_url": "", "post_date": "2025-07-19T23:30:19.360000", "content": "That makes sense, thanks! I didn't see errors indicative of systemic problems so I'm going to begin rerunning older submissions. The actual results won't be available until early next week.
", "votes": 0, "replies": [] } ] } ] }, { "id": 3250972, "author_name": "Gordon Yip", "author_url": "", "post_date": "2025-07-19T15:51:54.337000", "content": "Hi everyone -
\nHost here - We know how frustrating it must be to lose work you’ve put time and effort into due to our data refresh issues. We completely understand your disappointment.
\nWe take full responsibility for this situation. We had planned for the data refresh to happen seamlessly so you could submit to the new dataset immediately, but technical difficulties prevented this from working as intended.
\nWe are working with Kaggle to see what can be done.
\nIn hindsight, we should have given advance warning about the data refresh, and we sincerely apologise for the inconvenience caused by our mistake.
", "votes": 2, "replies": [ { "id": 3251006, "author_name": "Horikita Saku", "author_url": "", "post_date": "2025-07-19T17:06:27.723000", "content": "So, hasn't the current test set been updated yet?
", "votes": 0, "replies": [ { "id": 3251027, "author_name": "Sohier Dane", "author_url": "", "post_date": "2025-07-19T18:08:01.597000", "content": "@horikitasaku the test set has been updated.
", "votes": 1, "replies": [] } ] } ] }, { "id": 3250948, "author_name": "SolverWorld", "author_url": "", "post_date": "2025-07-19T14:50:50.583000", "content": "Is the original dataset useful additional training data, albeit with incorrect transit periods? Or did other parameters change so much that it is completely out-of-band?
", "votes": 2, "replies": [ { "id": 3250970, "author_name": "Gordon Yip", "author_url": "", "post_date": "2025-07-19T15:49:33.857000", "content": "I think the nature of the problem didnt change between the two datasets - since the objective is to measure the transit depth accurately, changing the transit period will only affect the duration (width of the transit), as shown in the figure above.
", "votes": 2, "replies": [] } ] }, { "id": 3252504, "author_name": "AI-Cat", "author_url": "", "post_date": "2025-07-22T22:02:11.983000", "content": "An interesting case happened. Yesterday the programs were recalculated, in one the result became smaller. All the programs were glowing green, except for my new program, it did not go with the new data. This evening, all the programs that were recalculated yesterday suddenly turned red, but the result remained. How can this be explained?
\n
In the following message.
\nHowever there are still some old scores remain valid and i think some scores on lb have not changed. So I'm not sure if there is no problem.
\n", "votes": 0, "replies": [] } ] }, { "id": 3252493, "author_name": "Sohier Dane", "author_url": "", "post_date": "2025-07-22T21:35:20.620000", "content": "Due to an administrative error on my part, rerun submissions were entered both as new submissions and updates of the old submissions. The duplicate copies have been invalidated. They'll clutter some lists of submissions but can be safely ignored. Apologies for the confusion and to all teams that were temporarily blocked from submitting.
\n
Due to an administrative error on my part, rerun submissions were entered both as new submissions and updates of the old submissions. The duplicate copies have been invalidated. They'll clutter some lists of submissions but can be safely ignored. Apologies for the confusion and to all teams that were temporarily blocked from submitting.
", "votes": 0, "replies": [ { "id": 3252515, "author_name": "", "author_url": "", "post_date": "2025-07-22T23:29:36.073000", "content": "", "votes": 0, "replies": [] }, { "id": 3252517, "author_name": "Horikita Saku", "author_url": "", "post_date": "2025-07-22T23:35:34.277000", "content": "But there are still old scores being used?
\nI noticed that some of my old scores are still valid, and some scores on lb have not changed.
\nhttps://www.kaggle.com/competitions/ariel-data-challenge-2025/discussion/590773
@sohier The leaderboard still shows many scores obtained with the old dataset. Can you remove all the old scores? It is rather simple IMHO: remove all subs that were submitted before the dataset update.
", "votes": 2, "replies": [] }, { "id": 3253545, "author_name": "Sohier Dane", "author_url": "", "post_date": "2025-07-24T22:12:54.337000", "content": "I believe the leaderboard is in a good state now. Apologies for the confusion, that update turned out to be much more painful than it needed to be. I've proposed some upgrades to our internal tooling to prevent similar issues in the future.
", "votes": 2, "replies": [ { "id": 3253590, "author_name": "Horikita Saku", "author_url": "", "post_date": "2025-07-25T00:56:19.113000", "content": "Great thanks to your effort
", "votes": 1, "replies": [] } ] } ] }, { "id": 3251814, "author_name": "Sohier Dane", "author_url": "", "post_date": "2025-07-21T15:53:07.637000", "content": "Updated scores for older submissions are now live on the leaderboard.
", "votes": 0, "replies": [ { "id": 3251826, "author_name": "Devin Anzelmo", "author_url": "", "post_date": "2025-07-21T16:17:56.153000", "content": "@sohier The old scores are still on leaderboard, and I know that one of my notebooks which definitely error on new data still says it was successful with previous score from old data.
", "votes": 1, "replies": [ { "id": 3251827, "author_name": "", "author_url": "", "post_date": "2025-07-21T16:19:09.077000", "content": "", "votes": 0, "replies": [] }, { "id": 3252520, "author_name": "Sohier Dane", "author_url": "", "post_date": "2025-07-22T23:41:54.360000", "content": "@devinanzelmo thanks for reporting. Those notebooks should now being showing the correct errors.
", "votes": 0, "replies": [] } ] }, { "id": 3251828, "author_name": "Horikita Saku", "author_url": "", "post_date": "2025-07-21T16:19:20.377000", "content": "Great thanks! what about the old scores?
\nwill you delete them or just make them unvalid
Are they? LB is the same as yesterday basically. It looks like old scores are shown.
", "votes": 0, "replies": [] }, { "id": 3251845, "author_name": "Horikita Saku", "author_url": "", "post_date": "2025-07-21T16:54:38.383000", "content": "Moreover, it seems that re-running the test was also regarded as my submission, which led to an increase in the number of my submissions.
", "votes": 0, "replies": [] }, { "id": 3252038, "author_name": "Timmy Juicehouse", "author_url": "", "post_date": "2025-07-21T23:24:40.820000", "content": "But I feel like nothing has changed？
", "votes": 0, "replies": [] }, { "id": 3252653, "author_name": "Pascal Pfeiffer", "author_url": "", "post_date": "2025-07-23T06:21:39.213000", "content": "seems they only have been updated (invalidated) today?
", "votes": 0, "replies": [] } ] }, { "id": 3251598, "author_name": "Timmy Juicehouse", "author_url": "", "post_date": "2025-07-21T07:15:29.990000", "content": "my current lb score is 0.398
\nwhile an almost identical code with fixes for extracting ingress and egress : 0.365
\nI'm not sure if there's still an issue with extracting ingress and egress. I'm still checking it through EDA.
", "votes": 0, "replies": [] }, { "id": 3251372, "author_name": "Navneet", "author_url": "", "post_date": "2025-07-20T15:43:39.477000", "content": "Has the current test set been updated?
", "votes": 0, "replies": [ { "id": 3251378, "author_name": "Horikita Saku", "author_url": "", "post_date": "2025-07-20T16:00:02.923000", "content": "yes
\nit has been updated
Thank you. No wonder I just found that the same code has different results.
", "votes": 0, "replies": [ { "id": 3250870, "author_name": "Horikita Saku", "author_url": "", "post_date": "2025-07-19T10:54:32.807000", "content": "In addition, regardless of what has been submitted, the new test set will be used in the current new submission, right?
", "votes": 0, "replies": [ { "id": 3250924, "author_name": "Horikita Saku", "author_url": "", "post_date": "2025-07-19T13:42:35.197000", "content": "I can run normally on the new training set, but failed on the test set …
", "votes": 0, "replies": [] }, { "id": 3251029, "author_name": "Sohier Dane", "author_url": "", "post_date": "2025-07-19T18:12:36.647000", "content": "@horikitasaku I can confirm that there have been successful submissions using the updated dataset. However, I'm going to hold off on rerunning older submissions on the new dataset until I've had a chance to look into some of the new submission errors you and other competitors are seeing in more detail. I can confirm that some of them are issues that reasonably occur with new data (such as out of memory or GPU memory) but I want to make sure there's nothing else going on.
", "votes": 1, "replies": [] }, { "id": 3251054, "author_name": "Devin Anzelmo", "author_url": "", "post_date": "2025-07-19T20:33:28.053000", "content": "@sohier There are several examples in the train where data starts at the end of ingress(or there is no ingress). Code that ran fine on old data now errors. I am sure this is happening to a lot of other solutions as well. Not sure if this is an issue or not, we can adapt code to handle it pretty easily.
", "votes": 3, "replies": [] }, { "id": 3251187, "author_name": "Viji", "author_url": "", "post_date": "2025-07-20T06:22:57.193000", "content": "@sohier - I am able to resubmit old submission without error and with lower score. Not yet tried training with new dataset.
", "votes": 0, "replies": [] }, { "id": 3251271, "author_name": "Horikita Saku", "author_url": "", "post_date": "2025-07-20T09:46:59.650000", "content": "thanks guys
\nI repaired the detection of ingress and egress and successfully submitted it.
\n@sohier @devinanzelmo @viji1609