\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F11676771%2F33288e8f45188ec7a47666acf91b7183%2F963cfb4e-514c-4c68-bbb3-5f389dc7812d.png?generation=1753227908749515&alt=media\")
I noticed that those successful re-runs and repeated commits were set to invalid. But are there still past submissions that are valid, and are all the current scores using the new dataset?
\n
At the same time I noticed that some scores on the LB remained completely unchanged. In my opinion, the probability of getting exactly the same score on different datasets is very small. So, are all the scores now new ones?
\nDo others still have past scores which are valid?
At least for me old submissions have not been disabled yet. My score on new data for example is 0.407.
", "rawMarkdown": "At least for me old submissions have not been disabled yet. My score on new data for example is 0.407. ", "votes": 3, "replies": [ { "id": 3252647, "postDate": "2025-07-23T06:11:10.743Z", "content": "Thats wired
\nMy score of 0.433 was set as invalid
\nI think the staff still haven't adjusted the score properly
mine all turned red this night except one
", "rawMarkdown": "mine all turned red this night except one", "votes": 1, "replies": [ { "id": 3252655, "postDate": "2025-07-23T06:28:54.400Z", "content": "mine currently are the new score on the new data.
\nI guess your current score should be the rerun of the baseline?
\nfrom 0.328โ0.287
yes, that is the only green submission from me now: https://www.kaggle.com/code/ilu000/ariel25-baseline-submission-1d-modelling?scriptVersionId=249429083
\nfunny that it is the notebook that I made public.
And there is another entry of that one in my list, which failed
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2675447%2F319f0a8fccb35e5efcca470d53a80f03%2FBildschirmfoto%20vom%202025-07-23%2009-00-22.png?generation=1753254075442372&alt=media\")
really a mess ๐
", "rawMarkdown": "really a mess ๐", "votes": 2 } ] }, { "id": 3252872, "postDate": "2025-07-23T15:35:31.360Z", "content": "@christofhenkel Thanks for flagging this. The next steps for investigating and remediating any remaining issues will take some time; unfortunately I don't expect to have any updates today.
", "rawMarkdown": "@christofhenkel Thanks for flagging this. The next steps for investigating and remediating any remaining issues will take some time; unfortunately I don't expect to have any updates today.", "votes": 1, "replies": [ { "id": 3252966, "postDate": "2025-07-23T19:11:31.550Z", "content": "It seems the LB is now updated with scores form after the dataset update. This is great.
", "rawMarkdown": "It seems the LB is now updated with scores form after the dataset update. This is great." }, { "id": 3252977, "postDate": "2025-07-23T19:48:48.547Z", "content": "yes looks great
", "rawMarkdown": "yes looks great" } ] } ] }, { "id": 3252522, "postDate": "2025-07-22T23:45:57.350Z", "content": "I noticed that those successful re-runs and repeated commits were set to invalid. But are there still past submissions that are valid, and are all the current scores using the new dataset?
\n
At the same time I noticed that some scores on the LB remained completely unchanged. In my opinion, the probability of getting exactly the same score on different datasets is very small. So, are all the scores now new ones?
\nDo others still have past scores which are valid?
27 July, old scores must be removed in Public LB. My Public LB 0.303 โ 0.229
", "rawMarkdown": "27 July, old scores must be removed in Public LB. My Public LB 0.303 โ 0.229" }, { "id": 3252524, "postDate": "2025-07-23T00:06:59.690Z", "content": "The most obvious scores are those 0.331 0.292 and 0.328 from the excellent public notebook.
\nhttps://www.kaggle.com/code/vitalykudelya/neurips-non-ml-transit-curve-fitting?scriptVersionId=250419499
\nhttps://www.kaggle.com/code/vitalykudelya/neurips-non-ml-transit-curve-fitting?scriptVersionId=249420717
\nhttps://www.kaggle.com/code/ilu000/ariel25-baseline-submission-1d-modelling
These scores remained completely still on the LB. Maybe there are others besides these.
\nSince I haven't submitted the exact same code as these notebooks, I'm not sure what the theoretical scores should be. Does anyone know?
Perhaps the safest way is for us to resubmit all the code on the new datasetโฆ
", "rawMarkdown": "The most obvious scores are those 0.331 0.292 and 0.328 from the excellent public notebook.\nhttps://www.kaggle.com/code/vitalykudelya/neurips-non-ml-transit-curve-fitting?scriptVersionId=250419499\nhttps://www.kaggle.com/code/vitalykudelya/neurips-non-ml-transit-curve-fitting?scriptVersionId=249420717\nhttps://www.kaggle.com/code/ilu000/ariel25-baseline-submission-1d-modelling\n\nThese scores remained completely still on the LB. Maybe there are others besides these.\nSince I haven't submitted the exact same code as these notebooks, I'm not sure what the theoretical scores should be. Does anyone know?\n\nPerhaps the safest way is for us to resubmit all the code on the new dataset...", "replies": [ { "id": 3252546, "postDate": "2025-07-23T01:32:18.257Z", "content": "Hey there! I was one of the folks who was trying to play around with the excellent public notebook by Vitaly with a score of 0.331. Currently my leaderboard score remains at 0.331 even though the new rerun score (which is the invalid rerun score) is much lower at 0.255. This is not displayed for the leaderboard though. Funnily enough some changes I made to this code that lowered leaderboard score on the old data actually now have resulted in increases to the score on the new data, huh.
", "rawMarkdown": "Hey there! I was one of the folks who was trying to play around with the excellent public notebook by Vitaly with a score of 0.331. Currently my leaderboard score remains at 0.331 even though the new rerun score (which is the invalid rerun score) is much lower at 0.255. This is not displayed for the leaderboard though. Funnily enough some changes I made to this code that lowered leaderboard score on the old data actually now have resulted in increases to the score on the new data, huh.", "votes": 2, "replies": [ { "id": 3252569, "postDate": "2025-07-23T02:50:32Z", "content": "Hi! thanks for your information
\nso in that case, I guess LB just didn't refresh
No wonder I found it strange that the code for 0.331 didn't change at all.
", "rawMarkdown": "No wonder I found it strange that the code for 0.331 didn't change at all.", "votes": 1 } ] } ] } ], "comments": [ { "id": 3252645, "author_name": "Dieter", "author_url": "", "post_date": "2025-07-23T06:07:48.800000", "content": "At least for me old submissions have not been disabled yet. My score on new data for example is 0.407.
", "votes": 3, "replies": [ { "id": 3252647, "author_name": "Horikita Saku", "author_url": "", "post_date": "2025-07-23T06:11:10.743000", "content": "Thats wired
\nMy score of 0.433 was set as invalid
\nI think the staff still haven't adjusted the score properly
mine all turned red this night except one
", "votes": 1, "replies": [ { "id": 3252655, "author_name": "Horikita Saku", "author_url": "", "post_date": "2025-07-23T06:28:54.400000", "content": "mine currently are the new score on the new data.
\nI guess your current score should be the rerun of the baseline?
\nfrom 0.328โ0.287
yes, that is the only green submission from me now: https://www.kaggle.com/code/ilu000/ariel25-baseline-submission-1d-modelling?scriptVersionId=249429083
\nfunny that it is the notebook that I made public.
And there is another entry of that one in my list, which failed
\n
really a mess ๐
", "votes": 2, "replies": [] } ] }, { "id": 3252872, "author_name": "Sohier Dane", "author_url": "", "post_date": "2025-07-23T15:35:31.360000", "content": "@christofhenkel Thanks for flagging this. The next steps for investigating and remediating any remaining issues will take some time; unfortunately I don't expect to have any updates today.
", "votes": 1, "replies": [ { "id": 3252966, "author_name": "CPMP", "author_url": "", "post_date": "2025-07-23T19:11:31.550000", "content": "It seems the LB is now updated with scores form after the dataset update. This is great.
", "votes": 0, "replies": [] }, { "id": 3252977, "author_name": "Horikita Saku", "author_url": "", "post_date": "2025-07-23T19:48:48.547000", "content": "yes looks great
", "votes": 0, "replies": [] } ] } ] }, { "id": 3254644, "author_name": "๐ข Jun Koda", "author_url": "", "post_date": "2025-07-26T23:52:23.673000", "content": "27 July, old scores must be removed in Public LB. My Public LB 0.303 โ 0.229
", "votes": 0, "replies": [] }, { "id": 3252524, "author_name": "Horikita Saku", "author_url": "", "post_date": "2025-07-23T00:06:59.690000", "content": "The most obvious scores are those 0.331 0.292 and 0.328 from the excellent public notebook.
\nhttps://www.kaggle.com/code/vitalykudelya/neurips-non-ml-transit-curve-fitting?scriptVersionId=250419499
\nhttps://www.kaggle.com/code/vitalykudelya/neurips-non-ml-transit-curve-fitting?scriptVersionId=249420717
\nhttps://www.kaggle.com/code/ilu000/ariel25-baseline-submission-1d-modelling
These scores remained completely still on the LB. Maybe there are others besides these.
\nSince I haven't submitted the exact same code as these notebooks, I'm not sure what the theoretical scores should be. Does anyone know?
Perhaps the safest way is for us to resubmit all the code on the new datasetโฆ
", "votes": 0, "replies": [ { "id": 3252546, "author_name": "Noah Stefancik", "author_url": "", "post_date": "2025-07-23T01:32:18.257000", "content": "Hey there! I was one of the folks who was trying to play around with the excellent public notebook by Vitaly with a score of 0.331. Currently my leaderboard score remains at 0.331 even though the new rerun score (which is the invalid rerun score) is much lower at 0.255. This is not displayed for the leaderboard though. Funnily enough some changes I made to this code that lowered leaderboard score on the old data actually now have resulted in increases to the score on the new data, huh.
", "votes": 2, "replies": [ { "id": 3252569, "author_name": "Horikita Saku", "author_url": "", "post_date": "2025-07-23T02:50:32", "content": "Hi! thanks for your information
\nso in that case, I guess LB just didn't refresh
No wonder I found it strange that the code for 0.331 didn't change at all.
", "votes": 1, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "3252645": "At least for me old submissions have not been disabled yet. My score on new data for example is 0.407. ", "3252522": "I noticed that those successful re-runs and repeated commits were set to invalid. But are there still past submissions that are valid, and are all the current scores using the new dataset?\n\n\nAt the same time I noticed that some scores on the LB remained completely unchanged. In my opinion, the probability of getting exactly the same score on different datasets is very small. So, are all the scores now new ones?\nDo others still have past scores which are valid?\n\n@sohier \n", "3254644": "27 July, old scores must be removed in Public LB. My Public LB 0.303 โ 0.229", "3252524": "The most obvious scores are those 0.331 0.292 and 0.328 from the excellent public notebook.\nhttps://www.kaggle.com/code/vitalykudelya/neurips-non-ml-transit-curve-fitting?scriptVersionId=250419499\nhttps://www.kaggle.com/code/vitalykudelya/neurips-non-ml-transit-curve-fitting?scriptVersionId=249420717\nhttps://www.kaggle.com/code/ilu000/ariel25-baseline-submission-1d-modelling\n\nThese scores remained completely still on the LB. Maybe there are others besides these.\nSince I haven't submitted the exact same code as these notebooks, I'm not sure what the theoretical scores should be. Does anyone know?\n\nPerhaps the safest way is for us to resubmit all the code on the new dataset..." } }