I know many of the submissions on the LB have try except in their code and that may cause some potential data issue to be ignored. The fix of the public LB score does not means all the issue is clear.
\nAs mentioned in title, I have wrote a custom io tool to deal with dicom files, which works well on all of the train series. But I get exception in submission after about 7 minutes….Which makes me feel that there is still some issues in the testset(Maybe not, but I have checked my code many times to figure out potential bug).
\nThe logic of my is tool is as follows:
\nConsidering the code worked for all the train series, obviously there is a data pattern which only occurs in the hidden testset. So here I have some questions hoping that the host may answer to help find out the potential outlier. @ryanholbrook @evancalabrese
\nforce=True in demo-submission.Personally, I suspect that the issue is on multiframe dicom, the logic of 2D dicom is pretty simple and only access to metadata like ImageOrientationPatient, ImagePositionPatient.
\nI appreciate for the work of kaggle and hope that the hidden testset can be proved good without a try exception workaround. If neccesary, I am glad to provide my io tool privately to help figure out the problem.
\nUpdate: The error arises from multiframe dicom in hidden testset. After I comment out the if route for multiframe, the \"Notebook threw Exception\" disappear. Chances are that it is a metadata issue. It is too frustrating to debug by comment out and submit…. Hoping the host to clarify the question 1 above, thanks again.
\nUpdate2: The error comes from dicom which lacks \"PerFrameFunctionalGroupsSequence\" Tag. We cannot identify the Position of each slice without this tag and can only have a workaround with SpacingBetweenSlices(if exists). Also Orientation cannot be identified if some other tags do not exist. Considering all the train series do not have this problem, I guess that there are some series from another site which never appear in train series. But, is this expected? I think it is better not to have an unknown dicom format which never occur in train set to appear in test set, at least for a competition.
\nUpdate3: The error still occurs after the update. Here are the possible reasons according to my code.
\nI know many of the submissions on the LB have try except in their code and that may cause some potential data issue to be ignored. The fix of the public LB score does not means all the issue is clear.
\nAs mentioned in title, I have wrote a custom io tool to deal with dicom files, which works well on all of the train series. But I get exception in submission after about 7 minutes….Which makes me feel that there is still some issues in the testset(Maybe not, but I have checked my code many times to figure out potential bug).
\nThe logic of my is tool is as follows:
\nConsidering the code worked for all the train series, obviously there is a data pattern which only occurs in the hidden testset. So here I have some questions hoping that the host may answer to help find out the potential outlier. @ryanholbrook @evancalabrese
\nforce=True in demo-submission.Personally, I suspect that the issue is on multiframe dicom, the logic of 2D dicom is pretty simple and only access to metadata like ImageOrientationPatient, ImagePositionPatient.
\nI appreciate for the work of kaggle and hope that the hidden testset can be proved good without a try exception workaround. If neccesary, I am glad to provide my io tool privately to help figure out the problem.
\nUpdate: The error arises from multiframe dicom in hidden testset. After I comment out the if route for multiframe, the \"Notebook threw Exception\" disappear. Chances are that it is a metadata issue. It is too frustrating to debug by comment out and submit…. Hoping the host to clarify the question 1 above, thanks again.
\nUpdate2: The error comes from dicom which lacks \"PerFrameFunctionalGroupsSequence\" Tag. We cannot identify the Position of each slice without this tag and can only have a workaround with SpacingBetweenSlices(if exists). Also Orientation cannot be identified if some other tags do not exist. Considering all the train series do not have this problem, I guess that there are some series from another site which never appear in train series. But, is this expected? I think it is better not to have an unknown dicom format which never occur in train set to appear in test set, at least for a competition.
\nUpdate3: The error still occurs after the update. Here are the possible reasons according to my code.
\nThanks so much for your detailed analysis and updates! We will take a closer look to see if indeed there is a test-set specific DICOM header pattern as you have suggested. In the short term, hopefully you can skip this case, which should not contribute meaningfully to your score. We will get back to you once we have reviewed.
", "rawMarkdown": "Thanks so much for your detailed analysis and updates! We will take a closer look to see if indeed there is a test-set specific DICOM header pattern as you have suggested. In the short term, hopefully you can skip this case, which should not contribute meaningfully to your score. We will get back to you once we have reviewed.", "votes": 3, "replies": [ { "id": 3281444, "postDate": "2025-09-04T10:01:26.323Z", "content": "Thanks!
\n\n\nwhich should not contribute meaningfully to your score
\n
Do you mean that multiframe dicoms are assumed to have \"PerFrameFunctionalGroupsSequence\" Tag and the case which hit the error is an outlier? Or do you mean that there are not many multiframe dicoms in testest?
\nMy concern is that in the train set, there are around 330 multiframe dicoms, which is 9% of the all. If the distribution is the same in testset and all the multiframe dicoms lack \"PerFrameFunctionalGroupsSequence\" Tag, ignoring all the multiframe dicoms will definitely damage the score.
", "rawMarkdown": "Thanks!\n>which should not contribute meaningfully to your score\n\nDo you mean that multiframe dicoms are assumed to have \"PerFrameFunctionalGroupsSequence\" Tag and the case which hit the error is an outlier? Or do you mean that there are not many multiframe dicoms in testest? \n\nMy concern is that in the train set, there are around 330 multiframe dicoms, which is 9% of the all. If the distribution is the same in testset and all the multiframe dicoms lack \"PerFrameFunctionalGroupsSequence\" Tag, ignoring all the multiframe dicoms will definitely damage the score.", "votes": 3 }, { "id": 3294253, "postDate": "2025-09-25T17:04:03.743Z", "content": "Hi! Is the issue in test set fixed?
", "rawMarkdown": "Hi! Is the issue in test set fixed?", "votes": 1, "isDeleted": true } ] }, { "id": 3294616, "postDate": "2025-09-26T12:36:44.153Z", "content": "Thanks for the detailed update @honglihang. I am assuming you are able to process all dicoms without error now?
", "rawMarkdown": "Thanks for the detailed update @honglihang. I am assuming you are able to process all dicoms without error now?", "replies": [ { "id": 3295055, "postDate": "2025-09-27T14:23:52.837Z", "content": "Hi, to tell the truth I am not sure enough. I finally added try exception to set the default value for the missing tag. After the test set update, I have not yet test the code without try exception. I have revert the code to that version and made a submission today, so I will update whether the code passed. Please wait.
", "rawMarkdown": "Hi, to tell the truth I am not sure enough. I finally added try exception to set the default value for the missing tag. After the test set update, I have not yet test the code without try exception. I have revert the code to that version and made a submission today, so I will update whether the code passed. Please wait.", "votes": 2 }, { "id": 3295065, "postDate": "2025-09-27T15:00:32.293Z", "content": "Hi, my submission failed after 13min with original code without try exception. Seems that still there are some information missing in dicom metadata. I have updated the discussion.
\n@evancalabrese
Thanks so much for your detailed analysis and updates! We will take a closer look to see if indeed there is a test-set specific DICOM header pattern as you have suggested. In the short term, hopefully you can skip this case, which should not contribute meaningfully to your score. We will get back to you once we have reviewed.
", "votes": 3, "replies": [ { "id": 3281444, "author_name": "RihanPiggy", "author_url": "", "post_date": "2025-09-04T10:01:26.323000", "content": "Thanks!
\n\n\nwhich should not contribute meaningfully to your score
\n
Do you mean that multiframe dicoms are assumed to have \"PerFrameFunctionalGroupsSequence\" Tag and the case which hit the error is an outlier? Or do you mean that there are not many multiframe dicoms in testest?
\nMy concern is that in the train set, there are around 330 multiframe dicoms, which is 9% of the all. If the distribution is the same in testset and all the multiframe dicoms lack \"PerFrameFunctionalGroupsSequence\" Tag, ignoring all the multiframe dicoms will definitely damage the score.
", "votes": 3, "replies": [] }, { "id": 3294253, "author_name": "", "author_url": "", "post_date": "2025-09-25T17:04:03.743000", "content": "Hi! Is the issue in test set fixed?
", "votes": 1, "replies": [] } ] }, { "id": 3294616, "author_name": "Bartley", "author_url": "", "post_date": "2025-09-26T12:36:44.153000", "content": "Thanks for the detailed update @honglihang. I am assuming you are able to process all dicoms without error now?
", "votes": 0, "replies": [ { "id": 3295055, "author_name": "RihanPiggy", "author_url": "", "post_date": "2025-09-27T14:23:52.837000", "content": "Hi, to tell the truth I am not sure enough. I finally added try exception to set the default value for the missing tag. After the test set update, I have not yet test the code without try exception. I have revert the code to that version and made a submission today, so I will update whether the code passed. Please wait.
", "votes": 2, "replies": [] }, { "id": 3295065, "author_name": "RihanPiggy", "author_url": "", "post_date": "2025-09-27T15:00:32.293000", "content": "Hi, my submission failed after 13min with original code without try exception. Seems that still there are some information missing in dicom metadata. I have updated the discussion.
\n@evancalabrese