{
  "id": 431836,
  "title": "How do load dcm files from test folder",
  "url": "/competitions/rsna-2023-abdominal-trauma-detection/discussion/431836",
  "author_name": "Chau YH",
  "post_date": "2023-08-15T06:17:44.088000",
  "votes": 6,
  "comment_count": 0,
  "views": 0,
  "content": "<p>This is <strong>not</strong> a question about how do convert .dcm files to images, as it is already addressed in <a href=\"https://www.kaggle.com/theoviel\" target=\"_blank\">@theoviel</a>'s <a href=\"https://www.kaggle.com/competitions/rsna-2023-abdominal-trauma-detection/discussion/427427\" target=\"_blank\">notebook</a> and <a href=\"https://www.kaggle.com/competitions/rsna-2023-abdominal-trauma-detection/discussion/427217\" target=\"_blank\">huiminglin's discussion</a>.</p>\n<p>I am having difficulties in loading the dcm files in the <em>hidden test set</em> for my submission, and I could not find out the problem after a few days of probing into the test set. I think I have narrowed down the problem, but still can't find out exactly what the problem is:</p>\n<h1>Whether all files in the folder are .dcm files (yes)</h1>\n<p><a href=\"https://www.kaggle.com/code/chauyh/rsna-files-ends-with-dcm-success\" target=\"_blank\">https://www.kaggle.com/code/chauyh/rsna-files-ends-with-dcm-success</a></p>\n<h1>Whether it is possible to load pixel_array from each .dcm (no)</h1>\n<p><a href=\"https://www.kaggle.com/code/chauyh/rsna-pixel-array-loading-fail\" target=\"_blank\">https://www.kaggle.com/code/chauyh/rsna-pixel-array-loading-fail</a><br>\nRuns for ~20mins and crashes when submitting.</p>\n<h1>Whether CT images in folders that only have one .dcm file must be 3D (yes)</h1>\n<p><a href=\"https://www.kaggle.com/code/chauyh/rsna-shape-3d-success\" target=\"_blank\">https://www.kaggle.com/code/chauyh/rsna-shape-3d-success</a></p>\n<p>If I correctly recall, all the dcm files in the training set are 2D. I have successfully stacked them together to create a 3D npy array for each series in the training set.</p>\n<p>This suggests that the format of the .dcm files in the training set differ from those of the test set. Of course the result may be vacuous as there might not be any folders with only one .dcm file, but there are 3 folders with only 1 .dcm file each in the visible test set folder.</p>\n<p>I asked GPT and found out that there's two types of CT images, namely CT Image Storage and Enhanced CT Image Storage, the former stores a 2D slice in each .dcm, while the latter stores a 3D volume in a single .dcm. Not sure if hallucinating or not.</p>\n<h1>Whether the SOP Class UID belongs to CT images (no)</h1>\n<p><a href=\"https://www.kaggle.com/code/chauyh/rsna-ct-types-check-fail\" target=\"_blank\">https://www.kaggle.com/code/chauyh/rsna-ct-types-check-fail</a></p>\n<p>Lookup for SOP Class UID: <a href=\"https://dicom.nema.org/dicom/2013/output/chtml/part04/sect_B.5.html\" target=\"_blank\">https://dicom.nema.org/dicom/2013/output/chtml/part04/sect_B.5.html</a></p>\n<p>I think that the error in my submission code is because of non existent pixel_array in some .dcm files, because their types are not CT scans (SOP Class UID). Or there could be some other error in my code. I would appreciate it if some help could be given.</p>",
  "messages": [
    {
      "id": 2391383,
      "postDate": "2023-08-15T06:17:44.090Z",
      "content": "<p>This is <strong>not</strong> a question about how do convert .dcm files to images, as it is already addressed in <a href=\"https://www.kaggle.com/theoviel\" target=\"_blank\">@theoviel</a>'s <a href=\"https://www.kaggle.com/competitions/rsna-2023-abdominal-trauma-detection/discussion/427427\" target=\"_blank\">notebook</a> and <a href=\"https://www.kaggle.com/competitions/rsna-2023-abdominal-trauma-detection/discussion/427217\" target=\"_blank\">huiminglin's discussion</a>.</p>\n<p>I am having difficulties in loading the dcm files in the <em>hidden test set</em> for my submission, and I could not find out the problem after a few days of probing into the test set. I think I have narrowed down the problem, but still can't find out exactly what the problem is:</p>\n<h1>Whether all files in the folder are .dcm files (yes)</h1>\n<p><a href=\"https://www.kaggle.com/code/chauyh/rsna-files-ends-with-dcm-success\" target=\"_blank\">https://www.kaggle.com/code/chauyh/rsna-files-ends-with-dcm-success</a></p>\n<h1>Whether it is possible to load pixel_array from each .dcm (no)</h1>\n<p><a href=\"https://www.kaggle.com/code/chauyh/rsna-pixel-array-loading-fail\" target=\"_blank\">https://www.kaggle.com/code/chauyh/rsna-pixel-array-loading-fail</a><br>\nRuns for ~20mins and crashes when submitting.</p>\n<h1>Whether CT images in folders that only have one .dcm file must be 3D (yes)</h1>\n<p><a href=\"https://www.kaggle.com/code/chauyh/rsna-shape-3d-success\" target=\"_blank\">https://www.kaggle.com/code/chauyh/rsna-shape-3d-success</a></p>\n<p>If I correctly recall, all the dcm files in the training set are 2D. I have successfully stacked them together to create a 3D npy array for each series in the training set.</p>\n<p>This suggests that the format of the .dcm files in the training set differ from those of the test set. Of course the result may be vacuous as there might not be any folders with only one .dcm file, but there are 3 folders with only 1 .dcm file each in the visible test set folder.</p>\n<p>I asked GPT and found out that there's two types of CT images, namely CT Image Storage and Enhanced CT Image Storage, the former stores a 2D slice in each .dcm, while the latter stores a 3D volume in a single .dcm. Not sure if hallucinating or not.</p>\n<h1>Whether the SOP Class UID belongs to CT images (no)</h1>\n<p><a href=\"https://www.kaggle.com/code/chauyh/rsna-ct-types-check-fail\" target=\"_blank\">https://www.kaggle.com/code/chauyh/rsna-ct-types-check-fail</a></p>\n<p>Lookup for SOP Class UID: <a href=\"https://dicom.nema.org/dicom/2013/output/chtml/part04/sect_B.5.html\" target=\"_blank\">https://dicom.nema.org/dicom/2013/output/chtml/part04/sect_B.5.html</a></p>\n<p>I think that the error in my submission code is because of non existent pixel_array in some .dcm files, because their types are not CT scans (SOP Class UID). Or there could be some other error in my code. I would appreciate it if some help could be given.</p>",
      "rawMarkdown": "This is **not** a question about how do convert .dcm files to images, as it is already addressed in @theoviel's [notebook](https://www.kaggle.com/competitions/rsna-2023-abdominal-trauma-detection/discussion/427427) and [huiminglin's discussion](https://www.kaggle.com/competitions/rsna-2023-abdominal-trauma-detection/discussion/427217).\n\nI am having difficulties in loading the dcm files in the *hidden test set* for my submission, and I could not find out the problem after a few days of probing into the test set. I think I have narrowed down the problem, but still can't find out exactly what the problem is:\n\n# Whether all files in the folder are .dcm files (yes)\nhttps://www.kaggle.com/code/chauyh/rsna-files-ends-with-dcm-success\n\n# Whether it is possible to load pixel_array from each .dcm (no)\nhttps://www.kaggle.com/code/chauyh/rsna-pixel-array-loading-fail\nRuns for ~20mins and crashes when submitting.\n\n# Whether CT images in folders that only have one .dcm file must be 3D (yes)\nhttps://www.kaggle.com/code/chauyh/rsna-shape-3d-success\n\nIf I correctly recall, all the dcm files in the training set are 2D. I have successfully stacked them together to create a 3D npy array for each series in the training set.\n\nThis suggests that the format of the .dcm files in the training set differ from those of the test set. Of course the result may be vacuous as there might not be any folders with only one .dcm file, but there are 3 folders with only 1 .dcm file each in the visible test set folder.\n\nI asked GPT and found out that there's two types of CT images, namely CT Image Storage and Enhanced CT Image Storage, the former stores a 2D slice in each .dcm, while the latter stores a 3D volume in a single .dcm. Not sure if hallucinating or not.\n\n# Whether the SOP Class UID belongs to CT images (no)\nhttps://www.kaggle.com/code/chauyh/rsna-ct-types-check-fail\n\nLookup for SOP Class UID: https://dicom.nema.org/dicom/2013/output/chtml/part04/sect_B.5.html\n\nI think that the error in my submission code is because of non existent pixel_array in some .dcm files, because their types are not CT scans (SOP Class UID). Or there could be some other error in my code. I would appreciate it if some help could be given.",
      "votes": 5
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "2391383": "This is **not** a question about how do convert .dcm files to images, as it is already addressed in @theoviel's [notebook](https://www.kaggle.com/competitions/rsna-2023-abdominal-trauma-detection/discussion/427427) and [huiminglin's discussion](https://www.kaggle.com/competitions/rsna-2023-abdominal-trauma-detection/discussion/427217).\n\nI am having difficulties in loading the dcm files in the *hidden test set* for my submission, and I could not find out the problem after a few days of probing into the test set. I think I have narrowed down the problem, but still can't find out exactly what the problem is:\n\n# Whether all files in the folder are .dcm files (yes)\nhttps://www.kaggle.com/code/chauyh/rsna-files-ends-with-dcm-success\n\n# Whether it is possible to load pixel_array from each .dcm (no)\nhttps://www.kaggle.com/code/chauyh/rsna-pixel-array-loading-fail\nRuns for ~20mins and crashes when submitting.\n\n# Whether CT images in folders that only have one .dcm file must be 3D (yes)\nhttps://www.kaggle.com/code/chauyh/rsna-shape-3d-success\n\nIf I correctly recall, all the dcm files in the training set are 2D. I have successfully stacked them together to create a 3D npy array for each series in the training set.\n\nThis suggests that the format of the .dcm files in the training set differ from those of the test set. Of course the result may be vacuous as there might not be any folders with only one .dcm file, but there are 3 folders with only 1 .dcm file each in the visible test set folder.\n\nI asked GPT and found out that there's two types of CT images, namely CT Image Storage and Enhanced CT Image Storage, the former stores a 2D slice in each .dcm, while the latter stores a 3D volume in a single .dcm. Not sure if hallucinating or not.\n\n# Whether the SOP Class UID belongs to CT images (no)\nhttps://www.kaggle.com/code/chauyh/rsna-ct-types-check-fail\n\nLookup for SOP Class UID: https://dicom.nema.org/dicom/2013/output/chtml/part04/sect_B.5.html\n\nI think that the error in my submission code is because of non existent pixel_array in some .dcm files, because their types are not CT scans (SOP Class UID). Or there could be some other error in my code. I would appreciate it if some help could be given."
  }
}