{
  "id": 361138,
  "title": "Data loading error",
  "url": "/competitions/rsna-2022-cervical-spine-fracture-detection/discussion/361138",
  "author_name": "Denison Patison",
  "post_date": "2022-10-20T01:05:36.841000",
  "votes": 3,
  "comment_count": 4,
  "views": 0,
  "content": "<p>Hello! Please check the mapping between the rows test.csv and folder names. <br>\nAre there any empty directories?<br>\nI am getting a read error via SimpleITK. <br>\nPreviously, I only got this error on the test due to incorrect mapping, but now I always get it.</p>\n<p>I use this:</p>\n<pre><code>KAGGLE_DATA = r'../input/rsna-2022-cervical-spine-fracture-detection'\ntest_df = read_csv(join(KAGGLE_DATA, \"test.csv\"))\ntest_img_path = join(KAGGLE_DATA, \"test_images\")\n\n....\n\nclass RSNADataset:\n    def load_data(self):\n        return self.data_path\n\n    def __init__(self, csv_data, img_path, transform=None, target_cols=None, device='cpu'):\n        self.target_cols = target_cols\n        self.img_path = img_path\n        self.device = device\n        self.dataset = csv_data\n        self.transform = transform\n        self.reader = sitk.ImageSeriesReader()\n        self.client_ids = self.dataset['StudyInstanceUID'].tolist()\n\n    def __len__(self):\n        return self.dataset.shape[0]\n\n    def __getitem__(self, index):\n        self.reader.SetFileNames(self.reader.GetGDCMSeriesFileNames(join(self.img_path, self.client_ids[index])))\n        client_imgs = sitk.GetArrayFromImage(\n            sitk.Cast(\n                sitk.RescaleIntensity(self.reader.Execute(), 0, 255),\n                sitk.sitkUInt8\n            )\n        )\n        client_imgs = Tensor(client_imgs).to(self.device)\n        return self.transform(client_imgs), self.dataset.iloc[index, :].to_dict()\n</code></pre>\n<p>Take error from test sample:<br>\nWARNING: In /tmp/SimpleITK-build/ITK/Modules/IO/GDCM/src/itkGDCMSeriesFileNames.cxx, line 67<br>\nGDCMSeriesFileNames (0x55ea6b354280): ../input/rsna-2022-cervical-spine-fracture-detection/test_images/1.2.826.0.1.3680043.10197 is not a directory</p>\n<p>WARNING: In /tmp/SimpleITK-build/ITK/Modules/IO/GDCM/src/itkGDCMSeriesFileNames.cxx, line 113<br>\nGDCMSeriesFileNames (0x55ea6b354280): No Series can be found, make sure your restrictions are not too strong</p>",
  "messages": [
    {
      "id": 1995954,
      "postDate": "2022-10-20T01:05:36.843Z",
      "content": "<p>Hello! Please check the mapping between the rows test.csv and folder names. <br>\nAre there any empty directories?<br>\nI am getting a read error via SimpleITK. <br>\nPreviously, I only got this error on the test due to incorrect mapping, but now I always get it.</p>\n<p>I use this:</p>\n<pre><code>KAGGLE_DATA = r'../input/rsna-2022-cervical-spine-fracture-detection'\ntest_df = read_csv(join(KAGGLE_DATA, \"test.csv\"))\ntest_img_path = join(KAGGLE_DATA, \"test_images\")\n\n....\n\nclass RSNADataset:\n    def load_data(self):\n        return self.data_path\n\n    def __init__(self, csv_data, img_path, transform=None, target_cols=None, device='cpu'):\n        self.target_cols = target_cols\n        self.img_path = img_path\n        self.device = device\n        self.dataset = csv_data\n        self.transform = transform\n        self.reader = sitk.ImageSeriesReader()\n        self.client_ids = self.dataset['StudyInstanceUID'].tolist()\n\n    def __len__(self):\n        return self.dataset.shape[0]\n\n    def __getitem__(self, index):\n        self.reader.SetFileNames(self.reader.GetGDCMSeriesFileNames(join(self.img_path, self.client_ids[index])))\n        client_imgs = sitk.GetArrayFromImage(\n            sitk.Cast(\n                sitk.RescaleIntensity(self.reader.Execute(), 0, 255),\n                sitk.sitkUInt8\n            )\n        )\n        client_imgs = Tensor(client_imgs).to(self.device)\n        return self.transform(client_imgs), self.dataset.iloc[index, :].to_dict()\n</code></pre>\n<p>Take error from test sample:<br>\nWARNING: In /tmp/SimpleITK-build/ITK/Modules/IO/GDCM/src/itkGDCMSeriesFileNames.cxx, line 67<br>\nGDCMSeriesFileNames (0x55ea6b354280): ../input/rsna-2022-cervical-spine-fracture-detection/test_images/1.2.826.0.1.3680043.10197 is not a directory</p>\n<p>WARNING: In /tmp/SimpleITK-build/ITK/Modules/IO/GDCM/src/itkGDCMSeriesFileNames.cxx, line 113<br>\nGDCMSeriesFileNames (0x55ea6b354280): No Series can be found, make sure your restrictions are not too strong</p>",
      "rawMarkdown": "Hello! Please check the mapping between the rows test.csv and folder names. \nAre there any empty directories?\nI am getting a read error via SimpleITK. \nPreviously, I only got this error on the test due to incorrect mapping, but now I always get it.\n\nI use this:\n\n```\nKAGGLE_DATA = r'../input/rsna-2022-cervical-spine-fracture-detection'\ntest_df = read_csv(join(KAGGLE_DATA, \"test.csv\"))\ntest_img_path = join(KAGGLE_DATA, \"test_images\")\n\n....\n\nclass RSNADataset:\n    def load_data(self):\n        return self.data_path\n\n    def __init__(self, csv_data, img_path, transform=None, target_cols=None, device='cpu'):\n        self.target_cols = target_cols\n        self.img_path = img_path\n        self.device = device\n        self.dataset = csv_data\n        self.transform = transform\n        self.reader = sitk.ImageSeriesReader()\n        self.client_ids = self.dataset['StudyInstanceUID'].tolist()\n\n    def __len__(self):\n        return self.dataset.shape[0]\n\n    def __getitem__(self, index):\n        self.reader.SetFileNames(self.reader.GetGDCMSeriesFileNames(join(self.img_path, self.client_ids[index])))\n        client_imgs = sitk.GetArrayFromImage(\n            sitk.Cast(\n                sitk.RescaleIntensity(self.reader.Execute(), 0, 255),\n                sitk.sitkUInt8\n            )\n        )\n        client_imgs = Tensor(client_imgs).to(self.device)\n        return self.transform(client_imgs), self.dataset.iloc[index, :].to_dict()\n```\n\nTake error from test sample:\nWARNING: In /tmp/SimpleITK-build/ITK/Modules/IO/GDCM/src/itkGDCMSeriesFileNames.cxx, line 67\nGDCMSeriesFileNames (0x55ea6b354280): ../input/rsna-2022-cervical-spine-fracture-detection/test_images/1.2.826.0.1.3680043.10197 is not a directory\n\nWARNING: In /tmp/SimpleITK-build/ITK/Modules/IO/GDCM/src/itkGDCMSeriesFileNames.cxx, line 113\nGDCMSeriesFileNames (0x55ea6b354280): No Series can be found, make sure your restrictions are not too strong",
      "votes": 3
    },
    {
      "id": 1996808,
      "postDate": "2022-10-20T12:40:02.850Z",
      "content": "<p>do not use <code>test.csv</code><br>\ntry using glob to scan the test folder.</p>",
      "rawMarkdown": "do not use `test.csv`\ntry using glob to scan the test folder.",
      "votes": 1
    },
    {
      "id": 1999552,
      "postDate": "2022-10-22T12:54:51.463Z",
      "content": "<p>Use something like:<br>\n<code>os.listdir('../input/rsna-2022-cervical-spine-fracture-detection/test_images')</code><br>\nto get list of StudyInstanceUIDs.</p>\n<p>To get paths of image files of a study with a set StudyInstanceUID, <em>StudyInstanceUID_i</em>, use:<br>\n<code>glob.glob(f'../input/rsna-2022-cervical-spine-fracture-detection/test_images/{StudyInstanceUID_i}/*.dcm')</code></p>",
      "rawMarkdown": "Use something like:\n`os.listdir('../input/rsna-2022-cervical-spine-fracture-detection/test_images')`\nto get list of StudyInstanceUIDs.\n\nTo get paths of image files of a study with a set StudyInstanceUID, *StudyInstanceUID_i*, use:\n`glob.glob(f'../input/rsna-2022-cervical-spine-fracture-detection/test_images/{StudyInstanceUID_i}/*.dcm')`"
    },
    {
      "id": 1997610,
      "postDate": "2022-10-21T03:38:09.560Z",
      "content": "<p><a href=\"https://www.kaggle.com/haqishen\" target=\"_blank\">@haqishen</a> thank you very much, no error is thrown</p>",
      "rawMarkdown": "@haqishen thank you very much, no error is thrown"
    },
    {
      "id": 1995985,
      "postDate": "2022-10-20T01:47:57.147Z",
      "content": "<p><a href=\"https://www.kaggle.com/sohier\" target=\"_blank\">@sohier</a> please, need you help</p>",
      "rawMarkdown": "@sohier please, need you help"
    }
  ],
  "comments": [
    {
      "id": 1996808,
      "author_name": "Qishen Ha",
      "author_url": "",
      "post_date": "2022-10-20T12:40:02.850000",
      "content": "<p>do not use <code>test.csv</code><br>\ntry using glob to scan the test folder.</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1999552,
      "author_name": "pushkin05",
      "author_url": "",
      "post_date": "2022-10-22T12:54:51.463000",
      "content": "<p>Use something like:<br>\n<code>os.listdir('../input/rsna-2022-cervical-spine-fracture-detection/test_images')</code><br>\nto get list of StudyInstanceUIDs.</p>\n<p>To get paths of image files of a study with a set StudyInstanceUID, <em>StudyInstanceUID_i</em>, use:<br>\n<code>glob.glob(f'../input/rsna-2022-cervical-spine-fracture-detection/test_images/{StudyInstanceUID_i}/*.dcm')</code></p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1997610,
      "author_name": "Denison Patison",
      "author_url": "",
      "post_date": "2022-10-21T03:38:09.560000",
      "content": "<p><a href=\"https://www.kaggle.com/haqishen\" target=\"_blank\">@haqishen</a> thank you very much, no error is thrown</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 1995985,
      "author_name": "Denison Patison",
      "author_url": "",
      "post_date": "2022-10-20T01:47:57.147000",
      "content": "<p><a href=\"https://www.kaggle.com/sohier\" target=\"_blank\">@sohier</a> please, need you help</p>",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1995954": "Hello! Please check the mapping between the rows test.csv and folder names. \nAre there any empty directories?\nI am getting a read error via SimpleITK. \nPreviously, I only got this error on the test due to incorrect mapping, but now I always get it.\n\nI use this:\n\n```\nKAGGLE_DATA = r'../input/rsna-2022-cervical-spine-fracture-detection'\ntest_df = read_csv(join(KAGGLE_DATA, \"test.csv\"))\ntest_img_path = join(KAGGLE_DATA, \"test_images\")\n\n....\n\nclass RSNADataset:\n    def load_data(self):\n        return self.data_path\n\n    def __init__(self, csv_data, img_path, transform=None, target_cols=None, device='cpu'):\n        self.target_cols = target_cols\n        self.img_path = img_path\n        self.device = device\n        self.dataset = csv_data\n        self.transform = transform\n        self.reader = sitk.ImageSeriesReader()\n        self.client_ids = self.dataset['StudyInstanceUID'].tolist()\n\n    def __len__(self):\n        return self.dataset.shape[0]\n\n    def __getitem__(self, index):\n        self.reader.SetFileNames(self.reader.GetGDCMSeriesFileNames(join(self.img_path, self.client_ids[index])))\n        client_imgs = sitk.GetArrayFromImage(\n            sitk.Cast(\n                sitk.RescaleIntensity(self.reader.Execute(), 0, 255),\n                sitk.sitkUInt8\n            )\n        )\n        client_imgs = Tensor(client_imgs).to(self.device)\n        return self.transform(client_imgs), self.dataset.iloc[index, :].to_dict()\n```\n\nTake error from test sample:\nWARNING: In /tmp/SimpleITK-build/ITK/Modules/IO/GDCM/src/itkGDCMSeriesFileNames.cxx, line 67\nGDCMSeriesFileNames (0x55ea6b354280): ../input/rsna-2022-cervical-spine-fracture-detection/test_images/1.2.826.0.1.3680043.10197 is not a directory\n\nWARNING: In /tmp/SimpleITK-build/ITK/Modules/IO/GDCM/src/itkGDCMSeriesFileNames.cxx, line 113\nGDCMSeriesFileNames (0x55ea6b354280): No Series can be found, make sure your restrictions are not too strong",
    "1996808": "do not use `test.csv`\ntry using glob to scan the test folder.",
    "1999552": "Use something like:\n`os.listdir('../input/rsna-2022-cervical-spine-fracture-detection/test_images')`\nto get list of StudyInstanceUIDs.\n\nTo get paths of image files of a study with a set StudyInstanceUID, *StudyInstanceUID_i*, use:\n`glob.glob(f'../input/rsna-2022-cervical-spine-fracture-detection/test_images/{StudyInstanceUID_i}/*.dcm')`",
    "1997610": "@haqishen thank you very much, no error is thrown",
    "1995985": "@sohier please, need you help"
  }
}