Hi there,
\nI'm looking at the training data and have a few questions:
Hi, thank you for your great questions!
\nRegarding the DICOM tags, you are correct.
\nWe are missing some DICOM tags to convert to nifti. I suspect the missing tag is Spacing Between Slices, because the other ones related to spacial resolution are present.
\nI suspect the missing tag will make training 3D models more challenging, but not as much for 2D models. The LB results are good so far, but I suspect they could be even better if we had the missing tag.
\nI will try to get access to the original files to read that tag. Let me know if you miss any other tag.
\nRegarding the transformation in the pixel data, the complete code for facial deidentification is here: https://github.com/kitamura-felipe/face_deid_ct
\nIn a nutshell, the code fills the air with pixels that have values in the range of those of the patient's skin. By doing that, we reduce the chance that the process can be perfectly reverted. In some cases, it removed part of the skull as well. That is a limitation of this method.
\nWe believe that models trained with images using that defacing technique will generalize if we apply the same function to newer studies on which we want to do inference. The results of the LB show that is true so far.
\nThank you for bringing that up!
", "rawMarkdown": "Hi, thank you for your great questions!\n\nRegarding the DICOM tags, you are correct. \n\nWe are missing some DICOM tags to convert to nifti. I suspect the missing tag is Spacing Between Slices, because the other ones related to spacial resolution are present. \n\nI suspect the missing tag will make training 3D models more challenging, but not as much for 2D models. The LB results are good so far, but I suspect they could be even better if we had the missing tag. \n\nI will try to get access to the original files to read that tag. Let me know if you miss any other tag.\n\nRegarding the transformation in the pixel data, the complete code for facial deidentification is here: https://github.com/kitamura-felipe/face_deid_ct\n\nIn a nutshell, the code fills the air with pixels that have values in the range of those of the patient's skin. By doing that, we reduce the chance that the process can be perfectly reverted. In some cases, it removed part of the skull as well. That is a limitation of this method.\n\nWe believe that models trained with images using that defacing technique will generalize if we apply the same function to newer studies on which we want to do inference. The results of the LB show that is true so far.\n\nThank you for bringing that up!", "votes": 1, "replies": [ { "id": 2720785, "postDate": "2024-03-28T15:16:16.743Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2722030, "postDate": "2024-03-29T11:17:19.897Z", "content": "Only some of the studies had the tag SpaceBetweenSlices. I am uploading two CSV files with that information for the training and test sets. It should be available in 4 hours.
", "rawMarkdown": "Only some of the studies had the tag SpaceBetweenSlices. I am uploading two CSV files with that information for the training and test sets. It should be available in 4 hours.", "votes": 2 }, { "id": 2722045, "postDate": "2024-03-29T11:25:15.430Z", "content": "we can create 3d array from dicoms .
\n ():\n \n img = (img*slope +intercept) \n img_min = window_center - window_width// \n img_max = window_center + window_width// \n img[img<img_min] = img_min \n img[img>img_max] = img_max \n rescale: \n img = (img - img_min) / (img_max - img_min)* \n img\n\n ():\n \n image = image - np.(image)\n image / np.(image)\n ():\n \n :\n image = dcm.pixel_array\n :\n \n window_center window_width :\n\n image = window_image(image, window_center, window_width, dcm.RescaleIntercept, dcm.RescaleSlope)\n\n dcm.PhotometricInterpretation == :\n image = np.invert(image)\n\n normalized_image = normalize_image(image)\n normalized_image\n ():\n\n t_paths = (glob.glob(os.path.join(path, ))[:], key= x: (x.split()[-].split()[]))\n\n\n dicoms = [pydicom.dcmread(d, force= ) d t_paths]\n sereis_instance_uid = [d.SeriesInstanceUID d dicoms]\n ((sereis_instance_uid))\n\n images = {i:[] i (sereis_instance_uid)}\n z_pos = {i:[] i (sereis_instance_uid)}\n dcm,s_id (dicoms,sereis_instance_uid):\n d = load_dicom(dcm,,)\n d :\n \n images[s_id].append(d)\n z_pos[s_id].append((dcm.ImagePositionPatient[-]))\n\n \n s_id (sereis_instance_uid):\n images[s_id] = np.stack(images[s_id], -)[:,:,np.argsort(z_pos[s_id])]\n z_pos[s_id] = np.sort(z_pos[s_id])\n images,z_pos\nHi Felipe,
\nThank you for help and explanation. I have explored less common converters and the Fiji is recognizing a lot of the sets. here is an example:
\n(Fiji Is Just) ImageJ 2.14.0/1.54f; Java 1.8.0_202 [64-bit];
Title: 002360
\nWidth: 249.9999 mm (512)
\nHeight: 249.9999 mm (512)
\nDepth: 173.75 mm (139)
\nSize: 70MB
\nResolution: 2.0480 pixels per mm
\nVoxel size: 0.4883x0.4883x1.25 mm^3
\nID: -212
\nBits per pixel: 16 (unsigned, grayscale LUT)
\nDisplay range: -15 - 95
\nRaw pixel value range: 33069 - 35553
\nImage: 82/139 (1.2.840.12345.4549484626668932613792536314089807458768514401)
\nNo threshold
\nScaleToFit: false
So the voxel size can be read, but the typical location for shape definition has been erased like you mentioned.
\nIs the gender encoded in the dicom real or false?
Great to hear you are making progress.
\nThe gender is real and can be used as independent variable.
", "rawMarkdown": "Great to hear you are making progress.\n\nThe gender is real and can be used as independent variable." } ] } ] }, { "id": 2720280, "postDate": "2024-03-28T07:55:30.160Z", "content": "Hi there,
\nI'm looking at the training data and have a few questions:
Hi, thank you for your great questions!
\nRegarding the DICOM tags, you are correct.
\nWe are missing some DICOM tags to convert to nifti. I suspect the missing tag is Spacing Between Slices, because the other ones related to spacial resolution are present.
\nI suspect the missing tag will make training 3D models more challenging, but not as much for 2D models. The LB results are good so far, but I suspect they could be even better if we had the missing tag.
\nI will try to get access to the original files to read that tag. Let me know if you miss any other tag.
\nRegarding the transformation in the pixel data, the complete code for facial deidentification is here: https://github.com/kitamura-felipe/face_deid_ct
\nIn a nutshell, the code fills the air with pixels that have values in the range of those of the patient's skin. By doing that, we reduce the chance that the process can be perfectly reverted. In some cases, it removed part of the skull as well. That is a limitation of this method.
\nWe believe that models trained with images using that defacing technique will generalize if we apply the same function to newer studies on which we want to do inference. The results of the LB show that is true so far.
\nThank you for bringing that up!
", "votes": 1, "replies": [ { "id": 2720785, "author_name": "", "author_url": "", "post_date": "2024-03-28T15:16:16.743000", "content": "", "votes": 0, "replies": [] }, { "id": 2722030, "author_name": "FelipeKitamura, MD, PhD", "author_url": "", "post_date": "2024-03-29T11:17:19.897000", "content": "Only some of the studies had the tag SpaceBetweenSlices. I am uploading two CSV files with that information for the training and test sets. It should be available in 4 hours.
", "votes": 2, "replies": [] }, { "id": 2722045, "author_name": "patriot", "author_url": "", "post_date": "2024-03-29T11:25:15.430000", "content": "we can create 3d array from dicoms .
\n ():\n \n img = (img*slope +intercept) \n img_min = window_center - window_width// \n img_max = window_center + window_width// \n img[img<img_min] = img_min \n img[img>img_max] = img_max \n rescale: \n img = (img - img_min) / (img_max - img_min)* \n img\n\n ():\n \n image = image - np.(image)\n image / np.(image)\n ():\n \n :\n image = dcm.pixel_array\n :\n \n window_center window_width :\n\n image = window_image(image, window_center, window_width, dcm.RescaleIntercept, dcm.RescaleSlope)\n\n dcm.PhotometricInterpretation == :\n image = np.invert(image)\n\n normalized_image = normalize_image(image)\n normalized_image\n ():\n\n t_paths = (glob.glob(os.path.join(path, ))[:], key= x: (x.split()[-].split()[]))\n\n\n dicoms = [pydicom.dcmread(d, force= ) d t_paths]\n sereis_instance_uid = [d.SeriesInstanceUID d dicoms]\n ((sereis_instance_uid))\n\n images = {i:[] i (sereis_instance_uid)}\n z_pos = {i:[] i (sereis_instance_uid)}\n dcm,s_id (dicoms,sereis_instance_uid):\n d = load_dicom(dcm,,)\n d :\n \n images[s_id].append(d)\n z_pos[s_id].append((dcm.ImagePositionPatient[-]))\n\n \n s_id (sereis_instance_uid):\n images[s_id] = np.stack(images[s_id], -)[:,:,np.argsort(z_pos[s_id])]\n z_pos[s_id] = np.sort(z_pos[s_id])\n images,z_pos\nHi Felipe,
\nThank you for help and explanation. I have explored less common converters and the Fiji is recognizing a lot of the sets. here is an example:
\n(Fiji Is Just) ImageJ 2.14.0/1.54f; Java 1.8.0_202 [64-bit];
Title: 002360
\nWidth: 249.9999 mm (512)
\nHeight: 249.9999 mm (512)
\nDepth: 173.75 mm (139)
\nSize: 70MB
\nResolution: 2.0480 pixels per mm
\nVoxel size: 0.4883x0.4883x1.25 mm^3
\nID: -212
\nBits per pixel: 16 (unsigned, grayscale LUT)
\nDisplay range: -15 - 95
\nRaw pixel value range: 33069 - 35553
\nImage: 82/139 (1.2.840.12345.4549484626668932613792536314089807458768514401)
\nNo threshold
\nScaleToFit: false
So the voxel size can be read, but the typical location for shape definition has been erased like you mentioned.
\nIs the gender encoded in the dicom real or false?
Great to hear you are making progress.
\nThe gender is real and can be used as independent variable.
", "votes": 0, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2720742": "Hi, thank you for your great questions!\n\nRegarding the DICOM tags, you are correct. \n\nWe are missing some DICOM tags to convert to nifti. I suspect the missing tag is Spacing Between Slices, because the other ones related to spacial resolution are present. \n\nI suspect the missing tag will make training 3D models more challenging, but not as much for 2D models. The LB results are good so far, but I suspect they could be even better if we had the missing tag. \n\nI will try to get access to the original files to read that tag. Let me know if you miss any other tag.\n\nRegarding the transformation in the pixel data, the complete code for facial deidentification is here: https://github.com/kitamura-felipe/face_deid_ct\n\nIn a nutshell, the code fills the air with pixels that have values in the range of those of the patient's skin. By doing that, we reduce the chance that the process can be perfectly reverted. In some cases, it removed part of the skull as well. That is a limitation of this method.\n\nWe believe that models trained with images using that defacing technique will generalize if we apply the same function to newer studies on which we want to do inference. The results of the LB show that is true so far.\n\nThank you for bringing that up!", "2720280": "Hi there, \nI'm looking at the training data and have a few questions:\n1. Can you list the modifications of the raw CT data in the dataset?\n2. Why are the dicom tags removed so that the most common dicom to nifti converters (itk and dcm2niix) do not recognize data as dicom? (I understand anonymization, but have a problem with spatial reconstruction)\n3. What are the parameters of the noise in the images?\n4. I guess that you have computed a mask for the head region of interest to modify background noise. Are you going to share it?\n5. Did the mask affect the skull bones in any way? (Bone changes in the frontal bones and temporal bones are associated with age)\n6. Can you provide the data in nifti format? if not can you provide the range of resolutions and acquisition protocols for the datasets\n7. Do you think that the results will be generelizable without knowing the above?\n" } }