![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F2a3da61fd65ba4d3625ffb757a011092%2FSelection_999(3022).png?generation=1693682184651228&alt=media\")
('inverted', patient_id,series_id)\n \n \n \n \n\n should use patentient position to check:\n, sy0, sz0 =\n, syN, szN =\n\n szN > sz0: ....\n
('inverted', patient_id,series_id)\n \n \n \n \n\n should use patentient position to check:\n, sy0, sz0 =\n, syN, szN =\n\n szN > sz0: ....\nIf you sort slices by patient_id, scan_id, and file name and then calculate z position diff, there are 95 scans that have positive diff values. Those are the ones that are inverted.
\ndf_train_dicom_tags.loc[df_train_dicom_tags[] > , ].unique()\narray([ , , , , , , , , ,\n , , , , , , , , ,\n , , , , , , , , ,\n , , , , , , , , ,\n , , , , , , , , ,\n , , , , , , , , ,\n , , , , , , , , ,\n , , , , , , , , ,\n , , , , , , , , ,\n , , , , , , , , ,\n , , , , ], dtype=uint64)\nSome scans are acquired this way. They start at the pelvis and move upward toward the chest. Abdomen/pelvic scans are typically done from the chest down though. Don't trust the instance number, or filename. Check the Z coordinates of ImagePositionPatient to know.
", "rawMarkdown": "Some scans are acquired this way. They start at the pelvis and move upward toward the chest. Abdomen/pelvic scans are typically done from the chest down though. Don't trust the instance number, or filename. Check the Z coordinates of ImagePositionPatient to know.", "votes": 6, "replies": [ { "id": 2420981, "postDate": "2023-09-03T04:53:59.813Z", "content": "What about PatientPosition attribute? I was flipping HFS scans on z and x axes.
\n![\"1\"](\"https://i.ibb.co/42GsVnB/dicom-tags-Patient-Position-value-counts.png\")
Wow I was not aware of this at all, does HFS refer to scan from head to toe?
", "rawMarkdown": "Wow I was not aware of this at all, does HFS refer to scan from head to toe?" }, { "id": 2421010, "postDate": "2023-09-03T05:27:10.830Z", "content": "Yep that's what I found on the internet. Check head first and feet first supine https://dicom.nema.org/medical/dicom/current/output/html/figures/PS3.3_C.7.3.1.1.2-1.svg
", "rawMarkdown": "Yep that's what I found on the internet. Check head first and feet first supine https://dicom.nema.org/medical/dicom/current/output/html/figures/PS3.3_C.7.3.1.1.2-1.svg", "votes": 2 }, { "id": 2421278, "postDate": "2023-09-03T09:21:43.933Z", "content": "I checked last week to see if all the patients are in a supine position.
\nhttps://www.kaggle.com/code/enriquezaf/rsna-atd-all-supine-check/notebook
\nThe organizers were benevolent and only put HFS/FFS in the testing dataset.
", "rawMarkdown": "I checked last week to see if all the patients are in a supine position.\n\nhttps://www.kaggle.com/code/enriquezaf/rsna-atd-all-supine-check/notebook\n\nThe organizers were benevolent and only put HFS/FFS in the testing dataset.", "votes": 2 }, { "id": 2421753, "postDate": "2023-09-03T14:00:10.310Z", "content": "I have never seen an abdomen/pelvic CT done in any other position than supine. I'm sure someone has done a CT in the prone or decubitus positions, but those would be rare in the wild.
\nThe PositionPatient tag only tells us which way the patient is oriented to the gantry. It doesn't tell us the direction of travel of the patient, or help to determine the order of slices.
", "rawMarkdown": "I have never seen an abdomen/pelvic CT done in any other position than supine. I'm sure someone has done a CT in the prone or decubitus positions, but those would be rare in the wild. \n\nThe PositionPatient tag only tells us which way the patient is oriented to the gantry. It doesn't tell us the direction of travel of the patient, or help to determine the order of slices.", "votes": 2 }, { "id": 2421801, "postDate": "2023-09-03T14:24:32.590Z", "content": "I see… that's why z position diff can be both positive and negative for both HFS and FFS. PatientPosition doesn't mean anything then because regardless of the patient's position, the scan can still be from top to bottom or vice versa. Is that correct?
", "rawMarkdown": "I see... that's why z position diff can be both positive and negative for both HFS and FFS. PatientPosition doesn't mean anything then because regardless of the patient's position, the scan can still be from top to bottom or vice versa. Is that correct?" }, { "id": 2421834, "postDate": "2023-09-03T14:35:31.967Z", "content": "That is correct.
", "rawMarkdown": "That is correct.", "votes": 2 }, { "id": 2421946, "postDate": "2023-09-03T15:42:33.850Z", "content": "Very useful information, thanks.
", "rawMarkdown": "Very useful information, thanks." }, { "id": 2422385, "postDate": "2023-09-04T01:38:36.323Z", "content": "i am thinking i amy want to train a simple classifier to check correct scan orientation based on
\nscan.mean(0),scan.mean(1),scan.mean(2) where scan is DxHxW volume.
@davidbroberts I've done some experiments with finding Z0 and Zn to calculate rotation and in both the first and last dicom way and min and max Z way from ImagePatientPosition, values were the same. Maybe this dataset is not as hard as you describe it
", "rawMarkdown": "@davidbroberts I've done some experiments with finding Z0 and Zn to calculate rotation and in both the first and last dicom way and min and max Z way from ImagePatientPosition, values were the same. Maybe this dataset is not as hard as you describe it" }, { "id": 2424713, "postDate": "2023-09-05T12:50:03.143Z", "content": "Ordering the slices by the Z component of ImagePositionPatient will put them in the correct order regardless of the scan direction.
", "rawMarkdown": "Ordering the slices by the Z component of ImagePositionPatient will put them in the correct order regardless of the scan direction.", "votes": 2 }, { "id": 2424717, "postDate": "2023-09-05T12:54:59.327Z", "content": "What does ordering mean for negative values? If Z is negative, should ordering be from the biggest value (so the smallest number in terms of standard ordering) to the smallest one (so the biggest number in terms of standard ordering) or the opposite?
\nI'm struggling with understanding. Let's take FFS as an example. If Z diff is positive, does it mean feet go first into the gantry and if Z diff is negative, does it mean head go first? And for HFS in the opposite way?
\nYou say ordering by the Z component will work. Let's assume the nose tip has some constant coordinates xn, yn, zn. Does it mean that whatever patient position (FFS or HFS) and Z diff are, will those coordinates be always the same in the ImagePositionPatient (for the given patient)?
", "rawMarkdown": "What does ordering mean for negative values? If Z is negative, should ordering be from the biggest value (so the smallest number in terms of standard ordering) to the smallest one (so the biggest number in terms of standard ordering) or the opposite?\n\nI'm struggling with understanding. Let's take FFS as an example. If Z diff is positive, does it mean feet go first into the gantry and if Z diff is negative, does it mean head go first? And for HFS in the opposite way?\n\nYou say ordering by the Z component will work. Let's assume the nose tip has some constant coordinates xn, yn, zn. Does it mean that whatever patient position (FFS or HFS) and Z diff are, will those coordinates be always the same in the ImagePositionPatient (for the given patient)?" }, { "id": 2424728, "postDate": "2023-09-05T13:04:36.147Z", "content": "Order by Z descending (largest number on top), and the images will be stacked from head-to-feet regardless of scan direction. HFS means the patient is in the gantry head-first. FFS = feet first. PatientPosition is independent of the direction the scanner moves, which is the issue here.
", "rawMarkdown": "Order by Z descending (largest number on top), and the images will be stacked from head-to-feet regardless of scan direction. HFS means the patient is in the gantry head-first. FFS = feet first. PatientPosition is independent of the direction the scanner moves, which is the issue here.", "votes": 3 }, { "id": 2424790, "postDate": "2023-09-05T13:48:02.070Z", "content": "Sorry if I'm annoying but just for final clarification:
\nIt does not matter what the PatientPosition is (FFS, HFS) and what Z diff is (positive or negative). If I will sort images by Z descending I will always end up with head-to-feet volume.
All I have to do next to that is a mirror on the X-axis for the one chosen PatientPosition.
", "rawMarkdown": "Sorry if I'm annoying but just for final clarification:\nIt does not matter what the PatientPosition is (FFS, HFS) and what Z diff is (positive or negative). If I will sort images by Z descending I will always end up with head-to-feet volume.\n\nAll I have to do next to that is a mirror on the X-axis for the one chosen PatientPosition." }, { "id": 2424893, "postDate": "2023-09-05T14:34:34.047Z", "content": "Not annoying at all. Generally speaking, on modern scanners .. when the technologist is planning the scan and specifies the patient's position (HFS,FFS), the couch location is \"zeroed\" (ImagePositionPatient[z] = 0.0).
\nNormally, the \"Z\" zero location is set near the patient's head, but not always. What's important is that the Z direction of travel is flipped according to the PatientPosition. This aligns the patient with the DICOM patient coordinate system LPH, where Z increases toward the head.
\nI have not verified it but I suspect this dataset conforms to that standard (some older scanners may not).
\nAbdomen scans are normally done from top to bottom, but sometimes for clinical reasons, they're done from bottom to top. I think it's safe to say .. \"If the first instance's Z is greater than the last instance's Z, the scan direction is reversed\" .. I think.
", "rawMarkdown": "Not annoying at all. Generally speaking, on modern scanners .. when the technologist is planning the scan and specifies the patient's position (HFS,FFS), the couch location is \"zeroed\" (ImagePositionPatient[z] = 0.0). \n\nNormally, the \"Z\" zero location is set near the patient's head, but not always. What's important is that the Z direction of travel is flipped according to the PatientPosition. This aligns the patient with the DICOM patient coordinate system LPH, where Z increases toward the head. \n\nI have not verified it but I suspect this dataset conforms to that standard (some older scanners may not).\n\nAbdomen scans are normally done from top to bottom, but sometimes for clinical reasons, they're done from bottom to top. I *think* it's safe to say .. \"If the first instance's Z is greater than the last instance's Z, the scan direction is reversed\" .. I think." } ] } ] }, { "id": 2420661, "postDate": "2023-09-02T19:26:00.307Z", "content": "tip:
\ni find this one of the base way to normalise intensity:
\n = read series of dicom file\n = torch.quantile(image, )\n = image[image > th]\n = torch.mean(sample)\n = torch.std(sample)\n = mean - * std\n = mean + * std\n = (image - tmin) / (tmax - tmin + ) \n\n\n = np_sigmoid((image - ) * )\nThis gave me slight validation loss boost. What's the intuition behind it?
", "rawMarkdown": "This gave me slight validation loss boost. What's the intuition behind it?" }, { "id": 2421971, "postDate": "2023-09-03T16:11:31.200Z", "content": "if you trust the dicom tag, you can use the usual processing
\n =\n s in range(slice_min, slice_max):\n = f'{dcm_dir}/{s}.dcm'\n\n \n \n \n\n = dicomsdl.open(f)\n = dicomsdl_to_numpy_image(dcm)\n dcm.PixelRepresentation == :\n = dcm.BitsAllocated - dcm.BitsStored\n = pixel_array.dtype\n = (pixel_array << bit_shift).astype(dtype) >> bit_shift\n\n \n = pixel_array.astype(np.float32)\n = dcm.RescaleSlope * pixel_array + dcm.RescaleIntercept\n = dcm.WindowCenter-.-(dcm.WindowWidth-)* .\n = dcm.WindowCenter-.+(dcm.WindowWidth-)* .\n = np.empty_like(pixel_array, dtype=np.uint8)\n .util.convert_to_uint8(pixel_array, norm, xmin, xmax)\n\n dcm.PhotometricInterpretation == 'MONOCHROME1':\n = - norm\n .append(norm)\ni am afraid that the dicom tag are not reliable. i check the intensity histogram
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fc3425937bdb3832bc781f0ea0068ccd1%2FSelection_999(3034).png?generation=1693757371400046&alt=media\")
basically for all the train dicom, there are 2 peaks as shown.
\nThe code below select the second peak.
\n = torch.quantile(image, )\n = image[image > th]\nthen you normalised to mean +/- factor*std of the second peak.
\ni did not clip the values
The image in your histogram appears to be MONOCHROME2, so the second peak represents the less intense (higher value) pixels. If you apply this to a MONOCHROME1 image, you'll be normalizing only on the darker pixels .. which are mostly air.
\nThis normalization seems the same as what increasing/decreasing the WindowCenter would do.
", "rawMarkdown": "The image in your histogram appears to be MONOCHROME2, so the second peak represents the less intense (higher value) pixels. If you apply this to a MONOCHROME1 image, you'll be normalizing only on the darker pixels .. which are mostly air.\n\nThis normalization seems the same as what increasing/decreasing the WindowCenter would do.", "votes": 2 } ] }, { "id": 2426738, "postDate": "2023-09-06T19:20:30.573Z", "content": "Thank you for sharing, very useful! But why do you share it here?
", "rawMarkdown": "Thank you for sharing, very useful! But why do you share it here?" } ] }, { "id": 2422242, "postDate": "2023-09-03T20:22:14.890Z", "content": "I'm just using monai.transforms reared, it does read in the right order. And it has a lot of useful tools.
\nhttps://monai.io/
Do you mean that not converting to png just use dicom?
", "rawMarkdown": "Do you mean that not converting to png just use dicom?", "replies": [ { "id": 2424928, "postDate": "2023-09-05T14:50:26.253Z", "content": "Yes, it directly reads dicoms
", "rawMarkdown": "Yes, it directly reads dicoms " } ] }, { "id": 2426969, "postDate": "2023-09-07T02:33:00.073Z", "content": "I am having trouble with monai, can you share how to read dicom from this competition?
", "rawMarkdown": "I am having trouble with monai, can you share how to read dicom from this competition?", "replies": [ { "id": 2426976, "postDate": "2023-09-07T02:45:27.273Z", "content": "import monai\nimport monai.transforms MT\nnull_transform = MT.Compose([\n MT.LoadImaged(=[, ]),\n MT.EnsureChannelFirstd(=[, ]), \n MT.Orientationd(=[, ], axcodes=),\n MT.Spacingd(\n =[, ],\n pixdim=(, , ),\n =(, ),\n ),\n MT.DivisiblePadd(\n =[, ],\n =,\n = ,\n ),\n])\n\n#e.g\nd = null_transform({\n = \n = \n})\n(d[].shape)\n(d[].shape)\nmonai is good for initial experiments.
\nbut i suggest read from scratch in final development.
\nhttps://www.kaggle.com/code/hengck23/lb0-55-2-5d-1d-sample-model
align nii mask to dicom image
\nhttps://www.kaggle.com/code/franklinshih0617/learning-nii-file-overlay-segmentation-on-dicom
this takes 1 minute + per dicom, I tried to process normally which is much faster but segmentation alignment isn't 100% working on every series_id. Thanks for this, let me try to re-build my processing pipeline, I think i'll make my own segmentation model.
", "rawMarkdown": "this takes 1 minute + per dicom, I tried to process normally which is much faster but segmentation alignment isn't 100% working on every series_id. Thanks for this, let me try to re-build my processing pipeline, I think i'll make my own segmentation model." }, { "id": 2428674, "postDate": "2023-09-08T05:10:42.743Z", "content": "monai have PersistentDataset,etc which store intermediate resuts on disk and/or ram to seedup data creation.
\nfor me, i code from from scratch and store the resuts as well.
\n(the downside is tat you need huge disk space)
so you can use multiprocessing to store monai results
", "rawMarkdown": "monai have PersistentDataset,etc which store intermediate resuts on disk and/or ram to seedup data creation.\nfor me, i code from from scratch and store the resuts as well.\n(the downside is tat you need huge disk space)\n\nso you can use multiprocessing to store monai results", "votes": 1 }, { "id": 2428676, "postDate": "2023-09-08T05:12:59.843Z", "content": "![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2Fca935b8055f3c041a4f1249d936011d3%2FSelection_999(3145).png?generation=1694149970180082&alt=media\")
green is cached
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F113660%2F1525f50456edb6a964936e753ea3073c%2FSelection_999(3146).png?generation=1694150109754336&alt=media\")
I just realized that you should also be aware that monai doesn't have that standardize_pixel_array thing discussed here:
\nhttps://www.kaggle.com/competitions/rsna-2023-abdominal-trauma-detection/discussion/427217
I just manually added it to monai.data.image_reader.
", "rawMarkdown": "I just realized that you should also be aware that monai doesn't have that standardize_pixel_array thing discussed here:\nhttps://www.kaggle.com/competitions/rsna-2023-abdominal-trauma-detection/discussion/427217\n\nI just manually added it to monai.data.image_reader." } ] } ] } ], "comments": [ { "id": 2421918, "author_name": "Gunes Evitan", "author_url": "", "post_date": "2023-09-03T15:10:38.157000", "content": "If you sort slices by patient_id, scan_id, and file name and then calculate z position diff, there are 95 scans that have positive diff values. Those are the ones that are inverted.
\ndf_train_dicom_tags.loc[df_train_dicom_tags[] > , ].unique()\narray([ , , , , , , , , ,\n , , , , , , , , ,\n , , , , , , , , ,\n , , , , , , , , ,\n , , , , , , , , ,\n , , , , , , , , ,\n , , , , , , , , ,\n , , , , , , , , ,\n , , , , , , , , ,\n , , , , , , , , ,\n , , , , ], dtype=uint64)\nSome scans are acquired this way. They start at the pelvis and move upward toward the chest. Abdomen/pelvic scans are typically done from the chest down though. Don't trust the instance number, or filename. Check the Z coordinates of ImagePositionPatient to know.
", "votes": 6, "replies": [ { "id": 2420981, "author_name": "Gunes Evitan", "author_url": "", "post_date": "2023-09-03T04:53:59.813000", "content": "What about PatientPosition attribute? I was flipping HFS scans on z and x axes.
\n
Wow I was not aware of this at all, does HFS refer to scan from head to toe?
", "votes": 0, "replies": [] }, { "id": 2421010, "author_name": "Gunes Evitan", "author_url": "", "post_date": "2023-09-03T05:27:10.830000", "content": "Yep that's what I found on the internet. Check head first and feet first supine https://dicom.nema.org/medical/dicom/current/output/html/figures/PS3.3_C.7.3.1.1.2-1.svg
", "votes": 2, "replies": [] }, { "id": 2421278, "author_name": "Antonio Félix", "author_url": "", "post_date": "2023-09-03T09:21:43.933000", "content": "I checked last week to see if all the patients are in a supine position.
\nhttps://www.kaggle.com/code/enriquezaf/rsna-atd-all-supine-check/notebook
\nThe organizers were benevolent and only put HFS/FFS in the testing dataset.
", "votes": 2, "replies": [] }, { "id": 2421753, "author_name": "David Roberts", "author_url": "", "post_date": "2023-09-03T14:00:10.310000", "content": "I have never seen an abdomen/pelvic CT done in any other position than supine. I'm sure someone has done a CT in the prone or decubitus positions, but those would be rare in the wild.
\nThe PositionPatient tag only tells us which way the patient is oriented to the gantry. It doesn't tell us the direction of travel of the patient, or help to determine the order of slices.
", "votes": 2, "replies": [] }, { "id": 2421801, "author_name": "Gunes Evitan", "author_url": "", "post_date": "2023-09-03T14:24:32.590000", "content": "I see… that's why z position diff can be both positive and negative for both HFS and FFS. PatientPosition doesn't mean anything then because regardless of the patient's position, the scan can still be from top to bottom or vice versa. Is that correct?
", "votes": 0, "replies": [] }, { "id": 2421834, "author_name": "David Roberts", "author_url": "", "post_date": "2023-09-03T14:35:31.967000", "content": "That is correct.
", "votes": 2, "replies": [] }, { "id": 2421946, "author_name": "Antonio Félix", "author_url": "", "post_date": "2023-09-03T15:42:33.850000", "content": "Very useful information, thanks.
", "votes": 0, "replies": [] }, { "id": 2422385, "author_name": "hengck23", "author_url": "", "post_date": "2023-09-04T01:38:36.323000", "content": "i am thinking i amy want to train a simple classifier to check correct scan orientation based on
\nscan.mean(0),scan.mean(1),scan.mean(2) where scan is DxHxW volume.
@davidbroberts I've done some experiments with finding Z0 and Zn to calculate rotation and in both the first and last dicom way and min and max Z way from ImagePatientPosition, values were the same. Maybe this dataset is not as hard as you describe it
", "votes": 0, "replies": [] }, { "id": 2424713, "author_name": "David Roberts", "author_url": "", "post_date": "2023-09-05T12:50:03.143000", "content": "Ordering the slices by the Z component of ImagePositionPatient will put them in the correct order regardless of the scan direction.
", "votes": 2, "replies": [] }, { "id": 2424717, "author_name": "JanGlinko2", "author_url": "", "post_date": "2023-09-05T12:54:59.327000", "content": "What does ordering mean for negative values? If Z is negative, should ordering be from the biggest value (so the smallest number in terms of standard ordering) to the smallest one (so the biggest number in terms of standard ordering) or the opposite?
\nI'm struggling with understanding. Let's take FFS as an example. If Z diff is positive, does it mean feet go first into the gantry and if Z diff is negative, does it mean head go first? And for HFS in the opposite way?
\nYou say ordering by the Z component will work. Let's assume the nose tip has some constant coordinates xn, yn, zn. Does it mean that whatever patient position (FFS or HFS) and Z diff are, will those coordinates be always the same in the ImagePositionPatient (for the given patient)?
", "votes": 0, "replies": [] }, { "id": 2424728, "author_name": "David Roberts", "author_url": "", "post_date": "2023-09-05T13:04:36.147000", "content": "Order by Z descending (largest number on top), and the images will be stacked from head-to-feet regardless of scan direction. HFS means the patient is in the gantry head-first. FFS = feet first. PatientPosition is independent of the direction the scanner moves, which is the issue here.
", "votes": 3, "replies": [] }, { "id": 2424790, "author_name": "JanGlinko2", "author_url": "", "post_date": "2023-09-05T13:48:02.070000", "content": "Sorry if I'm annoying but just for final clarification:
\nIt does not matter what the PatientPosition is (FFS, HFS) and what Z diff is (positive or negative). If I will sort images by Z descending I will always end up with head-to-feet volume.
All I have to do next to that is a mirror on the X-axis for the one chosen PatientPosition.
", "votes": 0, "replies": [] }, { "id": 2424893, "author_name": "David Roberts", "author_url": "", "post_date": "2023-09-05T14:34:34.047000", "content": "Not annoying at all. Generally speaking, on modern scanners .. when the technologist is planning the scan and specifies the patient's position (HFS,FFS), the couch location is \"zeroed\" (ImagePositionPatient[z] = 0.0).
\nNormally, the \"Z\" zero location is set near the patient's head, but not always. What's important is that the Z direction of travel is flipped according to the PatientPosition. This aligns the patient with the DICOM patient coordinate system LPH, where Z increases toward the head.
\nI have not verified it but I suspect this dataset conforms to that standard (some older scanners may not).
\nAbdomen scans are normally done from top to bottom, but sometimes for clinical reasons, they're done from bottom to top. I think it's safe to say .. \"If the first instance's Z is greater than the last instance's Z, the scan direction is reversed\" .. I think.
", "votes": 0, "replies": [] } ] } ] }, { "id": 2420661, "author_name": "hengck23", "author_url": "", "post_date": "2023-09-02T19:26:00.307000", "content": "tip:
\ni find this one of the base way to normalise intensity:
\n = read series of dicom file\n = torch.quantile(image, )\n = image[image > th]\n = torch.mean(sample)\n = torch.std(sample)\n = mean - * std\n = mean + * std\n = (image - tmin) / (tmax - tmin + ) \n\n\n = np_sigmoid((image - ) * )\nThis gave me slight validation loss boost. What's the intuition behind it?
", "votes": 0, "replies": [] }, { "id": 2421971, "author_name": "hengck23", "author_url": "", "post_date": "2023-09-03T16:11:31.200000", "content": "if you trust the dicom tag, you can use the usual processing
\n =\n s in range(slice_min, slice_max):\n = f'{dcm_dir}/{s}.dcm'\n\n \n \n \n\n = dicomsdl.open(f)\n = dicomsdl_to_numpy_image(dcm)\n dcm.PixelRepresentation == :\n = dcm.BitsAllocated - dcm.BitsStored\n = pixel_array.dtype\n = (pixel_array << bit_shift).astype(dtype) >> bit_shift\n\n \n = pixel_array.astype(np.float32)\n = dcm.RescaleSlope * pixel_array + dcm.RescaleIntercept\n = dcm.WindowCenter-.-(dcm.WindowWidth-)* .\n = dcm.WindowCenter-.+(dcm.WindowWidth-)* .\n = np.empty_like(pixel_array, dtype=np.uint8)\n .util.convert_to_uint8(pixel_array, norm, xmin, xmax)\n\n dcm.PhotometricInterpretation == 'MONOCHROME1':\n = - norm\n .append(norm)\ni am afraid that the dicom tag are not reliable. i check the intensity histogram
\n
basically for all the train dicom, there are 2 peaks as shown.
\nThe code below select the second peak.
\n = torch.quantile(image, )\n = image[image > th]\nthen you normalised to mean +/- factor*std of the second peak.
\ni did not clip the values
The image in your histogram appears to be MONOCHROME2, so the second peak represents the less intense (higher value) pixels. If you apply this to a MONOCHROME1 image, you'll be normalizing only on the darker pixels .. which are mostly air.
\nThis normalization seems the same as what increasing/decreasing the WindowCenter would do.
", "votes": 2, "replies": [] } ] }, { "id": 2426738, "author_name": "Gyula Maloveczky4", "author_url": "", "post_date": "2023-09-06T19:20:30.573000", "content": "Thank you for sharing, very useful! But why do you share it here?
", "votes": 0, "replies": [] } ] }, { "id": 2422242, "author_name": "AnatoliK", "author_url": "", "post_date": "2023-09-03T20:22:14.890000", "content": "I'm just using monai.transforms reared, it does read in the right order. And it has a lot of useful tools.
\nhttps://monai.io/
Do you mean that not converting to png just use dicom?
", "votes": 0, "replies": [ { "id": 2424928, "author_name": "AnatoliK", "author_url": "", "post_date": "2023-09-05T14:50:26.253000", "content": "Yes, it directly reads dicoms
", "votes": 0, "replies": [] } ] }, { "id": 2426969, "author_name": "outwrest", "author_url": "", "post_date": "2023-09-07T02:33:00.073000", "content": "I am having trouble with monai, can you share how to read dicom from this competition?
", "votes": 0, "replies": [ { "id": 2426976, "author_name": "hengck23", "author_url": "", "post_date": "2023-09-07T02:45:27.273000", "content": "import monai\nimport monai.transforms MT\nnull_transform = MT.Compose([\n MT.LoadImaged(=[, ]),\n MT.EnsureChannelFirstd(=[, ]), \n MT.Orientationd(=[, ], axcodes=),\n MT.Spacingd(\n =[, ],\n pixdim=(, , ),\n =(, ),\n ),\n MT.DivisiblePadd(\n =[, ],\n =,\n = ,\n ),\n])\n\n#e.g\nd = null_transform({\n = \n = \n})\n(d[].shape)\n(d[].shape)\nmonai is good for initial experiments.
\nbut i suggest read from scratch in final development.
\nhttps://www.kaggle.com/code/hengck23/lb0-55-2-5d-1d-sample-model
align nii mask to dicom image
\nhttps://www.kaggle.com/code/franklinshih0617/learning-nii-file-overlay-segmentation-on-dicom
this takes 1 minute + per dicom, I tried to process normally which is much faster but segmentation alignment isn't 100% working on every series_id. Thanks for this, let me try to re-build my processing pipeline, I think i'll make my own segmentation model.
", "votes": 0, "replies": [] }, { "id": 2428674, "author_name": "hengck23", "author_url": "", "post_date": "2023-09-08T05:10:42.743000", "content": "monai have PersistentDataset,etc which store intermediate resuts on disk and/or ram to seedup data creation.
\nfor me, i code from from scratch and store the resuts as well.
\n(the downside is tat you need huge disk space)
so you can use multiprocessing to store monai results
", "votes": 1, "replies": [] }, { "id": 2428676, "author_name": "hengck23", "author_url": "", "post_date": "2023-09-08T05:12:59.843000", "content": "
green is cached
\n
I just realized that you should also be aware that monai doesn't have that standardize_pixel_array thing discussed here:
\nhttps://www.kaggle.com/competitions/rsna-2023-abdominal-trauma-detection/discussion/427217
I just manually added it to monai.data.image_reader.
", "votes": 0, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2420652": "1. most public kernel code load dcm in sorted instance numbers (e.g 5.dcm, 6.dcm, 7.dcm, ...) But this can be inverted:\n\n.png?generation=1693682184651228&alt=media)\n\n```\n\nprint('inverted', patient_id,series_id)\ninverted 11139 24201\ninverted 12487 8913\ninverted 13615 4619\ninverted 13620 30803\n\nyou should use patentient position to check:\nsx0, sy0, sz0 = [float(v) for v in dcm0.ImagePositionPatient]\nsxN, syN, szN = [float(v) for v in dcmN.ImagePositionPatient]\n\nif szN > sz0: ....\n\n```", "2421918": "If you sort slices by patient_id, scan_id, and file name and then calculate z position diff, there are 95 scans that have positive diff values. Those are the ones that are inverted.\n\n```python\n>>> df_train_dicom_tags.loc[df_train_dicom_tags['z_position_diff'] > 0, 'scan_id'].unique()\narray([ 242, 427, 1628, 2839, 2871, 4619, 4735, 4857, 4876,\n 5363, 5829, 6387, 6469, 8437, 8913, 9712, 9765, 10921,\n 11056, 11535, 11538, 12082, 12869, 13271, 13823, 14480, 14668,\n 17473, 17868, 19353, 20158, 21248, 21894, 23281, 24201, 25834,\n 26191, 26811, 27566, 28267, 28457, 29152, 29547, 29653, 29949,\n 30155, 30279, 30335, 30471, 30803, 31265, 31825, 32240, 32734,\n 32907, 33093, 33211, 34388, 34693, 34993, 37372, 38062, 38639,\n 38807, 39850, 41997, 42392, 43041, 43593, 43613, 43792, 45247,\n 45306, 45790, 47288, 47344, 50282, 51023, 51752, 54975, 55234,\n 57157, 58230, 58431, 59280, 59559, 59734, 60088, 60813, 61007,\n 62580, 62870, 63087, 63104, 63585], dtype=uint64)\n```", "2420656": "Some scans are acquired this way. They start at the pelvis and move upward toward the chest. Abdomen/pelvic scans are typically done from the chest down though. Don't trust the instance number, or filename. Check the Z coordinates of ImagePositionPatient to know.", "2420661": "tip:\n\ni find this one of the base way to normalise intensity:\n\n```\nimage = read series of dicom file\n\t\tth = torch.quantile(image, 0.5)\n\t\tsample = image[image > th]\n\t\tmean = torch.mean(sample)\n\t\tstd = torch.std(sample)\n\t\ttmin = mean - 2.5 * std\n\t\ttmax = mean + 2.5 * std\n\t\timage = (image - tmin) / (tmax - tmin + 0.001) #use this for input to neural nets\n\n#use this for visualisation\nv = np_sigmoid((image - 0.5) * 8)\n\n\n```", "2422242": "I'm just using `monai.transforms` reared, it does read in the right order. And it has a lot of useful tools.\nhttps://monai.io/ " } }