I was browsing the segmentation images and bounding box data to see if I could utilize data other than images.
\nThen I realized that the annotation of one segmentation was wrong.
\nThe StudyInstanceUID of 1.2.826.0.1.3680043.1363 has the annotation in the .nii file reversed.
\nIf you are interested in using segmentation you need to be careful.
\nI could only find one mistake, but other files could also be wrong.
I have created an animation based on RSNA - CT gifs.
\nA Segmentation is in reverse order
\nI wish you good luck!
I was browsing the segmentation images and bounding box data to see if I could utilize data other than images.
\nThen I realized that the annotation of one segmentation was wrong.
\nThe StudyInstanceUID of 1.2.826.0.1.3680043.1363 has the annotation in the .nii file reversed.
\nIf you are interested in using segmentation you need to be careful.
\nI could only find one mistake, but other files could also be wrong.
I have created an animation based on RSNA - CT gifs.
\nA Segmentation is in reverse order
\nI wish you good luck!
Here is the list of inverted (z-axis) segmentation masks:
\n1363 \n20120\n2243\n23904\n24606\n32071\nDoes this happen only in segmentation masks or there might be inverted labels in classification labels?
\n@felipekitamura
\n@sohier
Regardless of the mask, in some cases the order of feet β head and head β feet is reversed, so I sort all examples using the position on the z-axis.
\ndef load_dicom_array(f):\n dicom_files = glob.glob(f\"{f}/*.dcm\")\n dicoms = [pydicom.dcmread(d) for d in tqdm(dicom_files,total=len(dicom_files))]\n M = float(dicoms[0].RescaleSlope)\n B = float(dicoms[0].RescaleIntercept)\n # Assume all images are axial\n z_pos = np.array([float(d.ImagePositionPatient[-1]) for d in dicoms])#different from patients\n z_inter = np.sort(z_pos)[3]-np.sort(z_pos)[2]\n\n dicoms = np.asarray([d.pixel_array for d in dicoms])\n dicoms = dicoms[np.argsort(-z_pos)]\n dicoms = dicoms * M\n dicoms = dicoms + B\n return dicoms, np.asarray(dicom_files)[np.argsort(-z_pos)],z_inter\n197/2019
\nreverse_ids = [128, 324, 575, 611, 925, 1102, 1177, 1187, 1363, 1381, 1479, 1632, 1836, 1880, 2243, 2374, 2792, 2967, 3072, 3121, 3163, 3271, 3399, 3542, 3717, 3959, 4110, 4216, 4359, 4553, 4722, 4740, 5055, 5393, 5474, 5541, 5699, 5949, 6287, 6409, 6555, 6620, 6862, 6917, 7550, 7680, 8049, 8089, 8362, 8611, 8907, 8939, 9647, 10360, 10443, 10449, 10608, 11026, 11300, 11401, 11506, 11515, 11654, 11683, 12109, 12140, 12145, 12718, 12767, 13324, 13571, 13587, 13913, 14087, 14186, 14345, 14421, 14464, 14833, 14916, 15000, 15213, 15593, 15641, 15776, 15952, 16348, 16386, 16451, 16534, 16833, 17210, 17242, 17284, 17359, 17495, 17964, 18077, 18219, 18534, 18861, 18971, 19304, 19537, 19644, 19675, 19688, 19700, 19705, 20038, 20087, 20120, 20180, 20515, 20574, 20976, 21239, 21312, 21526, 21628, 21684, 21982, 22310, 22358, 22438, 22678, 22980, 23251, 23257, 23325, 23410, 23422, 23697, 23904, 23944, 24018, 24206, 24264, 24316, 24570, 24606, 24878, 24962, 25172, 25278, 25493, 25812, 25834, 25919, 25987, 26040, 26177, 26217, 26306, 26514, 26781, 26933, 27030, 27079, 27262, 27275, 27426, 27436, 27975, 28019, 28355, 28527, 28753, 28794, 28865, 28873, 29218, 29378, 29469, 29986, 30134, 30238, 30307, 30539, 30610, 31114, 31205, 31409, 31419, 31450, 32005, 32017, 32020, 32023, 32071, 32291, 32357, 32387, 32458, 32480, 32627, 32721]
Thank you for sharing the list. Obviously the cause of this problem is while some scans are upside down, the segmentation masks are all in right direction.
", "rawMarkdown": "Thank you for sharing the list. Obviously the cause of this problem is while some scans are upside down, the segmentation masks are all in right direction.", "votes": 3 }, { "id": 1919070, "postDate": "2022-08-30T05:32:58.630Z", "content": "Yes, you are right. Also, the opposite 6 ids seems to be a bit lower in label quality. It may just be the quality of the labels is lower, but perhaps it needs to be addressed more than simply reversing them.
", "rawMarkdown": "Yes, you are right. Also, the opposite 6 ids seems to be a bit lower in label quality. It may just be the quality of the labels is lower, but perhaps it needs to be addressed more than simply reversing them.", "votes": 2 }, { "id": 1954204, "postDate": "2022-09-25T05:03:18.617Z", "content": "can somebody explain what these lines are for please?
\n dicoms = dicoms[np.argsort(-z_pos)]\n dicoms = dicoms * M\n dicoms = dicoms + B\nThose are the slope (M) and intercept (B) for converting the raw pixel values to Hounsfield Units (HU). Different materials/parts of the body have different HU (https://en.wikipedia.org/wiki/Hounsfield_scale). Air is -1000 HU, pure water is 0 HU. Fat is a little negative. Soft tissue is otherwise positive up to a few hundred. Different types of bone are between a few hundred to a few thousand HU. Metal (e.g. some types of implants) are often several thousand.
", "rawMarkdown": "Those are the slope (M) and intercept (B) for converting the raw pixel values to Hounsfield Units (HU). Different materials/parts of the body have different HU (https://en.wikipedia.org/wiki/Hounsfield_scale). Air is -1000 HU, pure water is 0 HU. Fat is a little negative. Soft tissue is otherwise positive up to a few hundred. Different types of bone are between a few hundred to a few thousand HU. Metal (e.g. some types of implants) are often several thousand.", "votes": 2 }, { "id": 1954994, "postDate": "2022-09-25T15:55:46.793Z", "content": "thanks for replying! whats the advantage to converting to HU? sorry im just really new to thisπ π
", "rawMarkdown": "thanks for replying! whats the advantage to converting to HU? sorry im just really new to thisπ π " }, { "id": 1955178, "postDate": "2022-09-25T19:21:37.233Z", "content": "There are probably a few possible advantages and they resolve around the different thresholds that you may want to apply to the images once you have them in HU. When humans are interpreting the images, they often view the same image with different thresholds to better highlight the different tissue types (i.e. they evaluate the bones using thresholds that emphasize bones; they evaluate soft tissue using thresholds the emphasize the soft tissue). For this challenge, I suspect that the bulk of the information regarding the presence of a fracture will be found in the appearance of the bones, however, it's not uncommon for there to be nearby soft tissue changes (e.g. swelling, hematoma) that may not be seen if only the bones are highlighted well. In the setting of questionable bone findings, there may be additional information in the appearance of the adjacent soft tissue (although those same soft tissue findings can be present due to soft tissue/ligament injuries in the absence of a fracture).
\nIn the space of computer vision, for example, you could imagine a crude/naive way to segment out the bones by only keeping the pixels that are between the typical HU limits for bones (say 300 - 3000). This won't fully account for things like dense soft tissue creeping up above 300 HU, osteopenic/osteoporotic bones that dip below 300, image noise, image artifacts, dense implants that have HUs in the range of bones (e.g. lines or tubes), etc.
\nSome folks have suggested and successfully used a few different threshold levels to create different versions of each image that can serve as channels (which somewhat mimics the typical 3-channel RGB images that are commonly used as input to pre-trained state of the art CNNs (e.g. Resnet).
", "rawMarkdown": "There are probably a few possible advantages and they resolve around the different thresholds that you may want to apply to the images once you have them in HU. When humans are interpreting the images, they often view the same image with different thresholds to better highlight the different tissue types (i.e. they evaluate the bones using thresholds that emphasize bones; they evaluate soft tissue using thresholds the emphasize the soft tissue). For this challenge, I suspect that the bulk of the information regarding the presence of a fracture will be found in the appearance of the bones, however, it's not uncommon for there to be nearby soft tissue changes (e.g. swelling, hematoma) that may not be seen if only the bones are highlighted well. In the setting of questionable bone findings, there may be additional information in the appearance of the adjacent soft tissue (although those same soft tissue findings can be present due to soft tissue/ligament injuries in the absence of a fracture).\n\nIn the space of computer vision, for example, you could imagine a crude/naive way to segment out the bones by only keeping the pixels that are between the typical HU limits for bones (say 300 - 3000). This won't fully account for things like dense soft tissue creeping up above 300 HU, osteopenic/osteoporotic bones that dip below 300, image noise, image artifacts, dense implants that have HUs in the range of bones (e.g. lines or tubes), etc.\n\nSome folks have suggested and successfully used a few different threshold levels to create different versions of each image that can serve as channels (which somewhat mimics the typical 3-channel RGB images that are commonly used as input to pre-trained state of the art CNNs (e.g. Resnet). ", "votes": 4 }, { "id": 1955542, "postDate": "2022-09-26T04:00:12.770Z", "content": "thanks so much!
", "rawMarkdown": "thanks so much!" } ] }, { "id": 1989876, "postDate": "2022-10-16T08:01:44.673Z", "content": "I want to know whether there is such a reverse order in the hidden data
", "rawMarkdown": "I want to know whether there is such a reverse order in the hidden data" } ], "comments": [ { "id": 1918318, "author_name": "RabotniKuma", "author_url": "", "post_date": "2022-08-29T13:45:07.853000", "content": "Here is the list of inverted (z-axis) segmentation masks:
\n1363 \n20120\n2243\n23904\n24606\n32071\nDoes this happen only in segmentation masks or there might be inverted labels in classification labels?
\n@felipekitamura
\n@sohier
Regardless of the mask, in some cases the order of feet β head and head β feet is reversed, so I sort all examples using the position on the z-axis.
\ndef load_dicom_array(f):\n dicom_files = glob.glob(f\"{f}/*.dcm\")\n dicoms = [pydicom.dcmread(d) for d in tqdm(dicom_files,total=len(dicom_files))]\n M = float(dicoms[0].RescaleSlope)\n B = float(dicoms[0].RescaleIntercept)\n # Assume all images are axial\n z_pos = np.array([float(d.ImagePositionPatient[-1]) for d in dicoms])#different from patients\n z_inter = np.sort(z_pos)[3]-np.sort(z_pos)[2]\n\n dicoms = np.asarray([d.pixel_array for d in dicoms])\n dicoms = dicoms[np.argsort(-z_pos)]\n dicoms = dicoms * M\n dicoms = dicoms + B\n return dicoms, np.asarray(dicom_files)[np.argsort(-z_pos)],z_inter\n197/2019
\nreverse_ids = [128, 324, 575, 611, 925, 1102, 1177, 1187, 1363, 1381, 1479, 1632, 1836, 1880, 2243, 2374, 2792, 2967, 3072, 3121, 3163, 3271, 3399, 3542, 3717, 3959, 4110, 4216, 4359, 4553, 4722, 4740, 5055, 5393, 5474, 5541, 5699, 5949, 6287, 6409, 6555, 6620, 6862, 6917, 7550, 7680, 8049, 8089, 8362, 8611, 8907, 8939, 9647, 10360, 10443, 10449, 10608, 11026, 11300, 11401, 11506, 11515, 11654, 11683, 12109, 12140, 12145, 12718, 12767, 13324, 13571, 13587, 13913, 14087, 14186, 14345, 14421, 14464, 14833, 14916, 15000, 15213, 15593, 15641, 15776, 15952, 16348, 16386, 16451, 16534, 16833, 17210, 17242, 17284, 17359, 17495, 17964, 18077, 18219, 18534, 18861, 18971, 19304, 19537, 19644, 19675, 19688, 19700, 19705, 20038, 20087, 20120, 20180, 20515, 20574, 20976, 21239, 21312, 21526, 21628, 21684, 21982, 22310, 22358, 22438, 22678, 22980, 23251, 23257, 23325, 23410, 23422, 23697, 23904, 23944, 24018, 24206, 24264, 24316, 24570, 24606, 24878, 24962, 25172, 25278, 25493, 25812, 25834, 25919, 25987, 26040, 26177, 26217, 26306, 26514, 26781, 26933, 27030, 27079, 27262, 27275, 27426, 27436, 27975, 28019, 28355, 28527, 28753, 28794, 28865, 28873, 29218, 29378, 29469, 29986, 30134, 30238, 30307, 30539, 30610, 31114, 31205, 31409, 31419, 31450, 32005, 32017, 32020, 32023, 32071, 32291, 32357, 32387, 32458, 32480, 32627, 32721]
Thank you for sharing the list. Obviously the cause of this problem is while some scans are upside down, the segmentation masks are all in right direction.
", "votes": 3, "replies": [] }, { "id": 1919070, "author_name": "YujiAriyasu", "author_url": "", "post_date": "2022-08-30T05:32:58.630000", "content": "Yes, you are right. Also, the opposite 6 ids seems to be a bit lower in label quality. It may just be the quality of the labels is lower, but perhaps it needs to be addressed more than simply reversing them.
", "votes": 2, "replies": [] }, { "id": 1954204, "author_name": "JINO ROHIT", "author_url": "", "post_date": "2022-09-25T05:03:18.617000", "content": "can somebody explain what these lines are for please?
\n dicoms = dicoms[np.argsort(-z_pos)]\n dicoms = dicoms * M\n dicoms = dicoms + B\nThose are the slope (M) and intercept (B) for converting the raw pixel values to Hounsfield Units (HU). Different materials/parts of the body have different HU (https://en.wikipedia.org/wiki/Hounsfield_scale). Air is -1000 HU, pure water is 0 HU. Fat is a little negative. Soft tissue is otherwise positive up to a few hundred. Different types of bone are between a few hundred to a few thousand HU. Metal (e.g. some types of implants) are often several thousand.
", "votes": 2, "replies": [] }, { "id": 1954994, "author_name": "JINO ROHIT", "author_url": "", "post_date": "2022-09-25T15:55:46.793000", "content": "thanks for replying! whats the advantage to converting to HU? sorry im just really new to thisπ π
", "votes": 0, "replies": [] }, { "id": 1955178, "author_name": "Greg Wehner", "author_url": "", "post_date": "2022-09-25T19:21:37.233000", "content": "There are probably a few possible advantages and they resolve around the different thresholds that you may want to apply to the images once you have them in HU. When humans are interpreting the images, they often view the same image with different thresholds to better highlight the different tissue types (i.e. they evaluate the bones using thresholds that emphasize bones; they evaluate soft tissue using thresholds the emphasize the soft tissue). For this challenge, I suspect that the bulk of the information regarding the presence of a fracture will be found in the appearance of the bones, however, it's not uncommon for there to be nearby soft tissue changes (e.g. swelling, hematoma) that may not be seen if only the bones are highlighted well. In the setting of questionable bone findings, there may be additional information in the appearance of the adjacent soft tissue (although those same soft tissue findings can be present due to soft tissue/ligament injuries in the absence of a fracture).
\nIn the space of computer vision, for example, you could imagine a crude/naive way to segment out the bones by only keeping the pixels that are between the typical HU limits for bones (say 300 - 3000). This won't fully account for things like dense soft tissue creeping up above 300 HU, osteopenic/osteoporotic bones that dip below 300, image noise, image artifacts, dense implants that have HUs in the range of bones (e.g. lines or tubes), etc.
\nSome folks have suggested and successfully used a few different threshold levels to create different versions of each image that can serve as channels (which somewhat mimics the typical 3-channel RGB images that are commonly used as input to pre-trained state of the art CNNs (e.g. Resnet).
", "votes": 4, "replies": [] }, { "id": 1955542, "author_name": "JINO ROHIT", "author_url": "", "post_date": "2022-09-26T04:00:12.770000", "content": "thanks so much!
", "votes": 0, "replies": [] } ] }, { "id": 1989876, "author_name": "qiucen", "author_url": "", "post_date": "2022-10-16T08:01:44.673000", "content": "I want to know whether there is such a reverse order in the hidden data
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1918290": "I was browsing the segmentation images and bounding box data to see if I could utilize data other than images.\nThen I realized that the annotation of one segmentation was wrong.\n**The StudyInstanceUID of 1.2.826.0.1.3680043.1363 has the annotation in the .nii file reversed.**\nIf you are interested in using segmentation you need to be careful.\nI could only find one mistake, but other files could also be wrong.\n\nI have created an animation based on [RSNA - CT gifs](https://www.kaggle.com/code/samuelcortinhas/rsna-ct-gifs).\n[A Segmentation is in reverse order](https://www.kaggle.com/code/itsuki9180/a-segmentation-is-in-reverse-order)\nI wish you good luck!", "1918318": "Here is the list of inverted (z-axis) segmentation masks: \n```\n1363 \n20120\n2243\n23904\n24606\n32071\n```\n\nDoes this happen only in segmentation masks or there might be inverted labels in classification labels?\n@felipekitamura \n@sohier ", "1989876": "I want to know whether there is such a reverse order in the hidden data" } }