{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"
\n

RSNA 2022 Cervical Spine Fracture Detection

\n

Understanding the Dataset

\n

(as a beginner)

\n @shreydan\n
\n\n### Please make sure to checkout the credits and references at the end!\n___","metadata":{}},{"cell_type":"markdown","source":"# Installations & Imports\n___\n### from [@andradaolteanu](https://www.kaggle.com/andradaolteanu), [@ipythonx](https://www.kaggle.com/datasets/ipythonx/for-pydicom)","metadata":{}},{"cell_type":"markdown","source":"### Online","metadata":{}},{"cell_type":"code","source":"# !pip install -qU \"python-gdcm\" pydicom pylibjpeg \"opencv-python-headless\"","metadata":{"_kg_hide-output":true,"execution":{"iopub.status.busy":"2022-10-04T15:15:05.889093Z","iopub.execute_input":"2022-10-04T15:15:05.889759Z","iopub.status.idle":"2022-10-04T15:15:05.918197Z","shell.execute_reply.started":"2022-10-04T15:15:05.889513Z","shell.execute_reply":"2022-10-04T15:15:05.917185Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Offline","metadata":{}},{"cell_type":"code","source":"!pip install -qU ../input/for-pydicom/python_gdcm-3.0.14-cp37-cp37m-manylinux_2_17_x86_64.manylinux2014_x86_64.whl ../input/for-pydicom/pylibjpeg-1.4.0-py3-none-any.whl --find-links frozen_packages --no-index","metadata":{"execution":{"iopub.status.busy":"2022-10-04T15:15:05.920224Z","iopub.execute_input":"2022-10-04T15:15:05.920887Z","iopub.status.idle":"2022-10-04T15:15:19.960005Z","shell.execute_reply.started":"2022-10-04T15:15:05.920851Z","shell.execute_reply":"2022-10-04T15:15:19.958859Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport plotly.express as px\nfrom plotly.offline import init_notebook_mode, iplot, plot\nfrom tqdm.notebook import tqdm\nfrom pathlib import Path\nfrom collections import Counter\n\ntqdm.pandas()","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-10-04T15:15:19.961613Z","iopub.execute_input":"2022-10-04T15:15:19.962023Z","iopub.status.idle":"2022-10-04T15:15:21.820616Z","shell.execute_reply.started":"2022-10-04T15:15:19.961983Z","shell.execute_reply":"2022-10-04T15:15:21.818547Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# for dcm and nii\nimport pydicom\nimport nibabel as nib\nfrom pydicom.pixel_data_handlers.util import apply_voi_lut","metadata":{"execution":{"iopub.status.busy":"2022-10-04T15:15:21.825617Z","iopub.execute_input":"2022-10-04T15:15:21.826163Z","iopub.status.idle":"2022-10-04T15:15:22.169276Z","shell.execute_reply.started":"2022-10-04T15:15:21.826112Z","shell.execute_reply":"2022-10-04T15:15:22.167892Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"paths = {\n 'train_df': Path('../input/rsna-2022-cervical-spine-fracture-detection/train.csv'),\n 'train_bbox': Path('../input/rsna-2022-cervical-spine-fracture-detection/train_bounding_boxes.csv'),\n 'train_images': Path('../input/rsna-2022-cervical-spine-fracture-detection/train_images'),\n 'train_nifti_segments': Path('../input/rsna-2022-cervical-spine-fracture-detection/segmentations'),\n 'test_df': Path('../input/rsna-2022-cervical-spine-fracture-detection/test.csv'),\n 'test_images': Path('../input/rsna-2022-cervical-spine-fracture-detection/test_images')\n}","metadata":{"execution":{"iopub.status.busy":"2022-10-04T15:15:22.171857Z","iopub.execute_input":"2022-10-04T15:15:22.173081Z","iopub.status.idle":"2022-10-04T15:15:22.178774Z","shell.execute_reply.started":"2022-10-04T15:15:22.173039Z","shell.execute_reply":"2022-10-04T15:15:22.177896Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# **Preparing Main DataFrame**\n___","metadata":{}},{"cell_type":"code","source":"train_df = pd.read_csv(paths['train_df'])\ntest_df = pd.read_csv(paths['test_df'])","metadata":{"execution":{"iopub.status.busy":"2022-10-04T15:15:22.18015Z","iopub.execute_input":"2022-10-04T15:15:22.180517Z","iopub.status.idle":"2022-10-04T15:15:22.21961Z","shell.execute_reply.started":"2022-10-04T15:15:22.180484Z","shell.execute_reply":"2022-10-04T15:15:22.218103Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Column Information\n___\n\n- `StudyInstanceUID` - The study ID. There is one unique study ID for each patient scan.\n- `patient_overall` - One of the target columns. The patient level outcome, i.e. if any of the vertebrae are fractured.\n- `C[1-7]` - The other target columns. Whether the given vertebrae is fractured. See this diagram for the real location of each vertbrae in the spine.\n","metadata":{}},{"cell_type":"code","source":"train_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-10-04T15:15:22.221127Z","iopub.execute_input":"2022-10-04T15:15:22.221519Z","iopub.status.idle":"2022-10-04T15:15:22.248583Z","shell.execute_reply.started":"2022-10-04T15:15:22.221483Z","shell.execute_reply":"2022-10-04T15:15:22.247323Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Ignoring a specific StudyInstanceUID\nDiscussion Link: [The scan 1.2.826.0.1.3680043.20574 does not include a full cervical spine and should be ignored. Thanks to @kretes for catching this.](https://www.kaggle.com/competitions/rsna-2022-cervical-spine-fracture-detection/discussion/344862)","metadata":{}},{"cell_type":"code","source":"train_df[train_df['StudyInstanceUID'] == '1.2.826.0.1.3680043.20574']","metadata":{"execution":{"iopub.status.busy":"2022-10-04T15:15:22.250117Z","iopub.execute_input":"2022-10-04T15:15:22.250643Z","iopub.status.idle":"2022-10-04T15:15:22.271894Z","shell.execute_reply.started":"2022-10-04T15:15:22.250599Z","shell.execute_reply":"2022-10-04T15:15:22.270882Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.drop(index=1135, inplace=True)\ntrain_df.reset_index(drop=True, inplace=True)","metadata":{"execution":{"iopub.status.busy":"2022-10-04T15:15:22.273323Z","iopub.execute_input":"2022-10-04T15:15:22.273672Z","iopub.status.idle":"2022-10-04T15:15:22.28412Z","shell.execute_reply.started":"2022-10-04T15:15:22.273641Z","shell.execute_reply":"2022-10-04T15:15:22.282788Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-10-04T15:15:22.288738Z","iopub.execute_input":"2022-10-04T15:15:22.289167Z","iopub.status.idle":"2022-10-04T15:15:22.30555Z","shell.execute_reply.started":"2022-10-04T15:15:22.28913Z","shell.execute_reply":"2022-10-04T15:15:22.303828Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## adding `total_fractures` column to indicate how many cervical vertebrae are fractured","metadata":{}},{"cell_type":"code","source":"train_df['total_fractures'] = train_df.loc[:,[f\"C{i}\" for i in range(1,8)]].sum(axis=1)","metadata":{"execution":{"iopub.status.busy":"2022-10-04T15:15:22.306984Z","iopub.execute_input":"2022-10-04T15:15:22.307396Z","iopub.status.idle":"2022-10-04T15:15:22.32044Z","shell.execute_reply.started":"2022-10-04T15:15:22.307359Z","shell.execute_reply":"2022-10-04T15:15:22.31924Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.head(2)","metadata":{"execution":{"iopub.status.busy":"2022-10-04T15:15:22.322234Z","iopub.execute_input":"2022-10-04T15:15:22.322668Z","iopub.status.idle":"2022-10-04T15:15:22.341015Z","shell.execute_reply.started":"2022-10-04T15:15:22.322633Z","shell.execute_reply":"2022-10-04T15:15:22.339769Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"len(train_df)","metadata":{"execution":{"iopub.status.busy":"2022-10-04T15:15:22.342377Z","iopub.execute_input":"2022-10-04T15:15:22.342732Z","iopub.status.idle":"2022-10-04T15:15:22.353834Z","shell.execute_reply.started":"2022-10-04T15:15:22.34268Z","shell.execute_reply":"2022-10-04T15:15:22.352641Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## **Now in total, We've 2018 study IDs**","metadata":{}},{"cell_type":"markdown","source":"___","metadata":{}},{"cell_type":"markdown","source":"# **Understanding CT Scans**\n\n...A CT scan generates images that can be reformatted in multiple planes. It can even generate three-dimensional images...[read more](https://www.radiologyinfo.org/en/info/bodyct)\n\nThis allows multiple planes of observation, the most common one being axial.\n\n![CT Scan Planes](https://www.ipfradiologyrounds.com/_images/image-reconstruction-planes.png)\n\n[Image Source](https://www.ipfradiologyrounds.com/hrct-primer/image-reconstruction/)\n\n### The data in `train_images` and `test_images` is segmented as per the axial plane.","metadata":{}},{"cell_type":"markdown","source":"## Adding segmentations path of each patient to the dataframe","metadata":{}},{"cell_type":"code","source":"train_df['segment_path'] = train_df['StudyInstanceUID'].map(lambda x: paths['train_images']/x)","metadata":{"execution":{"iopub.status.busy":"2022-10-04T15:44:52.741974Z","iopub.execute_input":"2022-10-04T15:44:52.742553Z","iopub.status.idle":"2022-10-04T15:44:52.801905Z","shell.execute_reply.started":"2022-10-04T15:44:52.742477Z","shell.execute_reply":"2022-10-04T15:44:52.800688Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df['segment_path']","metadata":{"execution":{"iopub.status.busy":"2022-10-04T16:53:54.287762Z","iopub.execute_input":"2022-10-04T16:53:54.288727Z","iopub.status.idle":"2022-10-04T16:53:54.298953Z","shell.execute_reply.started":"2022-10-04T16:53:54.288674Z","shell.execute_reply":"2022-10-04T16:53:54.297611Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Each patient has different number of slices, adding `num_slices` to each patient","metadata":{}},{"cell_type":"code","source":"def num_slices(path):\n slices = list(path.glob('*'))\n return len(slices)\n\ntrain_df['num_slices'] = train_df['segment_path'].progress_map(num_slices)\ntrain_df['num_slices'] = train_df['num_slices'].astype('int')","metadata":{"execution":{"iopub.status.busy":"2022-10-04T16:56:33.411034Z","iopub.execute_input":"2022-10-04T16:56:33.411729Z","iopub.status.idle":"2022-10-04T16:57:03.312188Z","shell.execute_reply.started":"2022-10-04T16:56:33.411667Z","shell.execute_reply":"2022-10-04T16:57:03.310798Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-10-04T15:17:07.424019Z","iopub.execute_input":"2022-10-04T15:17:07.42491Z","iopub.status.idle":"2022-10-04T15:17:07.441521Z","shell.execute_reply.started":"2022-10-04T15:17:07.424854Z","shell.execute_reply":"2022-10-04T15:17:07.44004Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.head(2)","metadata":{"execution":{"iopub.status.busy":"2022-10-04T15:17:07.443434Z","iopub.execute_input":"2022-10-04T15:17:07.444349Z","iopub.status.idle":"2022-10-04T15:17:07.464605Z","shell.execute_reply.started":"2022-10-04T15:17:07.444294Z","shell.execute_reply":"2022-10-04T15:17:07.463632Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"___\n# **Some basic EDA**\n___","metadata":{}},{"cell_type":"code","source":"fig = px.pie(train_df['patient_overall'].value_counts().reset_index(), \n names=['not_fractured','fractured'], # [0,1]\n values='patient_overall',\n color_discrete_sequence = px.colors.qualitative.Pastel,\n title='Patient Fracture Overall Distribution'\n ).update_traces(textinfo='label+percent')\niplot(fig)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-10-04T15:17:07.465852Z","iopub.execute_input":"2022-10-04T15:17:07.466982Z","iopub.status.idle":"2022-10-04T15:17:08.687713Z","shell.execute_reply.started":"2022-10-04T15:17:07.466944Z","shell.execute_reply":"2022-10-04T15:17:08.686469Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fractures = train_df.loc[:,[f\"C{i}\" for i in range(1,8)]].sum().reset_index()\nfractures['not_fractured'] = fractures[0].map(lambda x:len(train_df)-x)\nfractures.rename(columns={'index':'vertebra',0:'fractured'}, inplace=True)\nfig = px.bar(fractures, \n x='vertebra', \n y=['fractured','not_fractured'],\n title='Fractures as per Vertebra',\n color_discrete_sequence = ['#e0465f','#afde2c'],\n )\niplot(fig)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-10-04T15:17:08.68947Z","iopub.execute_input":"2022-10-04T15:17:08.690351Z","iopub.status.idle":"2022-10-04T15:17:08.838839Z","shell.execute_reply.started":"2022-10-04T15:17:08.6903Z","shell.execute_reply":"2022-10-04T15:17:08.837596Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = px.bar(train_df['total_fractures'].value_counts().reset_index(), \n x='index', \n y='total_fractures',\n title='No. of fractures amongst patients',\n text='total_fractures',\n color_discrete_sequence = ['#d82cde'],\n )\niplot(fig)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-10-04T15:17:08.840159Z","iopub.execute_input":"2022-10-04T15:17:08.840554Z","iopub.status.idle":"2022-10-04T15:17:08.932718Z","shell.execute_reply.started":"2022-10-04T15:17:08.840518Z","shell.execute_reply":"2022-10-04T15:17:08.93156Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"fig = px.histogram(train_df, x=\"num_slices\")\niplot(fig)","metadata":{"execution":{"iopub.status.busy":"2022-10-04T15:17:08.934302Z","iopub.execute_input":"2022-10-04T15:17:08.93466Z","iopub.status.idle":"2022-10-04T15:17:09.051473Z","shell.execute_reply.started":"2022-10-04T15:17:08.934628Z","shell.execute_reply":"2022-10-04T15:17:09.050165Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"___","metadata":{}},{"cell_type":"markdown","source":"# **Understanding the different medical-imaging file formats**\n\n- `.dcm` - [DICOM](https://fileinfo.com/extension/dcm) file: A DCM file is an image file saved in the Digital Imaging and Communications in Medicine (DICOM) image format. It stores a medical image, such as a CT scan or ultrasound, and may also include patient information to pair the image with the patient. \n\n- `.nii` - [NIFTI file format](https://brainder.org/2012/09/23/the-nifti-file-format/) - [A nibabel image](https://nipy.org/nibabel/nibabel_images.html) object is the association of three things:\n - an N-D array containing the image data;\n - a (4, 4) affine matrix mapping array coordinates to coordinates in some RAS+ world coordinate space (Coordinate systems and affines);\n - image metadata in the form of a header.\n","metadata":{}},{"cell_type":"markdown","source":"___\n\n# **Loading DICOM and NIFTI files**\n### from [@andradaolteanu](https://www.kaggle.com/andradaolteanu) and [@harshitsheoran](https://www.kaggle.com/harshitsheoran)","metadata":{}},{"cell_type":"markdown","source":"## **Let's try opening a random `.dcm` and a `.nii` file**","metadata":{}},{"cell_type":"markdown","source":"## **.dcm**","metadata":{}},{"cell_type":"code","source":"random_dcm_file = list(train_df['segment_path'][123].glob('*'))[0]\nprint(random_dcm_file)\nrandom_dcm_file = pydicom.dcmread(random_dcm_file)\nprint(random_dcm_file)","metadata":{"execution":{"iopub.status.busy":"2022-10-04T15:17:09.053219Z","iopub.execute_input":"2022-10-04T15:17:09.053716Z","iopub.status.idle":"2022-10-04T15:17:09.076051Z","shell.execute_reply.started":"2022-10-04T15:17:09.05365Z","shell.execute_reply":"2022-10-04T15:17:09.07481Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# getting metadata with .get(key)\nprint(\"Instance Number:\",random_dcm_file.get('InstanceNumber'))\nprint(\"Rows x Columns:\", random_dcm_file.get(\"Rows\"), random_dcm_file.get(\"Columns\"))\nprint(\"Image Position (Patient):\", random_dcm_file.get(\"ImagePositionPatient\"))","metadata":{"execution":{"iopub.status.busy":"2022-10-04T15:17:09.077402Z","iopub.execute_input":"2022-10-04T15:17:09.077785Z","iopub.status.idle":"2022-10-04T15:17:09.08486Z","shell.execute_reply.started":"2022-10-04T15:17:09.077741Z","shell.execute_reply":"2022-10-04T15:17:09.083658Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## from the above metadata:\n- patient ID = patient Name\n- `Slice Thickness` = 1.0 mm [might not be same for all patients]\n- `Instance Number` = slice number\n- `Image Position (Patient)`: gives an array of values [x-axis, y-axis, z-axis], here z-axis denotes the position in the sagittal plane\n\n`apply_voi_lut()` applies a windowing function to the image - [read more here](https://dicom.innolitics.com/ciods/ct-image/voi-lut/00281056)","metadata":{}},{"cell_type":"code","source":"dcm_image = apply_voi_lut(random_dcm_file.pixel_array, random_dcm_file)","metadata":{"execution":{"iopub.status.busy":"2022-10-04T15:17:09.086343Z","iopub.execute_input":"2022-10-04T15:17:09.086721Z","iopub.status.idle":"2022-10-04T15:17:09.106484Z","shell.execute_reply.started":"2022-10-04T15:17:09.086671Z","shell.execute_reply":"2022-10-04T15:17:09.105159Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.imshow(dcm_image, cmap='bone')","metadata":{"execution":{"iopub.status.busy":"2022-10-04T15:17:09.10956Z","iopub.execute_input":"2022-10-04T15:17:09.110479Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"___\n\n## **.nii**","metadata":{}},{"cell_type":"code","source":"random_nii_file = list(paths['train_nifti_segments'].glob('*'))[10]\nprint(random_nii_file)\n\ndef open_nii_file(path):\n f = nib.load(path)\n segmentations = f.get_fdata()[:, ::-1, ::-1].transpose(2, 1, 0)\n return segmentations\n\nnii_segments = open_nii_file(random_nii_file)\nprint(nii_segments.shape, \"=> (num_slices, height, width)\")\nplt.imshow(nii_segments[123,:,:],cmap='bone')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### A caveat of axial plane is that we are not aware of which cervical vertebra we are looking at, the above `nii_segments` array can be reused to get the vertebra index using `np.unique()`","metadata":{}},{"cell_type":"code","source":"# np.unique(nii_segments[slice_number])\nprint(\"[background, *, vertebra]\")\nprint(\"here 4. denotes that the slice is a part of C4 vertebra\")\nnp.unique(nii_segments[100])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"___","metadata":{}},{"cell_type":"markdown","source":"## not all study UIDs have nii segments, for the ones we do have, adding their path to df","metadata":{}},{"cell_type":"code","source":"f\"We only have {len(list(paths['train_nifti_segments'].glob('*')))} nii files / {len(train_df)} studies\"","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def add_nii_segment_path(uid):\n base_path = paths['train_nifti_segments']\n # path if exists else None\n path = base_path/(uid+'.nii')\n if path.exists():\n return path\n return None\n\ntrain_df['nii_segments_path'] = train_df['StudyInstanceUID'].map(add_nii_segment_path)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"___","metadata":{}},{"cell_type":"code","source":"def get_dcm_images(path):\n paths = list(path.glob('*'))\n paths.sort(key=lambda x:int(x.stem)) # sort based on slice index which is the filename: index.dcm\n data = [pydicom.dcmread(f) for f in paths]\n images = [apply_voi_lut(dcm.pixel_array, dcm) for dcm in data]\n return images","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def get_nii_segments(path):\n f = nib.load(path)\n segmentations = f.get_fdata()[:, ::-1, ::-1].transpose(2, 1, 0)\n return segmentations","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sample_idx = 99\ndcm_images = get_dcm_images(train_df['segment_path'][sample_idx])\nnii_segments = get_nii_segments(train_df['nii_segments_path'][sample_idx])\nprint((len(dcm_images), *dcm_images[0].shape), nii_segments.shape)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def plot_slice(idx, dcm_images=dcm_images, nii_segments=nii_segments):\n fig, (ax1, ax2) = plt.subplots(1, 2)\n ax1.axis('off'); ax2.axis('off')\n fig.suptitle(f'Slice {idx}')\n ax1.imshow(dcm_images[idx], cmap='bone')\n ax2.imshow(nii_segments[idx,:,:], cmap='bone')\n\nfor i in range(123,128):\n plot_slice(i)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"___\n\n## all UIDs which have segmentations:","metadata":{}},{"cell_type":"code","source":"uids_with_segments = train_df[train_df['nii_segments_path'].notnull()]\nuids_with_segments.head()","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"___\n\n# **some scans maybe in reverse order**\n\n[A Segmentation is in reverse order by @itsuki9180](https://www.kaggle.com/code/itsuki9180/a-segmentation-is-in-reverse-order/comments) and [based on this comment by @abebe9849](https://www.kaggle.com/competitions/rsna-2022-cervical-spine-fracture-detection/discussion/348658#1918410)\n\n#### based on [this discussion](https://www.kaggle.com/code/itsuki9180/a-segmentation-is-in-reverse-order/comments#1919812), for the z_pos value in ImagePositionPatient. If z_pos of last slice is greater than z_pos of first slice, then we've to reverse the order since positive is towards the head of the patient\n___","metadata":{}},{"cell_type":"code","source":"def check_reverse_required(path):\n paths = list(path.glob('*'))\n paths.sort(key=lambda x:int(x.stem))\n z_first = pydicom.dcmread(paths[0]).get(\"ImagePositionPatient\")[-1]\n z_last = pydicom.dcmread(paths[-1]).get(\"ImagePositionPatient\")[-1]\n if z_last < z_first:\n return False\n return True","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"checks = train_df['segment_path'].progress_map(check_reverse_required)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df['reverse_required'] = checks","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Counter(checks)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"indices_where_reverse_required = [i for i,req in (checks.reset_index()).values if req is True]\nprint(indices_where_reverse_required)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"___\n\n# **Generating Sagittal View**\n### based on [this comment by @harshitsheoran](https://www.kaggle.com/competitions/rsna-2022-cervical-spine-fracture-detection/discussion/340612#1891123) which gives a decently usable saggital view","metadata":{}},{"cell_type":"code","source":"print(train_df.loc[41,:])\ndcm_images = get_dcm_images(train_df['segment_path'][41]) # 0 fractures\ndcm_np = np.array(dcm_images)\nif train_df['reverse_required'][41] == True:\n dcm_np = dcm_np[::-1]\nsaggital_view = dcm_np[:,:,256]\nplt.imshow(saggital_view, cmap='bone')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(train_df.loc[60,:])\ndcm_images = get_dcm_images(train_df['segment_path'][60]) # 4 fractures\ndcm_np = np.array(dcm_images)\nif train_df['reverse_required'][60] == True:\n dcm_np = dcm_np[::-1]\nsaggital_view = dcm_np[:,:,256]\nplt.imshow(saggital_view, cmap='bone')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(train_df.loc[4,:])\ndcm_images = get_dcm_images(train_df['segment_path'][4]) # 4 fractures\ndcm_np = np.array(dcm_images)\nif train_df['reverse_required'][4] == True:\n dcm_np = dcm_np[::-1]\nsaggital_view = dcm_np[:,:,256]\nplt.imshow(saggital_view, cmap='bone')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## You can run this to get all the saggital views\n","metadata":{}},{"cell_type":"code","source":"# !mkdir saggital_views\n\n# saggital_path = Path('./saggital_views')\n# for uid, path, rev in tqdm(train_df.loc[:,['StudyInstanceUID','segment_path','reverse_required']].values):\n# dcm_images = get_dcm_images(path)\n# dcm_np = np.array(dcm_images)\n# if rev == True:\n# dcm_np = dcm_np[::-1]\n# saggital_view = dcm_np[:,:,256]\n# plt.imsave(saggital_path/(uid+'.png'),saggital_view,cmap='bone')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Saving","metadata":{}},{"cell_type":"code","source":"train_df.to_csv('train_new.csv')\nuids_with_segments.to_csv('train_with_segments.csv')","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"___","metadata":{}},{"cell_type":"markdown","source":"# **Animation**\n___","metadata":{}},{"cell_type":"code","source":"sample_index = train_df.iloc[99,:]\ndcm_images = get_dcm_images(sample_index['segment_path'])\nif sample_index['reverse_required'] == True:\n dcm_images.sort(reverse=True)\nsegments = get_nii_segments(sample_index['nii_segments_path'])","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"from matplotlib import animation, rc\nrc('animation', html='jshtml')\n\nfig, [ax1,ax2] = plt.subplots(1,2)\nax1.axis('off')\nax2.axis('off')\nimages = []\nfor i in tqdm(range(len(dcm_images))):\n im1 = ax1.imshow(dcm_images[i], animated=True, cmap='bone')\n im2 = ax2.imshow(segments[i,:,:], animated=True, cmap='bone')\n if i==0:\n ax1.imshow(dcm_images[i], cmap='bone')\n ax2.imshow(segments[i,:,:], cmap='bone')\n images.append([im1,im2])\n \n\nani = animation.ArtistAnimation(fig, images, interval=50, blit=True,\n repeat_delay=1000)\nplt.close()\nani","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"___","metadata":{}},{"cell_type":"markdown","source":"# **References**\n\n> This community is incredible, a few days ago I had no idea what DICOM files were but today I have a basic understanding of how medical imaging works. I went through many notebooks and discussions to understand how everyone was handling the dataset.\n\n## These are some references I used:\nIf I missed anything here, they should be in the notebook in the above markdown cells somewhere\n\n- ### Discussions\n - #### [Explaining Data and Submission in detail](https://www.kaggle.com/competitions/rsna-2022-cervical-spine-fracture-detection/discussion/340612) by [@harshitsheoran](https://www.kaggle.com/harshitsheoran)\n > this was very helpful. @harshitsheoran explained everything perfectly which gave me the confidence to try out this dataset\n - #### [Order of Slices](https://www.kaggle.com/code/itsuki9180/a-segmentation-is-in-reverse-order/comments#1919812) by [@solverworld](https://www.kaggle.com/solverworld)\n - #### [Sorting the DICOM files](https://www.kaggle.com/competitions/rsna-2022-cervical-spine-fracture-detection/discussion/348658#1918410) by [@abebe9849](https://www.kaggle.com/abebe9849)\n - #### [Generating Saggital Views](https://www.kaggle.com/competitions/rsna-2022-cervical-spine-fracture-detection/discussion/340612#1891123) by [@harshitsheoran](https://www.kaggle.com/harshitsheoran)\n\n- ### Notebooks\n - #### [rsna-fracture-detection-dicom-images-explore](https://www.kaggle.com/code/andradaolteanu/rsna-fracture-detection-dicom-images-explore) by [@andradaolteanu](https://www.kaggle.com/andradaolteanu)\n - #### [rsna-fracture-detection-in-depth-eda](https://www.kaggle.com/code/samuelcortinhas/rsna-fracture-detection-in-depth-eda) by [@samuelcortinhas](https://www.kaggle.com/samuelcortinhas)\n - #### [spine-fracture-eda-loading-dicom-3d-browse](https://www.kaggle.com/code/jirkaborovec/spine-fracture-eda-loading-dicom-3d-browse) by [@jirkaborovec](https://www.kaggle.com/jirkaborovec)\n - #### [a-segmentation-is-in-reverse-order](https://www.kaggle.com/code/itsuki9180/a-segmentation-is-in-reverse-order) by [@itsuki9180](https://www.kaggle.com/itsuki9180)\n \n\n\n\n## Thank you to all the users for their work and help. It was fun to try this out :)\n","metadata":{}}]}