{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Introduction\n\nMedical Open Network for Artificial Intelligence ([MONAI](https://monai.io)) is an open-source framework for deep-learning in healthcare imaging. MONAI is packed with useful features that can ease the workflow of working with medical images, including pathology whole slide images. The package has several superclasses that take care of IO, preprocessing, model training, inference, and post-processing. \n\nIn this tutorial, we will use MONAI to read the competition data, including the DICOM files and segmentation masks, and then will go over saving the images in NIfTI format for future use.","metadata":{}},{"cell_type":"markdown","source":"# Installation\n\nMONAI is based on PyTorch framework and only requires `pytorch` and `numpy` to have its baseline functionality. However, for reading DICOM files, it relies on `itk`. I have packaged these two in a kaggle dataset ([MONAI + ITK](https://www.kaggle.com/datasets/bardiakh/monaiitk)) so they can be easily installed in the offline inference code. \n\n*Be sure to include the dataset before running the next cell block.*","metadata":{}},{"cell_type":"code","source":"!pip install itk --no-index --find-links=file:///kaggle/input/monaiitk/itk -q\n!pip install monai --no-index --find-links=file:///kaggle/input/monaiitk/monai -q","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","execution":{"iopub.status.busy":"2022-07-31T17:32:01.549166Z","iopub.execute_input":"2022-07-31T17:32:01.549761Z","iopub.status.idle":"2022-07-31T17:32:51.700066Z","shell.execute_reply.started":"2022-07-31T17:32:01.549709Z","shell.execute_reply":"2022-07-31T17:32:51.699021Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Start Coding!\n\nIn this notebook, we will be using IPython widgets to visualize CT slices and their overlying masks.","metadata":{}},{"cell_type":"code","source":"import os\nfrom typing import Union\nimport pandas as pd\nimport numpy as np\nimport monai as mn\nimport torch\nfrom ipywidgets import interactive, widgets, fixed\nimport matplotlib.pyplot as plt\n%matplotlib inline","metadata":{"execution":{"iopub.status.busy":"2022-07-31T17:36:40.381203Z","iopub.execute_input":"2022-07-31T17:36:40.383446Z","iopub.status.idle":"2022-07-31T17:36:40.397589Z","shell.execute_reply.started":"2022-07-31T17:36:40.383364Z","shell.execute_reply":"2022-07-31T17:36:40.396251Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Constants\n\nCT scan pixel values are based on Hounsfield Units (HU) and are pretty much standard across different devices. HU can range from -1000 (air) to more than +10,000 (metals). Radiologists use **windowing** to focus on specific tissues. To window a CT scan, you need two values, `WINDOW_WIDTH` and `WINDOW_LEVEL`. By windowing an image, you change the pixel values of an image so that the maximum gray level is `WINDOW_LEVEL + (WINDOW_WIDTH/2)` and the minimum gray level is `WINDOW_LEVEL - (WINDOW_WIDTH/2)` and anything outside this range would be clipped. To better visualize vertebrae, we can use the width of 1800 and level 400. We defined these constants here and will use that in our pipeline.","metadata":{}},{"cell_type":"code","source":"DATA_PATH = \"../input/rsna-2022-cervical-spine-fracture-detection\"\nWINDOW_WIDTH = 1800\nWINDOW_LEVEL = 400","metadata":{"execution":{"iopub.status.busy":"2022-07-31T17:33:10.133497Z","iopub.execute_input":"2022-07-31T17:33:10.133947Z","iopub.status.idle":"2022-07-31T17:33:10.139998Z","shell.execute_reply.started":"2022-07-31T17:33:10.133912Z","shell.execute_reply":"2022-07-31T17:33:10.138512Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Data Preparation\n\nIn this tutorial, we will use only a subset of studies with segmentation masks. This is because we want to show MONAI's capabilities. You can later change this in your code.","metadata":{}},{"cell_type":"code","source":"train_df = pd.read_csv(f\"{DATA_PATH}/train.csv\")\ntrain_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-31T17:33:16.973149Z","iopub.execute_input":"2022-07-31T17:33:16.973556Z","iopub.status.idle":"2022-07-31T17:33:17.012521Z","shell.execute_reply.started":"2022-07-31T17:33:16.973524Z","shell.execute_reply":"2022-07-31T17:33:17.011123Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Now that we have the list of all studies in the training set, we only select those with a segmentation mask in the */segmentations* directory.","metadata":{}},{"cell_type":"code","source":"segmented_studies = [f[:-4] for f in os.listdir(f\"{DATA_PATH}/segmentations/\") if f.endswith(\".nii\")]\ntrain_df.drop(train_df[~train_df['StudyInstanceUID'].isin(segmented_studies)].index, inplace=True)\ntrain_df.reset_index(drop=True, inplace=True)\ntrain_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-07-31T17:33:19.273195Z","iopub.execute_input":"2022-07-31T17:33:19.273613Z","iopub.status.idle":"2022-07-31T17:33:19.327643Z","shell.execute_reply.started":"2022-07-31T17:33:19.273565Z","shell.execute_reply":"2022-07-31T17:33:19.326638Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Creating the Dictionary\n\nOne cool thing about MONAI transforms is that it can be configured to accept a dictionary as input. We will create a list of dictionaries (one dictionary for each row of the dataframe) and then manually add `image` and `mask` keys that point to the image directory and segmentation *.nii* file, respectively.  ","metadata":{}},{"cell_type":"code","source":"train_dict = train_df.to_dict(orient=\"records\")\nfor i,item in enumerate(train_dict):\n    train_dict[i][\"image\"] = f\"{DATA_PATH}/train_images/{item['StudyInstanceUID']}/\"\n    train_dict[i][\"mask\"] = f\"{DATA_PATH}/segmentations/{item['StudyInstanceUID']}.nii\"","metadata":{"execution":{"iopub.status.busy":"2022-07-31T17:33:19.959056Z","iopub.execute_input":"2022-07-31T17:33:19.959714Z","iopub.status.idle":"2022-07-31T17:33:19.96803Z","shell.execute_reply.started":"2022-07-31T17:33:19.959676Z","shell.execute_reply":"2022-07-31T17:33:19.967019Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Stacing Transforms\n\nMONAI uses an API interface that is very similar to `torchvision` or `albumentations` libraries. You can stack several transforms, which will be sequentially applied to the input. As mentioned, MONAI can do its magic on a dictionary. Basically, each transform gets a dictionary and the corresponding key, applies its operation on that key's value, and then replaces the value with the transformation output. \n\n*To learn more about MONAI's transformation pipeline, check out Chapter 6 of the [MIDeL MOOC](https://mayo-radiology-informatics-lab.github.io/MIDeL/chapters.html).*","metadata":{}},{"cell_type":"code","source":"transforms = mn.transforms.Compose([\n    mn.transforms.LoadImageD(keys=[\"image\",\"mask\"], reader=[\"ITKReader\",\"NibabelReader\"]),\n    mn.transforms.AddChannelD(keys=[\"image\",\"mask\"]),\n    mn.transforms.TransposeD(keys=[\"image\"], indices=(0,2,1,3)),\n    mn.transforms.Rotate90D(keys=[\"mask\"], k=1),\n    mn.transforms.ScaleIntensityRangeD(keys=[\"image\"],a_min=WINDOW_LEVEL-(WINDOW_WIDTH/2), a_max=WINDOW_LEVEL+(WINDOW_WIDTH/2),b_min=0.0, b_max=1.0, clip=True),\n    mn.transforms.ToTensorD(keys=[\"image\",\"mask\"]),\n])","metadata":{"execution":{"iopub.status.busy":"2022-07-31T18:43:43.219741Z","iopub.execute_input":"2022-07-31T18:43:43.220178Z","iopub.status.idle":"2022-07-31T18:43:43.2366Z","shell.execute_reply.started":"2022-07-31T18:43:43.220144Z","shell.execute_reply":"2022-07-31T18:43:43.235658Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"This pipeline will:\n1. Read image and mask keys (which were paths to the DICOM and NIfTI) and replace these keys with numpy arrays of the image and mask.\n2. Adds a channel to the beginning to create CHWD arrays.\n3. Transposes the image and rotates the mask by 90 degrees so that they overlap with each other.\n4. Windows the image based on the values of `WINDOW_LEVEL` and `WINDOW_WIDTH`.\n5. Converts the image to tensors.","metadata":{"execution":{"iopub.status.busy":"2022-07-31T18:43:43.942773Z","iopub.execute_input":"2022-07-31T18:43:43.943617Z","iopub.status.idle":"2022-07-31T18:43:43.955103Z","shell.execute_reply.started":"2022-07-31T18:43:43.943548Z","shell.execute_reply":"2022-07-31T18:43:43.953034Z"}}},{"cell_type":"markdown","source":"## Creating a Dataset\n\nLet's create a dataset using MONAI's API so that we can use the images for visualization.","metadata":{}},{"cell_type":"code","source":"dataset = mn.data.Dataset(data=train_dict, transform=transforms)\nprint(dataset[0].keys())","metadata":{"execution":{"iopub.status.busy":"2022-07-31T18:54:37.201529Z","iopub.execute_input":"2022-07-31T18:54:37.202103Z","iopub.status.idle":"2022-07-31T18:54:44.832556Z","shell.execute_reply.started":"2022-07-31T18:54:37.20206Z","shell.execute_reply":"2022-07-31T18:54:44.830956Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"As you see, the output of the dataset is a dictionary with the same keys as the input (plus some meta-data keys), but the `image` and `mask` keys now contain the image and mask arrays rather than the directory.","metadata":{}},{"cell_type":"markdown","source":"## Visualization\n\nWe will use IPython interactive widgets to show different slices in an image. \n\n**You have to clone this notebook to use the plot interactively.** ","metadata":{}},{"cell_type":"code","source":"def plot_overlayed_mask(slice: int, image: Union[np.ndarray, torch.Tensor], mask: Union[np.ndarray, torch.Tensor]) -> None:\n    \"\"\"Plots an image and a mask overlayed on top of it.\n\n    Args:\n        slice (int): slice number\n        image (Union[np.ndarray, torch.Tensor]): image array\n        mask (Union[np.ndarray, torch.Tensor]): mask array\n    \"\"\"\n    slice-=1 # slice number starts at 0\n    plt.imshow(image[0,:,:,slice], cmap=\"gray\")\n    plt.imshow(mask[0,:,:,slice], cmap=\"viridis\", alpha=0.6)\n    plt.show()","metadata":{"execution":{"iopub.status.busy":"2022-07-31T18:54:44.835477Z","iopub.execute_input":"2022-07-31T18:54:44.836187Z","iopub.status.idle":"2022-07-31T18:54:44.847382Z","shell.execute_reply.started":"2022-07-31T18:54:44.836145Z","shell.execute_reply":"2022-07-31T18:54:44.845782Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"sample_image = dataset[0]\nsample_image_slices = sample_image['image'].shape[-1]\n\ninteractive_plot = interactive(\n    plot_overlayed_mask,\n    slice=widgets.IntSlider(min=1, max=sample_image_slices, step=1, value=1),\n    image=fixed(sample_image['image']),\n    mask=fixed(sample_image['mask'])\n)\ninteractive_plot","metadata":{"execution":{"iopub.status.busy":"2022-07-31T18:54:44.849822Z","iopub.execute_input":"2022-07-31T18:54:44.850814Z","iopub.status.idle":"2022-07-31T18:54:48.979347Z","shell.execute_reply.started":"2022-07-31T18:54:44.850757Z","shell.execute_reply":"2022-07-31T18:54:48.977952Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Saving\n\nWe just add a `SaveImageD` transform to the pipeline, which will take care of saving the transformation results! We will use **.nii** format so that we preserve the affine information for future transforms.","metadata":{}},{"cell_type":"code","source":"os.makedirs(\"./images_nii\", exist_ok=True)\nos.makedirs(\"./masks_nii\", exist_ok=True)\n\ndef save_to_nii(item: dict, image_directory: str = \"./images_nii\", mask_directory: str = \"./masks_nii\") -> None:\n    \"\"\"Saving a MONAI output dictionay to nii files.\n\n    Args:\n        item (dict): dictionary containing image and mask keys.\n        image_directory (str, optional): directory to save images. Defaults to \"./images_nii\".\n        mask_directory (str, optional): directory to save masks. Defaults to \"./masks_nii\".\n    \"\"\"\n    \n    writer = mn.data.NibabelWriter()\n    study_id = item['StudyInstanceUID']  \n\n    if \"image\" in item:\n        writer.set_data_array(item['image'], channel_dim=0)\n        writer.set_metadata({\"affine\": item['image_meta_dict']['affine'], \"original_affine\": item['image_meta_dict']['original_affine']})\n        writer.write(f\"{image_directory}/{study_id}.nii.gz\", verbose=True)\n\n    if \"mask\" in item:\n        writer.set_data_array(item['mask'], channel_dim=0)\n        writer.set_metadata({\"affine\": item['mask_meta_dict']['affine'], \"original_affine\": item['mask_meta_dict']['original_affine']})\n        writer.write(f\"{mask_directory}/{study_id}.nii.gz\", verbose=True)\n","metadata":{"execution":{"iopub.status.busy":"2022-07-31T20:06:25.880777Z","iopub.execute_input":"2022-07-31T20:06:25.881307Z","iopub.status.idle":"2022-07-31T20:06:25.894977Z","shell.execute_reply.started":"2022-07-31T20:06:25.881269Z","shell.execute_reply":"2022-07-31T20:06:25.893472Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"save_to_nii(sample_image)","metadata":{"execution":{"iopub.status.busy":"2022-07-31T20:06:26.568958Z","iopub.execute_input":"2022-07-31T20:06:26.570723Z","iopub.status.idle":"2022-07-31T20:06:41.388691Z","shell.execute_reply.started":"2022-07-31T20:06:26.570666Z","shell.execute_reply":"2022-07-31T20:06:41.385112Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Wrapping Up\n\nFrom here, you can use MONAI to augment our data and start training segmentation/classifier models.","metadata":{}},{"cell_type":"markdown","source":"**If you liked this tutorial, please upvote it; I will try to continue this series in my spare time.**","metadata":{}}]}