{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"👋 In this kernel I will show how to use `SOLT` to augment the competition data. `SOLT` (streaming over lightweight data \ntransformations) is a fast data augmentation library, with support for images, segmentation masks, keypoints and data \nlabels.","metadata":{}},{"cell_type":"markdown","source":"<div class=\"alert alert-block alert-success\">\n    <b>Data source:</b> Radiological Society of North America. (2022, Nov 29). RSNA Screening Mammography Breast Cancer Detection, Version 1. Retrieved 2023 Feb 9 from [https://www.kaggle.com/competitions/rsna-breast-cancer-detection/data].\n</div>","metadata":{}},{"cell_type":"markdown","source":"# Prerequisites\n\n`SOLT` uses `DataContainer`'s to wrap data into a single object. This is done by specifying the data that we are inputting into the `DataContainer`. `I` stands for image, `M` stands for segmentation mask, `P` for keypoints, and `L` for labels. Multiple items can be can be inputted as a `tuple` and a string with equal amount of letters to specify the type of data at each `tuple` position.\n\n`SOLT` has a broad selection of augmentations and a convenient way to combine them, called `Stream`. `Stream` returns `torch` tensors by default. What is more, images are preprocessed by subtracting the ImageNet mean. This might not be the desired course of action in all applications and therefore it is possible also to return a `DataContainer`.\n\nApplying transformations is effortless after defining what to include in the transformation stream. Stream can be be defined for example by including the transforms within your pipeline or they can also be read from a `YAML` file. It is also possible to serialize a stream for example into a `YAML` file.","metadata":{}},{"cell_type":"markdown","source":"# Install `SOLT`","metadata":{}},{"cell_type":"code","source":"!pip install solt==0.1.9","metadata":{"execution":{"iopub.status.busy":"2023-02-11T02:43:47.133126Z","iopub.execute_input":"2023-02-11T02:43:47.133999Z","iopub.status.idle":"2023-02-11T02:44:02.90532Z","shell.execute_reply.started":"2023-02-11T02:43:47.133893Z","shell.execute_reply":"2023-02-11T02:44:02.9037Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Imports","metadata":{}},{"cell_type":"code","source":"import glob\nimport pathlib\n\nimport pydicom\nfrom pydicom.errors import InvalidDicomError\nfrom pydicom.pixel_data_handlers.util import apply_voi_lut\n\nimport numpy as np\nimport pandas as pd\n\nimport matplotlib.pyplot as plt\n\nimport solt\nimport solt.transforms as slt","metadata":{"execution":{"iopub.status.busy":"2023-02-11T02:44:02.907924Z","iopub.execute_input":"2023-02-11T02:44:02.908299Z","iopub.status.idle":"2023-02-11T02:44:05.835563Z","shell.execute_reply.started":"2023-02-11T02:44:02.908264Z","shell.execute_reply":"2023-02-11T02:44:05.834253Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Read DICOM","metadata":{}},{"cell_type":"code","source":"dcm_loc = pathlib.Path('/kaggle/input/rsna-breast-cancer-detection/train_images/')","metadata":{"execution":{"iopub.status.busy":"2023-02-11T02:44:05.836965Z","iopub.execute_input":"2023-02-11T02:44:05.837604Z","iopub.status.idle":"2023-02-11T02:44:05.845386Z","shell.execute_reply.started":"2023-02-11T02:44:05.837569Z","shell.execute_reply":"2023-02-11T02:44:05.842302Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"patient_id = 10006  # folder name, corresponds to patient_id","metadata":{"execution":{"iopub.status.busy":"2023-02-11T02:44:05.848908Z","iopub.execute_input":"2023-02-11T02:44:05.849358Z","iopub.status.idle":"2023-02-11T02:44:05.859754Z","shell.execute_reply.started":"2023-02-11T02:44:05.849322Z","shell.execute_reply":"2023-02-11T02:44:05.858617Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"filelist = list(dcm_loc.glob(f'{patient_id}/*.dcm'))\nfilelist","metadata":{"execution":{"iopub.status.busy":"2023-02-11T02:44:05.861384Z","iopub.execute_input":"2023-02-11T02:44:05.862536Z","iopub.status.idle":"2023-02-11T02:44:05.889442Z","shell.execute_reply.started":"2023-02-11T02:44:05.862455Z","shell.execute_reply":"2023-02-11T02:44:05.888241Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_meta = pd.read_csv(\"/kaggle/input/rsna-breast-cancer-detection/train.csv\")\ntrain_meta.head()","metadata":{"execution":{"iopub.status.busy":"2023-02-11T02:44:05.890917Z","iopub.execute_input":"2023-02-11T02:44:05.89155Z","iopub.status.idle":"2023-02-11T02:44:06.039806Z","shell.execute_reply.started":"2023-02-11T02:44:05.891514Z","shell.execute_reply":"2023-02-11T02:44:06.038858Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"image_dict = {}\nfor i, file in enumerate(filelist):\n    try:\n        ds = pydicom.dcmread(str(file))\n    except InvalidDicomError:\n        print(f'Invalid DICOM file: {str(file)}')\n        continue\n    \n    # print(ds)\n    \n    if hasattr(ds, 'SOPClassUID') and not ds.SOPClassUID == '1.2.840.10008.5.1.4.1.1.1.2':\n        print(f'File has unknown SOPClassUID: {ds.SOPClassUID}')\n        continue\n    \n    if hasattr(ds, 'PresentationIntentType') and not ds.PresentationIntentType == 'FOR PROCESSING':\n        print(f'Invalid Presentation Intent Type: {ds.PresentationIntentType}')\n        continue\n\n    try:\n        arr = ds.pixel_array.copy()  # get pixel array\n    except AttributeError:\n        print(f'AttributeError: Unable to return pixel array.')\n        continue\n    except ValueError:\n        print(f'ValueError: Unable to return pixel array.')\n        continue\n\n    try:\n        out = apply_voi_lut(arr, ds)  # apply VOI LUT (default LUT) or windowing operation\n    except AttributeError:\n        print(f'AttributeError: Unable to apply VOI LUT or windowing.')\n        continue\n    \n    if hasattr(ds, 'PhotometricInterpretation') and not (ds.PhotometricInterpretation == 'MONOCHROME1' or ds.PhotometricInterpretation == 'MONOCHROME2'):\n        print(f'Unknown photometric interpretation: {ds.PhotometricInterpretation}')\n        continue\n    elif not hasattr(ds, 'PhotometricInterpretation'):\n        print(f'Missing photometric interpretation.')\n        continue\n\n    if ds.PhotometricInterpretation == 'MONOCHROME1':  # ranges from bright to dark with ascending pixel values\n        out = out.max() - out\n    elif ds.PhotometricInterpretation == 'MONOCHROME2':  # ranges from dark to bright with ascending pixel values\n        pass\n\n    height = ds.Rows\n    width = ds.Columns\n    \n    out = out.reshape((height, width))  # (height, width)\n\n    if hasattr(ds, 'ImageLaterality'):\n        laterality = ds.ImageLaterality\n    else:\n        print(f'Unknown or invalid Laterality.')\n        continue\n    \n    patient = ds.PatientID\n    \n    try:\n        view = train_meta.loc[( train_meta['patient_id'] == int(patient) ) & ( train_meta['image_id'] == int(file.stem) ), 'view'].item()\n    except ValueError:\n        print(f'ValueError: Unable to return View.')\n        continue\n\n    bitdepth = ds.BitsAllocated\n    # bitsstored = ds.BitsStored\n\n    out = out.astype(np.float64)\n    out /= out.max()\n    out *= pow(2, bitdepth) - 1\n    out = out.astype(np.uint16)\n\n    if laterality == 'R' and view == 'MLO':\n        image_dict['R-MLO'] = out\n    elif laterality == 'L' and view == 'MLO':\n        image_dict['L-MLO'] = out\n    elif laterality == 'R' and view == 'CC':\n        image_dict['R-CC'] = out\n    elif laterality == 'L' and view == 'CC':\n        image_dict['L-CC'] = out\n    \n    # study = ds.StudyInstanceUID","metadata":{"execution":{"iopub.status.busy":"2023-02-11T02:44:06.041421Z","iopub.execute_input":"2023-02-11T02:44:06.042101Z","iopub.status.idle":"2023-02-11T02:44:15.857508Z","shell.execute_reply.started":"2023-02-11T02:44:06.042064Z","shell.execute_reply":"2023-02-11T02:44:15.856061Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"image_dict","metadata":{"execution":{"iopub.status.busy":"2023-02-11T02:44:15.85956Z","iopub.execute_input":"2023-02-11T02:44:15.859969Z","iopub.status.idle":"2023-02-11T02:44:15.869595Z","shell.execute_reply.started":"2023-02-11T02:44:15.859932Z","shell.execute_reply":"2023-02-11T02:44:15.868396Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for standard_view in ['R-MLO', 'L-MLO', 'R-CC', 'L-CC']:\n    img = image_dict[standard_view]\n    fig = plt.figure(figsize=(7, 7))\n    ax = fig.add_subplot(1,1,1)\n    ax.imshow(img, cmap='gray')\n    plt.show()\n    \n    unique_colors = len(np.unique(img))\n    mi = np.min(img)\n    ma = np.max(img)\n    print(f'Unique colors: {unique_colors}; Min value: {mi}; Max value: {ma}')\n    \n    del img","metadata":{"execution":{"iopub.status.busy":"2023-02-11T02:44:15.871264Z","iopub.execute_input":"2023-02-11T02:44:15.872306Z","iopub.status.idle":"2023-02-11T02:44:27.001984Z","shell.execute_reply.started":"2023-02-11T02:44:15.872266Z","shell.execute_reply":"2023-02-11T02:44:27.000531Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Augmentations\n\nThe example is adapted from `SOLT` documentation, which can be found [here](https://oulu-imeds.github.io/solt/index.html).","metadata":{}},{"cell_type":"code","source":"dc = solt.DataContainer((image_dict['R-MLO'], ), 'I', transform_settings={0: {'interpolation': 'bilinear'}})  # I stands for \"image\"","metadata":{"execution":{"iopub.status.busy":"2023-02-11T02:44:27.006177Z","iopub.execute_input":"2023-02-11T02:44:27.006879Z","iopub.status.idle":"2023-02-11T02:44:27.013511Z","shell.execute_reply.started":"2023-02-11T02:44:27.00684Z","shell.execute_reply":"2023-02-11T02:44:27.012449Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"image_size = image_dict['R-MLO'].shape[:2]\n\nprob = 0.5\n\nrotation_range_neg = -5\nrotation_range_pos = 5\n\nscale_range_lower = 0.7\nscale_range_upper = 1.3\n\nshear_range_lower = -0.5\nshear_range_upper = 0.5\n\ntranslation_range=50\n\nv_range_min = 0.0000001\nv_range_max=0.00009\n\nuse_cutout = np.random.choice([False, True], p=[1-prob, prob])  # p=[1-p1, p1]\ncutout_proportion = 0.2\ncutout_size = (int(cutout_proportion * image_size[0]), int(cutout_proportion * image_size[1]))\ncutout = slt.CutOut(cutout_size=cutout_size, p=prob)\n\nstream = solt.Stream([\n    slt.Resize(resize_to=image_size),  # here the image is already this fixed size\n    slt.Projection(affine_transforms=solt.Stream([\n        slt.Rotate(angle_range=(rotation_range_neg, rotation_range_pos), p=prob),\n        slt.Scale(range_x=(scale_range_lower, scale_range_upper),\n                  range_y=(scale_range_lower, scale_range_upper),\n                  same=False,\n                  p=prob),\n        slt.Shear(range_x=(shear_range_lower, shear_range_upper),\n                  range_y=(shear_range_lower, shear_range_upper),\n                  interpolation='bilinear',\n                  padding='z',\n                  p=prob),\n        slt.Translate(range_x=translation_range,\n                      range_y=translation_range,\n                      p=prob),\n    ]), v_range=(v_range_min, v_range_max)),\n    # Spatial\n    slt.Pad(pad_to=image_size),\n    slt.Crop(crop_mode='r', crop_to=image_size),\n    slt.Flip(p=prob, axis=1),\n    # Cutout\n    solt.SelectiveStream([\n        cutout if use_cutout else solt.Stream(),\n    ]),\n], ignore_fast_mode=False)","metadata":{"execution":{"iopub.status.busy":"2023-02-11T02:44:27.015107Z","iopub.execute_input":"2023-02-11T02:44:27.015533Z","iopub.status.idle":"2023-02-11T02:44:27.176377Z","shell.execute_reply.started":"2023-02-11T02:44:27.015501Z","shell.execute_reply":"2023-02-11T02:44:27.175087Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(stream.to_yaml())","metadata":{"execution":{"iopub.status.busy":"2023-02-11T02:44:27.177962Z","iopub.execute_input":"2023-02-11T02:44:27.178317Z","iopub.status.idle":"2023-02-11T02:44:27.207074Z","shell.execute_reply.started":"2023-02-11T02:44:27.178286Z","shell.execute_reply":"2023-02-11T02:44:27.205607Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for i in range(10):\n    res = stream(dc, return_torch=False)  # False is set as we don't want to subtract the ImageNet mean here\n    assert isinstance(res, solt.DataContainer)\n\n    img_res = res.data[0]\n    img = img_res.squeeze()\n    \n    fig = plt.figure(figsize=(5, 5))\n    ax = fig.add_subplot(1,1,1)\n    ax.imshow(img, cmap='gray')\n    plt.show()\n    \n    unique_colors = len(np.unique(img))\n    mi = np.min(img)\n    ma = np.max(img)\n    print(f'Unique colors: {unique_colors}; Min value: {mi}; Max value: {ma}')\n    \n    del img\n    del res\n    del img_res","metadata":{"execution":{"iopub.status.busy":"2023-02-11T02:44:27.208416Z","iopub.execute_input":"2023-02-11T02:44:27.208772Z","iopub.status.idle":"2023-02-11T02:44:51.849674Z","shell.execute_reply.started":"2023-02-11T02:44:27.208712Z","shell.execute_reply":"2023-02-11T02:44:51.848312Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Concluding remarks\n\nI hope this library will give you some additional speed needed in this competition. 👍","metadata":{}}]}