{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceId":13451,"databundleVersionId":1188070,"sourceType":"competition"},{"sourceId":187731,"sourceType":"datasetVersion","datasetId":80814}],"dockerImageVersionId":29507,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# RSNA Intracranial Hemorrhage Detection\n## Identify acute intracranial hemorrhage and its subtypes\n\n\n![](https://media.springernature.com/lw685/springer-static/image/art%3A10.1038%2Fs41746-017-0015-z/MediaObjects/41746_2017_15_Fig3_HTML.jpg)\n\n
\n\n**Intracranial hemorrhage**, bleeding that occurs inside the cranium, is a serious health problem requiring rapid and often intensive medical treatment. For example, intracranial hemorrhages account for approximately 10% of strokes in the U.S., where stroke is the fifth-leading cause of death. Identifying the location and type of any hemorrhage present is a critical step in treating the patient.\n\n**Diagnosis** requires an urgent procedure. When a patient shows acute neurological symptoms such as severe headache or loss of consciousness, highly trained specialists review medical images of the patient’s cranium to look for the presence, location and type of hemorrhage. The process is complicated and often time consuming.\n\n**The challenge** is to build an algorithm to detect acute intracranial hemorrhage and its subtypes. \n\n
\n### IMPORTANT: I'll update this kernels almost every day, stay tuned :)\n
\n\n# Table of Contents\n\n1. [EDA](#EDA)\n2. [Visualization & Augmentations](#Visualization)\n3. [Model](#Model)\n4. [Submission](#Submission)\n\n
\n\n### References:\n\n- [Basic EDA + Data Visualization đŸ§  ](https://www.kaggle.com/marcovasquez/basic-eda-data-visualization)\n- [Simple EDA](https://www.kaggle.com/currypurin/simple-eda)\n- [Basic EDA + albumentations augs](https://www.kaggle.com/alimbekovkz/basic-eda-albumentations-augs)\n\n
\n## Hemorrhage Types\n\n**You can find more information [here](https://www.kaggle.com/c/rsna-intracranial-hemorrhage-detection/overview/hemorrhage-types)**\n\n> Hemorrhage in the head (intracranial hemorrhage) is a relatively common condition that has many causes ranging from trauma, stroke, aneurysm, vascular malformations, high blood pressure, illicit drugs and blood clotting disorders. The neurologic consequences also vary extensively depending upon the size, type of hemorrhage and location ranging from headache to death. The role of the Radiologist is to detect the hemorrhage, characterize the hemorrhage subtype, its size and to determine if the hemorrhage might be jeopardizing critical areas of the brain that might require immediate surgery. \n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F603584%2F56162e47358efd77010336a373beb0d2%2Fsubtypes-of-hemorrhage.png?generation=1568657910458946&alt=media)\n\n
\n\n## Model example\n\n![](https://s3.amazonaws.com/zapnito/uploads/bcf25032e0801dcfebd72ff2f6a2a064/95a8da83-49b5-4081-af62-371a460fc9f0.jpeg)\n\n> [Explainable, Radiologist Mimicking, Deep-Learning for Detection of Acute Intracranial Haemorrhage from Small CT Datasets](https://bioengineeringcommunity.nature.com/users/203140-michael-h-lev-md-faha-facr/posts/42310-explainable-radiologist-mimicking-deep-learning-for-detection-of-acute-intracranial-haemorrhage-from-small-ct-datasets)\n\n
\n\n## Metric\n\n**Weighted multi-label logarithmic loss**\n\n- [What is Log Loss?](https://www.kaggle.com/dansbecker/what-is-log-loss)\n- [sklearn.metrics.log_loss](https://scikit-learn.org/stable/modules/generated/sklearn.metrics.log_loss.html)","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport os\nimport pydicom\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport json\nimport cv2\n\nprint ('Packages ready!')","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"ls ../input/rsna-intracranial-hemorrhage-detection/","metadata":{"trusted":true,"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Load data","metadata":{}},{"cell_type":"code","source":"train = pd.read_csv(\"../input/rsna-intracranial-hemorrhage-detection/stage_1_train.csv\")\nsub = pd.read_csv(\"../input/rsna-intracranial-hemorrhage-detection/stage_1_sample_submission.csv\")\ntrain_images = os.listdir(\"../input/rsna-intracranial-hemorrhage-detection/stage_1_train_images/\")\ntest_images = os.listdir(\"../input/rsna-intracranial-hemorrhage-detection/stage_1_test_images/\")\nprint ('Train:', train.shape[0])\nprint ('Sub:', sub.shape[0])","metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true,"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# EDA\n\n### [Data Description](https://www.kaggle.com/c/rsna-intracranial-hemorrhage-detection/data)\n\nThe training data is provided as a set of image Ids and multiple labels, one for each of five sub-types of hemorrhage, plus an additional label for any, which should always be true if any of the sub-type labels is true.\n\nThere is also a target column, ```Label```, indicating the probability of whether that type of hemorrhage exists in the indicated image.\n\nThere will be **6** rows per image ```Id```. The label indicated by a particular row will look like ```[Image Id]_[Sub-type Name]```, as follows:\n\n```\nId,Label\n1_epidural_hemorrhage,0\n1_intraparenchymal_hemorrhage,0\n1_intraventricular_hemorrhage,0\n1_subarachnoid_hemorrhage,0.6\n1_subdural_hemorrhage,0\n1_any,0.9\n```","metadata":{}},{"cell_type":"code","source":"train['type'] = train['ID'].str.split(\"_\", n = 3, expand = True)[2]\ntrain['PatientID'] = train['ID'].str.split(\"_\", n = 3, expand = True)[1]\ntrain['filename'] = train['ID'].apply(lambda st: \"ID_\" + st.split('_')[1] + \".png\")\n\nsub['filename'] = sub['ID'].apply(lambda st: \"ID_\" + st.split('_')[1] + \".png\")\nsub['type'] = sub['ID'].apply(lambda st: st.split('_')[2])\n\ntrain.head()","metadata":{"_kg_hide-input":true,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"print ('Train type =', list(train.type.unique()))\nprint ('Train label =', list(train.Label.unique()))\n#train.to_csv('train.csv', index=False)","metadata":{"_kg_hide-input":true,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"### Basic Counts","metadata":{}},{"cell_type":"code","source":"print ('Number of Patients: ', train.PatientID.nunique())","metadata":{"_kg_hide-input":true,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**Type freq**\n\n- We have the same amount of pictures per type! ","metadata":{}},{"cell_type":"code","source":"train.type.value_counts()","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**Labels**\n\n> **imbalanced data !**","metadata":{}},{"cell_type":"code","source":"print(train.Label.value_counts())\nsns.countplot(x='Label', data=train)","metadata":{"_kg_hide-input":true,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"But, let's see better **Labels per Type** ...","metadata":{}},{"cell_type":"code","source":"train.groupby('type').Label.value_counts()","metadata":{"_kg_hide-input":true,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"sns.countplot(x=\"Label\", hue=\"type\", data=train)","metadata":{"_kg_hide-input":true,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Visualization\n\nAs you can read at the post [Window level and width on CT](https://www.kaggle.com/c/rsna-intracranial-hemorrhage-detection/discussion/109328#latest-629856)\n> Intracranial hemorrhages are better visualized with a brain window (level = 40, width = 80) than the default non normalized HU values.\n\nSee: https://www.kaggle.com/omission/eda-view-dicom-images-with-correct-windowing\n\n","metadata":{}},{"cell_type":"code","source":"TRAIN_IMG_PATH = \"../input/rsna-intracranial-hemorrhage-detection/stage_1_train_images/\"\nTEST_IMG_PATH = \"../input/rsna-intracranial-hemorrhage-detection/stage_1_test_images/\"\nBASE_PATH = '/kaggle/input/rsna-intracranial-hemorrhage-detection/'\nTRAIN_DIR = 'stage_1_train_images/'\nTEST_DIR = 'stage_1_test_images/'\n\ndef window_image(img, window_center,window_width, intercept, slope, rescale=True):\n\n img = (img*slope +intercept)\n img_min = window_center - window_width//2\n img_max = window_center + window_width//2\n img[imgimg_max] = img_max\n \n if rescale:\n # Extra rescaling to 0-1, not in the original notebook\n img = (img - img_min) / (img_max - img_min)\n \n return img\n \ndef get_first_of_dicom_field_as_int(x):\n #get x[0] as in int is x is a 'pydicom.multival.MultiValue', otherwise get int(x)\n if type(x) == pydicom.multival.MultiValue:\n return int(x[0])\n else:\n return int(x)\n\ndef get_windowing(data):\n dicom_fields = [data[('0028','1050')].value, #window center\n data[('0028','1051')].value, #window width\n data[('0028','1052')].value, #intercept\n data[('0028','1053')].value] #slope\n return [get_first_of_dicom_field_as_int(x) for x in dicom_fields]\n\n \n \ndef view_images(images, title = '', aug = None):\n width = 5\n height = 2\n fig, axs = plt.subplots(height, width, figsize=(15,5))\n \n for im in range(0, height * width):\n data = pydicom.read_file(os.path.join(TRAIN_IMG_PATH,'ID_'+images[im]+ '.dcm'))\n image = data.pixel_array\n window_center , window_width, intercept, slope = get_windowing(data)\n image_windowed = window_image(image, window_center, window_width, intercept, slope)\n\n\n i = im // width\n j = im % width\n axs[i,j].imshow(image_windowed, cmap=plt.cm.bone) \n axs[i,j].axis('off')\n \n plt.suptitle(title)\n plt.show()","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"case = 5\ndata = pydicom.dcmread(TRAIN_IMG_PATH+train_images[case])\n\nprint(data)\nwindow_center , window_width, intercept, slope = get_windowing(data)\n\n\n#displaying the image\nimg = pydicom.read_file(TRAIN_IMG_PATH+train_images[case]).pixel_array\n\nimg = window_image(img, window_center, window_width, intercept, slope)\nplt.imshow(img, cmap=plt.cm.bone)\nplt.grid(False)\n","metadata":{"_kg_hide-input":true,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"view_images(train[(train['type'] == 'epidural') & (train['Label'] == 1)][:10].PatientID.values, title = 'Images with epidural')","metadata":{"trusted":true,"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"view_images(train[(train['type'] == 'intraparenchymal') & (train['Label'] == 1)][:10].PatientID.values, title = 'Images with intraparenchymal')","metadata":{"trusted":true,"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"view_images(train[(train['type'] == 'subarachnoid') & (train['Label'] == 1)][:10].PatientID.values, title = 'Images with subarachnoid')","metadata":{"_kg_hide-input":true,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"view_images(train[(train['type'] == 'subdural') & (train['Label'] == 1)][:10].PatientID.values, title = 'Images with subdural')","metadata":{"_kg_hide-input":true,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"# Model\n\n- Reference: **[RSNA Intracranial: Simple DenseNet in Keras](https://www.kaggle.com/xhlulu/rsna-intracranial-simple-densenet-in-keras)** by @xhlulu\n\nI'm going to spend here a lot of quota ;)","metadata":{}},{"cell_type":"code","source":"from keras import layers\nfrom keras.applications import DenseNet121\nfrom keras.preprocessing.image import ImageDataGenerator\nfrom keras.callbacks import Callback, ModelCheckpoint\nfrom keras.preprocessing.image import ImageDataGenerator\nfrom keras.models import Sequential\nfrom keras.optimizers import Adam\nfrom tqdm import tqdm","metadata":{"_kg_hide-input":true,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"test = pd.DataFrame(sub.filename.unique(), columns=['filename'])\nprint ('Test:', test.shape[0])\ntest.head()","metadata":{"_kg_hide-input":true,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"np.random.seed(1234)\nsample_files = np.random.choice(os.listdir(TRAIN_IMG_PATH), 200000)\nsample_df = train[train.filename.apply(lambda x: x.replace('.png', '.dcm')).isin(sample_files)]\n\npivot_df = sample_df[['Label', 'filename', 'type']].drop_duplicates().pivot(\n index='filename', columns='type', values='Label').reset_index()\nprint(pivot_df.shape)\npivot_df.head()","metadata":{"_kg_hide-input":true,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def save_and_resize(filenames, load_dir): \n save_dir = '/kaggle/tmp/'\n if not os.path.exists(save_dir):\n os.makedirs(save_dir)\n\n for filename in tqdm(filenames):\n path = load_dir + filename\n new_path = save_dir + filename.replace('.dcm', '.png')\n \n dcm = pydicom.dcmread(path)\n window_center , window_width, intercept, slope = get_windowing(dcm)\n img = dcm.pixel_array\n img = window_image(img, window_center, window_width, intercept, slope)\n \n resized = cv2.resize(img, (224, 224))\n res = cv2.imwrite(new_path, resized)\n if not res:\n print('Failed')","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"save_and_resize(filenames=sample_files, load_dir=BASE_PATH + TRAIN_DIR)\nsave_and_resize(filenames=os.listdir(BASE_PATH + TEST_DIR), load_dir=BASE_PATH + TEST_DIR)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Data Generator","metadata":{}},{"cell_type":"code","source":"BATCH_SIZE = 32\n\ndef create_datagen():\n return ImageDataGenerator(\n zoom_range=0.1, # set range for random zoom\n # set mode for filling points outside the input boundaries\n fill_mode='constant',\n cval=0., # value used for fill_mode = \"constant\"\n horizontal_flip=True, # randomly flip images\n vertical_flip=True, # randomly flip images,\n validation_split=0.2\n )\n\ndef create_test_gen():\n return ImageDataGenerator().flow_from_dataframe(\n test,\n directory='/kaggle/tmp/',\n x_col='filename',\n class_mode=None,\n target_size=(224, 224),\n batch_size=BATCH_SIZE,\n shuffle=False\n )\n\ndef create_flow(datagen, subset):\n return datagen.flow_from_dataframe(\n pivot_df, \n directory='/kaggle/tmp/',\n x_col='filename', \n y_col=['any', 'epidural', 'intraparenchymal', \n 'intraventricular', 'subarachnoid', 'subdural'],\n class_mode='other',\n target_size=(224, 224),\n batch_size=BATCH_SIZE,\n subset=subset\n )\n\n# Using original generator\ndata_generator = create_datagen()\ntrain_gen = create_flow(data_generator, 'training')\nval_gen = create_flow(data_generator, 'validation')\ntest_gen = create_test_gen()","metadata":{"_kg_hide-input":true,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## DenseNet Model ","metadata":{}},{"cell_type":"code","source":"densenet = DenseNet121(\n weights='../input/densenet-keras/DenseNet-BC-121-32-no-top.h5',\n include_top=False,\n input_shape=(224,224,3)\n)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def build_model():\n model = Sequential()\n model.add(densenet)\n model.add(layers.GlobalAveragePooling2D())\n model.add(layers.Dropout(0.5))\n model.add(layers.Dense(6, activation='sigmoid'))\n \n model.compile(\n loss='binary_crossentropy',\n optimizer=Adam(lr=0.001),\n metrics=['accuracy']\n )\n \n return model","metadata":{"_kg_hide-input":true,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"model = build_model()\nmodel.summary()","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Training","metadata":{}},{"cell_type":"code","source":"checkpoint = ModelCheckpoint(\n 'model.h5', \n monitor='val_loss', \n verbose=0, \n save_best_only=True, \n save_weights_only=False,\n mode='auto'\n)\n\ntotal_steps = sample_files.shape[0] / BATCH_SIZE\n\nhistory = model.fit_generator(\n train_gen,\n steps_per_epoch=total_steps * 0.85,\n validation_data=val_gen,\n validation_steps=total_steps * 0.15,\n callbacks=[checkpoint],\n epochs=11\n)","metadata":{"_kg_hide-input":true,"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"with open('history.json', 'w') as f:\n json.dump(history.history, f)\n\nhistory_df = pd.DataFrame(history.history)\nhistory_df[['loss', 'val_loss']].plot()\nhistory_df[['acc', 'val_acc']].plot()","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"
\n# Submission\n\nThere will be **6 rows** per image ```Id```. The label indicated by a particular row will look like ```[Image Id]_[Sub-type Name]```, as follows\n\nThere is also a target column, ```Label```, indicating the probability of whether that type of hemorrhage exists in the indicated image.","metadata":{}},{"cell_type":"code","source":"model.load_weights('model.h5')\ny_test = model.predict_generator(\n test_gen,\n steps=len(test_gen),\n verbose=1\n)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"test_df = test.join(pd.DataFrame(y_test, columns = ['any', 'epidural', 'intraparenchymal', \n 'intraventricular', 'subarachnoid', 'subdural']))\n\n# Unpivot table\ntest_df = test_df.melt(id_vars=['filename'])\n\n# Combine the filename column with the variable column\ntest_df['ID'] = test_df.filename.apply(lambda x: x.replace('.png', '')) + '_' + test_df.variable\ntest_df['Label'] = test_df['value']\n\ntest_df[['ID', 'Label']].to_csv('submission.csv', index=False)","metadata":{"trusted":true,"_kg_hide-input":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"test_df[['ID', 'Label']].head(10)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":" ","metadata":{}}]}