{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# Convolutional Neural Network Model for Classifying Blood Clot Origins in Ischemic Strokes\n### Written by Jonathan Ma - 08/22/2022\n\nThe goal of this notebook is to create a CNN model which can classify blood clot origins in ischemic stroke. \n\nGiven whole slide digital pathology images, and using Tensorflow, this model aims to differentiate between the two major acute ischemic stroke (AIS) etiology subtypes: cardiac and large artery atherosclerosis.","metadata":{}},{"cell_type":"code","source":"import warnings\nwarnings.filterwarnings('ignore')\nimport os\nos.environ['TF_CPP_MIN_LOG_LEVEL'] = '3'\n\nimport tensorflow as tf\nimport numpy as np \nimport pandas as pd \nimport matplotlib.pyplot as plt\n%matplotlib inline\nimport openslide\nfrom openslide import OpenSlide\nimport tensorflow as tf\nfrom tensorflow.keras import Model\nfrom tensorflow.keras import layers\nfrom tensorflow.keras.models import Sequential\nfrom tensorflow.keras.layers import Conv2D, MaxPooling2D, Dropout, Flatten, Dense\nfrom sklearn.model_selection import train_test_split\n\nfrom tqdm import tqdm","metadata":{"execution":{"iopub.status.busy":"2022-11-02T13:02:25.945051Z","iopub.execute_input":"2022-11-02T13:02:25.945719Z","iopub.status.idle":"2022-11-02T13:02:31.653313Z","shell.execute_reply.started":"2022-11-02T13:02:25.945634Z","shell.execute_reply":"2022-11-02T13:02:31.652006Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df = pd.read_csv(\"/kaggle/input/mayo-clinic-strip-ai/train.csv\")\ntest_df = pd.read_csv(\"/kaggle/input/mayo-clinic-strip-ai/test.csv\")","metadata":{"execution":{"iopub.status.busy":"2022-11-02T13:02:31.65527Z","iopub.execute_input":"2022-11-02T13:02:31.657967Z","iopub.status.idle":"2022-11-02T13:02:31.695214Z","shell.execute_reply.started":"2022-11-02T13:02:31.657921Z","shell.execute_reply":"2022-11-02T13:02:31.694325Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Preliminary Modifications","metadata":{}},{"cell_type":"code","source":"train_df.tail()","metadata":{"execution":{"iopub.status.busy":"2022-11-02T13:02:31.698162Z","iopub.execute_input":"2022-11-02T13:02:31.701032Z","iopub.status.idle":"2022-11-02T13:02:31.726651Z","shell.execute_reply.started":"2022-11-02T13:02:31.700983Z","shell.execute_reply":"2022-11-02T13:02:31.725583Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We will need to preprocess our data by adding the actual file addresses of each patient's corresponding digital slide pathiology image. \n\n\nAdditionally, because we are building a binary classifier, we will label our examples with 1 for cardiac embolisms, and 0 for large artery atheroschlerosis.","metadata":{}},{"cell_type":"code","source":"train_df[\"file_path\"] = train_df[\"image_id\"].apply(lambda x: \"../input/mayo-clinic-strip-ai/train/\" + x + \".tif\")\ntest_df[\"file_path\"]  = test_df[\"image_id\"].apply(lambda x: \"../input/mayo-clinic-strip-ai/test/\" + x + \".tif\")\ntrain_df[\"Y\"] = train_df[\"label\"].apply(lambda x : 1 if x==\"CE\" else 0) # Creating truth labels","metadata":{"execution":{"iopub.status.busy":"2022-11-02T13:02:31.72952Z","iopub.execute_input":"2022-11-02T13:02:31.729867Z","iopub.status.idle":"2022-11-02T13:02:31.74112Z","shell.execute_reply.started":"2022-11-02T13:02:31.729832Z","shell.execute_reply":"2022-11-02T13:02:31.740013Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train_df.head()","metadata":{"execution":{"iopub.status.busy":"2022-11-02T13:02:31.743042Z","iopub.execute_input":"2022-11-02T13:02:31.74421Z","iopub.status.idle":"2022-11-02T13:02:31.758886Z","shell.execute_reply.started":"2022-11-02T13:02:31.744165Z","shell.execute_reply":"2022-11-02T13:02:31.758054Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"This is what our input data looks like. We will only be using the labels and images addresses.","metadata":{}},{"cell_type":"markdown","source":"# Previewing our slides","metadata":{}},{"cell_type":"code","source":"slide = OpenSlide(train_df.loc[142, \"file_path\"])\nregion = (0, 0)\nsize = (10000, 10000)\nregion = slide.read_region(region, 0, size)\nplt.figure(figsize=(8, 8))\nplt.imshow(region)\nplt.show()  ","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-11-02T13:02:31.761064Z","iopub.execute_input":"2022-11-02T13:02:31.761678Z","iopub.status.idle":"2022-11-02T13:02:51.859575Z","shell.execute_reply.started":"2022-11-02T13:02:31.761642Z","shell.execute_reply":"2022-11-02T13:02:51.858712Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"These are huge images, and would take an obsene amount of time to train on without compression. So let's compress these images abit. \nWe will reduce the size of our images from 10000 x 100000 to 5000 x 5000, and we will resize our images down to 512 x 512.\n\n\n*This step usually takes about 50 minutes.* \n\n\nI've preprocessed the images and published them [here:](https://www.kaggle.com/datasets/jonathanma02/mayo-clinic-strip-ai-preprocessed-images)\n\n","metadata":{}},{"cell_type":"code","source":"%%time\ndef preprocess(image_path):\n    slide = OpenSlide(image_path)\n    region = (1000,1000)    \n    size = (5000, 5000)\n    image = slide.read_region(region, 0, size)\n    image = tf.image.resize(image, (512, 512))\n    image = np.array(image)    \n    return image\n\n\ntrain_x=[]\nfor i in tqdm(train_df['file_path']): \n    x1=preprocess(i)\n    train_x.append(x1)\n\n\ntrain_x = np.array(train_x)/255.0\ntrain_y = train_df[\"Y\"]\n# np.save(\"preprocessed\", train_x) # testing purposes\n","metadata":{"execution":{"iopub.status.busy":"2022-11-02T13:02:51.860615Z","iopub.execute_input":"2022-11-02T13:02:51.860921Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# Retrieving preprocessed images from published dataset\n# Will not work for submission, as this requires internet access\n\n# For testing purposes only\n\"\"\"\n!pip install opendatasets --upgrade --quiet\nimport opendatasets as od\nod.download(\"https://www.kaggle.com/datasets/jonathanma02/mayo-clinic-strip-ai-preprocessed-images\")\n\"\"\"\npass\n","metadata":{"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Creating Our Model\n\nWe are going to implement a Sequential Tensorflow Model. We will use:\n\n- Adam optimization\n- ~~Learning rate decay (monitored)~~ $\\leftarrow$ *wasn't doing anything meaningful*\n- Dropout regularization\n- L2 regularization","metadata":{}},{"cell_type":"code","source":"#train_x = np.load(\"mayo-clinic-strip-ai-preprocessed-images/preprocessed.npy\")/255.0 #testing purposes only\n\nmodel = Sequential()\ninput_shape = (512, 512, 4)\n\nmodel.add(Conv2D(filters=32, kernel_size = (3,3), strides =2, padding = 'same', activation = 'relu', input_shape = input_shape))\nmodel.add(Conv2D(filters=64, kernel_size = (3,3), strides =2, padding = 'same', activation = 'relu'))\nmodel.add(Conv2D(filters=32, kernel_size = (3,3), strides =2, padding = 'same', activation = 'relu'))\n# no pooling layer -- features are too minute, instead we are using a convolution layers with strides of 2\nmodel.add(Flatten())\nmodel.add(Dense(128, activation = 'relu',kernel_regularizer=tf.keras.regularizers.l2(0.1)))\nmodel.add(Dropout(0.50))\n\nmodel.add(Dense(1))\n\nmodel.compile(\n    loss = tf.keras.losses.MeanSquaredError(),    \n    metrics=[tf.keras.metrics.RootMeanSquaredError(name=\"rmse\"), \n             tf.keras.metrics.BinaryAccuracy(name=\"accuracy\")],\n    optimizer = tf.keras.optimizers.Adam(1e-4))","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Below is a callback function which will plot our model's metrics using matplotlib.","metadata":{}},{"cell_type":"code","source":"import numpy as np \nfrom tensorflow import keras\nfrom matplotlib import pyplot as plt\nfrom IPython.display import clear_output\n\nclass PlotLearning(keras.callbacks.Callback):\n    \"\"\"\n    Callback to plot the learning curves of the model during training.\n    \"\"\"\n    def on_train_begin(self, logs={}):\n        self.metrics = {}\n        self.metrics[\"loss\"]=[]\n        self.metrics[\"val_loss\"]=[]\n        self.metrics[\"accuracy\"]=[]\n        self.metrics[\"val_accuracy\"]=[]\n        self.metrics[\"lr\"]=[]\n            \n\n    def on_epoch_end(self, epoch, logs={}):\n        # Storing metrics\n        for metric in logs:\n            if metric in self.metrics:\n                self.metrics[metric].append(logs.get(metric))\n        \n        # Plotting\n        metrics = [x for x in logs if x in self.metrics and \"val\" not in x]\n        \n        f, axs = plt.subplots(1, len(metrics), figsize=(15,5))\n        clear_output(wait=True)\n\n        for i, metric in enumerate(metrics):\n            axs[i].plot(range(1, epoch + 2), \n                    self.metrics[metric], \n                    label=metric)\n            if metric != \"lr\" and logs['val_' + metric]:\n                axs[i].plot(range(1, epoch + 2), \n                            self.metrics['val_' + metric], \n                            label='val_' + metric)\n                \n            axs[i].legend()\n            axs[i].grid()\n            if metric == \"lr\":\n                axs[i].set_ylim(bottom=0, top=0.0015)\n            else:\n                axs[i].set_ylim(bottom=0, top=1)\n\n        plt.tight_layout()\n        plt.show()","metadata":{"_kg_hide-output":true,"_kg_hide-input":true,"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Training","metadata":{}},{"cell_type":"code","source":"%%time\ntrain_y = train_df[\"Y\"]\ntrain_df = None\ntrain_x,test_x,train_y,test_y=train_test_split(train_x,train_y,test_size=0.2)\n\nimport math\nfrom tensorflow.keras.callbacks import LearningRateScheduler, EarlyStopping, Callback, ReduceLROnPlateau \n\n\n#lrate = ReduceLROnPlateau(monitor='val_loss', factor=0.2,\n#                              patience=8, min_lr=0.0001) # not doing anything meaningful for us\n\nearstop = EarlyStopping(monitor = 'val_loss', min_delta = 0, patience = 16)\n\n\nhistory = model.fit(\n    train_x,\n    train_y,\n    epochs = 1000,\n    batch_size=32,\n    validation_data = (test_x,test_y),\n    shuffle=True,\n    verbose = 1,\n    callbacks = [PlotLearning(), earstop] #lrate]\n)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print(f\"Epochs: {len(history.history['accuracy'])}\")\nprint(f\"Accuracy: {history.history['accuracy'][-1]}\")\nprint(f\"Validation Accuracy: {history.history['val_accuracy'][-1]}\")\nprint(f\"Loss: {history.history['loss'][-1]}\")\nprint(f\"Validation Loss: {history.history['val_loss'][-1]}\")","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Observations\nWe are well fitted. I have run some additional models without early stopping, and it looks like early stopping succesfully prevents us from mildly overfitting. There is generally a critical point where validation acc/loss diverges from training acc/loss. Some ideas which may improve this model include:\n\n- Experimenting with different CNN architecture\n- Data augmentation\n- More computational power: larger batch sizes, more convolution layers\n\nUnfortunately, these ideas are out of the breadth of this notebook.\n\n\n Overall, looks like our model did decently well. Our validation accuracy was steadily around 70%.","metadata":{}},{"cell_type":"markdown","source":"# Submission","metadata":{}},{"cell_type":"code","source":"test_s=[]\nfor i in test_df['file_path']:\n    x1=preprocess(i)\n    test_s.append(x1)\ntest_s=np.array(test_s)/255.0\n\nsub_pred=model.predict(test_s)","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission = pd.DataFrame(test_df[\"patient_id\"].copy())\nsubmission[\"CE\"] = sub_pred\nsubmission[\"CE\"] = submission[\"CE\"].apply(lambda x : 0 if x<0 else x)\nsubmission[\"CE\"] = submission[\"CE\"].apply(lambda x : 1 if x>1 else x)\nsubmission[\"LAA\"] = 1- submission[\"CE\"]\n\nsubmission = submission.groupby(\"patient_id\").mean()\nsubmission = submission[[\"CE\", \"LAA\"]].round(6).reset_index()\nsubmission","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission.to_csv(\"submission.csv\", index = False)\n!head submission.csv","metadata":{"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Thank you for taking the time to read my notebook. Through the construction of this model, I hoped to gain useful experience with Tensorflow, CNNs, and the machine learning process, as well as have fun playing with a meaningful dataset. \n\n\nFor the past few months, I have been taking Andrew Ng's Deep Learning courses, and after each class, I like to try and complete a Kaggle competition to apply what I've learned. This project has been the most fun I've had, and I believe it's because now I have enough knowledge and experience to know with more confidence what I am doing.\n\n\nThe road to data science mastery is a long journey, as it is with all things, but I can confidentally say that this is not my last stop. \n\n\n# References\n\n\nAndrew Ng's [\"Improving Deep Neural Networks\"](https://www.coursera.org/learn/deep-neural-network) course\n\n\nJunji Takeshima's [CNN Notebook Starter Template](https://www.kaggle.com/code/junjitakeshima/mayo-simple-cnn-starter-eng)","metadata":{}}]}