{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"\n\n","metadata":{}},{"cell_type":"code","source":"import warnings\nwarnings.filterwarnings(\"ignore\", category=DeprecationWarning)\nwarnings.filterwarnings(\"ignore\", category=UserWarning)\nwarnings.filterwarnings(\"ignore\", category=FutureWarning)\n\nfrom IPython.display import HTML","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-31T18:42:36.905976Z","iopub.execute_input":"2022-08-31T18:42:36.906563Z","iopub.status.idle":"2022-08-31T18:42:36.936483Z","shell.execute_reply.started":"2022-08-31T18:42:36.906465Z","shell.execute_reply":"2022-08-31T18:42:36.935536Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Competition goal\n\n* Classifiy the etiology (origin) of a blood clot shown in a tissue slice of a patient that has experienced an acute ischemic stroke. \n* It's a **binary classification task** with labels CE (cardioembolic) or LAA (Large Artery Atherosclerosis)\n* There are supplemental slides with a either an **unknown etiology or an etiology other than CE or LAA**! \n\nAha! ... :-O Do you know the difference between CE and LAA or what is meant by an acute ischemic stroke? ... Neither do I! ;-) ","metadata":{}},{"cell_type":"code","source":"HTML('<iframe width=\"800\" height=\"444\" src=\"https://www.youtube.com/embed/abxcrAvw-O0\" title=\"YouTube video player\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen></iframe>')","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-31T18:42:36.938342Z","iopub.execute_input":"2022-08-31T18:42:36.9393Z","iopub.status.idle":"2022-08-31T18:42:36.950985Z","shell.execute_reply.started":"2022-08-31T18:42:36.939263Z","shell.execute_reply":"2022-08-31T18:42:36.950019Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Table of contents\n\n1. [Prepare to start](#prepare)\n2. [Exploratory analysis - EDA](#eda)\n    * [What do we know about the training data?](#train_data)\n3. [The nightmare of overfitting](#nightmare)\n4. [Tell me what do you see?](#explain)\n    * [Choosing a model](#choose_model)\n5. [Explaining predictions with LIME](#lime)\n","metadata":{}},{"cell_type":"markdown","source":"# Prepare to start <a class=\"anchor\" id=\"prepare\"></a>\n\n## Loading packages","metadata":{}},{"cell_type":"code","source":"import os\n\nimport pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\n%matplotlib inline\n\nfrom PIL import Image\nimport cv2\nfrom skimage.segmentation import mark_boundaries\n\nimport tensorflow as tf\n\nfrom tqdm.notebook import tqdm\nimport torch\nfrom torch.utils.data import Dataset, DataLoader\nfrom torch import nn\n\nimport seaborn as sns\nsns.set()\n\nfrom lime import lime_image\n\nimport shap","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-31T18:42:36.952523Z","iopub.execute_input":"2022-08-31T18:42:36.953122Z","iopub.status.idle":"2022-08-31T18:42:47.637568Z","shell.execute_reply.started":"2022-08-31T18:42:36.953087Z","shell.execute_reply":"2022-08-31T18:42:47.636627Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Loading data","metadata":{}},{"cell_type":"code","source":"train = pd.read_csv(\"../input/mayo-clinic-strip-ai/train.csv\")","metadata":{"execution":{"iopub.status.busy":"2022-08-31T18:42:47.64058Z","iopub.execute_input":"2022-08-31T18:42:47.641437Z","iopub.status.idle":"2022-08-31T18:42:47.656661Z","shell.execute_reply.started":"2022-08-31T18:42:47.641393Z","shell.execute_reply":"2022-08-31T18:42:47.65555Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Exploratory analysis <a class=\"anchor\" id=\"eda\"></a>","metadata":{}},{"cell_type":"markdown","source":"## What do we know about our training data? <a class=\"anchor\" id=\"train_data\"></a>\n","metadata":{}},{"cell_type":"code","source":"train.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-31T18:42:47.658206Z","iopub.execute_input":"2022-08-31T18:42:47.658885Z","iopub.status.idle":"2022-08-31T18:42:47.674466Z","shell.execute_reply.started":"2022-08-31T18:42:47.658846Z","shell.execute_reply":"2022-08-31T18:42:47.673436Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"train.shape","metadata":{"execution":{"iopub.status.busy":"2022-08-31T18:42:47.676206Z","iopub.execute_input":"2022-08-31T18:42:47.676849Z","iopub.status.idle":"2022-08-31T18:42:47.682995Z","shell.execute_reply.started":"2022-08-31T18:42:47.676811Z","shell.execute_reply":"2022-08-31T18:42:47.681967Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"* The **image_id** consists of two different parts: *{patient_id}_{image_number}*\n* The **center_id** defines the medical center or institute where the image was taken\n* **patient_id** is the id of the patient\n* **image_num** is a counter for slices of a patient\n* **label** is our target\n    * CE: cardioembolic\n    * LAA: Large artery atherosclerosis\n    ","metadata":{}},{"cell_type":"code","source":"train.patient_id.nunique()","metadata":{"execution":{"iopub.status.busy":"2022-08-31T18:42:47.684638Z","iopub.execute_input":"2022-08-31T18:42:47.685605Z","iopub.status.idle":"2022-08-31T18:42:47.698516Z","shell.execute_reply.started":"2022-08-31T18:42:47.685564Z","shell.execute_reply":"2022-08-31T18:42:47.69752Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"632 patients in train.","metadata":{}},{"cell_type":"code","source":"plt.figure(figsize=(10,5))\nsns.countplot(train.groupby(\"patient_id\").image_num.size(), palette=\"Greens_r\")\nplt.xlabel(\"Number of images per patient\")\nplt.title(\"Max image number per patient in train\");","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-31T18:42:47.700321Z","iopub.execute_input":"2022-08-31T18:42:47.700755Z","iopub.status.idle":"2022-08-31T18:42:47.94447Z","shell.execute_reply.started":"2022-08-31T18:42:47.700713Z","shell.execute_reply":"2022-08-31T18:42:47.943455Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We can clearly see that most patients only have one slice in the training data and only a few show more than one. Maybe these are hard cases where it's more difficult to see the etiology?! Or does it depend on the medical center?","metadata":{}},{"cell_type":"code","source":"train.groupby(\"patient_id\").center_id.nunique().max()","metadata":{"execution":{"iopub.status.busy":"2022-08-31T18:42:47.946311Z","iopub.execute_input":"2022-08-31T18:42:47.946694Z","iopub.status.idle":"2022-08-31T18:42:47.956078Z","shell.execute_reply.started":"2022-08-31T18:42:47.946656Z","shell.execute_reply":"2022-08-31T18:42:47.954878Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Ok, all patients belong to one center. There is no patient that shows images that were taken in different medial centers. How many do we have and how many patients do we have per center?","metadata":{}},{"cell_type":"code","source":"train.groupby(\"center_id\").patient_id.size().sort_values(ascending=False)","metadata":{"execution":{"iopub.status.busy":"2022-08-31T18:42:47.960626Z","iopub.execute_input":"2022-08-31T18:42:47.961373Z","iopub.status.idle":"2022-08-31T18:42:47.971142Z","shell.execute_reply.started":"2022-08-31T18:42:47.961339Z","shell.execute_reply":"2022-08-31T18:42:47.969522Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Definitely not balanced.","metadata":{}},{"cell_type":"markdown","source":"Let's take a look at the target distribution! :-)","metadata":{}},{"cell_type":"code","source":"sns.countplot(train.label, palette=\"Reds_r\")","metadata":{"execution":{"iopub.status.busy":"2022-08-31T18:42:47.973886Z","iopub.execute_input":"2022-08-31T18:42:47.974633Z","iopub.status.idle":"2022-08-31T18:42:48.149707Z","shell.execute_reply.started":"2022-08-31T18:42:47.974598Z","shell.execute_reply":"2022-08-31T18:42:48.148681Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Ok, it's an imbalanced classification problem. ","metadata":{}},{"cell_type":"code","source":"train.label.value_counts() / train.shape[0]","metadata":{"execution":{"iopub.status.busy":"2022-08-31T18:42:48.151287Z","iopub.execute_input":"2022-08-31T18:42:48.152403Z","iopub.status.idle":"2022-08-31T18:42:48.163802Z","shell.execute_reply.started":"2022-08-31T18:42:48.152366Z","shell.execute_reply":"2022-08-31T18:42:48.162561Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Ok we know that we have some patients with more than one slice. Is it possible they show more than one label per patient?","metadata":{}},{"cell_type":"code","source":"train.groupby(\"patient_id\").label.nunique().max()","metadata":{"execution":{"iopub.status.busy":"2022-08-31T18:42:48.165376Z","iopub.execute_input":"2022-08-31T18:42:48.166616Z","iopub.status.idle":"2022-08-31T18:42:48.177604Z","shell.execute_reply.started":"2022-08-31T18:42:48.166579Z","shell.execute_reply":"2022-08-31T18:42:48.176624Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"No! :-)","metadata":{}},{"cell_type":"code","source":"test = pd.read_csv(\"../input/mayo-clinic-strip-ai/test.csv\")\ntest[\"image_path\"] = test[\"image_id\"].apply(lambda x: os.path.join(\"../input/mayo-clinic-strip-ai/test/\", x+\".tif\"))\ntest.head()","metadata":{"execution":{"iopub.status.busy":"2022-08-31T18:42:48.17913Z","iopub.execute_input":"2022-08-31T18:42:48.179597Z","iopub.status.idle":"2022-08-31T18:42:48.20112Z","shell.execute_reply.started":"2022-08-31T18:42:48.179554Z","shell.execute_reply":"2022-08-31T18:42:48.20014Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## The nightmare of overfitting ;-) <a class=\"anchor\" id=\"nightmare\"></a>","metadata":{}},{"cell_type":"markdown","source":"Can't wait anymore! Let's start to discover the images! ","metadata":{}},{"cell_type":"code","source":"from PIL import Image\n\nImage.MAX_IMAGE_PIXELS = None","metadata":{"execution":{"iopub.status.busy":"2022-08-31T18:42:48.202383Z","iopub.execute_input":"2022-08-31T18:42:48.203022Z","iopub.status.idle":"2022-08-31T18:42:48.208025Z","shell.execute_reply.started":"2022-08-31T18:42:48.202972Z","shell.execute_reply":"2022-08-31T18:42:48.207018Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"example = \"../input/mayo-clinic-strip-ai/train/006388_0.tif\"\nimg = Image.open(example)\nimg.size","metadata":{"execution":{"iopub.status.busy":"2022-08-31T18:42:48.209397Z","iopub.execute_input":"2022-08-31T18:42:48.210112Z","iopub.status.idle":"2022-08-31T18:42:48.286926Z","shell.execute_reply.started":"2022-08-31T18:42:48.210078Z","shell.execute_reply":"2022-08-31T18:42:48.285985Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Uff!","metadata":{}},{"cell_type":"code","source":"x1 = img.size[0]\nx2 = img.size[1]\nsc = x2/x1","metadata":{"execution":{"iopub.status.busy":"2022-08-31T18:42:48.288414Z","iopub.execute_input":"2022-08-31T18:42:48.289017Z","iopub.status.idle":"2022-08-31T18:42:48.293669Z","shell.execute_reply.started":"2022-08-31T18:42:48.288978Z","shell.execute_reply":"2022-08-31T18:42:48.292705Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"factor = 1/10\nresized_img = img.resize((int(x1*factor), int(sc*x1*factor)))","metadata":{"execution":{"iopub.status.busy":"2022-08-31T18:42:48.295249Z","iopub.execute_input":"2022-08-31T18:42:48.296047Z","iopub.status.idle":"2022-08-31T18:43:53.566923Z","shell.execute_reply.started":"2022-08-31T18:42:48.296011Z","shell.execute_reply":"2022-08-31T18:43:53.565908Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"resized_img.size","metadata":{"execution":{"iopub.status.busy":"2022-08-31T18:43:53.568465Z","iopub.execute_input":"2022-08-31T18:43:53.569063Z","iopub.status.idle":"2022-08-31T18:43:53.577372Z","shell.execute_reply.started":"2022-08-31T18:43:53.569024Z","shell.execute_reply":"2022-08-31T18:43:53.57628Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"resized_img.rotate(90, expand=True)","metadata":{"execution":{"iopub.status.busy":"2022-08-31T18:43:53.57882Z","iopub.execute_input":"2022-08-31T18:43:53.579329Z","iopub.status.idle":"2022-08-31T18:43:56.09819Z","shell.execute_reply.started":"2022-08-31T18:43:53.579293Z","shell.execute_reply":"2022-08-31T18:43:56.096933Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Oh this will be a nightmare for validation.... such a complex feature space... I can't see the blood clot, do you? How should we prevent our model from overfitting? Seems hopeless. Now, imagine that medical centers might also use different methods or protocols to obtain these slices... ohoh! The number of patients given such a high feature space is low and the public test data seems to be quite small as well. Hmm.","metadata":{}},{"cell_type":"markdown","source":"# Tell me, what do you see? <a class=\"anchor\" id=\"explain\"></a>\n\nOk, so far I can't imagine that any model will be able to see something useful in this complex feature space... but let's better try to understand and explain what our models \"see\". For this purpose I like to use and learn more about some machine learning explainability tools for computer vision.","metadata":{}},{"cell_type":"markdown","source":"## What is meant by Machine Learning Explainability?","metadata":{}},{"cell_type":"markdown","source":"## Choosing a model <a class=\"anchor\" id=\"choose_model\"></a>\n\n\nLet's pick one of the best scoring public models so far with the hope that it has found an interesting and useful signal. At the moment it's this one:\n\n* https://www.kaggle.com/code/realneuralnetwork/cnn-strip-ai-inference/notebook","metadata":{}},{"cell_type":"code","source":"class ImgDataset(Dataset):\n    \n    def __init__(self, df):\n        self.df = df \n        self.train = 'label' in df.columns\n        \n    def __len__(self):\n        return len(self.df)\n    \n    def __getitem__(self, index):\n        paths = [\"../input/jpg-images-strip-ai/test/\", \"../input/jpg-images-strip-ai/train/\"]\n        try:\n            image = cv2.imread(paths[self.train] + self.df.iloc[index].image_id + \".jpg\")\n        except:\n            image = np.zeros((512,512,3), np.uint8)\n        label = 0\n        try:\n            if len(image.shape) == 5:\n                image = image.squeeze().transpose(1, 2, 0)\n            image = cv2.resize(image, (512, 512)).transpose(2, 0, 1)\n        except:\n            image = np.zeros((3, 512, 512))\n        if(self.train):\n            label = {\"CE\" : 0, \"LAA\": 1}[self.df.iloc[index].label]\n        patient_id = self.df.iloc[index].patient_id\n        return image, label, patient_id\n    \n    \ndef predict(model, dataloader):\n    model.cuda()\n    model.eval()\n    dataloader = dataloader\n    outputs = []\n    s = nn.Softmax(dim=1)\n    ids = []\n    for item in tqdm(dataloader, leave=False):\n        patient_id = item[2][0]\n        try:\n            images = item[0].cuda().float()\n            ids.append(patient_id)\n            output = model(images)\n            outputs.append(s(output.cpu()[:,:2])[0].detach().numpy())\n        except:\n            ids.append(patient_id)\n            outputs.append(s(torch.tensor([[1, 1]]).float())[0].detach().numpy())\n    return np.array(outputs), ids","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-31T18:43:56.099825Z","iopub.execute_input":"2022-08-31T18:43:56.100287Z","iopub.status.idle":"2022-08-31T18:43:56.128163Z","shell.execute_reply.started":"2022-08-31T18:43:56.100238Z","shell.execute_reply":"2022-08-31T18:43:56.126885Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model = torch.jit.load('../input/strip-ai-models-to-explain/kabir_ivan_cnn_strip_ai_v1.pth')","metadata":{"execution":{"iopub.status.busy":"2022-08-31T18:43:56.129963Z","iopub.execute_input":"2022-08-31T18:43:56.142514Z","iopub.status.idle":"2022-08-31T18:43:57.898958Z","shell.execute_reply.started":"2022-08-31T18:43:56.142448Z","shell.execute_reply":"2022-08-31T18:43:57.897953Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"batch_size = 1\ntest_loader = DataLoader(\n    ImgDataset(test.iloc[0:4]), \n    batch_size=batch_size, \n    shuffle=False, \n    num_workers=1\n)","metadata":{"execution":{"iopub.status.busy":"2022-08-31T18:43:57.900587Z","iopub.execute_input":"2022-08-31T18:43:57.90099Z","iopub.status.idle":"2022-08-31T18:43:57.906697Z","shell.execute_reply.started":"2022-08-31T18:43:57.900946Z","shell.execute_reply":"2022-08-31T18:43:57.905554Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"anss, ids = predict(model, test_loader)","metadata":{"execution":{"iopub.status.busy":"2022-08-31T18:43:57.908219Z","iopub.execute_input":"2022-08-31T18:43:57.908974Z","iopub.status.idle":"2022-08-31T18:44:11.351792Z","shell.execute_reply.started":"2022-08-31T18:43:57.908932Z","shell.execute_reply":"2022-08-31T18:44:11.35061Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"anss","metadata":{"execution":{"iopub.status.busy":"2022-08-31T18:44:11.355702Z","iopub.execute_input":"2022-08-31T18:44:11.356015Z","iopub.status.idle":"2022-08-31T18:44:11.364675Z","shell.execute_reply.started":"2022-08-31T18:44:11.355985Z","shell.execute_reply":"2022-08-31T18:44:11.363485Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ids","metadata":{"execution":{"iopub.status.busy":"2022-08-31T18:44:11.366418Z","iopub.execute_input":"2022-08-31T18:44:11.367081Z","iopub.status.idle":"2022-08-31T18:44:11.375693Z","shell.execute_reply.started":"2022-08-31T18:44:11.367045Z","shell.execute_reply":"2022-08-31T18:44:11.374489Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"prob = pd.DataFrame({\"CE\" : anss[:,0], \"LAA\" : anss[:,1], \"id\" : ids}).groupby(\"id\").mean()\nsubmission = pd.read_csv(\"../input/mayo-clinic-strip-ai/sample_submission.csv\")\nsubmission.CE = prob.CE.to_list()\nsubmission.LAA = prob.LAA.to_list()\nsubmission.to_csv(\"submission.csv\", index = False)","metadata":{"execution":{"iopub.status.busy":"2022-08-31T18:44:11.377258Z","iopub.execute_input":"2022-08-31T18:44:11.378387Z","iopub.status.idle":"2022-08-31T18:44:11.409614Z","shell.execute_reply.started":"2022-08-31T18:44:11.378358Z","shell.execute_reply":"2022-08-31T18:44:11.408699Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Explaining predictions with LIME <a class=\"anchor\" id=\"lime\"></a>\n\nLIME stands for Local Interpretable Model-Agnoistic Explanations. You can read the paper published 2016 [here](https://arxiv.org/pdf/1602.04938.pdf). I know that there exist newer tools but I think it can be a good starting point to get started with the topic. ;-) Here is a great video to introduce LIME:","metadata":{}},{"cell_type":"code","source":"HTML('<iframe width=\"708\" height=\"400\" src=\"https://www.youtube.com/embed/hUnRCxnydCc\" title=\"YouTube video player\" frameborder=\"0\" allow=\"accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture\" allowfullscreen></iframe>')","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2022-08-31T18:44:11.41527Z","iopub.execute_input":"2022-08-31T18:44:11.416082Z","iopub.status.idle":"2022-08-31T18:44:11.424245Z","shell.execute_reply.started":"2022-08-31T18:44:11.416048Z","shell.execute_reply":"2022-08-31T18:44:11.423106Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"### Insights\n\nLet's collect a few key messages of the video and the paper:\n\n* LIME explains the predictions of any classifer (model) by treating it as a black box.\n* It trains an interpretable model locally around the prediction.\n* It helps to decide whether to trust a prediction or a model.\n* We need to keep in mind that the performance measured by your evaluation metric might not suite to the performance of our model on real-world data (pulic/private test data here) or even with good performance it can happen that your predictions doesn't make sense. \n* The explanation we get with LIME has to be simple enough for a human to understand regardless how many features were used by a model to make its prediction.\n* For an image classifier an interpretable representation can be a binary vector that indicates a presence of absence of a contiguous patch of similar pixels (super-pixel).\n\n","metadata":{}},{"cell_type":"markdown","source":"Ok let's start! ","metadata":{}},{"cell_type":"code","source":"def batch_predict(images):\n    s = nn.Softmax(dim=1)\n    images = images.transpose((0,3,1,2))\n    images = torch.tensor(images).cuda().float()\n    outputs = model(images)\n    preds = s(outputs.cpu()[:,:2]).detach().numpy()\n    return preds","metadata":{"execution":{"iopub.status.busy":"2022-08-31T18:46:18.800623Z","iopub.execute_input":"2022-08-31T18:46:18.801082Z","iopub.status.idle":"2022-08-31T18:46:18.81303Z","shell.execute_reply.started":"2022-08-31T18:46:18.801047Z","shell.execute_reply":"2022-08-31T18:46:18.811757Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"batch_size = 1\ntest_loader = DataLoader(\n    ImgDataset(test.iloc[0:2]), \n    batch_size=batch_size, \n    shuffle=False, \n    num_workers=1\n)\ntest_iterator = iter(test_loader)","metadata":{"execution":{"iopub.status.busy":"2022-08-31T18:46:19.286221Z","iopub.execute_input":"2022-08-31T18:46:19.286664Z","iopub.status.idle":"2022-08-31T18:46:19.528307Z","shell.execute_reply.started":"2022-08-31T18:46:19.286628Z","shell.execute_reply":"2022-08-31T18:46:19.525371Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"As we have only two classes we can set top_labels with the value 2:","metadata":{}},{"cell_type":"code","source":"model.cuda()\nmodel.eval()\nbatch = next(test_iterator)\npatient_ids = batch[2]\nimage = batch[0].cpu().detach()\nlm_image = image.squeeze().numpy().transpose((1,2,0)) \n\nexplainer = lime_image.LimeImageExplainer()\n\n\nexplanation = explainer.explain_instance(lm_image, \n                                     batch_predict, \n                                     top_labels=2, \n                                     hide_color=0, batch_size=1)\noutput = model(image.cuda().float())\ns = nn.Softmax(dim=1)\nprobas = s(output[:,:2])[0].cpu().detach().numpy()","metadata":{"execution":{"iopub.status.busy":"2022-08-31T18:46:23.656612Z","iopub.execute_input":"2022-08-31T18:46:23.657562Z","iopub.status.idle":"2022-08-31T18:47:18.472025Z","shell.execute_reply.started":"2022-08-31T18:46:23.65752Z","shell.execute_reply":"2022-08-31T18:47:18.470731Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"probas","metadata":{"execution":{"iopub.status.busy":"2022-08-31T18:48:58.875953Z","iopub.execute_input":"2022-08-31T18:48:58.876533Z","iopub.status.idle":"2022-08-31T18:48:58.888985Z","shell.execute_reply.started":"2022-08-31T18:48:58.876466Z","shell.execute_reply":"2022-08-31T18:48:58.887581Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"We can see that the probability of the first label (CE) is higher than for the second one (LAA). To understand which parts of the image caused the model to make this prediction, we can take a look at the superpixels for the top_label (CE in this case). The tissues of these images are very detailed and I expect that the signals of the origin of the stroke are very tiny. For this reason I like to choose a high number of superpixels given by the num_features attribute. We only want to see which superpixels contribute to the top_label prediction (CE) and hide the rest of the image. And this is what we get:","metadata":{}},{"cell_type":"code","source":"temp, mask = explanation.get_image_and_mask(explanation.top_labels[0], positive_only=True, num_features=100, hide_rest=True)\nimg_boundry1 = mark_boundaries(temp/255.0, mask)\n\n\nimages = batch[0].cpu().detach().numpy()\nprint(images.shape)\nimages = images.transpose((0,2,3,1))\n\nfig, ax = plt.subplots(1,2,figsize=(20,10))\nax[0].imshow(images[0])\n\nax[0].set_title(\"Patient {}\".format(patient_ids[0]))\nax[1].imshow(img_boundry1)\n\nfor n in range(2):\n    ax[n].axis(\"off\");","metadata":{"execution":{"iopub.status.busy":"2022-08-31T18:58:50.454311Z","iopub.execute_input":"2022-08-31T18:58:50.454694Z","iopub.status.idle":"2022-08-31T18:58:51.049243Z","shell.execute_reply.started":"2022-08-31T18:58:50.454664Z","shell.execute_reply":"2022-08-31T18:58:51.048326Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Conclusion \n\n* We can't trust our model! Do you see it? We can clearly see that empty superpixels contributed to the top_label prediction. This doesn't make sense! \n* There are also a few regions that belong to tissue but so far it's not clear why. It could be the color or the way the image was streched or distorted during preprocessing... who knows?! \n\nIn my opinion we should try to understand the problem better. Can we really see differenes in these tissues given the origin of the blood clot after an acute ischemic stroke? How do medical experts perform this task? Do they have more information available when they need to make a choice?","metadata":{}}]}