{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"#### The Problem\n\nIntracranial hemorrhage detection is a critical problem that requires immediate diagnosis and treatment. Traditionally, doctors have to diagnose a hemorrhage by visiaully inspecting images from angiograms and MRI scans. The symptoms for a hemmorhage could be as simple as an intense headache which may or may not actually be an intracranial hemmorhage thereby wasting time and resources.","metadata":{}},{"cell_type":"markdown","source":"#### The Data\n\nThe Radiological Society of North America (RSNA) has provided collection of over 750,000 images of brain scans through Kaggle. This data comes in DICOM format so they also contain associated metadata such as patient ID and appropriate labels. The data comes with one of 6 labels: epidural, subdural, subarachnoid, intraparenchymal, intraventricular hemorrhages, and ‘any’ if any of the preceding ones are 1. A description of each type of hemorrhage follows. ","metadata":{}},{"cell_type":"markdown","source":"#### The Approach\n\nThe aim is to try to find a model that will classify the images based on the existence of a hemorrhage and then, if there is one, into each of the subtypes described above. We use png data from Kaggle because the file sizes are smaller. ","metadata":{}},{"cell_type":"code","source":"ls ../input/rsna-intracranial-hemorrhage-detection/rsna-intracranial-hemorrhage-detection/","metadata":{"execution":{"iopub.status.busy":"2021-07-13T06:41:39.091955Z","iopub.execute_input":"2021-07-13T06:41:39.09227Z","iopub.status.idle":"2021-07-13T06:41:39.743101Z","shell.execute_reply.started":"2021-07-13T06:41:39.092215Z","shell.execute_reply":"2021-07-13T06:41:39.742348Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!pip install pretrainedmodels\n","metadata":{"execution":{"iopub.status.busy":"2021-07-13T07:06:34.71309Z","iopub.execute_input":"2021-07-13T07:06:34.713434Z","iopub.status.idle":"2021-07-13T07:06:43.051878Z","shell.execute_reply.started":"2021-07-13T07:06:34.713377Z","shell.execute_reply":"2021-07-13T07:06:43.051053Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"import os\nimport time\nimport copy\nimport glob\nimport numpy as np\nimport pandas as pd\nfrom tqdm import tqdm\nfrom ipywidgets import IntProgress\n\n#Computer Vision Libraries\nimport cv2\nfrom albumentations import Compose, ShiftScaleRotate, Resize\nfrom albumentations.pytorch import ToTensor\n\n#Deep Learning\nimport torch\nimport torch.nn as nn\nimport torch.optim as optim\nfrom torch.optim import lr_scheduler\nimport pretrainedmodels\nimport torchvision\nfrom torchvision import datasets, models, transforms\nfrom torch.utils.data import Dataset, random_split\n\nfrom matplotlib import pyplot as plt","metadata":{"execution":{"iopub.status.busy":"2021-07-13T07:10:11.09525Z","iopub.execute_input":"2021-07-13T07:10:11.09555Z","iopub.status.idle":"2021-07-13T07:10:11.977504Z","shell.execute_reply.started":"2021-07-13T07:10:11.095499Z","shell.execute_reply":"2021-07-13T07:10:11.976653Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Load Training Data","metadata":{}},{"cell_type":"code","source":"data_path = '../input/rsna-intracranial-hemorrhage-detection/rsna-intracranial-hemorrhage-detection/'\ntrain_data_images = '../input/rsna-train-stage-1-images-png-224x/stage_1_train_png_224x/'\n\ntrain = pd.read_csv(os.path.join(data_path,'stage_2_train.csv')) #Read file\ntrain[['ID','Image','Diagnosis']]=train['ID'].str.split('_',expand= True) #Split the ID column at each _\ntrain = train[['Image','Diagnosis','Label']] #reorder the columns\ntrain.drop_duplicates(inplace= True)  #drop duplicates\ntrain = train.pivot(index = 'Image' , columns = 'Diagnosis', values = 'Label').reset_index() #Reorganizes csv to make columns with labels instead of 6 rows for each image\ntrain['Image'] = 'ID_' + train['Image'] #Put ID_ back with picture ID's\n\n#Remove files that aren't of png type\npng = glob.glob(os.path.join(train_data_images, '*.png')) #list of paths to the pictures\npng = [os.path.basename(png)[:-4] for png in png] #drop the .png at the end \npng = np.array(png) #convert to a NumPy array\n\n\ntrain = train[train['Image'].isin(png)] # Reconcile the lists and images\ntrain = train[:300000] #Take the first 300k pictures\ntrain.to_csv('train.csv', index = False)\nprint(train.shape) #just to know shape of dataset ","metadata":{"execution":{"iopub.status.busy":"2021-07-13T06:45:24.650056Z","iopub.execute_input":"2021-07-13T06:45:24.650378Z","iopub.status.idle":"2021-07-13T06:46:14.268236Z","shell.execute_reply.started":"2021-07-13T06:45:24.650323Z","shell.execute_reply":"2021-07-13T06:46:14.267163Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"#### Load Testing Data","metadata":{}},{"cell_type":"code","source":"#Same code as before, just changing names\ndata_path = '../input/rsna-intracranial-hemorrhage-detection/rsna-intracranial-hemorrhage-detection/'\ntest_data_images = '../input/stage-2-png/'\n\ntest = pd.read_csv(os.path.join(data_path,'stage_2_sample_submission.csv')) #Read file\ntest[['ID','Image','Diagnosis']]=test['ID'].str.split('_',expand= True) #Split the ID column at each _\ntest['Image'] = 'ID_' + test['Image'] #Put ID_ back with picture ID's\ntest = test [['Image','Label']]\ntest.drop_duplicates(inplace= True)  #drop duplicates\ntest.to_csv('test.csv', index = False)\nprint(test.shape) #just to know shape of dataset ","metadata":{"execution":{"iopub.status.busy":"2021-07-13T06:47:05.316836Z","iopub.execute_input":"2021-07-13T06:47:05.317143Z","iopub.status.idle":"2021-07-13T06:47:09.766611Z","shell.execute_reply.started":"2021-07-13T06:47:05.317092Z","shell.execute_reply":"2021-07-13T06:47:09.765717Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"class RSNA(Dataset):\n\n    def __init__(self, csv_file, path, labels, transform=None):       \n        self.path = path\n        self.data = pd.read_csv(csv_file)\n        self.transform = transform\n        self.labels = labels\n\n    def __len__(self):        \n        return len(self.data)\n\n    def __getitem__(self, idx):\n        img_name = os.path.join(self.path, self.data.loc[idx, 'Image'] + '.png')\n        img = cv2.imread(img_name)   \n        \n        if self.transform:                   \n            augmented = self.transform(image=img)\n            img = augmented['image']   \n            \n        if self.labels:            \n            labels = torch.tensor(\n                self.data.loc[idx, ['epidural', 'intraparenchymal', 'intraventricular', 'subarachnoid', 'subdural', 'any']])\n            return {'image': img, 'labels': labels}    \n        \n        else:                  \n            return {'image': img}","metadata":{"execution":{"iopub.status.busy":"2021-07-13T06:47:12.80525Z","iopub.execute_input":"2021-07-13T06:47:12.805548Z","iopub.status.idle":"2021-07-13T06:47:12.814678Z","shell.execute_reply.started":"2021-07-13T06:47:12.805497Z","shell.execute_reply":"2021-07-13T06:47:12.813751Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"transform_train = Compose([ShiftScaleRotate(),ToTensor()])\ntransform_test = Compose([ToTensor()])\n\ntrain_dataset = RSNA(csv_file='train.csv', path=train_data_images, transform=transform_train, labels=True)\ntest_dataset = RSNA(csv_file='test.csv', path=test_data_images, transform=transform_test, labels=False)\n\nbatch_size = 64\ndata_train_generator = torch.utils.data.DataLoader(dataset=train_dataset,batch_size=batch_size,shuffle=True)\ndata_test_generator = torch.utils.data.DataLoader(dataset=test_dataset,batch_size=batch_size,shuffle=False)","metadata":{"execution":{"iopub.status.busy":"2021-07-13T07:45:46.844103Z","iopub.execute_input":"2021-07-13T07:45:46.844878Z","iopub.status.idle":"2021-07-13T07:45:47.281602Z","shell.execute_reply.started":"2021-07-13T07:45:46.844463Z","shell.execute_reply":"2021-07-13T07:45:47.280586Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\nmodel0 = models.densenet201(pretrained = False) \nmodel = torch.nn.Sequential(model0, torch.nn.Linear(1000, 6)).to(device)\n\n#model = {alexnet,vgg, resnet18}","metadata":{"execution":{"iopub.status.busy":"2021-07-13T07:00:36.365623Z","iopub.execute_input":"2021-07-13T07:00:36.365926Z","iopub.status.idle":"2021-07-13T07:00:37.613304Z","shell.execute_reply.started":"2021-07-13T07:00:36.365872Z","shell.execute_reply":"2021-07-13T07:00:37.612567Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"num_epochs = 3\noptimizer = optim.Adam(model.parameters(), lr=4e-5)\ncriterion = torch.nn.BCEWithLogitsLoss()","metadata":{"execution":{"iopub.status.busy":"2021-06-29T20:26:36.798848Z","iopub.execute_input":"2021-06-29T20:26:36.799139Z","iopub.status.idle":"2021-06-29T20:26:36.809416Z","shell.execute_reply.started":"2021-06-29T20:26:36.799091Z","shell.execute_reply":"2021-06-29T20:26:36.808825Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"def train_model(model,criterion,optimizer,num_epochs=1):\n    for epoch in range(1, num_epochs+1):\n\n        print('Epoch {}/{}'.format(epoch, num_epochs))\n        print('-' * 10)\n\n        model.train()    \n        tr_loss = 0\n\n        tk0 = tqdm(data_train_generator, desc=\"Iteration\")\n\n        for step, batch in enumerate(tk0):\n\n            inputs = batch[\"image\"]\n            labels = batch[\"labels\"]\n\n            inputs = inputs.to(device, dtype=torch.float)\n            labels = labels.to(device, dtype=torch.float)\n\n            outputs = model(inputs)\n            loss = criterion(outputs, labels)\n\n            loss.backward()\n\n            tr_loss += loss.item()\n\n            optimizer.step()\n            optimizer.zero_grad()\n\n        torch.save(model.state_dict(), f'resnext50_{epoch}.pth') \n\n        epoch_loss = tr_loss / len(data_train_generator)\n        print('Training Loss: {:.4f}'.format(epoch_loss))\n        \n    return model\n","metadata":{"execution":{"iopub.status.busy":"2021-06-29T20:26:36.810922Z","iopub.execute_input":"2021-06-29T20:26:36.811427Z","iopub.status.idle":"2021-06-29T20:26:37.0058Z","shell.execute_reply.started":"2021-06-29T20:26:36.811334Z","shell.execute_reply":"2021-06-29T20:26:37.004623Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"model_trained = train_model(model,criterion,optimizer,num_epochs)","metadata":{"execution":{"iopub.status.busy":"2021-06-29T20:26:37.009657Z","iopub.execute_input":"2021-06-29T20:26:37.009919Z","iopub.status.idle":"2021-06-30T04:11:17.876025Z","shell.execute_reply.started":"2021-06-29T20:26:37.009874Z","shell.execute_reply":"2021-06-30T04:11:17.875234Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#model_name = 'se_resnext101_32x4d' # could be fbresnet152 or inceptionresnetv2\n#model = pretrainedmodels.__dict__[model_name](num_classes=1000)","metadata":{"execution":{"iopub.status.busy":"2021-07-13T07:42:46.984051Z","iopub.execute_input":"2021-07-13T07:42:46.984504Z","iopub.status.idle":"2021-07-13T07:42:47.592344Z","shell.execute_reply.started":"2021-07-13T07:42:46.984314Z","shell.execute_reply":"2021-07-13T07:42:47.591173Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"#weights = torch.load('../input/rsna-model110-2/model110/fold7_ep3.pt')\n#model.load_state_dict(weights, strict=False)","metadata":{"execution":{"iopub.status.busy":"2021-07-13T07:45:22.857346Z","iopub.execute_input":"2021-07-13T07:45:22.857636Z","iopub.status.idle":"2021-07-13T07:45:23.191559Z","shell.execute_reply.started":"2021-07-13T07:45:22.857588Z","shell.execute_reply":"2021-07-13T07:45:23.190545Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"for param in model.parameters():\n   param.requires_grad = False\n\nmodel.eval()\n\ntest_pred = np.zeros((len(test_dataset) * 6, 1))\n\nfor i, batch_ in enumerate(tqdm(data_test_generator)):\n    batch_ = batch_[\"image\"]\n    batch_ = batch_.to(device, dtype=torch.float)\n    \n    with torch.no_grad():\n        \n        pred = model(batch_)\n        \n        test_pred[(i * batch_size * 6):((i + 1) * batch_size * 6)] = torch.sigmoid(\n            pred).detach().cpu().reshape((len(batch_) * 6, 1))  ","metadata":{"execution":{"iopub.status.busy":"2021-07-13T07:45:54.865958Z","iopub.execute_input":"2021-07-13T07:45:54.866281Z","iopub.status.idle":"2021-07-13T07:45:55.251458Z","shell.execute_reply.started":"2021-07-13T07:45:54.866227Z","shell.execute_reply":"2021-07-13T07:45:55.25017Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"ls ../input/rsna-intracranial-hemorrhage-detection/rsna-intracranial-hemorrhage-detection/","metadata":{"execution":{"iopub.status.busy":"2021-06-30T04:42:17.147938Z","iopub.execute_input":"2021-06-30T04:42:17.149674Z","iopub.status.idle":"2021-06-30T04:42:18.162815Z","shell.execute_reply.started":"2021-06-30T04:42:17.14962Z","shell.execute_reply":"2021-06-30T04:42:18.161762Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"submission =  pd.read_csv(os.path.join(data_path, 'stage_2_sample_submission.csv'))\nsubmission = pd.concat([submission.drop(columns=['Label']), pd.DataFrame(test_pred)], axis=1)\nsubmission.columns = ['ID', 'Label']\nsubmission.to_csv('submission-1.csv', index=False)","metadata":{"execution":{"iopub.status.busy":"2021-06-30T04:42:18.165724Z","iopub.execute_input":"2021-06-30T04:42:18.166049Z","iopub.status.idle":"2021-06-30T04:42:22.25736Z","shell.execute_reply.started":"2021-06-30T04:42:18.165996Z","shell.execute_reply":"2021-06-30T04:42:22.256606Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}