{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Introduction to this Kernel ","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"The Sole intention of this kernel is to provide the reader with a basic yet robust to all situations pipeline for approaching almost any Image Classifiaction Problem. This is a very basic and generalized kernel and I've tried to keep it easily intergrable with almost any other pytorch code. Happy Kaggling :-)","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"**If you like the content in this notebook, Do Upvote the notebook and show me your support**","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Note: I have shown this kernel with no feature engineering no hyperparameter tuning, if you use N_tile feature engineering along with some other preprocessing and tune the hyper parameters well, you can get a decent score","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Problem Statement in a nutshell","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Given some 10,000 images predict the class they belong to, where the class is isup_score. We are given a csv file by the name train, which contains the names of the images and some extra information. \n\nIt can be interpreted like a normal image classification task. Simple isn't it? ","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Loading the Important Libraries","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"First things first, lets load all the important libraries first","execution_count":null},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true},"cell_type":"code","source":"#Basic Python and Machine learning libraries\nimport os, sys, warnings, random, time, cv2\nimport pandas as pd\nimport numpy as np\nimport matplotlib.pyplot as plt\nimport seaborn as sns\nimport skimage.io\nfrom PIL import Image\nfrom scipy import stats\nfrom sklearn import metrics\nfrom sklearn.model_selection import StratifiedKFold\nfrom IPython.display import display\nfrom tqdm import tqdm_notebook as tqdm\n\n#Pytorch and Albumentations(Data Augmentation Library)\nimport torch\nimport albumentations\nfrom albumentations.pytorch import ToTensorV2\nfrom torch import nn, optim\nfrom torch.optim import lr_scheduler\nfrom torch.functional import F \nfrom torch.utils.data import Dataset, DataLoader\nfrom torchvision import transforms, models\n\n#A piece of code that will allow me to use and retrain Efficient Nets from my input folder\npackage_path = '../input/efficientnet-pytorch/EfficientNet-PyTorch/EfficientNet-PyTorch-master'\nsys.path.append(package_path)\nfrom efficientnet_pytorch import model as enet\n\n#Mandatory Lines of code when working with jupyter notebooks\n%matplotlib inline\nwarnings.filterwarnings('ignore')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# I Just wanted my plots to look neat hence fixing some plot styles\nplt.style.use('seaborn-whitegrid')\nplt.rcParams['lines.linewidth'] = 2\nplt.rcParams['font.sans-serif'] = 'Arial'\nplt.rcParams['text.color'] = 'black'\nplt.rcParams['axes.labelcolor']= 'black'\nplt.rcParams['xtick.color'] = 'black'\nplt.rcParams['ytick.color'] = 'black'\nplt.rcParams['font.size'] = 12","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Fixing Config","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Putting some global variables in CONFIG class so that whenever I wish to make a change in their values, I've to change them here only, not everywhere in my code","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"class Config:\n    \n    DEBUG = False\n    pwd = '/kaggle/working/'\n    data_dir = '../input/prostate-cancer-grade-assessment/'\n    train_img_dir = os.path.join(data_dir, 'train_images')\n    test_img_dir = os.path.join(data_dir, 'test_images')\n    num_images_to_plot = 16\n    num_folds = 2 if DEBUG else 5\n    height = 256\n    width = 256\n    out_dim = 6\n    batch_size = 16\n    num_workers = 4\n    num_epochs = 2 if DEBUG else 10\n    SEED = 713\n    device = torch.device('cuda:0' if torch.cuda.is_available() else 'cpu')\n    #Image-net standard mean and std\n    mean = [0.485, 0.456, 0.406]\n    std = [0.229, 0.224, 0.225]\n    pretrainied_models = {\n        'efficientnet-b0': '../input/efficientnet-pytorch/efficientnet-b0-08094119.pth'\n    }","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Fixing seed so that our results are always reproducible\ndef seed_everything(seed):\n    random.seed(seed)\n    os.environ['PYTHONHASHSEED'] = str(seed)\n    np.random.seed(seed)\n    torch.manual_seed(seed)\n    torch.cuda.manual_seed(seed)\n    torch.backends.cudnn.deterministic = True\n    torch.backends.cudnn.benchmark = True\n\nseed_everything(Config.SEED)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's Just Peek into our train.csv","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df = pd.read_csv(Config.data_dir+'train.csv')\ntrain_df = train_df.sample(100).reset_index(drop=True) if Config.DEBUG else train_df\ndisplay(train_df.head())\nlen(train_df)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Basically DEBUG = True ensures that we can take a quick test run if our code is working or not, for Committing and Submitting keep DEBUG = False","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Building Dataset","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Now we are going to make a Class Build_Dataset, which inherits from torch.utils.data.Data set and returns us pytorch Dataset object.\n\nIf the above line is too technical for you, then simply what this class does is goes through our image directory and keeps all images ready for use whenever we need to use it, without throwing them into RAM. Imagine throwing 380GB in our RAM, catastrophic isn't it? This is why you should learn to build your custom dataset classes.\n\nIn this class it'll return me the image and the label corresponding to the index in my pandas dataframe whenever I request it to do so, we just need to over write the three methods of the pytorch's Dataset class, the __ init__() method, the __ len__() method and __ getitem__() method","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"class Build_Dataset(Dataset):\n    '''Builds Dataset to be fed to Neural Network\n       :param df: train_df or test_df\n       :param resize: tuple, eg(256, 256)\n       :param mode: string train or test \n       :param: augmentations: Image augmentations\n    '''\n    #Here I am defining my constructor\n    def __init__(self, df, resize=None, mode='train', augmentations=None):\n        self.df = df\n        self.resize = resize\n        self.mode = mode\n        self.augmentations = augmentations\n      \n    # This method returns the total length of the dataset\n    def __len__(self):\n        return len(self.df)\n    \n    # This method returns (image, label) at index idx\n    def __getitem__(self, idx):\n        if self.mode == 'train':\n            img_path = os.path.join(Config.train_img_dir, self.df['image_id'].values[idx]) + '.tiff'\n            image = skimage.io.MultiImage(img_path)\n            label = self.df['isup_grade'].values[idx]\n            \n        if self.mode == 'test':\n            img_path = os.path.join(Config.test_img_dir, self.df['image_id'].values[idx]) + '.tiff'\n            image = skimage.io.MultiImage(img_path)\n            label = -1\n            \n        if self.resize is not None:\n            image = cv2.resize(image[-1], (self.resize[0], self.resize[1]))\n            image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n            \n        image = np.array(image)\n        \n        if self.augmentations is not None:\n            augmented = self.augmentations(image=image)\n            image = augmented[\"image\"]\n            \n        return image, label\n            ","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Basic EDA time","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Let's have a glance at the Images given, this will also make us sure that my Build_Dataset code is working(I'm always skeptical about my code, specially when it's so long)","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"# The below code will plot down some images for you, given a list of images\ndef plot_images(images):\n\n    n_images = len(images)\n\n    rows = int(np.sqrt(n_images))\n    cols = int(np.sqrt(n_images))\n\n    fig = plt.figure(figsize=(20,10))\n    for i in range(rows*cols):\n        ax = fig.add_subplot(rows, cols, i+1)\n        ax.set_title('ISUP: '+str(images[i][1]))\n        ax.imshow(np.array(images[i][0]))\n        ax.axis('off')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Well if you are anything like me who has no domain experience in biology(I even skipped my biology lectures and exams but that's a different story all together), It's hard to say which images are of benign cells and which images are of cancerous(my best guess is all kinda thick slides are cancerous :p). Even if it wasn't a code competition, hand labelling was never an option for me :-/","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"train_data = Build_Dataset(train_df, resize=(Config.height, Config.width), mode='train')\nimages = [(image, label) for image, label in [train_data[i] for i in range(Config.num_images_to_plot)]] \nplot_images(images)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now let's have a look how is our target class distribution, I just hope that it is not heavily skewed","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"sns.countplot(train_df.isup_grade)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Okay, that's an imbalanced distribution, when we see imbalanced classes, what do we do? \n\nStratified K-fold cross validation, right? Let's create some folds now","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"skf = StratifiedKFold(Config.num_folds, shuffle=True, random_state=Config.SEED)\ntrain_df['fold'] = -1\nfor i, (tr_idx, val_idx) in enumerate(skf.split(train_df, train_df['isup_grade'])):\n    train_df.loc[val_idx, 'fold'] = i\ntrain_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Pre-processing The Data","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Ahh The Preprocessing, commomly hated by all researchers.\n\nLet's for the sake of simple kernel, not dive into the preprocessing and delete 'data_provider' and 'gleason_score' from training data to make our baseline pipeline simpler","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"train_df.drop(columns=['data_provider', 'gleason_score'], inplace=True)\ntrain_df.head()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Let's apply some augmentation to our images, normalize them and convert them to pytorch tensors. All standard stuff\n\nIt is so simple with the help of Albumentation","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"#Defining train and test transforms\ntrain_transforms = albumentations.Compose([\n    albumentations.HorizontalFlip(p=0.5),\n    albumentations.VerticalFlip(p=0.5),\n    albumentations.Normalize(mean=Config.mean, std=Config.std, always_apply=True),\n    albumentations.pytorch.ToTensorV2(),\n])\ntest_transforms = albumentations.Compose([\n    albumentations.Normalize(mean=Config.mean, std=Config.std, always_apply=True),\n    ToTensorV2(),\n])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Building Model","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"***Spoiler Alert*** : Fun Begins","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Let's use the pretrained Efficient Net b-0. Whenever we write our own custom model class, we need to inherit from nn.Module and overwrite __ init__() and forward() method","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"class enetv2(nn.Module):\n    def __init__(self, backbone, out_dim):\n        super(enetv2, self).__init__()\n        self.enet = enet.EfficientNet.from_name(backbone)\n        self.enet.load_state_dict(torch.load(Config.pretrainied_models[backbone]))\n        self.myfc = nn.Linear(self.enet._fc.in_features, out_dim)\n        self.enet._fc = nn.Identity()\n    \n    def extract(self, x):\n        return self.enet(x)\n    \n    def forward(self, x):\n        x = self.extract(x)\n        x = self.myfc(x)\n        return x","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#These are just 2 helpful functions I like using\ndef count_parameters(model):\n    return sum(p.numel() for p in model.parameters() if p.requires_grad)\n\ndef epoch_time(start_time, end_time):\n    elapsed_time = end_time - start_time\n    elapsed_mins = int(elapsed_time / 60)\n    elapsed_secs = int(elapsed_time - (elapsed_mins * 60))\n    return elapsed_mins, elapsed_secs\n\nmodel = enetv2('efficientnet-b0', Config.out_dim).to(Config.device)\nprint(f'The model has {count_parameters(model):,} trainable parameters')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Defining Training and Validation epochs","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"It looks pretty complicated but it is all pretty simple stuff.","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"All you need to do in training is\n\n- put our model into train mode\n- iterate over our dataloader, returning batches of (image, label)\n- place the batch on to our GPU, if we have one\n- clear the gradients calculated from the last batch\n- pass our batch of images, x, through to model to get predictions, y_pred\n- calculate the loss between our predictions and the actual labels\n- calculate and backpropagate the gradients of each parameter\n- update the parameters by taking an optimizer step\n- update our metrics","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"All you need to do in validation is \n\n- put our model into evaluation mode with model.eval()\n- wrap the iterations inside a with torch.no_grad()\n- pass our batch of images, x, through to model to get predictions, y_pred\n- do not zero gradients as we are not calculating any\n- do not calculate gradients as we are not updating parameters\n- do not take an optimizer step as we are not calculating gradients\n- update our metrics","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def train(model, iterator, optimizer, criterion, device):\n    \n    epoch_loss = 0\n    model.train()\n    \n    for (x, y) in iterator:\n        \n        x = x.to(device, dtype=torch.float)\n        y = y.to(device, dtype=torch.long)\n        optimizer.zero_grad()\n        y_pred = model(x)\n        loss = criterion(y_pred, y)\n        loss.backward()\n        optimizer.step()\n        loss_np = loss.detach().cpu().numpy()\n        epoch_loss += loss_np\n        \n    return epoch_loss/len(iterator)\n\ndef evaluate(model, iterator, criterion, device):\n    \n    epoch_loss = 0\n    preds = []\n    preds = np.array(preds)\n    targets = []\n    targets = np.array(targets)\n    model.eval()\n    \n    with torch.no_grad():\n        \n        for (x, y) in iterator:\n        \n            x = x.to(device, dtype=torch.float)\n            y = y.to(device, dtype=torch.long)\n            y_pred = model(x)\n            loss = criterion(y_pred, y)\n            loss_np = loss.detach().cpu().numpy()\n            epoch_loss += loss_np\n            preds = np.append(preds, np.argmax(y_pred.detach().cpu().numpy(), axis = 1))\n            targets = np.append(targets, y.detach().cpu().numpy())\n            \n    return epoch_loss/len(iterator), metrics.cohen_kappa_score(targets, preds, weights='quadratic')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Defining Training Loop","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Let's finally define our Training loop and now is the time to put out train and evaluate function to use","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"I am returning a pandas dataframe object as I intend to visualize the training curves","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"def fit_model(model, model_name, train_iterator, valid_iterator, optimizer, loss_criterion, device, num_epochs, fold):\n    \"\"\" Fits a dataset to model\"\"\"\n    #Setting best validation loss to infinity :p\n    best_valid_loss = float('inf')\n    \n    train_losses = []\n    valid_losses = []\n    valid_metric_scores = []\n    \n    #Let's loop through our data\n    for epoch in range(num_epochs):\n    \n        start_time = time.time()\n    \n        train_loss = train(model, train_iterator, optimizer, loss_criterion, device)\n        valid_loss, valid_metric_score = evaluate(model, valid_iterator, loss_criterion, device)\n        \n        train_losses.append(train_loss)\n        valid_losses.append(valid_loss)\n        valid_metric_scores.append(valid_metric_score)\n\n        #Let's keep updating our model, so that we save only the best one at the end\n        if valid_loss < best_valid_loss:\n            best_valid_loss = valid_loss\n            torch.save(model.state_dict(), f'{model_name}_fold_{fold}.pt')\n    \n        end_time = time.time()\n\n        epoch_mins, epoch_secs = epoch_time(start_time, end_time)\n        \n        #Printing and returning some important statistics\n        print(f'Epoch: {epoch+1:02} | Epoch Time: {epoch_mins}m {epoch_secs}s')\n        print(f'\\tTrain Loss: {train_loss:.3f}')\n        print(f'\\t Val. Loss: {valid_loss:.3f} |  Val. Metric Score: {valid_metric_score:.3f}')\n        \n    return pd.DataFrame({f'{model_name}_fold_{fold}_Training_Loss':train_losses,  \n                        f'{model_name}_fold_{fold}_Validation_Loss':valid_losses, \n                        f'{model_name}_fold_{fold}_Valid_Metric_Score':valid_metric_scores})","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#This will simply plot the training statistics we returned\ndef plot_training_statistics(train_stats, model_name, fold):\n    \n    fig, axes = plt.subplots(2, figsize=(15,15))\n    axes[0].plot(train_stats[f'{model_name}_fold_{fold}_Training_Loss'], label=f'{model_name}_fold_{fold}_Training_Loss')\n    axes[0].plot(train_stats[f'{model_name}_fold_{fold}_Validation_Loss'], label=f'{model_name}_fold_{fold}_Validation_Loss')\n    axes[1].plot(train_stats[f'{model_name}_fold_{fold}_Valid_Metric_Score'], label=f'{model_name}_fold_{fold}_Valid_Metric_Score')\n    \n    axes[0].set_xlabel(\"Number of Epochs\"), axes[0].set_ylabel(\"Loss\")\n    axes[1].set_xlabel(\"Number of Epochs\"), axes[1].set_ylabel(\"Score on Metric\")\n    \n    axes[0].legend(), axes[1].legend()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Training with K-Fold CV","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Finally The moment we have been waiting for has arrived. *Let's Train The Model*","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"This will output us k different models, by running the training look k times, so beware if you some computing constraints. But nevertheless, even I don't have a GPU at home or money and good stable internet connetion to work on cloud(poor me :-(  ). It works fine for me on the kaggle kernel, I hope it's gonna work well for you too.","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"for fold in range(Config.num_folds):\n    print(f\"Fitting on Fold {fold+1}\")\n    #Make Train and Valid DataFrame from fold\n    train_df_fold = train_df[train_df['fold'] != fold]\n    valid_df_fold = train_df[train_df['fold'] == fold]\n    \n    #Build and load Dataset\n    train_data = Build_Dataset(train_df_fold, resize=(Config.height, Config.width), mode='train', augmentations=train_transforms)\n    valid_data = Build_Dataset(valid_df_fold, resize=(Config.height, Config.width), mode='train', augmentations=test_transforms)\n    train_iterator = DataLoader(train_data, shuffle=True, batch_size=Config.batch_size, num_workers=Config.num_workers)\n    valid_iterator = DataLoader(valid_data, batch_size=Config.batch_size, num_workers=Config.num_workers)\n    \n    #Initialize model, loss and optimizer\n    model = enetv2('efficientnet-b0', out_dim=Config.out_dim).to(Config.device)\n    loss_criterion = nn.CrossEntropyLoss().to(Config.device)\n    optimizer=optim.Adam(model.parameters(), lr=0.001, betas=(0.9, 0.999))\n    \n    #Fit the model and visualize the training curves\n    train_stats = fit_model(model, 'efficientnet-b0', train_iterator, valid_iterator, \n              optimizer, loss_criterion, Config.device, Config.num_epochs, fold)\n    plot_training_statistics(train_stats, 'efficientnet-b0', fold)\n    \n    #Just making sure that the output looks neat\n    print('\\n')\n    print('-------------------------------------------------------')\n    print('\\n')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"markdown","source":"# Making submission to leaderboard","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"We are in the Endgame now.\n\nLe Me with Thor's voice in my head: **Bring me Models**","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Okay time to fetch all our models and combine their efforts and submit the predictions to the leaderboard!","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"k_fold_models = [Config.pwd+model for model in os.listdir(Config.pwd) if 'efficientnet-b0' in model]\nk_fold_models","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"#This bit of code is gonna fetch predictions for a single model\ndef get_predictions(model, iterator, device):\n    \n    preds = []\n    preds = np.array(preds)\n    model.eval()\n    \n    with torch.no_grad():\n        \n        for (x, y) in iterator:\n        \n            x = x.to(device, dtype=torch.float)\n            y = y.to(device, dtype=torch.long)\n            y_pred = model(x)\n            preds = np.append(preds, np.argmax(y_pred.detach().cpu().numpy(), axis = 1))\n            \n    return preds","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Yeah, I know where's the testing data, how the hell are we supposed to submit when there's no file containing test data, and all those questions.\n\nYou can just copy my code below and commit the notebook and submit to the competition, but why did organisers do this? \nI guess they wanted to avoid hand labelling and data leakage, and this is really a great attempt at doing that.\n\nBut what's happening? \n\nWhen you commit at that time too the test file won't be available so don't painc, just go to your outputs click on submit for that 3 rowed submission.csv. When you finally click submit with that, they re-run your notebook(but this time on their servers) and guess what, they put the test file in the input data, exactly the way we expect them to, hence your submission returns a score on public leaderboard and is a valid submission.\n\nFor more info on this follow [this thread](https://www.kaggle.com/c/prostate-cancer-grade-assessment/discussion/145219)","execution_count":null},{"metadata":{"trusted":true},"cell_type":"code","source":"test_df = pd.read_csv(Config.data_dir+'test.csv')\nsample = pd.read_csv('../input/prostate-cancer-grade-assessment/sample_submission.csv')\ntest_df.drop(columns=['data_provider'], inplace=True)\n\ndef submit(sample):\n    if os.path.exists(Config.test_img_dir):\n        test_data = Build_Dataset(test_df, resize=(Config.height, Config.width), mode='test', augmentations=test_transforms)\n        test_iterator = DataLoader(test_data, batch_size=Config.batch_size, num_workers=Config.num_workers)\n        \n        #Lets Ensemble all of our models and predict the most frequent(ie Voting ensemble)\n        final_preds = []\n        for fold in range(Config.num_folds):\n            model = enetv2('efficientnet-b0', out_dim=Config.out_dim).to(Config.device)\n            model.load_state_dict(torch.load(k_fold_models[fold], map_location=lambda storage, loc: storage))\n            model.to(Config.device)\n            preds = get_predictions(model, test_iterator, Config.device)\n            final_preds.append(preds)\n            \n        final_preds = np.array(final_preds)\n        final_preds = stats.mode(final_preds)\n        sample['isup_grade'] = final_preds[0][0]\n    return sample\n\nsubmission = submit(sample)\nsubmission['isup_grade'] = submission['isup_grade'].astype(int)\nsubmission.to_csv('submission.csv', index=False)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Acknowledgements","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"Last but not the least, I would like to thank Y.Nakama and Qishen Ha for their public kernels. Without their kernels, this simple kernel wouldn't have been possible.\n\nThe link to their work is\n\nhttps://www.kaggle.com/haqishen/train-efficientnet-b0-w-36-tiles-256-lb0-87\n\nhttps://www.kaggle.com/yasufuminakama/panda-submit-test","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"# Good Bye Fellas, Until next time","execution_count":null},{"metadata":{},"cell_type":"markdown","source":"**If the content in this notebook inspired you, Do Upvote the notebook and show me your support**","execution_count":null}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":4}