{"cells":[{"metadata":{},"cell_type":"markdown","source":"# Histogram Based Classification\nI alluded to this classification approach already in my data preprocessing notebook, and will use some of the functions here. Basically, I just want to try to do classification procedures just based on the value distributions inside the windowed CT images. It already worked in my opinion surprisingly well, but I wanted to extend the idea to classify different hemorrhage labels as well. So we might learn how different or similar certain lesions are. \n\n"},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"import numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\nimport matplotlib.pyplot as plt\nfrom glob import glob\nimport os\nimport gc\nimport seaborn as sns\nfrom tqdm import tqdm_notebook\nimport rsna_hemorrhage_detetction_preprocessing as rsna\n# Sklearn stuff\nfrom sklearn.linear_model import LogisticRegressionCV\nfrom sklearn.ensemble import RandomForestClassifier\nfrom sklearn.metrics import classification_report, balanced_accuracy_score, roc_auc_score\nfrom sklearn.externals.joblib import Parallel, delayed\nfrom scipy.stats import ttest_ind, f_oneway\n\n\n# Metric - see https://www.kaggle.com/c/rsna-intracranial-hemorrhage-detection/discussion/110461#latest-636870\nfrom sklearn.metrics import log_loss\nsample_weights = [2/7, 1/7, 1/7, 1/7, 1/7, 1/7]\n\nthreshold = 5\nthreshold2 = 0.001\ndata_samples = 2500\n\nsns.set_context('notebook')","execution_count":null,"outputs":[]},{"metadata":{"_uuid":"d629ff2d2480ee46fbb7e2d37f6b5fab8052498a","_cell_guid":"79c7e3d0-c299-4dcb-8224-4455121ee9b0","trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"PATH = '../input/rsna-intracranial-hemorrhage-detection/' # Set up the path.\n# Load the stage 1 file\ntrain_csv = pd.read_csv(f'{PATH}stage_1_train.csv')\n# Create a path to the train image location:\nimage_path = os.path.join(PATH, 'stage_1_train_images') + os.sep \nprint(image_path)","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# Check out this kernel: https://www.kaggle.com/currypurin/simple-eda \n# This is a really nice preprocessing of ID and labels :)\ntrain_csv['Image ID'] = train_csv['ID'].apply(lambda x: x.split('_')[1]) \ntrain_csv['Sub-type'] = train_csv['ID'].apply(lambda x: x.split('_')[2]) \ntrain_csv = pd.pivot_table(train_csv, index='Image ID', columns='Sub-type')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Investigation of Label Occurences\nMany notebooks did this before already, but we need to learn something about the label distribution anyways, so here is a little analysis of it."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"print(train_csv.sum(0))\nprint(f'Total number:{train_csv.shape[0]}')","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"fix, axes = plt.subplots(1,2, figsize=(12.5,5))\nplot_data = train_csv.sum(0).reset_index()\na = sns.barplot(data=plot_data, x='Sub-type', y=0, palette='viridis', ax=axes[0])\na.tick_params(axis='x', rotation=45)\na.set_ylabel('Counts')\nplot_data.loc[:, 0] = plot_data.loc[:, 0].values / train_csv.shape[0]\na = sns.barplot(data=plot_data, x='Sub-type', y=0, palette='viridis', ax=axes[1])\na.tick_params(axis='x', rotation=45)\na.set_ylabel('Percentage');","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"So at the first look, we see, that around 14% of the training data (CT slices) contain lesion information. The other lesions are roughly equally distributed, except for epidural, which are present in less than 2% of the data. \nSee the image from the description challenge description:\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F603584%2F56162e47358efd77010336a373beb0d2%2Fsubtypes-of-hemorrhage.png?generation=1568657910458946&alt=media)\n\nBased on this descriptio for intraparenchymal, intraventricular, and subarachnoid hemorrhages, we can assume, that there might be multiple hemorrhages in a single slice or patient. \nIn the next step we will only look slices, where a label is present (`any==1`). In the following graphic, we look at the co-occurences of hemorrhages. On the left there is the count, on the right there is the number of co-occurences divided by all "},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# Percentage of multiple labels: \ntrain_csv_dropped = train_csv.copy()\ntrain_csv_dropped.columns = train_csv_dropped.columns.droplevel(0)\nno_labels = train_csv_dropped.query('any == 1').shape[0]\nl_names = list(train_csv_dropped.columns[1:])\ncollect_data = []\n\nfor nn, lesion in enumerate(l_names) :\n    select_vec = np.ones(len(l_names) + 1).astype(bool)\n    select_vec[nn + 1 ] = 0\n    select_vec[0] = 0\n\n    plot_data = train_csv_dropped.query(f'{lesion}==1').sum(0).reset_index()\n    collect_data.append(plot_data[0].values)\n    \n# co-occurence matrix\ncollect_data = np.array(collect_data)\nfig, ax = plt.subplots(1,3, figsize=(15,15))\nsns.heatmap(collect_data[:, 1:], annot=True, fmt='', xticklabels=l_names, yticklabels=l_names, ax=ax[0], square=True, cbar=False, cmap='viridis')\nax[0].set_title(\"Count\")\n\nsns.heatmap(collect_data[:, 1:]/np.diag(collect_data[:,1:]).reshape(-1, 1), annot=True, fmt='.2', xticklabels=l_names, ax=ax[1], square=True, cbar=False, cmap='viridis')\nax[1].set_title(\"Co-occurences per label\")\n\nsns.heatmap(collect_data[:, 1:]/no_labels, annot=True, fmt='.2', xticklabels=l_names, ax=ax[2], square=True, cbar=False, cmap='viridis')\nax[2].set_title('Percentage of all positive labels')\n\nplt.tight_layout()","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"As we have seen in the description there are multi-label classes. Especially intraparenchymal and intraventricular hemorrhages often co-occur! "},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"fig, ax = plt.subplots()\ntemp_hist = plt.hist(train_csv_dropped.query(\"any==1\")[l_names].sum(1), bins=(np.arange(10))/2 + 1)\n[print(f' Overlapping labels: {i + 1}, count: {ii:4.2f}\\n') for i, ii in enumerate(temp_hist[0][::2])];","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We furthermore see, that in many cases there are multiple labels present. In about a quarter of the positive labels there are two labels, and in very very rare cases, there are 5. We will need to figure out, whether it is worth it, to also consider multilabel classes, rather than single label classes in the cross-validation procedure. "},{"metadata":{},"cell_type":"markdown","source":"# Voxel extraction - Histograms a features\nI have some functions from the previous kernel, and will use them here. Namely transformation to Hounfield units and windowing. "},{"metadata":{"trusted":true},"cell_type":"code","source":"plt.figure(figsize=(15,5))\nimg, dicom = rsna.image_to_hu(image_path, train_csv.index[5])\nplt.subplot(221)\nplt.title('Hounsfield Transformed Image')\nplt.imshow(img, cmap='bone')\nplt.subplot(223)\nplt.hist(img.ravel());\nplt.subplot(222)\nplt.title('Windowed Image')\nimg = rsna.image_windowed(img, 50, 130, False)\nplt.imshow(img, cmap='bone')\nplt.subplot(224)\nplt.hist(img.ravel());","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"min_img = np.min(img)\nmax_img = np.max(img)\nno_bins = (max_img - min_img) / 5\n# Create evenly spaced bins across histograms:\nhistogram_bins = np.linspace(min_img, max_img, np.int(no_bins + 1))\nsino_bins = np.linspace(-4000, 12000, 25)\n","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def get_histogram(image_path, img_id, train_csv, bin_spacing, sino_bins):\n    from skimage.transform import radon\n\n    img, dicom = rsna.image_to_hu(image_path, img_id)\n    img = rsna.image_resample(img, dicom)\n    img = rsna.image_windowed(img, 50, 130, False)\n    # Discard values outside of the window\n    histogram_vals = np.histogram(img.ravel(), histogram_bins)[0][1:-1]\n    \n    # Next to simple histograms, we are also applying the radon transform - i.e. creating the sinogram. \n    theta = np.linspace(0., 180., max(img.shape), endpoint=False)\n    sinogram = radon(img, theta=theta, circle=True)\n\n    sinogram_hist = np.histogram(sinogram.ravel(), bins=sino_bins)[0]\n    \n    if train_csv is not None:\n        labels = train_csv.loc[img_id].values\n    else:\n        labels = np.zeros(6)\n\n    return histogram_vals, sinogram_hist, labels","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Get IDs based on their labels:\nimageID_dict = {}\nfor les in l_names:\n    imageID_dict[les] = train_csv_dropped.query(f'{les}==1').index.values\n\nimageID_dict['nolabel'] = train_csv_dropped.query(f'any==0').index.values","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# 2000 samples of each hemorrhage + 10000 empty values\n# Set seed\nfrom sklearn.externals.joblib import Parallel, delayed\nnp.random.seed(2019)\n# If you run this, get a coffee or something ;) \nnoSamples = np.array([data_samples] * len(l_names) + [data_samples * 5])\n\n# Create empty X, and y arrays for training:\ndata = []\n\nc = 0\nfor noS, dict_key in zip(noSamples, imageID_dict.keys()):\n    \n    if noS < len(imageID_dict[dict_key]):\n        stepIDs = np.random.choice(imageID_dict[dict_key], noS, replace=False)\n    else:\n        stepIDs = np.random.choice(imageID_dict[dict_key], noS, replace=True)\n    \n    out = Parallel(n_jobs=-1, prefer=\"threads\")(delayed(get_histogram)(image_path, tmpid,\n                                               train_csv_dropped, histogram_bins, \n                                               sino_bins) for tmpid in tqdm_notebook(stepIDs))\n    data.extend(out)\n    gc.collect\n\nS = np.hstack([n + np.zeros(k) for n, k in enumerate(noSamples)])","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"X = np.zeros((len(data), data[0][0].shape[0] + data[0][1].shape[0]))\nY = np.zeros((len(data), data[0][2].shape[0]))\n\nfor n, (x1, x2, y1) in tqdm_notebook(enumerate(data)):\n    x1 = (x1 - np.mean(x1)) / np.std(x1)\n    x2 = (x2 - np.mean(x2)) / np.std(x2)\n    X[n, : x1.shape[0]] = x1\n    X[n, x1.shape[0] : ] = x2\n    Y[n, :] = y1\n\nX = np.nan_to_num(X, 0)","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# Seperation of Hemorrhages\nIs there something in the values, which we can use to separate different hemorrhages. "},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"HU_hist = np.histogram(np.arange(100), bins=histogram_bins)[1][1:-2]\nSIN_hist = np.histogram(np.arange(100), bins=sino_bins)[1][:-1]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"# Check if there's a difference between no lesion and lesions\nts = []\nfor ii in range(X.shape[1]):\n    ts.append(ttest_ind(X[S != 5, ii], X[S == 5, ii])[0])\n\nts = np.array(ts)\n\nhuTs = ts[: HU_hist.shape[0]]\nplot_dim = np.ceil(np.sqrt(huTs.shape))\n\nplt.figure(figsize=(15,15))\nfor nn, hidx in enumerate(np.arange(huTs.shape[0])[np.abs(huTs) > threshold]):\n    plt.subplot(plot_dim, plot_dim, nn + 1)\n    plt.hist(X[S!=5, hidx], alpha=0.5)\n    plt.hist(X[S==5, hidx], alpha=0.5)\n    plt.title(f'HU = {HU_hist[hidx]} \\n t={ts[hidx]:4.2f}')\n\nplt.tight_layout()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true,"_kg_hide-input":true},"cell_type":"code","source":"offset = HU_hist.shape[0]\nsinTs = ts[HU_hist.shape[0] :]\n\nplot_dim = np.ceil(np.sqrt(sinTs.shape))\n\nplt.figure(figsize=(15,15))\nfor nn, hidx in enumerate(np.arange(sinTs.shape[0])[np.abs(sinTs) > threshold]):\n    plt.subplot(plot_dim, plot_dim, nn + 1)\n    plt.hist(X[S!=5, hidx  + offset ], alpha=0.5)\n    plt.hist(X[S==5, hidx  + offset], alpha=0.5)\n    plt.title(f'SIN-Value = {np.round(SIN_hist[hidx])} \\n t={ts[hidx + offset]:4.2f}')\n\nplt.tight_layout()\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We find a series of histograms, which have values that are (significantly :P) differentiating between the presence (or absence) of a hemorrhage. \n\nSome of the significant values we find again in the Wikipedia article on Hounsfield units:\n\n|Substance\t|\t| HU |\n|-------------------\t|---------------------\t|------------------\t|\n| Subdural hematoma \t| First hours         \t| +75 to +100   \t|\n|   \t|   After 3 days       \t|        +65 to +85            \t|\n| \t|  After 10-14 days     \t|         +35 to +40           \t|\n| Other blood       \t| Unclotted           \t| +13 to +50 \t|\n|      \t|  Clotted      \t|      +50 to +75            \t| \nhttps://en.wikipedia.org/wiki/Hounsfield_scale\n\n"},{"metadata":{},"cell_type":"markdown","source":"## Can we differentiate between different Hemorrhage Subtypes?\nUsing the histogram bins, that seperated significantly between the absence and presence of a hemorrhage. I will now investigate, whether these can also be used to differentiate between the different subtypes. "},{"metadata":{"trusted":true},"cell_type":"code","source":"binsOfInterest = np.where(np.abs(ts) > threshold)[0]\nlabels = np.array(['HU_' + str(i) for i in HU_hist] + ['SIN_' + str(np.round(i)) for i in SIN_hist])\nlabels = labels[binsOfInterest]","execution_count":null,"outputs":[]},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"# First do an ANOVA to figure out which bins have significant differences between regions:\np_aov = np.zeros(binsOfInterest.shape)\nfor n, bOi in enumerate(binsOfInterest):\n    aov = f_oneway(X[S==0, bOi], X[S==1, bOi], X[S==2, bOi], X[S==3, bOi], X[S==4, bOi])\n    p_aov[n] = aov[1]","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"print(binsOfInterest[p_aov < threshold2])","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"We see that there is a smaller number now of bins which can different between the different hemorrhage type. In a further post hoc analysis, I will look at which bins can differentiate between which types of hemorrhages."},{"metadata":{"_kg_hide-input":true,"trusted":true},"cell_type":"code","source":"selected_binsOfInterest = binsOfInterest[p_aov < threshold2]\nselLabels = labels[p_aov < threshold2]\nsubplot_dims = np.ceil(np.sqrt(selected_binsOfInterest.shape[0]))\n\ndiffArrays = []\n\nplt.figure(figsize=(15,15))\nfor n, bOi in enumerate(selected_binsOfInterest):\n    plt.subplot(subplot_dims, subplot_dims, n + 1)\n    diffMap = np.zeros((5,5))\n    for rr in range(5):\n        for cc in range(5):\n            diffMap[rr, cc] = ttest_ind(X[S==rr, bOi], X[S==cc, bOi])[0]\n    \n    if n in [0, 4, 8, 12]:\n        yl = l_names\n    else:\n        yl = []\n    \n    if n >= 12:\n        xl = l_names\n    else:\n        xl = []\n        \n    sns.heatmap(np.abs(diffMap), annot=True, fmt='.2', square=True, cbar=False, xticklabels=xl, yticklabels=yl)\n    plt.title(selLabels[n])\n    diffArrays.append(diffMap)\n\nplt.suptitle(\"Absolut size of difference\\n abs(t)\")","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"## TODO: Interpretation\nWe see that certain bins are working quite well to seperate different hemorrhage subtypes. Other's don't. On a first glance epidural and subdural hemorrhages have profiles which are quite different from the others. One reason for this could be, that it is quite posible that there are classes with multiple labels in the analysis. I.e. a sample with the labels \"intraventricular\" and \"subarachnoid\" could be in both groups, therefore making the statistical analysis less valid. \n\n## Conclusion\nI think classifying \"any\" before the different subtypes of hemorrhages could be a valid approach. Later I'll try to implement this approach"},{"metadata":{},"cell_type":"markdown","source":"# Classification Approach"},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.model_selection import train_test_split, StratifiedKFold, StratifiedShuffleSplit\nXtrain, Xval, Ytrain, Yval, Strain, Sval = train_test_split(X, Y, S, stratify=S)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"from sklearn.pipeline import make_pipeline\nfrom sklearn.preprocessing import StandardScaler\nSKF = StratifiedKFold(5, random_state=24)\nCLF = make_pipeline(StandardScaler(), LogisticRegressionCV(cv=3))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Create different models:\nCLF_dict = {}\n\nc = 0\n\nfor trIdx, teIdx in SKF.split(Xtrain, Strain):\n    CLF_dict[c] = {}\n    \n    xTr = Xtrain[trIdx]; yTr = Ytrain[trIdx]\n    xTe = Xtrain[teIdx]; yTe = Ytrain[teIdx]\n    yPred = np.zeros(yTe.shape)\n    \n    CLF.fit(xTr[:, binsOfInterest], yTr[:, 0])\n    CLF_dict[c]['any'] = CLF\n    CLF_dict[c]['teIdx'] = teIdx\n    \n    yPred[:, 0] = CLF.predict_proba(xTe[:, binsOfInterest])[:, 1]\n    \n    for n, les in enumerate(l_names):\n        CLF.fit(xTr[:, selected_binsOfInterest], yTr[:, n+1])\n        CLF_dict[c][les] = CLF\n        yPred[:, n+1] = CLF.predict_proba(xTe[:, selected_binsOfInterest])[:, 1]\n        \n    CLF_dict[c]['Prediction'] = yPred\n    CLF_dict[c]['True'] = yTe\n    c += 1","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Classification accuracies:\nsomeNorm = 0\nbaac = { i : [] for i in range(6)}\nLL = []\n\nfor n, c in enumerate(CLF_dict.keys()):\n    tempPred = CLF_dict[c]['Prediction'].copy()\n    \n    if someNorm: \n        tempPred[tempPred[:, 0] < 0.5, 1:] = 0\n\n    for ii in range(6):\n            baac[ii].append(roc_auc_score(CLF_dict[c]['True'][:, ii], tempPred[:, ii]))\n    \n    ll= log_loss(CLF_dict[c][\"True\"].ravel(), tempPred.ravel(), sample_weight=sample_weights * CLF_dict[c][\"Prediction\"].shape[0])\n    LL.append(ll)\n\nfor n, les in enumerate(['any'] + l_names):\n    print(f'{les} - AUC: {np.mean(baac[n])} +/- {np.std(baac[n])}')\nprint(f'LL: {np.mean(LL)} +/- {np.std(LL)}')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"# TO-DO: Feature Importance and Missclassification Analysis"},{"metadata":{},"cell_type":"markdown","source":"# Test on hold out data"},{"metadata":{"trusted":true},"cell_type":"code","source":"baac = []\n\nyPred = np.zeros(Yval.shape)\n\nCLF.fit(Xtrain[:, binsOfInterest], Ytrain[:, 0])\n\nyPred[:, 0] = CLF.predict_proba(Xval[:, binsOfInterest])[:, 1]\n\nfor n, les in enumerate(l_names):\n    CLF.fit(Xtrain[:, selected_binsOfInterest], Ytrain[:, n+1])\n    yPred[:, n+1] = CLF.predict_proba(Xval[:, selected_binsOfInterest])[:, 1]\n\n\ntempPred = yPred.copy()\n\n\nif someNorm: \n    tempPred[tempPred[:, 0] < 0.5, 1:] = 0\n\nfor ii in range(6):\n        baac.append(roc_auc_score(Yval[:, ii], tempPred[:, ii]))\nLL= log_loss(Yval.ravel(), tempPred.ravel(), sample_weight=sample_weights * Yval.shape[0])\n\n\nfor n, les in enumerate(['any'] + l_names):\n    print(f'{les} - AUC: {np.mean(baac[n])} +/- {np.std(baac[n])}')\nprint(f'LL: {np.mean(LL)} +/- {np.std(LL)}')","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Pretty much the same performance, let's look at the models' performance on the leaderboard. Also to have some feeling for the actual meaning of these values. "},{"metadata":{},"cell_type":"markdown","source":"# Prediction on the Test-Set"},{"metadata":{},"cell_type":"markdown","source":"\nFor simplicity I will use 'any' as the maximum prediction value.\n"},{"metadata":{"trusted":true},"cell_type":"code","source":"sub_csv = pd.read_csv(f'{PATH}stage_1_sample_submission.csv')\n\nsub_csv['Image ID'] = sub_csv['ID'].apply(lambda x: x.split('_')[1]) \nsub_csv['Sub-type'] = sub_csv['ID'].apply(lambda x: x.split('_')[2]) \nsub_csv = pd.pivot_table(sub_csv, index='Image ID', columns='Sub-type')","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"sub_csv_dropped = sub_csv.copy()\nsub_csv_dropped.columns = sub_csv_dropped.columns.droplevel(0)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test_images = sub_csv_dropped.index.values\ntest_image_path =  os.path.join(PATH, 'stage_1_test_images') + os.sep ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"CLF_dictTest = {}\n\nCLF = RandomForestClassifier(n_estimators=10)\nCLF.fit(X[:, binsOfInterest], Y[:,0])\nCLF_dictTest['any'] = CLF\nfor n, l in enumerate(l_names):\n    CLF = RandomForestClassifier(n_estimators=10)\n    CLF.fit(X[:, selected_binsOfInterest], Y[:, n + 1])\n    CLF_dictTest[l] = CLF","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"import gc\ndel X\ndel Y\ndel Xtrain\ndel Xval\ngc.collect()","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"def calculate_test_sample(timg, binsOfInterest, histogram_bins, sino_bins, selected_binsOfInterest, CLF, test_image_path):\n    \n    pred = []\n    \n    tmpX = get_histogram(test_image_path, timg, None, histogram_bins, sino_bins)\n    tmpX = np.hstack([(tmpX[0] - np.mean(tmpX[0])) / np.std(tmpX[0]), \n                      (tmpX[1] - np.mean(tmpX[1])) / np.std(tmpX[1])])\n    tmpX = np.nan_to_num(tmpX, 0)\n    pred.append(CLF_dictTest['any'].predict_proba(tmpX[np.newaxis, binsOfInterest])[0, 1])\n    for les in  l_names:\n        pred.append(CLF_dictTest[les].predict_proba(tmpX[np.newaxis, selected_binsOfInterest])[0, 1])\n\n    return timg, np.hstack(pred)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"test_out = Parallel(n_jobs=-1, prefer=\"threads\")(delayed(calculate_test_sample)(timg, binsOfInterest, \n                                                                                histogram_bins, sino_bins, \n                                                                                selected_binsOfInterest, \n                                                                                CLF, test_image_path) \n                                                 for timg in tqdm_notebook(test_images))","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"for csv_id, preds in test_out:\n    sub_csv_dropped.loc[csv_id, :] = preds","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Inspired by https://www.kaggle.com/erikgaasedelen/pandas-tricks-with-averaged-baseline\n# Just far more hacky\nout = sub_csv_dropped.copy()\nout = out.reset_index()\n\nvalue_vars = out.columns[1:]\nout= pd.melt(out, id_vars=['Image ID'], value_vars=value_vars, value_name='Label')\nout['ID'] = out['Image ID'].str.cat(out['Sub-type'], sep='_')\nout.drop(columns=['Image ID', 'Sub-type'], inplace=True)\nout['ID'] = 'ID_' + out['ID']\nout = out[['ID', 'Label']] ","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"out.to_csv('submission_hist_sino.csv', index=False)","execution_count":null,"outputs":[]},{"metadata":{"trusted":true},"cell_type":"code","source":"# Quick Test if the csv contains all the information.\nsub_csv = pd.read_csv(f'{PATH}stage_1_sample_submission.csv')\nnp.sum(np.sort(sub_csv['ID']) == np.sort(out['ID'])) == out.shape[0]","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}