{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.6.6","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"gpu","dataSources":[{"sourceType":"competition","sourceId":13451,"datasetId":654585,"databundleVersionId":1188070},{"sourceType":"datasetVersion","sourceId":803887,"datasetId":327856,"databundleVersionId":825952}],"dockerImageVersionId":29662,"isInternetEnabled":true,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"## MY PLAN\n\n1. Using subset of the full data but with bit large image size 256x256\n2. Trying efficientnet (reason described below)\n3. Replacing batch normalization with group normalization for small batch size\n4. using radam instead of sgd or adam\n5. little bit of preprocessing","metadata":{}},{"cell_type":"markdown","source":"## UPDATE\n\n**in version 15**\n- reducing image size down to 224x224\n- subsample size 400000\n- epochs = 5\n- increasing batch size from 32 to 64\n- total_steps = 2000\n","metadata":{}},{"cell_type":"markdown","source":"## Recommendations \n* please don't forget to recommend any idea in the comment box to improve this kernel,your aid is highly highly appreciated,thanks in advance","metadata":{}},{"cell_type":"markdown","source":"**I hope this kernel helpful and some UPVOTES would be very much appreciated****","metadata":{}},{"cell_type":"markdown","source":"## why RSNA Intracranial Hemorrhage Detection is so important????","metadata":{}},{"cell_type":"markdown","source":"*An intracranial hemorrhage (ICH) is a condition in which a blood vessel erupts inside the brain, causing internal bleeding. If not treated correctly and immediately, a brain hemorrhage can be deadly. The type of hemorrhage is usually diagnosed using a CT or MRI scan. Some hemorrhages are also accompanied by cerebral edema – an excess accumulation of fluid in the intracellular or extracellular spaces of the brain.Edema is exceedingly difficult to identify, appearing as a subtle darker area surrounding the hemorrhage; it sometimes requires analysis of multiple sequential scans. Implementation and execution of a successful segmentation model in these situations requires expertise in all computer vision methods – both classical and deep learning based. RSIP Vision leverages both deep learning and classical computer vision techniques to provide a fully automated solution to this segmentation problem.*\nto know more,watch [this video](https://www.youtube.com/watch?v=7RqjrCSR8TE) ","metadata":{}},{"cell_type":"markdown","source":" Typically Intracranial Hemorrhage Detection is segmentation task like the diagram describing below \n\n","metadata":{}},{"cell_type":"markdown","source":"*source : https://www.rsipvision.com/intracranial-hemorrhage-and-edema-segmentation/*","metadata":{}},{"cell_type":"markdown","source":"## Types of Intracranial","metadata":{}},{"cell_type":"markdown","source":" ","metadata":{}},{"cell_type":"markdown","source":"**Necessary imports**","metadata":{}},{"cell_type":"code","source":"# This Python 3 environment comes with many helpful analytics libraries installed\n# It is defined by the kaggle/python docker image: https://github.com/kaggle/docker-python\n# For example, here's several helpful packages to load in \n\nimport numpy as np # linear algebra\nimport pandas as pd # data processing, CSV file I/O (e.g. pd.read_csv)\n\n# Input data files are available in the \"../input/\" directory.\n# For example, running this (by clicking run or pressing Shift+Enter) will list all files under the input directory\n\nimport os\n\n# Any results you write to the current directory are saved as output.\n\n\nimport seaborn as sns\nfrom sklearn.utils import shuffle\nfrom os.path import isfile, join\nimport keras\n\n# Standard dependencies\nimport cv2\nimport time\nimport scipy as sp\nimport numpy as np\nimport pandas as pd\nfrom tqdm import tqdm\nfrom PIL import Image\nfrom functools import partial\nimport matplotlib.pyplot as plt\n\n# Machine Learning\nimport tensorflow as tf\nimport keras\nfrom keras import initializers\nfrom keras import regularizers\nfrom keras import constraints\nfrom keras import backend as K\nfrom keras.activations import elu\nfrom keras.optimizers import Adam\nfrom keras.models import Sequential\nfrom keras.engine import Layer, InputSpec\nfrom keras.utils.generic_utils import get_custom_objects\nfrom keras.callbacks import Callback, EarlyStopping, ReduceLROnPlateau\nfrom keras.layers import Dense, Conv2D, Flatten, GlobalAveragePooling2D, Dropout\nfrom sklearn.metrics import cohen_kappa_score\nimport pydicom\n\nimport json\n\nimport cv2\nimport numpy as np\nimport pandas as pd\nimport matplotlib.pyplot as plt\nimport pydicom\nfrom keras import layers\nfrom keras.callbacks import Callback, ModelCheckpoint\nfrom keras.models import Sequential\nfrom keras.optimizers import Adam\nfrom tqdm import tqdm\n\n \nfrom keras import backend as K\nimport tensorflow as tf","metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true,"execution":{"iopub.status.busy":"2025-12-15T11:52:23.214328Z","iopub.execute_input":"2025-12-15T11:52:23.214621Z","iopub.status.idle":"2025-12-15T11:52:26.189388Z","shell.execute_reply.started":"2025-12-15T11:52:23.214581Z","shell.execute_reply":"2025-12-15T11:52:26.188609Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"!ls ../input\n","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-15T11:52:26.19104Z","iopub.execute_input":"2025-12-15T11:52:26.191238Z","iopub.status.idle":"2025-12-15T11:52:27.155459Z","shell.execute_reply.started":"2025-12-15T11:52:26.191204Z","shell.execute_reply":"2025-12-15T11:52:27.154785Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"os.listdir('/kaggle/input/rsna-intracranial-hemorrhage-detection/rsna-intracranial-hemorrhage-detection')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-15T11:52:27.158032Z","iopub.execute_input":"2025-12-15T11:52:27.1583Z","iopub.status.idle":"2025-12-15T11:52:27.178303Z","shell.execute_reply.started":"2025-12-15T11:52:27.15826Z","shell.execute_reply":"2025-12-15T11:52:27.177656Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"BASE_PATH = '/kaggle/input/rsna-intracranial-hemorrhage-detection/rsna-intracranial-hemorrhage-detection/'\nTRAIN_DIR = 'stage_2_train/'\nTEST_DIR = 'stage_2_test/'\ntrain_df = pd.read_csv(BASE_PATH + 'stage_2_train.csv')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-15T11:52:27.179601Z","iopub.execute_input":"2025-12-15T11:52:27.179912Z","iopub.status.idle":"2025-12-15T11:52:30.896659Z","shell.execute_reply.started":"2025-12-15T11:52:27.179839Z","shell.execute_reply":"2025-12-15T11:52:30.896083Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"train_df","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-15T11:52:30.899379Z","iopub.execute_input":"2025-12-15T11:52:30.89962Z","iopub.status.idle":"2025-12-15T11:52:30.918525Z","shell.execute_reply.started":"2025-12-15T11:52:30.899574Z","shell.execute_reply":"2025-12-15T11:52:30.917923Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"train_df.ID == 'ID_6431af929_intraparenchymal'","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-15T11:52:30.921363Z","iopub.execute_input":"2025-12-15T11:52:30.921699Z","iopub.status.idle":"2025-12-15T11:52:31.171816Z","shell.execute_reply.started":"2025-12-15T11:52:30.921648Z","shell.execute_reply":"2025-12-15T11:52:31.171236Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"sns.countplot(train_df.Label)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-15T11:52:31.173143Z","iopub.execute_input":"2025-12-15T11:52:31.173393Z","iopub.status.idle":"2025-12-15T11:52:31.632414Z","shell.execute_reply.started":"2025-12-15T11:52:31.173348Z","shell.execute_reply":"2025-12-15T11:52:31.631696Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"train_df.Label.value_counts()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-15T11:52:31.633436Z","iopub.execute_input":"2025-12-15T11:52:31.633625Z","iopub.status.idle":"2025-12-15T11:52:31.685288Z","shell.execute_reply.started":"2025-12-15T11:52:31.633592Z","shell.execute_reply":"2025-12-15T11:52:31.684505Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**As from above 2 cells we can see huge class imbalance problem in this competition**","metadata":{}},{"cell_type":"code","source":"sub_df = pd.read_csv(BASE_PATH + 'stage_2_sample_submission.csv')\n\ntrain_df['filename'] = train_df['ID'].apply(lambda st: \"ID_\" + st.split('_')[1] + \".png\")\ntrain_df['type'] = train_df['ID'].apply(lambda st: st.split('_')[2])\nsub_df['filename'] = sub_df['ID'].apply(lambda st: \"ID_\" + st.split('_')[1] + \".png\")\nsub_df['type'] = sub_df['ID'].apply(lambda st: st.split('_')[2])\n\nprint(train_df.shape)\ntrain_df.head()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-15T11:52:31.686338Z","iopub.execute_input":"2025-12-15T11:52:31.686548Z","iopub.status.idle":"2025-12-15T11:52:37.727816Z","shell.execute_reply.started":"2025-12-15T11:52:31.686509Z","shell.execute_reply":"2025-12-15T11:52:37.727162Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**The original image size from the [EfficientNet paper](https://arxiv.org/pdf/1905.11946.pdf) for EfficientNetB3 is 300x300x3. We are however not bound by this and can use a smaller size if we want. The original image sizes used for every version of EfficientNet are**\n\n* EfficientNetB0 - (224, 224, 3)\n* EfficientNetB1 - (240, 240, 3)\n* EfficientNetB2 - (260, 260, 3)\n* EfficientNetB3 - (300, 300, 3)\n* EfficientNetB4 - (380, 380, 3)\n* EfficientNetB5 - (456, 456, 3)\n* EfficientNetB6 - (528, 528, 3)\n* EfficientNetB7 - (600, 600, 3)","metadata":{}},{"cell_type":"markdown","source":"**Image: an overview of model architectures and their performance on [ImageNet](http://www.image-net.org/). We can see that EfficientNet achieves state-of-the-art and uses a lot less parameters than most modern CNN architectures.**","metadata":{}},{"cell_type":"markdown","source":"\n\n\n","metadata":{}},{"cell_type":"code","source":"test_df = pd.DataFrame(sub_df.filename.unique(), columns=['filename'])\nprint(test_df.shape)\ntest_df.head()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-15T11:52:37.729385Z","iopub.execute_input":"2025-12-15T11:52:37.729802Z","iopub.status.idle":"2025-12-15T11:52:37.799447Z","shell.execute_reply.started":"2025-12-15T11:52:37.729641Z","shell.execute_reply":"2025-12-15T11:52:37.798827Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"train_df.shape","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-15T11:52:37.800599Z","iopub.execute_input":"2025-12-15T11:52:37.800783Z","iopub.status.idle":"2025-12-15T11:52:37.80509Z","shell.execute_reply.started":"2025-12-15T11:52:37.80075Z","shell.execute_reply":"2025-12-15T11:52:37.804403Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"np.random.seed(2019)\nsample_files = np.random.choice(os.listdir(BASE_PATH + TRAIN_DIR), 400000)\nsample_df = train_df[train_df.filename.apply(lambda x: x.replace('.png', '.dcm')).isin(sample_files)]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-15T11:52:37.806195Z","iopub.execute_input":"2025-12-15T11:52:37.80643Z","iopub.status.idle":"2025-12-15T11:52:53.271873Z","shell.execute_reply.started":"2025-12-15T11:52:37.806386Z","shell.execute_reply":"2025-12-15T11:52:53.271245Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"pivot_df = sample_df[['Label', 'filename', 'type']].drop_duplicates().pivot(\n index='filename', columns='type', values='Label').reset_index()\nprint(pivot_df.shape)\npivot_df","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-15T11:52:53.273321Z","iopub.execute_input":"2025-12-15T11:52:53.273613Z","iopub.status.idle":"2025-12-15T11:52:55.837425Z","shell.execute_reply.started":"2025-12-15T11:52:53.273556Z","shell.execute_reply":"2025-12-15T11:52:55.836652Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## windowing","metadata":{}},{"cell_type":"markdown","source":"**Source: https://www.kaggle.com/omission/eda-view-dicom-images-with-correct-windowing**","metadata":{}},{"cell_type":"code","source":"def window_image(img, window_center,window_width, intercept, slope, rescale=True):\n\n img = (img*slope +intercept)\n img_min = window_center - window_width//2\n img_max = window_center + window_width//2\n img[imgimg_max] = img_max\n \n if rescale:\n # Extra rescaling to 0-1, not in the original notebook\n img = (img - img_min) / (img_max - img_min)\n \n return img\n \ndef get_first_of_dicom_field_as_int(x):\n #get x[0] as in int is x is a 'pydicom.multival.MultiValue', otherwise get int(x)\n if type(x) == pydicom.multival.MultiValue:\n return int(x[0])\n else:\n return int(x)\n\ndef get_windowing(data):\n dicom_fields = [data[('0028','1050')].value, #window center\n data[('0028','1051')].value, #window width\n data[('0028','1052')].value, #intercept\n data[('0028','1053')].value] #slope\n return [get_first_of_dicom_field_as_int(x) for x in dicom_fields]","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-15T11:52:55.83891Z","iopub.execute_input":"2025-12-15T11:52:55.839208Z","iopub.status.idle":"2025-12-15T11:52:55.84695Z","shell.execute_reply.started":"2025-12-15T11:52:55.839155Z","shell.execute_reply":"2025-12-15T11:52:55.84619Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**Resize 256x256**","metadata":{}},{"cell_type":"code","source":"def save_and_resize(filenames, load_dir): \n save_dir = '/kaggle/tmp/'\n if not os.path.exists(save_dir):\n os.makedirs(save_dir)\n\n for filename in tqdm(filenames):\n try:\n path = load_dir + filename\n new_path = save_dir + filename.replace('.dcm', '.png')\n dcm = pydicom.dcmread(path)\n window_center , window_width, intercept, slope = get_windowing(dcm)\n img = dcm.pixel_array\n img = window_image(img, window_center, window_width, intercept, slope)\n\n resized = cv2.resize(img, (224, 224))\n res = cv2.imwrite(new_path, resized)\n \n except ValueError:\n continue\n ","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-15T11:52:55.848143Z","iopub.execute_input":"2025-12-15T11:52:55.848382Z","iopub.status.idle":"2025-12-15T11:52:55.860579Z","shell.execute_reply.started":"2025-12-15T11:52:55.848316Z","shell.execute_reply":"2025-12-15T11:52:55.860101Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"save_and_resize(filenames=sample_files, load_dir=BASE_PATH + TRAIN_DIR)\nsave_and_resize(filenames=os.listdir(BASE_PATH + TEST_DIR), load_dir=BASE_PATH + TEST_DIR)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-15T11:52:55.862004Z","iopub.execute_input":"2025-12-15T11:52:55.862392Z","iopub.status.idle":"2025-12-15T14:25:07.604345Z","shell.execute_reply.started":"2025-12-15T11:52:55.862229Z","shell.execute_reply":"2025-12-15T14:25:07.601957Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**Implementation of EfficientNetB4 for this competition with Keras**","metadata":{}},{"cell_type":"code","source":"!pip install -U efficientnet==0.0.4","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-15T14:25:07.607197Z","iopub.execute_input":"2025-12-15T14:25:07.607981Z","iopub.status.idle":"2025-12-15T14:25:15.624492Z","shell.execute_reply.started":"2025-12-15T14:25:07.607928Z","shell.execute_reply":"2025-12-15T14:25:15.623558Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"from efficientnet import EfficientNetB4\n\nsize = 224","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-15T14:25:15.626026Z","iopub.execute_input":"2025-12-15T14:25:15.626247Z","iopub.status.idle":"2025-12-15T14:25:17.232441Z","shell.execute_reply.started":"2025-12-15T14:25:15.626211Z","shell.execute_reply":"2025-12-15T14:25:17.23188Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"#print(os.listdir(\"../input/efficientnetb0b7-keras-weights/efficientnet-b3_weights_tf_dim_ordering_tf_kernels_autoaugment_notop.h5\"))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-15T14:25:17.233542Z","iopub.execute_input":"2025-12-15T14:25:17.233791Z","iopub.status.idle":"2025-12-15T14:25:17.236622Z","shell.execute_reply.started":"2025-12-15T14:25:17.233744Z","shell.execute_reply":"2025-12-15T14:25:17.235966Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"effnet = EfficientNetB4(weights=None, include_top=False, input_shape=(size, size, 3))\n\neffnet.load_weights('../input/efficientnetb0b7-keras-weights/efficientnet-b4_weights_tf_dim_ordering_tf_kernels_autoaugment_notop.h5')","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-15T14:25:17.23796Z","iopub.execute_input":"2025-12-15T14:25:17.238145Z","iopub.status.idle":"2025-12-15T14:25:33.826682Z","shell.execute_reply.started":"2025-12-15T14:25:17.238113Z","shell.execute_reply":"2025-12-15T14:25:33.825979Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"pivot_df is simply sample_df reformatted so that each column is a label (this way, we can use multi-label in our data generator later).","metadata":{}},{"cell_type":"markdown","source":"**Batch Normalization becomes unstable with small batch sizes (<16) **","metadata":{}},{"cell_type":"markdown","source":"## Why Group Normalization(GN) ????","metadata":{}},{"cell_type":"markdown","source":"*Group Normalization(GN) divides the channels into groups and computes within each group the mean and variance for normalization. GN’s computation is independent of batch sizes, and its accuracy is stable in a wide range of batch sizes. On ResNet-50 trained in ImageNet, GN has 10.6% lower error than its BN counterpart when using a batch size of 2; when using typical batch sizes, GN is comparably good with batch normalization BN and outperforms other normalization variants. Moreover, GN can be naturally transferred from pre-training to fine-tuning. GN can outperform its BN based counterparts for object detection and segmentation.*\n\nto read more visit this link : https://analyticsindiamag.com/alternatives-batch-normalization-deep-learning/\n","metadata":{}},{"cell_type":"code","source":"class GroupNormalization(Layer):\n \"\"\"Group normalization layer\n Group Normalization divides the channels into groups and computes within each group\n the mean and variance for normalization. GN's computation is independent of batch sizes,\n and its accuracy is stable in a wide range of batch sizes\n # Arguments\n groups: Integer, the number of groups for Group Normalization.\n axis: Integer, the axis that should be normalized\n (typically the features axis).\n For instance, after a `Conv2D` layer with\n `data_format=\"channels_first\"`,\n set `axis=1` in `BatchNormalization`.\n epsilon: Small float added to variance to avoid dividing by zero.\n center: If True, add offset of `beta` to normalized tensor.\n If False, `beta` is ignored.\n scale: If True, multiply by `gamma`.\n If False, `gamma` is not used.\n When the next layer is linear (also e.g. `nn.relu`),\n this can be disabled since the scaling\n will be done by the next layer.\n beta_initializer: Initializer for the beta weight.\n gamma_initializer: Initializer for the gamma weight.\n beta_regularizer: Optional regularizer for the beta weight.\n gamma_regularizer: Optional regularizer for the gamma weight.\n beta_constraint: Optional constraint for the beta weight.\n gamma_constraint: Optional constraint for the gamma weight.\n # Input shape\n Arbitrary. Use the keyword argument `input_shape`\n (tuple of integers, does not include the samples axis)\n when using this layer as the first layer in a model.\n # Output shape\n Same shape as input.\n # References\n - [Group Normalization](https://arxiv.org/abs/1803.08494)\n \"\"\"\n\n def __init__(self,\n groups=32,\n axis=-1,\n epsilon=1e-5,\n center=True,\n scale=True,\n beta_initializer='zeros',\n gamma_initializer='ones',\n beta_regularizer=None,\n gamma_regularizer=None,\n beta_constraint=None,\n gamma_constraint=None,\n **kwargs):\n super(GroupNormalization, self).__init__(**kwargs)\n self.supports_masking = True\n self.groups = groups\n self.axis = axis\n self.epsilon = epsilon\n self.center = center\n self.scale = scale\n self.beta_initializer = initializers.get(beta_initializer)\n self.gamma_initializer = initializers.get(gamma_initializer)\n self.beta_regularizer = regularizers.get(beta_regularizer)\n self.gamma_regularizer = regularizers.get(gamma_regularizer)\n self.beta_constraint = constraints.get(beta_constraint)\n self.gamma_constraint = constraints.get(gamma_constraint)\n\n def build(self, input_shape):\n dim = input_shape[self.axis]\n\n if dim is None:\n raise ValueError('Axis ' + str(self.axis) + ' of '\n 'input tensor should have a defined dimension '\n 'but the layer received an input with shape ' +\n str(input_shape) + '.')\n\n if dim < self.groups:\n raise ValueError('Number of groups (' + str(self.groups) + ') cannot be '\n 'more than the number of channels (' +\n str(dim) + ').')\n\n if dim % self.groups != 0:\n raise ValueError('Number of groups (' + str(self.groups) + ') must be a '\n 'multiple of the number of channels (' +\n str(dim) + ').')\n\n self.input_spec = InputSpec(ndim=len(input_shape),\n axes={self.axis: dim})\n shape = (dim,)\n\n if self.scale:\n self.gamma = self.add_weight(shape=shape,\n name='gamma',\n initializer=self.gamma_initializer,\n regularizer=self.gamma_regularizer,\n constraint=self.gamma_constraint)\n else:\n self.gamma = None\n if self.center:\n self.beta = self.add_weight(shape=shape,\n name='beta',\n initializer=self.beta_initializer,\n regularizer=self.beta_regularizer,\n constraint=self.beta_constraint)\n else:\n self.beta = None\n self.built = True\n\n def call(self, inputs, **kwargs):\n input_shape = K.int_shape(inputs)\n tensor_input_shape = K.shape(inputs)\n\n # Prepare broadcasting shape.\n reduction_axes = list(range(len(input_shape)))\n del reduction_axes[self.axis]\n broadcast_shape = [1] * len(input_shape)\n broadcast_shape[self.axis] = input_shape[self.axis] // self.groups\n broadcast_shape.insert(1, self.groups)\n\n reshape_group_shape = K.shape(inputs)\n group_axes = [reshape_group_shape[i] for i in range(len(input_shape))]\n group_axes[self.axis] = input_shape[self.axis] // self.groups\n group_axes.insert(1, self.groups)\n\n # reshape inputs to new group shape\n group_shape = [group_axes[0], self.groups] + group_axes[2:]\n group_shape = K.stack(group_shape)\n inputs = K.reshape(inputs, group_shape)\n\n group_reduction_axes = list(range(len(group_axes)))\n group_reduction_axes = group_reduction_axes[2:]\n\n mean = K.mean(inputs, axis=group_reduction_axes, keepdims=True)\n variance = K.var(inputs, axis=group_reduction_axes, keepdims=True)\n\n inputs = (inputs - mean) / (K.sqrt(variance + self.epsilon))\n\n # prepare broadcast shape\n inputs = K.reshape(inputs, group_shape)\n outputs = inputs\n\n # In this case we must explicitly broadcast all parameters.\n if self.scale:\n broadcast_gamma = K.reshape(self.gamma, broadcast_shape)\n outputs = outputs * broadcast_gamma\n\n if self.center:\n broadcast_beta = K.reshape(self.beta, broadcast_shape)\n outputs = outputs + broadcast_beta\n\n outputs = K.reshape(outputs, tensor_input_shape)\n\n return outputs\n\n def get_config(self):\n config = {\n 'groups': self.groups,\n 'axis': self.axis,\n 'epsilon': self.epsilon,\n 'center': self.center,\n 'scale': self.scale,\n 'beta_initializer': initializers.serialize(self.beta_initializer),\n 'gamma_initializer': initializers.serialize(self.gamma_initializer),\n 'beta_regularizer': regularizers.serialize(self.beta_regularizer),\n 'gamma_regularizer': regularizers.serialize(self.gamma_regularizer),\n 'beta_constraint': constraints.serialize(self.beta_constraint),\n 'gamma_constraint': constraints.serialize(self.gamma_constraint)\n }\n base_config = super(GroupNormalization, self).get_config()\n return dict(list(base_config.items()) + list(config.items()))\n\n def compute_output_shape(self, input_shape):\n return input_shape","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-15T14:25:33.827911Z","iopub.execute_input":"2025-12-15T14:25:33.828129Z","iopub.status.idle":"2025-12-15T14:25:33.849235Z","shell.execute_reply.started":"2025-12-15T14:25:33.828093Z","shell.execute_reply":"2025-12-15T14:25:33.848403Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Replace all Batch Normalization layers by Group Normalization layers","metadata":{}},{"cell_type":"code","source":"\nfor i, layer in enumerate(effnet.layers):\n if \"batch_normalization\" in layer.name:\n effnet.layers[i] = GroupNormalization(groups=32, axis=-1, epsilon=0.00001)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-15T14:25:33.850525Z","iopub.execute_input":"2025-12-15T14:25:33.850813Z","iopub.status.idle":"2025-12-15T14:25:33.884715Z","shell.execute_reply.started":"2025-12-15T14:25:33.850762Z","shell.execute_reply":"2025-12-15T14:25:33.88422Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**BCE DICE LOSS**","metadata":{}},{"cell_type":"code","source":"from keras.losses import binary_crossentropy\ndef dice_coef(y_true, y_pred, smooth=1):\n y_true_f = K.flatten(y_true)\n y_pred_f = K.flatten(y_pred)\n intersection = K.sum(y_true_f * y_pred_f)\n return (2. * intersection + smooth) / (K.sum(y_true_f) + K.sum(y_pred_f) + smooth)\n\ndef dice_loss(y_true, y_pred):\n smooth = 1.\n y_true_f = K.flatten(y_true)\n y_pred_f = K.flatten(y_pred)\n intersection = y_true_f * y_pred_f\n score = (2. * K.sum(intersection) + smooth) / (K.sum(y_true_f) + K.sum(y_pred_f) + smooth)\n return 1. - score\n\ndef bce_dice_loss(y_true, y_pred):\n return binary_crossentropy(y_true, y_pred) + dice_loss(y_true, y_pred)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-15T14:25:33.8857Z","iopub.execute_input":"2025-12-15T14:25:33.885922Z","iopub.status.idle":"2025-12-15T14:25:33.891644Z","shell.execute_reply.started":"2025-12-15T14:25:33.885879Z","shell.execute_reply":"2025-12-15T14:25:33.891067Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"!pip install keras-rectified-adam\nfrom keras_radam import RAdam","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-15T14:25:33.892721Z","iopub.execute_input":"2025-12-15T14:25:33.89294Z","iopub.status.idle":"2025-12-15T14:25:40.231434Z","shell.execute_reply.started":"2025-12-15T14:25:33.892902Z","shell.execute_reply":"2025-12-15T14:25:40.230742Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def build_model():\n \"\"\"\n A custom implementation of EfficientNetB4\n \"\"\"\n model = Sequential()\n model.add(effnet)\n model.add(GlobalAveragePooling2D())\n model.add(Dropout(0.5))\n model.add(Dense(6, activation='sigmoid'))\n model.compile(loss=bce_dice_loss,\n optimizer=Adam(learning_rate=5e-5), \n metrics=['accuracy'])\n print(model.summary())\n return model","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-15T14:38:19.060256Z","iopub.execute_input":"2025-12-15T14:38:19.060537Z","iopub.status.idle":"2025-12-15T14:38:19.065604Z","shell.execute_reply.started":"2025-12-15T14:38:19.060504Z","shell.execute_reply":"2025-12-15T14:38:19.06493Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Initialize model\nmodel = build_model()","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"def preprocess_image(image, sigmaX=10):\n \"\"\"\n The whole preprocessing pipeline:\n 1. Read in image\n 2. Apply masks\n 3. Resize image to desired size\n 4. Add Gaussian noise to increase Robustness\n \"\"\"\n image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)\n image = crop_image_from_gray(image)\n image = cv2.resize(image, (size, size))\n image = cv2.addWeighted (image,4, cv2.GaussianBlur(image, (0,0) ,sigmaX), -4, 128)\n return image","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-15T14:38:43.238727Z","iopub.execute_input":"2025-12-15T14:38:43.239063Z","iopub.status.idle":"2025-12-15T14:38:43.243775Z","shell.execute_reply.started":"2025-12-15T14:38:43.239004Z","shell.execute_reply":"2025-12-15T14:38:43.242895Z"}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Attention please\n\nkeras team updated their utils.py file from keras-preprocessing github repository just 2 days ago(probably in 26th september,2019) (here:https://github.com/keras-team/keras-preprocessing/blob/master/keras_preprocessing/image/utils.py) so if you try to import and use ImageDataGenerator old from keras.preprocessing.image import ImageDataGenerator then you will get error saying \"ValueError: Invalid class_mode: other; expected one of: {'input', 'raw', None, 'multi_output', 'binary', 'sparse', 'categorical'}\"\n\ni got that error and spend few times on google to figure that out and solved it eventually\n\n**Solution : **\n\nas the keras-team just updated their utils file to use ImageDataGenerator now you will need to do !pip install git+https://github.com/keras-team/keras-preprocessing.git first then import ImageDataGenerator like this : from keras.preprocessing.image import ImageDataGenerator\nso, instead of previous keras.preprocessing.image we need to use keras_preprocessing.image now,lets do it ;)","metadata":{}},{"cell_type":"code","source":"!pip install git+https://github.com/keras-team/keras-preprocessing.git","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-15T14:38:47.948757Z","iopub.execute_input":"2025-12-15T14:38:47.949128Z","iopub.status.idle":"2025-12-15T14:38:54.477177Z","shell.execute_reply.started":"2025-12-15T14:38:47.949077Z","shell.execute_reply":"2025-12-15T14:38:54.476421Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"from keras_preprocessing.image import ImageDataGenerator","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-15T14:39:05.258471Z","iopub.execute_input":"2025-12-15T14:39:05.258832Z","iopub.status.idle":"2025-12-15T14:39:05.262768Z","shell.execute_reply.started":"2025-12-15T14:39:05.258772Z","shell.execute_reply":"2025-12-15T14:39:05.261998Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"BATCH_SIZE = 32\n\ndef create_datagen():\n return ImageDataGenerator(validation_split=0.15)\n\ndef create_test_gen():\n return ImageDataGenerator().flow_from_dataframe(\n sub_df,\n directory= '/kaggle/tmp/',\n x_col='filename',\n class_mode=None,\n target_size=(size, size),\n batch_size=BATCH_SIZE,\n shuffle=False\n )\n\ndef create_flow(datagen, subset):\n return datagen.flow_from_dataframe(\n pivot_df, \n directory='/kaggle/tmp/',\n \n x_col='filename', \n y_col=['any', 'epidural', 'intraparenchymal', \n 'intraventricular', 'subarachnoid', 'subdural'],\n class_mode='other',\n target_size=(size, size),\n batch_size=BATCH_SIZE,\n preprocessing_function=preprocess_image,\n rotation_range=360,\n horizontal_flip=True,\n validation_split=0.15,\n rescale=1 / 128.,\n subset=subset\n )\n\n# Using original generator\ndata_generator = create_datagen()\ntrain_gen = create_flow(data_generator, 'training')\nval_gen = create_flow(data_generator, 'validation')\ntest_gen = create_test_gen()","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2025-12-15T14:39:15.472654Z","iopub.execute_input":"2025-12-15T14:39:15.472961Z","iopub.status.idle":"2025-12-15T14:39:26.368775Z","shell.execute_reply.started":"2025-12-15T14:39:15.472913Z","shell.execute_reply":"2025-12-15T14:39:26.368154Z"}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"checkpoint = ModelCheckpoint(\n 'effnetb4.h5', \n monitor='val_loss', \n verbose=0, \n save_best_only=True, \n save_weights_only=False,\n mode='auto'\n)\n#train_length = len(train_df)\ntotal_steps = sample_files.shape[0] // BATCH_SIZE\ntotal_steps = total_steps // 4\nhistory = model.fit_generator(\n train_gen,\n steps_per_epoch = total_steps,\n validation_data=val_gen,\n validation_steps=total_steps * 0.15,\n callbacks=[checkpoint],\n epochs=1\n)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"**submission**","metadata":{}},{"cell_type":"code","source":"model.load_weights('effnetb4.h5')\ny_test = model.predict_generator(\n test_gen,\n steps=len(test_gen),\n verbose=1\n)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"code","source":"# Append the output predicts in the wide format to the y_test\ntest_df = test_df.join(pd.DataFrame(y_test, columns=[\n 'any', 'epidural', 'intraparenchymal', 'intraventricular', 'subarachnoid', 'subdural'\n]))\n\n# Unpivot table, i.e. wide (N x 6) to long format (6N x 1)\ntest_df = test_df.melt(id_vars=['filename'])\n\n# Combine the filename column with the variable column\ntest_df['ID'] = test_df.filename.apply(lambda x: x.replace('.png', '')) + '_' + test_df.variable\ntest_df['Label'] = test_df['value']\n\ntest_df[['ID', 'Label']].to_csv('submission.csv', index=False)","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## Future Plans\n\n1. trying focal instead of bce dice\n2. using equal number of positive and negative samples for training\n3. using bit more large size images","metadata":{}},{"cell_type":"markdown","source":"****I hope this kernel helpful and some UPVOTES would be very much appreciated******","metadata":{}},{"cell_type":"markdown","source":"**REFERENCES**\n\n- https://www.kaggle.com/xhlulu/rsna-intracranial-simple-densenet-in-keras\n- https://www.kaggle.com/carlolepelaars/efficientnetb5-with-keras-aptos-2019\n- https://www.kaggle.com/marcovasquez/basic-eda-data-visualization\n- https://www.kaggle.com/allunia/rsna-ih-detection-eda-baseline\n\n","metadata":{}},{"cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null}]}