{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"<center><img src=\"https://keras.io/img/logo-small.png\" alt=\"Keras logo\" width=\"100\"><br/>\nThis starter notebook is provided by the Keras team.</center>","metadata":{}},{"cell_type":"markdown","source":"# UBC Ovarian Cancer Subtype Classification and Outlier Detection (UBC-OCEAN) with [KerasCV](https://github.com/keras-team/keras-cv) and [Keras](https://github.com/keras-team/keras)\n\n> Your challenge in this competition is to classify the type of ovarian cancer from microscopy scans of biopsy samples.\n\nThis notebook walks you through how to train a **Convolutional Neural Network (CNN)** model (here ResNet) using KerasCV on the UBC-OCEAN dataset made available for this competition. In this notebook we specifically train on the `thumbnail` images provided.\n\nFun fact: This notebook is backend (tensorflow, pytorch, jax) agnostic. Using KerasCV and Keras we can choose a backend of our choise! Feel free to read on [Keras](https://keras.io/keras_core/announcement/) to know more.\n\nIn this notebook you will learn:\n\n* Loading the data using [`tf.data`](https://www.tensorflow.org/guide/data).\n* Create the model using KerasCV presets.\n* Train the model.\n* Submit to the competition.\n\n**Note**: [KerasCV guides](https://keras.io/guides/keras_cv/) is the place to go for a deeper understanding of KerasCV individually.","metadata":{}},{"cell_type":"markdown","source":"## Setup and Imports\n\nKeras is backend agnostic. This means that you can run keras on [TensorFlow](https://www.tensorflow.org/), [JAX](https://jax.readthedocs.io/en/latest/index.html), [PyTorch](https://pytorch.org/), or [Numpy](https://numpy.org/) (inference only). We will be using JAX as our backend. To switch backends set the `KERAS_BACKEND` varialbe to which backend to want.","metadata":{}},{"cell_type":"code","source":"import os\nos.environ[\"KERAS_BACKEND\"] = \"jax\" # or \"tensorflow\", \"torch\"\n\nimport cv2\nimport pickle\nimport numpy as np\nimport pandas as pd\nimport seaborn as sns\nimport matplotlib.pyplot as plt\n\n# Set the style for the plot\nsns.set(style=\"whitegrid\")\n\nimport tensorflow as tf\nimport keras_cv\nimport keras_core as keras\nfrom keras_core import ops","metadata":{"execution":{"iopub.status.busy":"2023-10-16T19:31:34.149167Z","iopub.execute_input":"2023-10-16T19:31:34.149502Z","iopub.status.idle":"2023-10-16T19:31:47.249545Z","shell.execute_reply.started":"2023-10-16T19:31:34.149473Z","shell.execute_reply":"2023-10-16T19:31:47.248607Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Configuration\n\nPlease feel free to change the configuration and run experiments.","metadata":{}},{"cell_type":"code","source":"class Config:\n    is_submission = True\n    \n    # Reproducibility\n    SEED = 42\n    \n    # Training\n    train_csv_path = \"/kaggle/input/UBC-OCEAN/train.csv\"\n    train_thumbnail_paths = \"/kaggle/input/UBC-OCEAN/train_thumbnails\"\n    batch_size = 8\n    learning_rate = 1e-3\n    epochs = 10\n    is_backbone_trainable = True\n    \n    # Inference\n    test_csv_path = \"/kaggle/input/UBC-OCEAN/test.csv\"\n    test_thumbnail_paths = \"/kaggle/input/UBC-OCEAN/test_thumbnails\"\n\nconfig = Config()","metadata":{"execution":{"iopub.status.busy":"2023-10-16T19:31:47.251453Z","iopub.execute_input":"2023-10-16T19:31:47.252244Z","iopub.status.idle":"2023-10-16T19:31:47.25756Z","shell.execute_reply.started":"2023-10-16T19:31:47.252209Z","shell.execute_reply":"2023-10-16T19:31:47.256665Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"To help with reproducibility we set the seed of the Pseudo Random Number Generator.","metadata":{}},{"cell_type":"code","source":"keras.utils.set_random_seed(seed=config.SEED)","metadata":{"execution":{"iopub.status.busy":"2023-10-16T19:31:47.258843Z","iopub.execute_input":"2023-10-16T19:31:47.259683Z","iopub.status.idle":"2023-10-16T19:31:47.282234Z","shell.execute_reply.started":"2023-10-16T19:31:47.259653Z","shell.execute_reply":"2023-10-16T19:31:47.281372Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Training","metadata":{}},{"cell_type":"markdown","source":"## Dataset\n\nIn this notebook we train on the thumbnails for quick iteration. Also note that we are NOT going to work on the anomaly detection part of the competition here. We will train a simple image classification model that will classify the scans of biopsy samples to their respective subtypes.","metadata":{}},{"cell_type":"code","source":"if not config.is_submission:\n    df = pd.read_csv(config.train_csv_path)\n\n    # Create the thumbnail df where is_tma == False\n    df = df[df[\"is_tma\"] == False]\n    \n    # Get basic statistics about the dataset\n    num_rows = df.shape[0]\n    num_unique_images = df['image_id'].nunique()\n    num_unique_labels = df['label'].nunique()\n    unique_labels = df['label'].unique()\n\n    print(f\"{num_rows=}\")\n    print(f\"{num_unique_images=}\")\n    print(f\"{num_unique_labels=}\")\n    print(f\"{unique_labels=}\")\n    \n    # Plot the distribution of the target classes\n    plt.figure(figsize=(10, 6))\n    sns.countplot(data=df, x='label', order=df['label'].value_counts().index)\n    plt.title('Distribution of Target Classes')\n    plt.xlabel('Label')\n    plt.ylabel('Count')\n    plt.show()","metadata":{"execution":{"iopub.status.busy":"2023-10-16T19:31:47.284647Z","iopub.execute_input":"2023-10-16T19:31:47.28526Z","iopub.status.idle":"2023-10-16T19:31:47.29577Z","shell.execute_reply.started":"2023-10-16T19:31:47.285229Z","shell.execute_reply":"2023-10-16T19:31:47.294874Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Note the imbalance in the distribution of target classes.","metadata":{}},{"cell_type":"markdown","source":"## Perform one-hot encoding","metadata":{}},{"cell_type":"code","source":"if not config.is_submission:\n    # Perform one-hot encoding of the 'label' column and explicitly convert to integer type\n    df_one_hot = pd.get_dummies(df[\"label\"], prefix=\"label\").astype(int)\n\n    # Concatenate the original DataFrame with the one-hot encoded labels\n    train_df = pd.concat([df[\"image_id\"], df_one_hot], axis=1)\n\n    # Get the thumbnail image paths\n    train_df[\"image_thumbnail_path\"] = train_df[\"image_id\"].apply(lambda x: f\"{config.train_thumbnail_paths}/{x}_thumbnail.png\")\n    \n    image_thumbnail_paths = train_df[\"image_thumbnail_path\"].values\n    labels = train_df[[col for col in train_df.columns if col.startswith(\"label_\")]].values\n\n    label_names = [col for col in train_df.columns if col.startswith(\"label_\")]\n    name_to_id = {key.replace(\"label_\", \"\"):value for value,key in enumerate(label_names)}\n    id_to_name = {key:value for value, key in name_to_id.items()}\n    \n    # Save to dictionary to disk\n    with open(\"id_to_name.pkl\", \"wb\") as f:\n        pickle.dump(id_to_name, f)","metadata":{"execution":{"iopub.status.busy":"2023-10-16T19:31:47.296974Z","iopub.execute_input":"2023-10-16T19:31:47.297793Z","iopub.status.idle":"2023-10-16T19:31:47.308226Z","shell.execute_reply.started":"2023-10-16T19:31:47.297764Z","shell.execute_reply":"2023-10-16T19:31:47.307323Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Taking pointers from the [image classification on imbalanced dataset guide](https://www.tensorflow.org/tutorials/structured_data/imbalanced_data), we create the class weights which will be used to train the model.","metadata":{}},{"cell_type":"code","source":"if not config.is_submission:\n    class_weights = np.sum(labels) - np.sum(labels, axis=0)\n    class_weights = class_weights / np.sum(class_weights) # Normalize the weights\n\n    class_weights = {idx:weight for idx, weight in enumerate(class_weights)}\n\n    for idx, weight in class_weights.items():\n        print(f\"{id_to_name[idx]}: {weight:0.2f}\")","metadata":{"execution":{"iopub.status.busy":"2023-10-16T19:31:47.309304Z","iopub.execute_input":"2023-10-16T19:31:47.310156Z","iopub.status.idle":"2023-10-16T19:31:47.325659Z","shell.execute_reply.started":"2023-10-16T19:31:47.310126Z","shell.execute_reply":"2023-10-16T19:31:47.324724Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Creating the `tf.data.Dataset` pipeline","metadata":{}},{"cell_type":"code","source":"def read_image(path):\n    file = tf.io.read_file(path)\n    image = tf.io.decode_png(file, 3)\n    image = tf.image.resize(image, (256, 256))\n    image = tf.image.per_image_standardization(image)\n    return image","metadata":{"execution":{"iopub.status.busy":"2023-10-16T19:31:47.326821Z","iopub.execute_input":"2023-10-16T19:31:47.327612Z","iopub.status.idle":"2023-10-16T19:31:47.337407Z","shell.execute_reply.started":"2023-10-16T19:31:47.327583Z","shell.execute_reply":"2023-10-16T19:31:47.336485Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"if not config.is_submission:\n    x = (\n        tf.data.Dataset.from_tensor_slices(image_thumbnail_paths)\n        .map(read_image, num_parallel_calls=tf.data.AUTOTUNE)\n    )\n    y = tf.data.Dataset.from_tensor_slices(labels)\n\n    # Zip the x and y together\n    ds = tf.data.Dataset.zip((x, y))\n    \n    # Create the training and validation splits\n    val_ds = (\n        ds\n        .take(50)\n        .batch(config.batch_size)\n        .prefetch(tf.data.AUTOTUNE)\n    )\n    train_ds = (\n        ds\n        .skip(50)\n        .shuffle(config.batch_size * 10)\n        .batch(config.batch_size)\n        .prefetch(tf.data.AUTOTUNE)\n    )","metadata":{"execution":{"iopub.status.busy":"2023-10-16T19:31:47.338542Z","iopub.execute_input":"2023-10-16T19:31:47.339307Z","iopub.status.idle":"2023-10-16T19:31:47.34879Z","shell.execute_reply.started":"2023-10-16T19:31:47.339279Z","shell.execute_reply":"2023-10-16T19:31:47.348005Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Visualizing the dataset is always fruitful. Below we will sample from the dataset and visualize some images.","metadata":{}},{"cell_type":"code","source":"if not config.is_submission:\n    images, labels = train_ds.take(1).get_single_element()\n\n    keras_cv.visualization.plot_image_gallery(\n        images,\n        value_range=(0, 1),\n        rows=2,\n        cols=2,\n    )","metadata":{"execution":{"iopub.status.busy":"2023-10-16T19:31:47.349878Z","iopub.execute_input":"2023-10-16T19:31:47.350163Z","iopub.status.idle":"2023-10-16T19:31:47.36349Z","shell.execute_reply.started":"2023-10-16T19:31:47.350138Z","shell.execute_reply":"2023-10-16T19:31:47.362584Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Build the Model\n\n[List of Keras CV models](https://keras.io/api/keras_cv/models/)","metadata":{}},{"cell_type":"code","source":"# Load the image and text backbones with presets\nresnet_backbone = keras_cv.models.ResNetV2Backbone.from_preset(\n    \"resnet152_v2\",\n)\nresnet_backbone.trainable = config.is_backbone_trainable\n\nimage_inputs = resnet_backbone.input\nimage_embeddings = resnet_backbone(image_inputs)\nimage_embeddings = keras.layers.GlobalAveragePooling2D()(image_embeddings)\n\nx = keras.layers.BatchNormalization(epsilon=1e-05, momentum=0.1)(image_embeddings)\nx = keras.layers.Dense(units=1024, activation=\"relu\")(x)\nx = keras.layers.Dropout(0.1)(x)\nx = keras.layers.Dense(units=512, activation=\"relu\")(x)\nx = keras.layers.Dropout(0.1)(x)\nx = keras.layers.Dense(units=256, activation=\"relu\")(x)\noutputs = keras.layers.Dense(units=5, activation=\"softmax\")(x)\n\n# Build the model with the Functional API\nmodel = keras.Model(\n    inputs=image_inputs,\n    outputs=outputs,\n)\n\nmodel.summary()","metadata":{"execution":{"iopub.status.busy":"2023-10-16T19:31:47.366261Z","iopub.execute_input":"2023-10-16T19:31:47.36668Z","iopub.status.idle":"2023-10-16T19:32:00.692184Z","shell.execute_reply.started":"2023-10-16T19:31:47.366647Z","shell.execute_reply":"2023-10-16T19:32:00.691285Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"if not config.is_submission:\n    model.compile(\n        optimizer=keras.optimizers.Adam(learning_rate=config.learning_rate),\n        loss=keras.losses.CategoricalCrossentropy(),\n        metrics=[\"accuracy\"],\n    )\n\n    history = model.fit(\n        train_ds,\n        epochs=config.epochs,\n        validation_data=val_ds,\n        class_weight=class_weights,\n    )\n    \n    model.save_weights(\"ucb_ocean_checkpoint.weights.h5\")","metadata":{"execution":{"iopub.status.busy":"2023-10-16T19:32:00.693363Z","iopub.execute_input":"2023-10-16T19:32:00.694226Z","iopub.status.idle":"2023-10-16T19:32:00.69999Z","shell.execute_reply.started":"2023-10-16T19:32:00.694195Z","shell.execute_reply":"2023-10-16T19:32:00.699114Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Inference","metadata":{}},{"cell_type":"code","source":"if config.is_submission:\n    df = pd.read_csv(config.test_csv_path)\n    df[\"image_path\"] = df[\"image_id\"].apply(lambda x: f\"{config.test_thumbnail_paths}/{x}_thumbnail.png\")\n    \n    # Load the model weights\n    model.load_weights(\"/kaggle/input/kerascv-train-and-infer-on-thumbnails/ucb_ocean_checkpoint.weights.h5\")\n    \n    # Load the id to name dictionary\n    with open(\"/kaggle/input/kerascv-train-and-infer-on-thumbnails/id_to_name.pkl\", \"rb\") as f:\n        id_to_name = pickle.load(f)","metadata":{"execution":{"iopub.status.busy":"2023-10-16T19:32:00.702094Z","iopub.execute_input":"2023-10-16T19:32:00.702984Z","iopub.status.idle":"2023-10-16T19:32:04.066824Z","shell.execute_reply.started":"2023-10-16T19:32:00.702951Z","shell.execute_reply":"2023-10-16T19:32:04.065905Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"if config.is_submission:\n    predicted_labels = []\n\n    for index, row in df.iterrows():\n        # Get the image path\n        image_path = row[\"image_path\"]\n\n        # Get the image\n        image = read_image(image_path)[None, ...]\n\n        # Predict the label\n        logits = model.predict(image)\n        pred = ops.argmax(logits, axis=-1).tolist()[0]\n\n        # Map the pred to the name\n        label = id_to_name[pred]\n\n        predicted_labels.append(label)\n\n    # Add the predicted labels to the csv\n    df[\"label\"] = predicted_labels","metadata":{"execution":{"iopub.status.busy":"2023-10-16T19:32:04.068103Z","iopub.execute_input":"2023-10-16T19:32:04.068651Z","iopub.status.idle":"2023-10-16T19:32:16.205769Z","shell.execute_reply.started":"2023-10-16T19:32:04.06862Z","shell.execute_reply":"2023-10-16T19:32:16.204827Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"if config.is_submission:\n    # Create the submission\n    submission_df = df[[\"image_id\", \"label\"]]\n    submission_df.to_csv(\"submission.csv\", index=False)","metadata":{"execution":{"iopub.status.busy":"2023-10-16T19:32:16.207224Z","iopub.execute_input":"2023-10-16T19:32:16.207579Z","iopub.status.idle":"2023-10-16T19:32:16.221311Z","shell.execute_reply.started":"2023-10-16T19:32:16.207535Z","shell.execute_reply":"2023-10-16T19:32:16.220162Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# Directions Ahead\n\nThere are a lot of directions one can take to iterate on top of this starter notebook. Some of them are:\n\n- Play with the various hyperparameters (in the configuration) to get better results\n- Come up with a strategy for anomaly detection\n- Use different models from the Keras CV presets","metadata":{}}]}