Now we have an idea on what is the distribution of the positive and negative files, let's explore one of the file. I will select file having
.
","metadata":{}},{"cell_type":"code","source":"tem_8 = np.load(\n \"/kaggle/input/google-research-identify-contrails-reduce-global-warming/train/1000216489776414077/band_08.npy\"\n)\n\nfig, ax = plt.subplots(nrows=1, ncols=2, sharey=True)\n\n# plt.plot(tem_8[0])\nax[0].imshow(tem_8[..., 0])\nax[1].imshow(tem_8[..., 1])\n\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-05-28T11:41:05.122263Z","iopub.execute_input":"2023-05-28T11:41:05.122732Z","iopub.status.idle":"2023-05-28T11:41:05.582044Z","shell.execute_reply.started":"2023-05-28T11:41:05.1227Z","shell.execute_reply":"2023-05-28T11:41:05.581089Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\nBefore we begin analyzing the data, it is important to understand where the data came from. The source of the data is essential to know when building a model. By understanding the source of the data and the method of data collection, we can extract more information from the dataset.\n
\n
\n
\nThe original satellite images were obtained from the
GOES-16 Advanced Baseline Imager (ABI), which is publicly available on
Google Cloud Storage. The original full-disk images were reprojected using bilinear resampling to generate a local scene image.\n\n
\n
Advanced Baseline Imager(ABI) & Spectral bands\nThe Advanced Baseline Imager (ABI) is an advanced weather satellite instrument used for Earth observation. It is one of the primary instruments onboard the Geostationary Operational Environmental Satellite (GOES) series, specifically designed for weather monitoring and forecasting. The ABI provides high-resolution, multi-channel imaging capabilities that significantly enhance weather analysis and forecasting capabilities.\n
\n
Spectral bands (
Source)
\nABI views Earth with 16 different spectral bands (compared to five on the previous generation of GOES), including two visible channels, four near-infrared channels, and 10 infrared channels. These different channels (wavelengths) are used by models and tools to indicate various elements on Earth’s surface or in the atmosphere, such as trees, water, clouds, moisture, or smoke.
\n
","metadata":{}},{"cell_type":"code","source":"# Idea borrowed from Pranav Nadimpalli @pranavnadimpali\nbands = range(8, 17)\ncolors = [\n \"red\",\n \"blue\",\n \"green\",\n \"orange\",\n \"purple\",\n \"cyan\",\n \"magenta\",\n \"yellow\",\n \"black\",\n]\n\nfor i, band in enumerate(bands):\n temp = np.load(f\"{TRAIN_PATH}/1000216489776414077/band_{band:02}.npy\")\n plt.plot(temp[0], color=colors[i], label=f\"Band {band:02}\")\n\n# plt.legend()\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-05-28T11:41:05.583555Z","iopub.execute_input":"2023-05-28T11:41:05.584516Z","iopub.status.idle":"2023-05-28T11:41:06.55436Z","shell.execute_reply.started":"2023-05-28T11:41:05.584477Z","shell.execute_reply":"2023-05-28T11:41:06.553477Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n
What are these spectral bands🤔🌈?\n
\nIn the given dataset we are provided images which are captured at 8 different spectral bands namely from band 8 to band 16. Each band represents a series of images captured at different wavelengths of light. For instance, band 08 consists of a series of images highlighting a specific wavelength, while band 16 contains images capturing a different wavelength. It is important to note that each band contains images of the same thing just at different wavelengths so each band will look slightly different and contain more information than others because contrails look different at different wavelengths.\n\nThe ABI captures images in separate bands at regular 10-minute intervals. Within each band, there are a total of 8 images taken over a span of 80 minutes, with a 10-minute gap between each image. For instance, when we analyze band 8, we are essentially observing a sequence of images, each captured 10 minutes apart from one another. This series of images offers valuable insights into the progression and transformations of contrails, enabling us to study their expansion and shape variations over time.\n\n
\n\nNow that we have got a bird's eye view of the spectral bands let's understanf what the two files :
human_pixel_masks and
human_individual_masks are trying to tell us.\n\n- human_pixel_masks : these file contain the ground truth labels. We can consider a label correct only if more than half of the labellers have aggried upon it. Doing so increases the rwliability of the results produced. Since the images are captured at three intervals namely n_times_before which is 4 and n_timed_after which is 3, this implies that the ground truth mask corresponds to the 5
th example.\n\n- human_individual_masks: The human_individual_masks represent labels generated by multiple labellers. These labels are then compared and combined to create a final ground truth, which can be found in the human_pixel_masks file. \n \n\n
","metadata":{}},{"cell_type":"markdown","source":"
\nLet's dig deep into individual images and find their metadata.\n
","metadata":{}},{"cell_type":"code","source":"band = 8\nsample_img = np.load(f\"{TRAIN_PATH}/1000216489776414077/band_{band:02}.npy\")\nprint(f\"{green}The image has a shape : {sample_img.shape}\")","metadata":{"execution":{"iopub.status.busy":"2023-05-28T11:41:06.555537Z","iopub.execute_input":"2023-05-28T11:41:06.55632Z","iopub.status.idle":"2023-05-28T11:41:06.567124Z","shell.execute_reply.started":"2023-05-28T11:41:06.556282Z","shell.execute_reply":"2023-05-28T11:41:06.565853Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n
Plot the images and visualize the change in the countrails over time.\n
\nThe plot will go from left to right which will tell us the images for different bands.
\nThe images from top to bottom will tell us the change over time steps.(10 min apart) \n
\n
","metadata":{}},{"cell_type":"code","source":"def plot_images(img_id: str, img_dir: str):\n \"\"\"\n Args:\n img_id = The ID of the image (e.g., '1000216489776414077')\n img_dir = The directory (e.g., 'train', 'test', or 'valid')\n \"\"\"\n\n fig, axs = plt.subplots(len(bands), 8, figsize=(16, 16))\n\n for i, band in enumerate(bands):\n img = np.load(f\"{BASE_PATH}/{img_dir}/{img_id}/band_{band:02}.npy\")\n for j in range(8):\n axs[i, j].imshow(img[..., j])\n axs[i, j].set_title(f\"Band {band}\\nTime Step {j-4}\")\n axs[i, j].set_xticks([])\n axs[i, j].set_yticks([])\n\n plt.tight_layout()\n plt.show()\n\n\nplot_images(\"1001792196232384964\", \"train\")","metadata":{"execution":{"iopub.status.busy":"2023-05-28T11:41:06.568751Z","iopub.execute_input":"2023-05-28T11:41:06.569103Z","iopub.status.idle":"2023-05-28T11:41:11.902606Z","shell.execute_reply.started":"2023-05-28T11:41:06.569072Z","shell.execute_reply":"2023-05-28T11:41:11.9012Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"
\n\nNow since we have got an overview of how are the images in several band we will move on to exploring the ground truth masks.\n
","metadata":{}},{"cell_type":"code","source":"band_8_human_individual = np.load(\n f\"{TRAIN_PATH}/1000603527582775543/human_individual_masks.npy\"\n)\nband_8_human_individual.shape","metadata":{"execution":{"iopub.status.busy":"2023-05-28T11:41:11.904109Z","iopub.execute_input":"2023-05-28T11:41:11.904489Z","iopub.status.idle":"2023-05-28T11:41:11.932987Z","shell.execute_reply.started":"2023-05-28T11:41:11.904456Z","shell.execute_reply":"2023-05-28T11:41:11.931939Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.imshow(band_8_human_individual[..., 0])\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2023-05-28T11:41:11.934746Z","iopub.execute_input":"2023-05-28T11:41:11.935492Z","iopub.status.idle":"2023-05-28T11:41:12.177592Z","shell.execute_reply.started":"2023-05-28T11:41:11.935454Z","shell.execute_reply":"2023-05-28T11:41:12.176471Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}
![](\"https://storage.googleapis.com/kaggle-media/competitions/Google-Contrails/waterdroplets.png\")
\n\nObjectives: In creating this notebook, my objectives are:\n \n