{
  "id": 430556,
  "title": "31th place solution",
  "url": "/competitions/google-research-identify-contrails-reduce-global-warming/discussion/430556",
  "author_name": "YumeNeko",
  "post_date": "2023-08-10T09:05:25.908000",
  "votes": 20,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Thank you to all the hosts and to all the participants for this interesting competition.   <br>\nIt was a very challenging, inventive and fun competition.  </p>\n<h1>Overview</h1>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3823496%2Fff0117d3aaa5cbb35e5d1de6a44bb4f4%2Foverview.jpg?generation=1691658120356042&amp;alt=media\" alt=\"Overview\"></p>\n<ul>\n<li><p>2-stage pipeline</p>\n<ul>\n<li>In the first phase, binary classification of whether the image contains contrail or not, using a time series-based classification model.</li>\n<li>In the second phase, only images predicted to include contrail are inferred by the segmentation model.Images predicted not to contain contrails are not done anything.  </li></ul></li>\n<li><p>Final Score  </p>\n<ul>\n<li>cv=0.694 / public=0.706 / private=0.699</li>\n<li>cv is calculated with data in the validation folder.  </li></ul></li>\n</ul>\n<h1>Solution pipeline</h1>\n<h2>1. Classification Model</h2>\n<ul>\n<li>According to the material provided by the organizers, we considered time series information important to determine if the shadows were contrail. We really wanted to reflect the time-series information in the segmentation model, but modeling and training were not successful, so we decided to use a classification model that binary classifies whether an image contains contrails or not.</li>\n<li>Training a classification model that combines encoder and LSTM with reference to <a href=\"https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/391779\" target=\"_blank\">the solution of the RSNA competition</a>.</li>\n<li>After verification in the environment at hand, we decided to use only images with prob&gt;0.2 as the inference target. This method improved cv by about 0.001.</li>\n<li>Detailed training settings are as follows  <ul>\n<li>encoder: convnext_large</li>\n<li>loss: BCEWithLogits</li></ul></li>\n</ul>\n<h2>2. Segmentation Model</h2>\n<ul>\n<li>A simple 2D model using Unet was used.</li>\n<li>A weighted ensemble of five models with different encoders and losses. Weights were calculated by Nelder-Mead to maximize cv.  </li>\n<li>Details are as follows<ul>\n<li>encoder: EfficientNet-b7 / convnext-s / convnext-l</li>\n<li>loss: diceloss / dice+bce</li>\n<li>image size: 512x512</li>\n<li>dataset: false rgb</li>\n<li>augmentation: v/h flip, rot90</li></ul></li>\n</ul>\n<h3>What worked</h3>\n<ul>\n<li>soft label using human_individual_masks  </li>\n<li>Pseudo Label for images other than t=4  <ul>\n<li>Pseudo-labels were generated by ensembling the results of efficientnet-b7 and resnet101E, which were trained on t=4 data only.</li>\n<li>First, the model is trained using only Pseudo Label. Next, fine tuning was performed using t=4 data with the results as initial weights.</li></ul></li>\n<li>Fold division without leakage  <ul>\n<li>Fold division based on latitude and longitude information in the metadata, with minimal effect of leakage.</li></ul></li>\n<li>TTA  <ul>\n<li>v/h flip, rot90 (same as augmentation)</li></ul></li>\n</ul>",
  "messages": [
    {
      "id": 2383246,
      "postDate": "2023-08-10T09:05:25.907Z",
      "content": "<p>Thank you to all the hosts and to all the participants for this interesting competition.   <br>\nIt was a very challenging, inventive and fun competition.  </p>\n<h1>Overview</h1>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3823496%2Fff0117d3aaa5cbb35e5d1de6a44bb4f4%2Foverview.jpg?generation=1691658120356042&amp;alt=media\" alt=\"Overview\"></p>\n<ul>\n<li><p>2-stage pipeline</p>\n<ul>\n<li>In the first phase, binary classification of whether the image contains contrail or not, using a time series-based classification model.</li>\n<li>In the second phase, only images predicted to include contrail are inferred by the segmentation model.Images predicted not to contain contrails are not done anything.  </li></ul></li>\n<li><p>Final Score  </p>\n<ul>\n<li>cv=0.694 / public=0.706 / private=0.699</li>\n<li>cv is calculated with data in the validation folder.  </li></ul></li>\n</ul>\n<h1>Solution pipeline</h1>\n<h2>1. Classification Model</h2>\n<ul>\n<li>According to the material provided by the organizers, we considered time series information important to determine if the shadows were contrail. We really wanted to reflect the time-series information in the segmentation model, but modeling and training were not successful, so we decided to use a classification model that binary classifies whether an image contains contrails or not.</li>\n<li>Training a classification model that combines encoder and LSTM with reference to <a href=\"https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/391779\" target=\"_blank\">the solution of the RSNA competition</a>.</li>\n<li>After verification in the environment at hand, we decided to use only images with prob&gt;0.2 as the inference target. This method improved cv by about 0.001.</li>\n<li>Detailed training settings are as follows  <ul>\n<li>encoder: convnext_large</li>\n<li>loss: BCEWithLogits</li></ul></li>\n</ul>\n<h2>2. Segmentation Model</h2>\n<ul>\n<li>A simple 2D model using Unet was used.</li>\n<li>A weighted ensemble of five models with different encoders and losses. Weights were calculated by Nelder-Mead to maximize cv.  </li>\n<li>Details are as follows<ul>\n<li>encoder: EfficientNet-b7 / convnext-s / convnext-l</li>\n<li>loss: diceloss / dice+bce</li>\n<li>image size: 512x512</li>\n<li>dataset: false rgb</li>\n<li>augmentation: v/h flip, rot90</li></ul></li>\n</ul>\n<h3>What worked</h3>\n<ul>\n<li>soft label using human_individual_masks  </li>\n<li>Pseudo Label for images other than t=4  <ul>\n<li>Pseudo-labels were generated by ensembling the results of efficientnet-b7 and resnet101E, which were trained on t=4 data only.</li>\n<li>First, the model is trained using only Pseudo Label. Next, fine tuning was performed using t=4 data with the results as initial weights.</li></ul></li>\n<li>Fold division without leakage  <ul>\n<li>Fold division based on latitude and longitude information in the metadata, with minimal effect of leakage.</li></ul></li>\n<li>TTA  <ul>\n<li>v/h flip, rot90 (same as augmentation)</li></ul></li>\n</ul>",
      "rawMarkdown": "Thank you to all the hosts and to all the participants for this interesting competition.   \nIt was a very challenging, inventive and fun competition.  \n\n# Overview\n![Overview](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3823496%2Fff0117d3aaa5cbb35e5d1de6a44bb4f4%2Foverview.jpg?generation=1691658120356042&alt=media)\n* 2-stage pipeline\n  * In the first phase, binary classification of whether the image contains contrail or not, using a time series-based classification model.\n  * In the second phase, only images predicted to include contrail are inferred by the segmentation model.Images predicted not to contain contrails are not done anything.  \n\n* Final Score  \n  * cv=0.694 / public=0.706 / private=0.699\n  * cv is calculated with data in the validation folder.  \n\n# Solution pipeline\n## 1. Classification Model\n* According to the material provided by the organizers, we considered time series information important to determine if the shadows were contrail. We really wanted to reflect the time-series information in the segmentation model, but modeling and training were not successful, so we decided to use a classification model that binary classifies whether an image contains contrails or not.\n* Training a classification model that combines encoder and LSTM with reference to [the solution of the RSNA competition](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/391779).\n* After verification in the environment at hand, we decided to use only images with prob>0.2 as the inference target. This method improved cv by about 0.001.\n* Detailed training settings are as follows  \n  * encoder: convnext_large\n  * loss: BCEWithLogits\n\n## 2. Segmentation Model\n* A simple 2D model using Unet was used.\n* A weighted ensemble of five models with different encoders and losses. Weights were calculated by Nelder-Mead to maximize cv.  \n* Details are as follows\n  * encoder: EfficientNet-b7 / convnext-s / convnext-l\n  * loss: diceloss / dice+bce\n  * image size: 512x512\n  * dataset: false rgb\n  * augmentation: v/h flip, rot90\n\n### What worked  \n* soft label using human_individual_masks  \n* Pseudo Label for images other than t=4  \n  * Pseudo-labels were generated by ensembling the results of efficientnet-b7 and resnet101E, which were trained on t=4 data only.\n  * First, the model is trained using only Pseudo Label. Next, fine tuning was performed using t=4 data with the results as initial weights.\n* Fold division without leakage  \n  * Fold division based on latitude and longitude information in the metadata, with minimal effect of leakage.\n* TTA  \n  * v/h flip, rot90 (same as augmentation)",
      "votes": 19
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "2383246": "Thank you to all the hosts and to all the participants for this interesting competition.   \nIt was a very challenging, inventive and fun competition.  \n\n# Overview\n![Overview](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F3823496%2Fff0117d3aaa5cbb35e5d1de6a44bb4f4%2Foverview.jpg?generation=1691658120356042&alt=media)\n* 2-stage pipeline\n  * In the first phase, binary classification of whether the image contains contrail or not, using a time series-based classification model.\n  * In the second phase, only images predicted to include contrail are inferred by the segmentation model.Images predicted not to contain contrails are not done anything.  \n\n* Final Score  \n  * cv=0.694 / public=0.706 / private=0.699\n  * cv is calculated with data in the validation folder.  \n\n# Solution pipeline\n## 1. Classification Model\n* According to the material provided by the organizers, we considered time series information important to determine if the shadows were contrail. We really wanted to reflect the time-series information in the segmentation model, but modeling and training were not successful, so we decided to use a classification model that binary classifies whether an image contains contrails or not.\n* Training a classification model that combines encoder and LSTM with reference to [the solution of the RSNA competition](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/391779).\n* After verification in the environment at hand, we decided to use only images with prob>0.2 as the inference target. This method improved cv by about 0.001.\n* Detailed training settings are as follows  \n  * encoder: convnext_large\n  * loss: BCEWithLogits\n\n## 2. Segmentation Model\n* A simple 2D model using Unet was used.\n* A weighted ensemble of five models with different encoders and losses. Weights were calculated by Nelder-Mead to maximize cv.  \n* Details are as follows\n  * encoder: EfficientNet-b7 / convnext-s / convnext-l\n  * loss: diceloss / dice+bce\n  * image size: 512x512\n  * dataset: false rgb\n  * augmentation: v/h flip, rot90\n\n### What worked  \n* soft label using human_individual_masks  \n* Pseudo Label for images other than t=4  \n  * Pseudo-labels were generated by ensembling the results of efficientnet-b7 and resnet101E, which were trained on t=4 data only.\n  * First, the model is trained using only Pseudo Label. Next, fine tuning was performed using t=4 data with the results as initial weights.\n* Fold division without leakage  \n  * Fold division based on latitude and longitude information in the metadata, with minimal effect of leakage.\n* TTA  \n  * v/h flip, rot90 (same as augmentation)"
  }
}