{ "id": 432254, "title": "13th Place Solution", "url": "/competitions/google-research-identify-contrails-reduce-global-warming/discussion/432254", "author_name": "Masaya", "post_date": "2023-08-16T16:03:39.633000", "votes": 10, "comment_count": 0, "views": 0, "content": "

13th Place Solution

\n

First of all, we would like to thank the organizers and the Kaggle team for hosting the competition.

\n

Overview

\n

We ensembled total 20 models including 2.5D-model and 2D model.

\n

We used the nelder-mead approach to determine the weight with constant threshold.

\n

I think taking an average of the individual labels is the best way for score.

\n

Here, the following shows Masaya's part in brief.

\n

Masaya part

\n

Data preprocessing

\n

I prepared 2 sets of data for evaluation, which are only validation data and cross-validation.
\nThis is because I also wanted to use validation data as training data while that was very correlated with Public LB.

\n

Maybe, the 2 most important thing to raise my score are

\n\n

Model

\n\n

Training

\n\n

Postprocess

\n

The threshold varied a lot, depending on how the CV was cut.
\nSo ensemble by average got less accurate.
\nTherefore, we fixed the threshold and determined the weight that maximized the validation by using the nelder-mead.

\n

Not work for me

\n", "messages": [ { "id": 2393941, "postDate": "2023-08-16T16:03:39.633Z", "content": "

13th Place Solution

\n

First of all, we would like to thank the organizers and the Kaggle team for hosting the competition.

\n

Overview

\n

We ensembled total 20 models including 2.5D-model and 2D model.

\n

We used the nelder-mead approach to determine the weight with constant threshold.

\n

I think taking an average of the individual labels is the best way for score.

\n

Here, the following shows Masaya's part in brief.

\n

Masaya part

\n

Data preprocessing

\n

I prepared 2 sets of data for evaluation, which are only validation data and cross-validation.
\nThis is because I also wanted to use validation data as training data while that was very correlated with Public LB.

\n

Maybe, the 2 most important thing to raise my score are

\n\n

Model

\n\n

Training

\n\n

Postprocess

\n

The threshold varied a lot, depending on how the CV was cut.
\nSo ensemble by average got less accurate.
\nTherefore, we fixed the threshold and determined the weight that maximized the validation by using the nelder-mead.

\n

Not work for me

\n", "rawMarkdown": "# 13th Place Solution\n\nFirst of all, we would like to thank the organizers and the Kaggle team for hosting the competition.\n\n## Overview\n\nWe ensembled total 20 models including 2.5D-model and 2D model.\n\nWe used the nelder-mead approach to determine the weight with constant threshold.\n\nI think taking an average of the individual labels is the best way for score.\n\nHere, the following shows Masaya's part in brief.\n\n## Masaya part\n\n### Data preprocessing\nI prepared 2 sets of data for evaluation, which are only validation data and cross-validation.\nThis is because I also wanted to use validation data as training data while that was very correlated with Public LB.\n\nMaybe, the 2 most important thing to raise my score are\n* label data are averaged over individual labels.\n* image resolution are set to 512pix or higher.\n\n### Model\n* EfficientNetb7-UNet\n* EfficientNetb7-Unet++\n* MiTb5-UNet\n\n### Training\n* amp (Sometimes the loss gets nan. Why?)\n* Optimizer: AdamW\n * No weight decay\n* Sheduler: CossineAnnealing with warm up\n * initial lr: 3e-4 ~ 5e-4\n* Loss: SoftBCE\n* CV strategy\n * hold out validation\n * StratifiedKFold\n * train data or validation data\n * including mask\n* Augmentation\n * hflip and vflip (long epoch worked)\n * only ShiftScaleRotate\n * none\n\n### Postprocess\nThe threshold varied a lot, depending on how the CV was cut. \nSo ensemble by average got less accurate. \nTherefore, we fixed the threshold and determined the weight that maximized the validation by using the nelder-mead.\n\n\n### Not work for me\n* Pseudo labels\n* training for only positive image\n* screening for positive label\n* 2nd stage model with segmentation prediction of 1st stage model\n* tta\n* another band image\n", "votes": 10 } ], "comments": [], "raw_markdown_by_id": { "2393941": "# 13th Place Solution\n\nFirst of all, we would like to thank the organizers and the Kaggle team for hosting the competition.\n\n## Overview\n\nWe ensembled total 20 models including 2.5D-model and 2D model.\n\nWe used the nelder-mead approach to determine the weight with constant threshold.\n\nI think taking an average of the individual labels is the best way for score.\n\nHere, the following shows Masaya's part in brief.\n\n## Masaya part\n\n### Data preprocessing\nI prepared 2 sets of data for evaluation, which are only validation data and cross-validation.\nThis is because I also wanted to use validation data as training data while that was very correlated with Public LB.\n\nMaybe, the 2 most important thing to raise my score are\n* label data are averaged over individual labels.\n* image resolution are set to 512pix or higher.\n\n### Model\n* EfficientNetb7-UNet\n* EfficientNetb7-Unet++\n* MiTb5-UNet\n\n### Training\n* amp (Sometimes the loss gets nan. Why?)\n* Optimizer: AdamW\n * No weight decay\n* Sheduler: CossineAnnealing with warm up\n * initial lr: 3e-4 ~ 5e-4\n* Loss: SoftBCE\n* CV strategy\n * hold out validation\n * StratifiedKFold\n * train data or validation data\n * including mask\n* Augmentation\n * hflip and vflip (long epoch worked)\n * only ShiftScaleRotate\n * none\n\n### Postprocess\nThe threshold varied a lot, depending on how the CV was cut. \nSo ensemble by average got less accurate. \nTherefore, we fixed the threshold and determined the weight that maximized the validation by using the nelder-mead.\n\n\n### Not work for me\n* Pseudo labels\n* training for only positive image\n* screening for positive label\n* 2nd stage model with segmentation prediction of 1st stage model\n* tta\n* another band image\n" } }