![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1745801%2Fdeb33f0f5a0349334ed541ec39c2e353%2F2023-08-10%2016.57.28.png?generation=1691664174212060&alt=media\")
First of all, we would like to thank the organizers of the competition and the Kaggle team for the competition hosting and all the participants who have shared their knowledge so generously.
\n
This pipeline has been used frequently in past semantic segmentation competitions.
\nThe advantage of this pipeline is that it creates two models, one trained on pos_only and the other on all data, thereby increasing the ensemble effect.
\nIt also saves inference time by allowing more models to be assigned to images for which screening has determined that a mask is present.
Both 2.5D and 2D models were trained for both hard and soft labels.
\nThe soft label is created as follows
label = self.np_load(\n os.path.join()\n).astype(np.float32)\nlabel = np.clip((label * ).(-) / label.shape[-], , )\nAll models were trained by all the data in train folder and validated by the one in validation folder.
\nWe used a modified 2.5D model used in Vesuvius Challenge.
\nThe main changes are as follows
Backbones used were resnetrs101, resnest101e, swin_base_patch4_window12, swinv2_base_window16, convnext_base, convnext_large, etc.
\nThe input frame is set to 5 or 7 frames, with the target frame in the middle.
\nInput resolution of 384 or 512 are used for the model.
\nIt was trained using BCE+Dice*0.25 for the pos only model and BCE for the all data model.
\nAlso, different augmentation is used for each of the classification model, all data seg model, and pos only seg model. (possibly not optimized and therefore not appropriate).
\nalbu.Flip(p=),\nalbu.RandomRotate90(p=),\nalbu.ShiftScaleRotate(p=, scale_limit=, shift_limit=),\nalbu.GridDistortion(num_steps=, distort_limit=, p=),\nalbu.CoarseDropout(max_height=, max_width=, fill_value=, mask_fill_value=, p=,)\n\n\nalbu.ShiftScaleRotate(p=, scale_limit=, shift_limit=),\n\n\nalbu.Flip(p=),\nalbu.RandomRotate90(p=),\nalbu.ShiftScaleRotate(p=, scale_limit=, shift_limit=, rotate_limit=),\nalbu.Rotate(limit=, p=),\nFor 2D models, we simply used Unet implemented in Segmentation Models Pytorch. The backbones used were res2net50d, regnetz_d8, regnetz_d32, and regnetz_e8.
\nCharacteristically, the 2D models were trained on 11 channels, i.e. band15 - band14, band14 - band11 and all 9 bands. Learning with this input does not perform well in short epochs (e.g. 20~30 epochs), but in long epochs (specifically 100 epochs), it obtained better validation scores than with false color input.
\nThe 11ch models also had one curious point. They had poor public score but good private score. I believe this is what pushed us to the prize zone.
\nOther training settings are as follows(shared by all data seg model and pos only seg one):
\n0.9*BCE + 0.1*Dicealbu.HorizontalFlip(p=),\nalbu.VerticalFlip(p=),\nalbu.ShiftScaleRotate(p=, rotate_limit=)\nalbu.RandomResizedCrop(p: , scale=[, ], height=, width=)\nFrom the time taken to inference, the percentage of empty masks in the entire test was found to be about the same as in validation set. (about 30%).
\nTherefore, we used validation set to optimize the percentile threshold for both the classification part and segmentation part.
\nAs a result, the percentile threshold is better than the optimized fixed threshold.
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F1745801%2F48a3c5b9e9ed108faaed1514dac63485%2F2023-08-10%2017.20.42.png?generation=1691664304846451&alt=media\")
We have released the training code and the inference notebook for the best submission.
\nFirst of all, we would like to thank the organizers of the competition and the Kaggle team for the competition hosting and all the participants who have shared their knowledge so generously.
\n
This pipeline has been used frequently in past semantic segmentation competitions.
\nThe advantage of this pipeline is that it creates two models, one trained on pos_only and the other on all data, thereby increasing the ensemble effect.
\nIt also saves inference time by allowing more models to be assigned to images for which screening has determined that a mask is present.
Both 2.5D and 2D models were trained for both hard and soft labels.
\nThe soft label is created as follows
label = self.np_load(\n os.path.join()\n).astype(np.float32)\nlabel = np.clip((label * ).(-) / label.shape[-], , )\nAll models were trained by all the data in train folder and validated by the one in validation folder.
\nWe used a modified 2.5D model used in Vesuvius Challenge.
\nThe main changes are as follows
Backbones used were resnetrs101, resnest101e, swin_base_patch4_window12, swinv2_base_window16, convnext_base, convnext_large, etc.
\nThe input frame is set to 5 or 7 frames, with the target frame in the middle.
\nInput resolution of 384 or 512 are used for the model.
\nIt was trained using BCE+Dice*0.25 for the pos only model and BCE for the all data model.
\nAlso, different augmentation is used for each of the classification model, all data seg model, and pos only seg model. (possibly not optimized and therefore not appropriate).
\nalbu.Flip(p=),\nalbu.RandomRotate90(p=),\nalbu.ShiftScaleRotate(p=, scale_limit=, shift_limit=),\nalbu.GridDistortion(num_steps=, distort_limit=, p=),\nalbu.CoarseDropout(max_height=, max_width=, fill_value=, mask_fill_value=, p=,)\n\n\nalbu.ShiftScaleRotate(p=, scale_limit=, shift_limit=),\n\n\nalbu.Flip(p=),\nalbu.RandomRotate90(p=),\nalbu.ShiftScaleRotate(p=, scale_limit=, shift_limit=, rotate_limit=),\nalbu.Rotate(limit=, p=),\nFor 2D models, we simply used Unet implemented in Segmentation Models Pytorch. The backbones used were res2net50d, regnetz_d8, regnetz_d32, and regnetz_e8.
\nCharacteristically, the 2D models were trained on 11 channels, i.e. band15 - band14, band14 - band11 and all 9 bands. Learning with this input does not perform well in short epochs (e.g. 20~30 epochs), but in long epochs (specifically 100 epochs), it obtained better validation scores than with false color input.
\nThe 11ch models also had one curious point. They had poor public score but good private score. I believe this is what pushed us to the prize zone.
\nOther training settings are as follows(shared by all data seg model and pos only seg one):
\n0.9*BCE + 0.1*Dicealbu.HorizontalFlip(p=),\nalbu.VerticalFlip(p=),\nalbu.ShiftScaleRotate(p=, rotate_limit=)\nalbu.RandomResizedCrop(p: , scale=[, ], height=, width=)\nFrom the time taken to inference, the percentage of empty masks in the entire test was found to be about the same as in validation set. (about 30%).
\nTherefore, we used validation set to optimize the percentile threshold for both the classification part and segmentation part.
\nAs a result, the percentile threshold is better than the optimized fixed threshold.
We have released the training code and the inference notebook for the best submission.
\nUpdate
\nWe have released the training code(2.5D model part) and the inference notebook for the best submission.
Sorry for late.
\nWe have released the training code for 2D model part:
\nhttps://github.com/tawatawara/kaggle-google-research-identify-contrails-reduce-global-warming
Congratulations and thanks a lot for sharing!
\nLet me ask you a couple of questions 😃:
\n\n\nFrom the time taken to inference, the percentage of empty masks in the entire test was found to be about the same as in validation set. (about 30%)
\n
\n\nIf I am correct, the purpose of classification models is to determine whether the predicted mask is empty or not, and they are only applied to segmentation models trained on all data. However, what is the role of the \"top 100 logits mean per image\"?
\n
\"top 100 logits mean per image\" is a variation of the classification model using the segmentation model.
\nA similar approach was used in a previous competition, and again the top 100 logits mean was more accurate for binary classification than the classification model.
\n\nCould you elaborate a bit more about the steps to find that?
\n
In binary classification, we have both submissions with a percentile threshold and sub with the percentile converted to a fixed value and used as the threshold. Comparing the execution times of the two submissions, we can see that the percentage of empty mask images differs when there is a change. (since the size of the test set and the validation set are the same)
", "rawMarkdown": ">If I am correct, the purpose of classification models is to determine whether the predicted mask is empty or not, and they are only applied to segmentation models trained on all data. However, what is the role of the \"top 100 logits mean per image\"?\n\n\n\"top 100 logits mean per image\" is a variation of the classification model using the segmentation model.\nA similar approach was used in a previous competition, and again the top 100 logits mean was more accurate for binary classification than the classification model.\n\n>Could you elaborate a bit more about the steps to find that?\n\nIn binary classification, we have both submissions with a percentile threshold and sub with the percentile converted to a fixed value and used as the threshold. Comparing the execution times of the two submissions, we can see that the percentage of empty mask images differs when there is a change. (since the size of the test set and the validation set are the same)", "votes": 2 } ] }, { "id": 2386573, "postDate": "2023-08-12T03:51:09.497Z", "content": "That is very interesting. Thank you for sharing your approach!
", "rawMarkdown": "That is very interesting. Thank you for sharing your approach!\n" }, { "id": 2383895, "postDate": "2023-08-10T16:11:51.447Z", "content": "Congratulations!
\nYour pipeline is very attractive, do you plan to publish your code in the future?
Yes, we plan to release the code for both learning and inference.
\nWe are in the process of organizing it, so please be patient.
I read your Vesuvius Challenge solution and tried to do something similar but what I got was something like score 0.5 ¯_(ツ)_/¯. Looking forward to seeing your code.
\nCongratulations to the gold shake up!
Thank you and congratulations to you too on your solo win. I will read your solution carefully.
\nThis may be due to the limited number of suitable backbones for training in my experiment. Our 2.5D UNet with resnetrs101 resulted in a gain of 0.01 compared to the 2D UNet.
", "rawMarkdown": "Thank you and congratulations to you too on your solo win. I will read your solution carefully.\n\nThis may be due to the limited number of suitable backbones for training in my experiment. Our 2.5D UNet with resnetrs101 resulted in a gain of 0.01 compared to the 2D UNet.", "votes": 1 } ] } ] }, { "id": 2383581, "postDate": "2023-08-10T13:03:12.347Z", "content": "Did you use 2d models for classification as well? What was the performance (accuracy) of pure classifications models?
", "rawMarkdown": "Did you use 2d models for classification as well? What was the performance (accuracy) of pure classifications models?", "replies": [ { "id": 2383595, "postDate": "2023-08-10T13:09:26.180Z", "content": "We did not add a 2D model to the binary classification model; use it as a variation of the segmentation model. The classification accuracy is about 0.95.
", "rawMarkdown": "We did not add a 2D model to the binary classification model; use it as a variation of the segmentation model. The classification accuracy is about 0.95.", "votes": 1 } ] }, { "id": 2383469, "postDate": "2023-08-10T11:29:20.767Z", "content": "Congrats @tattaka
", "rawMarkdown": "Congrats @tattaka " } ], "comments": [ { "id": 2386414, "author_name": "tattaka", "author_url": "", "post_date": "2023-08-11T23:28:54.980000", "content": "Update
\nWe have released the training code(2.5D model part) and the inference notebook for the best submission.
Sorry for late.
\nWe have released the training code for 2D model part:
\nhttps://github.com/tawatawara/kaggle-google-research-identify-contrails-reduce-global-warming
Congratulations and thanks a lot for sharing!
\nLet me ask you a couple of questions 😃:
\n\n\nFrom the time taken to inference, the percentage of empty masks in the entire test was found to be about the same as in validation set. (about 30%)
\n
\n\nIf I am correct, the purpose of classification models is to determine whether the predicted mask is empty or not, and they are only applied to segmentation models trained on all data. However, what is the role of the \"top 100 logits mean per image\"?
\n
\"top 100 logits mean per image\" is a variation of the classification model using the segmentation model.
\nA similar approach was used in a previous competition, and again the top 100 logits mean was more accurate for binary classification than the classification model.
\n\nCould you elaborate a bit more about the steps to find that?
\n
In binary classification, we have both submissions with a percentile threshold and sub with the percentile converted to a fixed value and used as the threshold. Comparing the execution times of the two submissions, we can see that the percentage of empty mask images differs when there is a change. (since the size of the test set and the validation set are the same)
", "votes": 2, "replies": [] } ] }, { "id": 2386573, "author_name": "NA", "author_url": "", "post_date": "2023-08-12T03:51:09.497000", "content": "That is very interesting. Thank you for sharing your approach!
", "votes": 0, "replies": [] }, { "id": 2383895, "author_name": "Never$", "author_url": "", "post_date": "2023-08-10T16:11:51.447000", "content": "Congratulations!
\nYour pipeline is very attractive, do you plan to publish your code in the future?
Yes, we plan to release the code for both learning and inference.
\nWe are in the process of organizing it, so please be patient.
I read your Vesuvius Challenge solution and tried to do something similar but what I got was something like score 0.5 ¯_(ツ)_/¯. Looking forward to seeing your code.
\nCongratulations to the gold shake up!
Thank you and congratulations to you too on your solo win. I will read your solution carefully.
\nThis may be due to the limited number of suitable backbones for training in my experiment. Our 2.5D UNet with resnetrs101 resulted in a gain of 0.01 compared to the 2D UNet.
", "votes": 1, "replies": [] } ] } ] }, { "id": 2383581, "author_name": "Man of the year", "author_url": "", "post_date": "2023-08-10T13:03:12.347000", "content": "Did you use 2d models for classification as well? What was the performance (accuracy) of pure classifications models?
", "votes": 0, "replies": [ { "id": 2383595, "author_name": "tattaka", "author_url": "", "post_date": "2023-08-10T13:09:26.180000", "content": "We did not add a 2D model to the binary classification model; use it as a variation of the segmentation model. The classification accuracy is about 0.95.
", "votes": 1, "replies": [] } ] }, { "id": 2383469, "author_name": "Vijay Joshi", "author_url": "", "post_date": "2023-08-10T11:29:20.767000", "content": "Congrats @tattaka
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2383422": "First of all, we would like to thank the organizers of the competition and the Kaggle team for the competition hosting and all the participants who have shared their knowledge so generously. \n\n# Summary\n* 2 stage pipeline: classification and segmentation\n* ensemble of 2.5D model using 5 or 7 frame and 11ch input 2D model\n* soft label using individual label\n* percentile threshold\n\n# Pipeline\n\n\nThis pipeline has been used frequently in past semantic segmentation competitions. \nThe advantage of this pipeline is that it creates two models, one trained on pos_only and the other on all data, thereby increasing the ensemble effect. \nIt also saves inference time by allowing more models to be assigned to images for which screening has determined that a mask is present. \n\n# Models Detail\n## Common Settings\nBoth 2.5D and 2D models were trained for both hard and soft labels. \nThe soft label is created as follows\n``` python\nlabel = self.np_load(\n os.path.join(\"mask_individual.npy\")\n).astype(np.float32)\nlabel = np.clip((label * 2).sum(-1) / label.shape[-1], 0, 1)\n```\n\nAll models were trained by all the data in train folder and validated by the one in validation folder. \n\n## 2.5D Model\nWe used a modified [2.5D model used in Vesuvius Challenge](https://www.kaggle.com/competitions/vesuvius-challenge-ink-detection/discussion/417255).\nThe main changes are as follows\n* Apply 3D CNN to feature maps of all resolutions\n* 3D For the output of the 3D CNN, feature maps corresponding to the target frame are extracted and given to UNet \n* In the binary classification model, the 3DCNN is applied to the smallest resolution feature map and input to the FC layer in the same way. \n\nBackbones used were resnetrs101, resnest101e, swin_base_patch4_window12, swinv2_base_window16, convnext_base, convnext_large, etc. \nThe input frame is set to 5 or 7 frames, with the target frame in the middle. \nInput resolution of 384 or 512 are used for the model. \nIt was trained using `BCE+Dice*0.25` for the pos only model and `BCE` for the all data model. \nAlso, different augmentation is used for each of the classification model, all data seg model, and pos only seg model. (possibly not optimized and therefore not appropriate). \n\n``` python\n# classification\nalbu.Flip(p=0.5),\nalbu.RandomRotate90(p=0.5),\nalbu.ShiftScaleRotate(p=0.5, scale_limit=0.3, shift_limit=0.1),\nalbu.GridDistortion(num_steps=5, distort_limit=0.3, p=0.5),\nalbu.CoarseDropout(max_height=16, max_width=16, fill_value=255, mask_fill_value=0, p=0.5,)\n\n# all data segmentation\nalbu.ShiftScaleRotate(p=0.5, scale_limit=0.3, shift_limit=0.1),\n\n# pos only segmentation\nalbu.Flip(p=0.25),\nalbu.RandomRotate90(p=0.25),\nalbu.ShiftScaleRotate(p=0.5, scale_limit=0.3, shift_limit=0.1, rotate_limit=0),\nalbu.Rotate(limit=45, p=0.25),\n```\n* EMA(decay=0.998)\n\n## 2D Model\nFor 2D models, we simply used Unet implemented in Segmentation Models Pytorch. The backbones used were res2net50d, regnetz_d8, regnetz_d32, and regnetz_e8.\n\nCharacteristically, the 2D models were trained on 11 channels, i.e. band15 - band14, band14 - band11 and all 9 bands. Learning with this input does not perform well in short epochs (e.g. 20~30 epochs), but in long epochs (specifically 100 epochs), it obtained better validation scores than with false color input.\n\nThe 11ch models also had one curious point. They had poor public score but good private score. I believe this is what pushed us to the prize zone.\n\nOther training settings are as follows(shared by all data seg model and pos only seg one):\n* Input\n * size: 384x384\n * each channels are standardized by subtracting the global mean and dividing by the global variance of the channel\n* Loss: `0.9*BCE + 0.1*Dice`\n* Data Augmentation\n``` python\nalbu.HorizontalFlip(p=0.5),\nalbu.VerticalFlip(p=0.5),\nalbu.ShiftScaleRotate(p=0.5, rotate_limit=90)\nalbu.RandomResizedCrop(p: 1.0, scale=[0.875, 1.0], height=256, width=256)\n```\n* EMA(decay=0.9999)\n\n# Postprocessing\nFrom the time taken to inference, the percentage of empty masks in the entire test was found to be about the same as in validation set. (about 30%). \nTherefore, we used validation set to optimize the percentile threshold for both the classification part and segmentation part. \nAs a result, the percentile threshold is better than the optimized fixed threshold. \n \n\n\n# Not Works\n* pseudo label・mean teacher\n* 2D pretrain -> 2.5D finetuning\n * unstable\n* more augmentations\n* mixup・cutmix・label smoothing\n* TTA\n* efficientnet\n\n# Training Code and Inference Notebook\nWe have released the training code and the inference notebook for the best submission.\n\n* training code (2.5D model part): https://github.com/tattaka/google-research-identify-contrails-reduce-global-warming\n* training code (2D model part): https://github.com/tawatawara/kaggle-google-research-identify-contrails-reduce-global-warming\n* inference code: https://www.kaggle.com/code/tattaka/contrail-submission-ensemble?scriptVersionId=139455030", "2386414": "**Update**\nWe have released the training code(2.5D model part) and the inference notebook for the best submission.\n\n* training code (2.5D model part): https://github.com/tattaka/google-research-identify-contrails-reduce-global-warming\n* inference code: https://www.kaggle.com/code/tattaka/contrail-submission-ensemble?scriptVersionId=139455030", "2386427": "Sorry for late.\n\nWe have released the training code for 2D model part: \nhttps://github.com/tawatawara/kaggle-google-research-identify-contrails-reduce-global-warming", "2393427": "Congratulations and thanks a lot for sharing!\n\nLet me ask you a couple of questions 😃:\n\n- If I am correct, the purpose of classification models is to determine whether the predicted mask is empty or not, and they are only applied to segmentation models trained on all data. However, what is the role of the \"top 100 logits mean per image\"?\n\n>From the time taken to inference, the percentage of empty masks in the entire test was found to be about the same as in validation set. (about 30%)\n\n- Could you elaborate a bit more about the steps to find that?\n", "2386573": "That is very interesting. Thank you for sharing your approach!\n", "2383895": "Congratulations!\nYour pipeline is very attractive, do you plan to publish your code in the future?", "2383581": "Did you use 2d models for classification as well? What was the performance (accuracy) of pure classifications models?", "2383469": "Congrats @tattaka " } }