![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2590337%2Fc04e6ac0c033d17bac52b6405ea828ee%2Fphoto_2023-08-05_00-49-53.jpg?generation=1692294981919634&alt=media\")
Welcome! Сongratulations to everyone on finishing this great competition. It was a honor for us to compete with you all. Makes me happy to see, that AI solutions can be beneficial for the environment and nature.
\nOur team ( @slavabarkov and me) would like to present our approach. We didn't get the SOTA result, but I believe, that we got some insights.
\nOur best solution was an ensemble of several models with equal weights: efficientnet-b5, b6, b7, b8 and eca_nfnet_l2 with the Unet decoder. All models were trained on 512x512 images (Ash Color images, shout-out to @shashwatraman) and DICE loss. We used threshold=0.3 for the pixels final prediction. It got 0.685 on the public lb and 0.681 on the private.
\nOur validation score correlated rather nice with the public lb. We used the StratifiedKFold validation (4 folds), stratified by the masks size. I'll describe our approach in the next paragraph more precisely.
\nWe had a straightforward robust approach, that has a nice performance, without using the SOTA models. We developed a thoughtful validation strategy, that can be used in the future competitions.
\nWe also had ideas that we did not manage to work right (we joined the competition last week). However, according to the winning solutions, we moved in the right direction.
\nI didn't like the idea of random folds, since folds might contain the images of different complexity level. Some folds might contain too many empty masks or too big masks. I thought, that models might struggle with prediction of empty masks, for example. Therefore, it was a nice approach to divide images equally on folds, based on their masks size. One needs just to sum the total amount of masks pixels. Then you should create bins to label ranges of masks sizes. For empty masks we created a separate bin. Then you just stratify data, based on these labels.
\nCode example:
\n\nskf = StratifiedKFold(\n n_splits=config[][],\n shuffle=,\n random_state=config[][],\n)\n\n\ndf[] = pd.qcut(\n df[df[] > ][],\n q=config[][] - ,\n labels=,\n)\n\ndf.loc[df[] == , ] = -\n\n\n\n fold_number, (train_index, val_index) (\n skf.split(df, df[])\n):\n df.loc[val_index, ] = (fold_number)\n\n\ndf.drop(columns=[], inplace=)\nFor segmentation models we used the smp torch library. We have tried different backbones from the timm library. However, we found out that an efficientnet family was the best, resnet was not successful in our case. We also liked the nfnet models, they were faster and also could be trained on bigger image sizes. However, 512x512 worked the best. We also found out, that models hyperparameters was not that important: lr, scheduler - does not mean too much.
\nUnet decoder also was the best one, others were worse or showed almost the same performance.
\nThat was a really cool idea to generate the similar background images by generative networks and draw masks on them to pretrain the models. I think that it is a very perspective idea in future works. In our case it didn't work good, trained models showed nice performance on these images, but this didn't improve the quality on the competition dataset.
\n
Instead of using the ground truth masks, we can average the predictions by annotators. We preprocessed the data and then just didn't have the time to test it. But this can work nice, since there are controversial images in the dataset.
\nThat's a great approach to make the data bigger. We did this for the 2,3,5,6 frames and trained models with the DICE loss. It was a total mistake. We trained on binary masks, but we should use BCE loss and soft labels in this case. Another mistake was that we used the whole ensemble to label this data, so we could not really validate the models on previous folds. Our validation score improved, but the lb did not. I believe it is due to the hard labels and knowledge (data) leakage from the models trained on other folds.
\nI wanted to train a classifier model just to predict, whether the image has the empty mask or not. I was really concerned about the empty masks, since the wrong predictions on these images will get the high error. I tried different backbones, got about 0.9 accuracy and in the pipeline it didn't work well then. Didn't have time to improve this step.
\nWe have created a python project with loggers (wandb), which runs from the command line with YAML config. You can find it on the github: https://github.com/25icecreamflavors/contrails
\nWelcome! Сongratulations to everyone on finishing this great competition. It was a honor for us to compete with you all. Makes me happy to see, that AI solutions can be beneficial for the environment and nature.
\nOur team ( @slavabarkov and me) would like to present our approach. We didn't get the SOTA result, but I believe, that we got some insights.
\nOur best solution was an ensemble of several models with equal weights: efficientnet-b5, b6, b7, b8 and eca_nfnet_l2 with the Unet decoder. All models were trained on 512x512 images (Ash Color images, shout-out to @shashwatraman) and DICE loss. We used threshold=0.3 for the pixels final prediction. It got 0.685 on the public lb and 0.681 on the private.
\nOur validation score correlated rather nice with the public lb. We used the StratifiedKFold validation (4 folds), stratified by the masks size. I'll describe our approach in the next paragraph more precisely.
\nWe had a straightforward robust approach, that has a nice performance, without using the SOTA models. We developed a thoughtful validation strategy, that can be used in the future competitions.
\nWe also had ideas that we did not manage to work right (we joined the competition last week). However, according to the winning solutions, we moved in the right direction.
\nI didn't like the idea of random folds, since folds might contain the images of different complexity level. Some folds might contain too many empty masks or too big masks. I thought, that models might struggle with prediction of empty masks, for example. Therefore, it was a nice approach to divide images equally on folds, based on their masks size. One needs just to sum the total amount of masks pixels. Then you should create bins to label ranges of masks sizes. For empty masks we created a separate bin. Then you just stratify data, based on these labels.
\nCode example:
\n\nskf = StratifiedKFold(\n n_splits=config[][],\n shuffle=,\n random_state=config[][],\n)\n\n\ndf[] = pd.qcut(\n df[df[] > ][],\n q=config[][] - ,\n labels=,\n)\n\ndf.loc[df[] == , ] = -\n\n\n\n fold_number, (train_index, val_index) (\n skf.split(df, df[])\n):\n df.loc[val_index, ] = (fold_number)\n\n\ndf.drop(columns=[], inplace=)\nFor segmentation models we used the smp torch library. We have tried different backbones from the timm library. However, we found out that an efficientnet family was the best, resnet was not successful in our case. We also liked the nfnet models, they were faster and also could be trained on bigger image sizes. However, 512x512 worked the best. We also found out, that models hyperparameters was not that important: lr, scheduler - does not mean too much.
\nUnet decoder also was the best one, others were worse or showed almost the same performance.
\nThat was a really cool idea to generate the similar background images by generative networks and draw masks on them to pretrain the models. I think that it is a very perspective idea in future works. In our case it didn't work good, trained models showed nice performance on these images, but this didn't improve the quality on the competition dataset.
\n
Instead of using the ground truth masks, we can average the predictions by annotators. We preprocessed the data and then just didn't have the time to test it. But this can work nice, since there are controversial images in the dataset.
\nThat's a great approach to make the data bigger. We did this for the 2,3,5,6 frames and trained models with the DICE loss. It was a total mistake. We trained on binary masks, but we should use BCE loss and soft labels in this case. Another mistake was that we used the whole ensemble to label this data, so we could not really validate the models on previous folds. Our validation score improved, but the lb did not. I believe it is due to the hard labels and knowledge (data) leakage from the models trained on other folds.
\nI wanted to train a classifier model just to predict, whether the image has the empty mask or not. I was really concerned about the empty masks, since the wrong predictions on these images will get the high error. I tried different backbones, got about 0.9 accuracy and in the pipeline it didn't work well then. Didn't have time to improve this step.
\nWe have created a python project with loggers (wandb), which runs from the command line with YAML config. You can find it on the github: https://github.com/25icecreamflavors/contrails
\n