![\"Ensemble](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5570735%2F98f1fa7711e8116a4b40933f13209bbd%2Ficrgw-v1.drawio.png?generation=1691625764325229&alt=media\")
First, thanks to Google Research for hosting an interesting and well-organized competition. I am always happy to try and build solutions that help reduce our environmental impact. Second, thanks to everyone who participated and shared ideas throughout this competition. I always learn a lot from the code and discussion tabs.
\nMy solution is an ensemble of Unet models with Timm backbones. The optimal threshold for each model was tuned using an out-of-fold validation set. No augmentation was applied to the images during training.
\n
In order of highest to lowest validation dice scores, the backbones I used were maxxvitv2_rmlp_base_rw_384, resnest269, maxvit_base_tf_512, and mit_b4. I also used stochastic weight averaging on the last 5 epochs of training for the maxvit models, and no SWA for the other models. All of my experiments were done on small models and image sizes so that I could experiment quickly. I only scaled up the image sizes and model parameters in the last week of the competition for the final submission.
\nPlease leave any criticisms/suggestions/questions. The more the better :)
\nEach item on this list either did not improve performance or was too computationally expensive for me to pursue.
\nModels: segmentation_models.pytorch,Timm
\nTraining: Lightning-AI
\nLogging: WandB
Finally, congrats to @tascj0 for 1 submission gold. Very impressive!
", "messages": [ { "id": 2382663, "postDate": "2023-08-10T00:04:01.110Z", "content": "First, thanks to Google Research for hosting an interesting and well-organized competition. I am always happy to try and build solutions that help reduce our environmental impact. Second, thanks to everyone who participated and shared ideas throughout this competition. I always learn a lot from the code and discussion tabs.
\nMy solution is an ensemble of Unet models with Timm backbones. The optimal threshold for each model was tuned using an out-of-fold validation set. No augmentation was applied to the images during training.
\n
In order of highest to lowest validation dice scores, the backbones I used were maxxvitv2_rmlp_base_rw_384, resnest269, maxvit_base_tf_512, and mit_b4. I also used stochastic weight averaging on the last 5 epochs of training for the maxvit models, and no SWA for the other models. All of my experiments were done on small models and image sizes so that I could experiment quickly. I only scaled up the image sizes and model parameters in the last week of the competition for the final submission.
\nPlease leave any criticisms/suggestions/questions. The more the better :)
\nEach item on this list either did not improve performance or was too computationally expensive for me to pursue.
\nModels: segmentation_models.pytorch,Timm
\nTraining: Lightning-AI
\nLogging: WandB
Finally, congrats to @tascj0 for 1 submission gold. Very impressive!
", "rawMarkdown": "### Overview\n\nFirst, thanks to Google Research for hosting an interesting and well-organized competition. I am always happy to try and build solutions that help reduce our environmental impact. Second, thanks to everyone who participated and shared ideas throughout this competition. I always learn a lot from the code and discussion tabs.\n\nMy solution is an ensemble of Unet models with Timm backbones. The optimal threshold for each model was tuned using an out-of-fold validation set. No augmentation was applied to the images during training.\n\n\n\nIn order of highest to lowest validation dice scores, the backbones I used were maxxvitv2_rmlp_base_rw_384, resnest269, maxvit_base_tf_512, and mit_b4. I also used stochastic weight averaging on the last 5 epochs of training for the maxvit models, and no SWA for the other models. All of my experiments were done on small models and image sizes so that I could experiment quickly. I only scaled up the image sizes and model parameters in the last week of the competition for the final submission.\n\nPlease leave any criticisms/suggestions/questions. The more the better :)\n\n### Other Attempts\n\nEach item on this list either did not improve performance or was too computationally expensive for me to pursue.\n\n- Losses: Tversky, LogCoshDice, BCE\n- Downsampling/Upsampling Interpolation Methods\n- Removing Islands \n- Openmmlab (struggled to get this working)\n- Deep supervision\n- MANet, Unet++, Unet+++, and others..\n\n### Frameworks\n\nModels: [segmentation_models.pytorch](https://github.com/qubvel/segmentation_models.pytorch),[Timm](https://github.com/huggingface/pytorch-image-models/tree/main/timm)\nTraining: [Lightning-AI](https://github.com/Lightning-AI/lightning)\nLogging: [WandB](https://wandb.ai/)\n\n---\n\nFinally, congrats to @tascj0 for 1 submission gold. Very impressive!\n", "votes": 11 } ], "comments": [], "raw_markdown_by_id": { "2382663": "### Overview\n\nFirst, thanks to Google Research for hosting an interesting and well-organized competition. I am always happy to try and build solutions that help reduce our environmental impact. Second, thanks to everyone who participated and shared ideas throughout this competition. I always learn a lot from the code and discussion tabs.\n\nMy solution is an ensemble of Unet models with Timm backbones. The optimal threshold for each model was tuned using an out-of-fold validation set. No augmentation was applied to the images during training.\n\n\n\nIn order of highest to lowest validation dice scores, the backbones I used were maxxvitv2_rmlp_base_rw_384, resnest269, maxvit_base_tf_512, and mit_b4. I also used stochastic weight averaging on the last 5 epochs of training for the maxvit models, and no SWA for the other models. All of my experiments were done on small models and image sizes so that I could experiment quickly. I only scaled up the image sizes and model parameters in the last week of the competition for the final submission.\n\nPlease leave any criticisms/suggestions/questions. The more the better :)\n\n### Other Attempts\n\nEach item on this list either did not improve performance or was too computationally expensive for me to pursue.\n\n- Losses: Tversky, LogCoshDice, BCE\n- Downsampling/Upsampling Interpolation Methods\n- Removing Islands \n- Openmmlab (struggled to get this working)\n- Deep supervision\n- MANet, Unet++, Unet+++, and others..\n\n### Frameworks\n\nModels: [segmentation_models.pytorch](https://github.com/qubvel/segmentation_models.pytorch),[Timm](https://github.com/huggingface/pytorch-image-models/tree/main/timm)\nTraining: [Lightning-AI](https://github.com/Lightning-AI/lightning)\nLogging: [WandB](https://wandb.ai/)\n\n---\n\nFinally, congrats to @tascj0 for 1 submission gold. Very impressive!\n" } }