I've learned a bunch from this contest - big thanks to the organizers and all who took a shot at solving the problem, and congrats to the winners! \nMy approach is nothing novel but here goes.
\n\nMy solution was an ensemble of semi-supervised ImageNet models based on @Iafoss' concat tile pooling
\n\nThe only thing I changed was removing the final dropout layer and training the head for a few epochs before unfreezing the model.\nAnd of course \n@haqishen's genius BCE loss.
\n\nI generated tile sizes 256 and 384 from the medium resolution based on @akensert's optimized tiling.
\n\nThere was a performance trade-off between selecting more tiles and larger batch size so I settled on randomly sampling k tiles from the top N tiles for each epoch.
\n\n| model | input size | k | N | bs\n| ----- | --- | -- | -- | --\n| resnext50_32x4d_ssl | 192 x 192 | 28 | 40 | 10\n| resnext50_32x4d_ssl | 256 x 256 | 32 | 40 | 6\n| resnext50_32x4d_swsl | 384 x 384 | 14 | 24| 6
\n\nTraining with a smaller size(128) seemed to overfit while the larger size(512) was unstable because I had to lower the batch size
\n\nHue/saturation augmentations didn't improve CV so I stuck to affine transforms - rotations, flips, zoom, warp - all from the default fastai transforms. \nRandomly shuffling the tiles every other epoch also seemed to help.
", "messages": [ { "id": 941472, "postDate": "2020-07-23T08:30:10.803Z", "content": "I've learned a bunch from this contest - big thanks to the organizers and all who took a shot at solving the problem, and congrats to the winners! \nMy approach is nothing novel but here goes.
\n\nMy solution was an ensemble of semi-supervised ImageNet models based on @Iafoss' concat tile pooling
\n\nThe only thing I changed was removing the final dropout layer and training the head for a few epochs before unfreezing the model.\nAnd of course \n@haqishen's genius BCE loss.
\n\nI generated tile sizes 256 and 384 from the medium resolution based on @akensert's optimized tiling.
\n\nThere was a performance trade-off between selecting more tiles and larger batch size so I settled on randomly sampling k tiles from the top N tiles for each epoch.
\n\n| model | input size | k | N | bs\n| ----- | --- | -- | -- | --\n| resnext50_32x4d_ssl | 192 x 192 | 28 | 40 | 10\n| resnext50_32x4d_ssl | 256 x 256 | 32 | 40 | 6\n| resnext50_32x4d_swsl | 384 x 384 | 14 | 24| 6
\n\nTraining with a smaller size(128) seemed to overfit while the larger size(512) was unstable because I had to lower the batch size
\n\nHue/saturation augmentations didn't improve CV so I stuck to affine transforms - rotations, flips, zoom, warp - all from the default fastai transforms. \nRandomly shuffling the tiles every other epoch also seemed to help.
", "rawMarkdown": "I've learned a bunch from this contest - big thanks to the organizers and all who took a shot at solving the problem, and congrats to the winners! \nMy approach is nothing novel but here goes.\n\n### Models\nMy solution was an ensemble of semi-supervised ImageNet models based on @Iafoss' [concat tile pooling ](https://www.kaggle.com/iafoss/panda-concat-tile-pooling-starter-0-79-lb)\n\n- resnext50_32x4d_ssl: input 192x192, 256x256\n- resnext50_32x4d_swsl: input size 384x384\n\nThe only thing I changed was removing the final dropout layer and training the head for a few epochs before unfreezing the model.\nAnd of course \n@haqishen's genius [BCE loss](https://www.kaggle.com/haqishen/train-efficientnet-b0-w-36-tiles-256-lb0-87).\n\n### Data\nI generated tile sizes 256 and 384 from the medium resolution based on @akensert's [optimized tiling](https://www.kaggle.com/akensert/panda-optimized-tiling-tf-data-dataset).\n\nThere was a performance trade-off between selecting more tiles and larger batch size so I settled on randomly sampling *k* tiles from the top *N* tiles for each epoch.\n\n| model | input size | k | N | bs\n| ----- | --- | -- | -- | --\n| resnext50_32x4d_ssl | 192 x 192 | 28 | 40 | 10\n| resnext50_32x4d_ssl | 256 x 256 | 32 | 40 | 6\n| resnext50_32x4d_swsl | 384 x 384 | 14 | 24| 6\n\nTraining with a smaller size(128) seemed to overfit while the larger size(512) was unstable because I had to lower the batch size\n\n### Augmentations\nHue/saturation augmentations didn't improve CV so I stuck to affine transforms - rotations, flips, zoom, warp - all from the default fastai transforms. \nRandomly shuffling the tiles every other epoch also seemed to help.\n", "votes": 11 }, { "id": 1217181, "postDate": "2021-02-24T20:17:06.890Z", "content": "Awesome approach! Would you like to share your codes?
", "rawMarkdown": "Awesome approach! Would you like to share your codes? " } ], "comments": [ { "id": 1217181, "author_name": "Swikwislkdjc", "author_url": "", "post_date": "2021-02-24T20:17:06.890000", "content": "Awesome approach! Would you like to share your codes?
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "941472": "I've learned a bunch from this contest - big thanks to the organizers and all who took a shot at solving the problem, and congrats to the winners! \nMy approach is nothing novel but here goes.\n\n### Models\nMy solution was an ensemble of semi-supervised ImageNet models based on @Iafoss' [concat tile pooling ](https://www.kaggle.com/iafoss/panda-concat-tile-pooling-starter-0-79-lb)\n\n- resnext50_32x4d_ssl: input 192x192, 256x256\n- resnext50_32x4d_swsl: input size 384x384\n\nThe only thing I changed was removing the final dropout layer and training the head for a few epochs before unfreezing the model.\nAnd of course \n@haqishen's genius [BCE loss](https://www.kaggle.com/haqishen/train-efficientnet-b0-w-36-tiles-256-lb0-87).\n\n### Data\nI generated tile sizes 256 and 384 from the medium resolution based on @akensert's [optimized tiling](https://www.kaggle.com/akensert/panda-optimized-tiling-tf-data-dataset).\n\nThere was a performance trade-off between selecting more tiles and larger batch size so I settled on randomly sampling *k* tiles from the top *N* tiles for each epoch.\n\n| model | input size | k | N | bs\n| ----- | --- | -- | -- | --\n| resnext50_32x4d_ssl | 192 x 192 | 28 | 40 | 10\n| resnext50_32x4d_ssl | 256 x 256 | 32 | 40 | 6\n| resnext50_32x4d_swsl | 384 x 384 | 14 | 24| 6\n\nTraining with a smaller size(128) seemed to overfit while the larger size(512) was unstable because I had to lower the batch size\n\n### Augmentations\nHue/saturation augmentations didn't improve CV so I stuck to affine transforms - rotations, flips, zoom, warp - all from the default fastai transforms. \nRandomly shuffling the tiles every other epoch also seemed to help.\n", "1217181": "Awesome approach! Would you like to share your codes? " } }