![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1943421%2F4e3b69d0933dc6eeb7686885cd97bac1%2FScreen%20Shot%202020-07-15%20at%205.22.54%20PM.png?generation=1594805015581551&alt=media\")
Hints here and there in the discussions suggest that TTA is an important component of improving model scores. My best single model is .86, and TTA+ensembling improved LB to my current position. Taken the different approaches to the problem, there are many ways to do TTA. How are you using TTA?
\n\nTo start off:\nI am currently using an approach somewhere between Iafoss's and Qishen Ha's, tile size 36x256x256.\n- 4xflip TTA and 1/2* (tile size) shift TTA improves score consistently by 0.01 (I am not sure if 4xflip contributes as much as the latter.
\n\nTwo TTAs which don't work (failed to improve LB under my current setup):\n- SafeRotate (rotate + appropriate padding) 0, 30, and 60 degrees. This improves submission time to 2 hrs but does not improve scores.\n- Tile size TTA, let model trained on 36x256x256 predict on 192x192, 320x320, and 384x384 tile sizes. Interestingly the CV is robust to a wide range of tile sizes (see below), but LB seems to think otherwise.
\n\nStill stuck at using shift TTA + flip TTA...how to ever get out of this labyrinth!
\n\n
Code snippet for SafeRotate, in case anyone needs it\n```
\n\ndef SafeRotate(img, degrees=45):\n w, h = img.shape[0], img.shape[1]\n (h, w) = img.shape[:2]\n center = (w // 2, h // 2) #11\n M = cv2.getRotationMatrix2D((w // 2, h // 2), degrees, 1.0)\n M_ = M[:, :2].T\n corners = np.array([[w//2, w//2], [h//2, -h//2]])\n new_corners = M_@corners\n del_y = max(np.abs(new_corners[0])) - w // 2\n del_x = max(np.abs(new_corners[1])) - h // 2\n pad_x, pad_y = int(max(del_x, 0)), int(max(del_y, 0))\n img = np.pad(img, ((pad_x, pad_x), (pad_y, pad_y), (0, 0)), constant_values=255)
\n\n(h, w) = img.shape[:2]\ncenter = (w // 2, h // 2) #11\nM = cv2.getRotationMatrix2D((w // 2, h // 2), degrees, 1.0)\nrotated = cv2.warpAffine(img, M, (w, h), cv2.INTER_LINEAR, borderMode=cv2.BORDER_CONSTANT, borderValue=(255, 255, 255))\nreturn rotated\n```
", "messages": [ { "id": 930219, "postDate": "2020-07-15T09:27:46.970Z", "content": "Hints here and there in the discussions suggest that TTA is an important component of improving model scores. My best single model is .86, and TTA+ensembling improved LB to my current position. Taken the different approaches to the problem, there are many ways to do TTA. How are you using TTA?
\n\nTo start off:\nI am currently using an approach somewhere between Iafoss's and Qishen Ha's, tile size 36x256x256.\n- 4xflip TTA and 1/2* (tile size) shift TTA improves score consistently by 0.01 (I am not sure if 4xflip contributes as much as the latter.
\n\nTwo TTAs which don't work (failed to improve LB under my current setup):\n- SafeRotate (rotate + appropriate padding) 0, 30, and 60 degrees. This improves submission time to 2 hrs but does not improve scores.\n- Tile size TTA, let model trained on 36x256x256 predict on 192x192, 320x320, and 384x384 tile sizes. Interestingly the CV is robust to a wide range of tile sizes (see below), but LB seems to think otherwise.
\n\nStill stuck at using shift TTA + flip TTA...how to ever get out of this labyrinth!
\n\n
Code snippet for SafeRotate, in case anyone needs it\n```
\n\ndef SafeRotate(img, degrees=45):\n w, h = img.shape[0], img.shape[1]\n (h, w) = img.shape[:2]\n center = (w // 2, h // 2) #11\n M = cv2.getRotationMatrix2D((w // 2, h // 2), degrees, 1.0)\n M_ = M[:, :2].T\n corners = np.array([[w//2, w//2], [h//2, -h//2]])\n new_corners = M_@corners\n del_y = max(np.abs(new_corners[0])) - w // 2\n del_x = max(np.abs(new_corners[1])) - h // 2\n pad_x, pad_y = int(max(del_x, 0)), int(max(del_y, 0))\n img = np.pad(img, ((pad_x, pad_x), (pad_y, pad_y), (0, 0)), constant_values=255)
\n\n(h, w) = img.shape[:2]\ncenter = (w // 2, h // 2) #11\nM = cv2.getRotationMatrix2D((w // 2, h // 2), degrees, 1.0)\nrotated = cv2.warpAffine(img, M, (w, h), cv2.INTER_LINEAR, borderMode=cv2.BORDER_CONSTANT, borderValue=(255, 255, 255))\nreturn rotated\n```
", "rawMarkdown": "Hints here and there in the discussions suggest that TTA is an important component of improving model scores. My best single model is .86, and TTA+ensembling improved LB to my current position. Taken the different approaches to the problem, there are many ways to do TTA. How are you using TTA?\n\nTo start off:\nI am currently using an approach somewhere between Iafoss's and Qishen Ha's, tile size 36x256x256.\n- 4xflip TTA and 1/2* (tile size) shift TTA **improves score consistently by 0.01** (I am not sure if 4xflip contributes as much as the latter.\n\nTwo TTAs which **don't work** (failed to improve LB under my current setup):\n- SafeRotate (rotate + appropriate padding) 0, 30, and 60 degrees. This improves submission time to 2 hrs but does not improve scores.\n- Tile size TTA, let model trained on 36x256x256 predict on 192x192, 320x320, and 384x384 tile sizes. Interestingly the CV is robust to a wide range of tile sizes (see below), but LB seems to think otherwise. \n\nStill stuck at using shift TTA + flip TTA...how to ever get out of this labyrinth!\n\n\n\nCode snippet for SafeRotate, in case anyone needs it\n```\n# Pad before rotation such that no information is lost\ndef SafeRotate(img, degrees=45):\n w, h = img.shape[0], img.shape[1]\n (h, w) = img.shape[:2]\n center = (w // 2, h // 2) #11\n M = cv2.getRotationMatrix2D((w // 2, h // 2), degrees, 1.0)\n M_ = M[:, :2].T\n corners = np.array([[w//2, w//2], [h//2, -h//2]])\n new_corners = M_@corners\n del_y = max(np.abs(new_corners[0])) - w // 2\n del_x = max(np.abs(new_corners[1])) - h // 2\n pad_x, pad_y = int(max(del_x, 0)), int(max(del_y, 0))\n img = np.pad(img, ((pad_x, pad_x), (pad_y, pad_y), (0, 0)), constant_values=255)\n \n (h, w) = img.shape[:2]\n center = (w // 2, h // 2) #11\n M = cv2.getRotationMatrix2D((w // 2, h // 2), degrees, 1.0)\n rotated = cv2.warpAffine(img, M, (w, h), cv2.INTER_LINEAR, borderMode=cv2.BORDER_CONSTANT, borderValue=(255, 255, 255))\n return rotated\n```", "votes": 22 }, { "id": 935257, "postDate": "2020-07-19T07:51:54.600Z", "content": "I tried scaling down tile size but it didn't work for me. My LB score is not affected by changing tile size.
", "rawMarkdown": "I tried scaling down tile size but it didn't work for me. My LB score is not affected by changing tile size.", "votes": 1 }, { "id": 937208, "postDate": "2020-07-20T20:11:58.227Z", "content": "DeepMind has a paper about detecting breast cancer from images. In the supplementary methods section they mention that they use 500 rounds of TTA using the same augmentations that were used during training. This is quite a bit more than I have ever seen before.
", "rawMarkdown": "DeepMind has a paper about detecting breast cancer from images. In the [supplementary methods](https://static-content.springer.com/esm/art%3A10.1038%2Fs41586-019-1799-6/MediaObjects/41586_2019_1799_MOESM1_ESM.pdf) section they mention that they use 500 rounds of TTA using the same augmentations that were used during training. This is quite a bit more than I have ever seen before.\n\n", "votes": 2, "replies": [ { "id": 937447, "postDate": "2020-07-21T03:04:51.867Z", "content": "Yeah, that's pretty nuts. They'd have a whole distribution per image. I guess maybe looking at the shape of that distro or feeding it into a secondary model might help though.
", "rawMarkdown": "Yeah, that's pretty nuts. They'd have a whole distribution per image. I guess maybe looking at the shape of that distro or feeding it into a secondary model might help though." } ] }, { "id": 931072, "postDate": "2020-07-16T00:22:27.813Z", "content": "Good sharing! May I ask you how many augmentations did you use in inference? What I mean is every time you choose 1 agumentation and do inference or you will enumerate all the combination of augmentations? (e.g. if you have n, you will apply 2^n augmentations)\nFor me, shiftscalerotate hurt my score, so I use flip(h, v), transpose, etc. And the way I choose the augmentations is by np.random.choice, lower=2, upper=len(tta-augmentations).
\n\nSide Note: tta improve my lb score <0.01(because the score isn't change, but I jump places :D)
", "rawMarkdown": "Good sharing! May I ask you how many augmentations did you use in inference? What I mean is every time you choose 1 agumentation and do inference or you will enumerate all the combination of augmentations? (e.g. if you have n, you will apply 2^n augmentations)\nFor me, shiftscalerotate hurt my score, so I use flip(h, v), transpose, etc. And the way I choose the augmentations is by np.random.choice, lower=2, upper=len(tta-augmentations).\n\nSide Note: tta improve my lb score <0.01(because the score isn't change, but I jump places :D)", "replies": [ { "id": 931155, "postDate": "2020-07-16T03:03:57.420Z", "content": "@cnzengshiyuan I use 8 augmentations for inference, (4x flipping) + tile shifting. Basically model sees each slide 8 times (in different forms) to make its prediction.
", "rawMarkdown": "@cnzengshiyuan I use 8 augmentations for inference, (4x flipping) + tile shifting. Basically model sees each slide 8 times (in different forms) to make its prediction.", "votes": 1 }, { "id": 931158, "postDate": "2020-07-16T03:06:04.510Z", "content": "I got it! Thanks for you reply😄
", "rawMarkdown": "I got it! Thanks for you reply😄 " }, { "id": 931780, "postDate": "2020-07-16T12:45:18.973Z", "content": "have you tried 8x flipping and how's the effect? I saw this in one paper.
", "rawMarkdown": "have you tried 8x flipping and how's the effect? I saw this in one paper.", "isDeleted": true }, { "id": 932071, "postDate": "2020-07-16T17:10:20.560Z", "content": "@roguekk007 \nwhat you mean by 1/2* (tile size) shift TTA
\nis it there in Albu?
@jaideepvalani No. It's self-implemented. See the code snippet below ;)
", "rawMarkdown": "@jaideepvalani No. It's self-implemented. See the code snippet below ;)" } ] }, { "id": 930252, "postDate": "2020-07-15T09:56:38.880Z", "content": "Hi @roguekk007 . What do you mean by \"1/2* (tile size) shift TTA\" ?
", "rawMarkdown": "Hi @roguekk007 . What do you mean by \"1/2* (tile size) shift TTA\" ?", "replies": [ { "id": 930262, "postDate": "2020-07-15T10:03:48.760Z", "content": "@vladvdv Maybe I have not made it clear enough. It is the same approach as used in Ha's public inference kernel Here. Basically, pad the whole image on both sides by 128 (if the tile size is 256) before tiling. This will generate different tiles. Hope this helps ;)
", "rawMarkdown": "@vladvdv Maybe I have not made it clear enough. It is the same approach as used in Ha's public inference kernel [Here](https://www.kaggle.com/haqishen/panda-inference-w-36-tiles-256). Basically, pad the whole image on both sides by 128 (if the tile size is 256) before tiling. This will generate different tiles. Hope this helps ;)", "votes": 3 }, { "id": 930320, "postDate": "2020-07-15T11:04:51.190Z", "content": "Got it. Good idea !
", "rawMarkdown": "Got it. Good idea !" }, { "id": 931097, "postDate": "2020-07-16T01:12:21.637Z", "content": "@roguekk007 Would you kindly elaborate in this part?
\n\n\npad the whole image on both sides by 128 (if the tile size is 256) before tiling
\n
I'm very new and tiling it's something that it's not in books or in pretty good tutorials.
\nP.S. Thanks for share your about TTA.
", "rawMarkdown": "@roguekk007 Would you kindly elaborate in this part?\n> pad the whole image on both sides by 128 (if the tile size is 256) before tiling\n\nI'm very new and **tiling** it's something that it's not in books or in pretty good tutorials.\n\nP.S. Thanks for share your about TTA. " }, { "id": 931121, "postDate": "2020-07-16T02:05:25.737Z", "content": "I guess it means that if you pad the whole slide image with extra (half the tile size) on both sides, all the tiles cropped will be shifted by (half the tile size) compared to without doing the padding. You can imagine that the very first tile at the top-left corner will be filled with extra pads of 3/4 of the tile(correct me if I'm wrong:D), hope that explains.
", "rawMarkdown": "I guess it means that if you pad the whole slide image with extra (half the tile size) on both sides, all the tiles cropped will be shifted by (half the tile size) compared to without doing the padding. You can imagine that the very first tile at the top-left corner will be filled with extra pads of 3/4 of the tile(correct me if I'm wrong:D), hope that explains.", "votes": 2 }, { "id": 931154, "postDate": "2020-07-16T03:02:50.560Z", "content": "@printer7580 @hiramcho That is a very clear clarification. Here is the augment snippet of my inference notebook.
\n\nI usually iterate from 0 to one, thus using two augmentations (flipping is done on tensors)\n\ndef Augment(img, mode=0):\n if mode % 2 == 1:\n img = np.pad(img, ((args['tile_size']//2, args['tile_size']//2), (args['tile_size']//2, args['tile_size']//2), (0, 0)), constant_values=255)\n return img
\n
Thanks to both of you.
", "rawMarkdown": "Thanks to both of you." } ] }, { "id": 931778, "postDate": "2020-07-16T12:44:25.047Z", "content": "pretty smart, I only think of keeping scaling down tile size rather than scale up. Gonna try. Thanks for sharing.
", "rawMarkdown": "pretty smart, I only think of keeping scaling down tile size rather than scale up. Gonna try. Thanks for sharing. ", "isDeleted": true, "replies": [ { "id": 931835, "postDate": "2020-07-16T13:37:02.083Z", "content": "@muerbingsha Glad to be of help;) do kindly share if it works out for you. Didn't find it working as of now.
", "rawMarkdown": "@muerbingsha Glad to be of help;) do kindly share if it works out for you. Didn't find it working as of now." } ] } ], "comments": [ { "id": 935257, "author_name": "RAHUL SINGH INDA", "author_url": "", "post_date": "2020-07-19T07:51:54.600000", "content": "I tried scaling down tile size but it didn't work for me. My LB score is not affected by changing tile size.
", "votes": 1, "replies": [] }, { "id": 937208, "author_name": "Josh Varty", "author_url": "", "post_date": "2020-07-20T20:11:58.227000", "content": "DeepMind has a paper about detecting breast cancer from images. In the supplementary methods section they mention that they use 500 rounds of TTA using the same augmentations that were used during training. This is quite a bit more than I have ever seen before.
", "votes": 2, "replies": [ { "id": 937447, "author_name": "عثمان", "author_url": "", "post_date": "2020-07-21T03:04:51.867000", "content": "Yeah, that's pretty nuts. They'd have a whole distribution per image. I guess maybe looking at the shape of that distro or feeding it into a secondary model might help though.
", "votes": 0, "replies": [] } ] }, { "id": 931072, "author_name": "Shiyuan Zeng", "author_url": "", "post_date": "2020-07-16T00:22:27.813000", "content": "Good sharing! May I ask you how many augmentations did you use in inference? What I mean is every time you choose 1 agumentation and do inference or you will enumerate all the combination of augmentations? (e.g. if you have n, you will apply 2^n augmentations)\nFor me, shiftscalerotate hurt my score, so I use flip(h, v), transpose, etc. And the way I choose the augmentations is by np.random.choice, lower=2, upper=len(tta-augmentations).
\n\nSide Note: tta improve my lb score <0.01(because the score isn't change, but I jump places :D)
", "votes": 0, "replies": [ { "id": 931155, "author_name": "Nicholas Lyu", "author_url": "", "post_date": "2020-07-16T03:03:57.420000", "content": "@cnzengshiyuan I use 8 augmentations for inference, (4x flipping) + tile shifting. Basically model sees each slide 8 times (in different forms) to make its prediction.
", "votes": 1, "replies": [] }, { "id": 931158, "author_name": "Shiyuan Zeng", "author_url": "", "post_date": "2020-07-16T03:06:04.510000", "content": "I got it! Thanks for you reply😄
", "votes": 0, "replies": [] }, { "id": 931780, "author_name": "", "author_url": "", "post_date": "2020-07-16T12:45:18.973000", "content": "have you tried 8x flipping and how's the effect? I saw this in one paper.
", "votes": 0, "replies": [] }, { "id": 932071, "author_name": "Jaideep", "author_url": "", "post_date": "2020-07-16T17:10:20.560000", "content": "@roguekk007 \nwhat you mean by 1/2* (tile size) shift TTA
\nis it there in Albu?
@jaideepvalani No. It's self-implemented. See the code snippet below ;)
", "votes": 0, "replies": [] } ] }, { "id": 930252, "author_name": "Vlad Vaduva", "author_url": "", "post_date": "2020-07-15T09:56:38.880000", "content": "Hi @roguekk007 . What do you mean by \"1/2* (tile size) shift TTA\" ?
", "votes": 0, "replies": [ { "id": 930262, "author_name": "Nicholas Lyu", "author_url": "", "post_date": "2020-07-15T10:03:48.760000", "content": "@vladvdv Maybe I have not made it clear enough. It is the same approach as used in Ha's public inference kernel Here. Basically, pad the whole image on both sides by 128 (if the tile size is 256) before tiling. This will generate different tiles. Hope this helps ;)
", "votes": 3, "replies": [] }, { "id": 930320, "author_name": "Vlad Vaduva", "author_url": "", "post_date": "2020-07-15T11:04:51.190000", "content": "Got it. Good idea !
", "votes": 0, "replies": [] }, { "id": 931097, "author_name": "Hiram Coria 🧬", "author_url": "", "post_date": "2020-07-16T01:12:21.637000", "content": "@roguekk007 Would you kindly elaborate in this part?
\n\n\npad the whole image on both sides by 128 (if the tile size is 256) before tiling
\n
I'm very new and tiling it's something that it's not in books or in pretty good tutorials.
\nP.S. Thanks for share your about TTA.
", "votes": 0, "replies": [] }, { "id": 931121, "author_name": "eyhsu", "author_url": "", "post_date": "2020-07-16T02:05:25.737000", "content": "I guess it means that if you pad the whole slide image with extra (half the tile size) on both sides, all the tiles cropped will be shifted by (half the tile size) compared to without doing the padding. You can imagine that the very first tile at the top-left corner will be filled with extra pads of 3/4 of the tile(correct me if I'm wrong:D), hope that explains.
", "votes": 2, "replies": [] }, { "id": 931154, "author_name": "Nicholas Lyu", "author_url": "", "post_date": "2020-07-16T03:02:50.560000", "content": "@printer7580 @hiramcho That is a very clear clarification. Here is the augment snippet of my inference notebook.
\n\nI usually iterate from 0 to one, thus using two augmentations (flipping is done on tensors)\n\ndef Augment(img, mode=0):\n if mode % 2 == 1:\n img = np.pad(img, ((args['tile_size']//2, args['tile_size']//2), (args['tile_size']//2, args['tile_size']//2), (0, 0)), constant_values=255)\n return img
\n
Thanks to both of you.
", "votes": 0, "replies": [] } ] }, { "id": 931778, "author_name": "", "author_url": "", "post_date": "2020-07-16T12:44:25.047000", "content": "pretty smart, I only think of keeping scaling down tile size rather than scale up. Gonna try. Thanks for sharing.
", "votes": 0, "replies": [ { "id": 931835, "author_name": "Nicholas Lyu", "author_url": "", "post_date": "2020-07-16T13:37:02.083000", "content": "@muerbingsha Glad to be of help;) do kindly share if it works out for you. Didn't find it working as of now.
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "930219": "Hints here and there in the discussions suggest that TTA is an important component of improving model scores. My best single model is .86, and TTA+ensembling improved LB to my current position. Taken the different approaches to the problem, there are many ways to do TTA. How are you using TTA?\n\nTo start off:\nI am currently using an approach somewhere between Iafoss's and Qishen Ha's, tile size 36x256x256.\n- 4xflip TTA and 1/2* (tile size) shift TTA **improves score consistently by 0.01** (I am not sure if 4xflip contributes as much as the latter.\n\nTwo TTAs which **don't work** (failed to improve LB under my current setup):\n- SafeRotate (rotate + appropriate padding) 0, 30, and 60 degrees. This improves submission time to 2 hrs but does not improve scores.\n- Tile size TTA, let model trained on 36x256x256 predict on 192x192, 320x320, and 384x384 tile sizes. Interestingly the CV is robust to a wide range of tile sizes (see below), but LB seems to think otherwise. \n\nStill stuck at using shift TTA + flip TTA...how to ever get out of this labyrinth!\n\n\n\nCode snippet for SafeRotate, in case anyone needs it\n```\n# Pad before rotation such that no information is lost\ndef SafeRotate(img, degrees=45):\n w, h = img.shape[0], img.shape[1]\n (h, w) = img.shape[:2]\n center = (w // 2, h // 2) #11\n M = cv2.getRotationMatrix2D((w // 2, h // 2), degrees, 1.0)\n M_ = M[:, :2].T\n corners = np.array([[w//2, w//2], [h//2, -h//2]])\n new_corners = M_@corners\n del_y = max(np.abs(new_corners[0])) - w // 2\n del_x = max(np.abs(new_corners[1])) - h // 2\n pad_x, pad_y = int(max(del_x, 0)), int(max(del_y, 0))\n img = np.pad(img, ((pad_x, pad_x), (pad_y, pad_y), (0, 0)), constant_values=255)\n \n (h, w) = img.shape[:2]\n center = (w // 2, h // 2) #11\n M = cv2.getRotationMatrix2D((w // 2, h // 2), degrees, 1.0)\n rotated = cv2.warpAffine(img, M, (w, h), cv2.INTER_LINEAR, borderMode=cv2.BORDER_CONSTANT, borderValue=(255, 255, 255))\n return rotated\n```", "935257": "I tried scaling down tile size but it didn't work for me. My LB score is not affected by changing tile size.", "937208": "DeepMind has a paper about detecting breast cancer from images. In the [supplementary methods](https://static-content.springer.com/esm/art%3A10.1038%2Fs41586-019-1799-6/MediaObjects/41586_2019_1799_MOESM1_ESM.pdf) section they mention that they use 500 rounds of TTA using the same augmentations that were used during training. This is quite a bit more than I have ever seen before.\n\n", "931072": "Good sharing! May I ask you how many augmentations did you use in inference? What I mean is every time you choose 1 agumentation and do inference or you will enumerate all the combination of augmentations? (e.g. if you have n, you will apply 2^n augmentations)\nFor me, shiftscalerotate hurt my score, so I use flip(h, v), transpose, etc. And the way I choose the augmentations is by np.random.choice, lower=2, upper=len(tta-augmentations).\n\nSide Note: tta improve my lb score <0.01(because the score isn't change, but I jump places :D)", "930252": "Hi @roguekk007 . What do you mean by \"1/2* (tile size) shift TTA\" ?", "931778": "pretty smart, I only think of keeping scaling down tile size rather than scale up. Gonna try. Thanks for sharing. " } }