![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F1212661%2Fe6fe32d759a28480343001aa3c661723%2FTILE.png?generation=1588094975239255&alt=media\")
Welcome to Prostate cANcer graDe Assessment (PANDA) Challenge. I have prepared a starter pack based on my idea of concat tile pooling (see the image below and this kernel). Specifically, I provide a data preprocessing kernel creating 16 128x128 tiles for each image selected based on the number of tissue pixels, model training kernel with detailed description of the proposed method, and inference kernel, which reached 0.79 public LB. I hope you will enjoy the competition and find these kernels to be useful to start with the challenge.
\n\n
Welcome to Prostate cANcer graDe Assessment (PANDA) Challenge. I have prepared a starter pack based on my idea of concat tile pooling (see the image below and this kernel). Specifically, I provide a data preprocessing kernel creating 16 128x128 tiles for each image selected based on the number of tissue pixels, model training kernel with detailed description of the proposed method, and inference kernel, which reached 0.79 public LB. I hope you will enjoy the competition and find these kernels to be useful to start with the challenge.
\n\n
Just an update on the number N:\n\nN=12 - 0.843 CV\n[[2290 424 106 41 11 1]\n [ 463 1563 455 116 17 2]\n [ 60 396 547 261 67 10]\n [ 26 81 213 397 395 114]\n [ 29 65 128 182 450 391]\n [ 13 31 42 91 270 768]]\n\n\nN=8 - 0.830 CV\n[[2176 519 126 37 12 3]\n [ 474 1512 490 107 30 3]\n [ 70 367 544 264 83 13]\n [ 30 78 206 396 388 128]\n [ 31 74 115 187 424 414]\n [ 24 30 43 82 261 775]]\n\n\nN=16 (smaller bs) - 0.840 CV\n[[2269 442 115 35 11 1]\n [ 435 1541 510 103 25 2]\n [ 59 333 565 284 87 13]\n [ 21 71 210 400 362 162]\n [ 32 64 111 185 407 446]\n [ 19 29 40 95 249 783]]\n\nThese results are obtained for 4-fold CV on 128x128 tiles, while LB is ~0.80.
New candidate at 0.90 LB single fold:\nCV 0.8911
\nEfficientNet-b0 256x256x16 Tiles\nLot of extra computations during inference (TTA and more).
Still using very naive head...
", "rawMarkdown": "New candidate at 0.90 LB single fold:\nCV 0.8911 \nEfficientNet-b0 256x256x16 Tiles\nLot of extra computations during inference (TTA and more).\n\nStill using very naive head...", "votes": 3, "replies": [ { "id": 910132, "postDate": "2020-07-01T02:25:22.377Z", "content": "Wow.
", "rawMarkdown": "Wow." }, { "id": 910133, "postDate": "2020-07-01T02:25:45.563Z", "content": "Intermediate Resolution or Mixture?
", "rawMarkdown": "Intermediate Resolution or Mixture?" } ] }, { "id": 827847, "postDate": "2020-04-30T15:21:48.127Z", "content": "To verify I am looking at the model correctly, you utilize the mask values to determine those tiles with tissue cells in them? Does this then effect the results as the test images as the tiles created are not scaled down for maximizing tissue presence?
", "rawMarkdown": "To verify I am looking at the model correctly, you utilize the mask values to determine those tiles with tissue cells in them? Does this then effect the results as the test images as the tiles created are not scaled down for maximizing tissue presence? ", "votes": 3, "replies": [ { "id": 827979, "postDate": "2020-04-30T17:22:39.877Z", "content": "I check how much the pixels are different from the background (255) and sort the tiles based on it, I do not use masks to create tiles.
", "rawMarkdown": "I check how much the pixels are different from the background (255) and sort the tiles based on it, I do not use masks to create tiles.", "votes": 3 }, { "id": 828610, "postDate": "2020-05-01T07:22:32.500Z", "content": "I think this is a very good think to investigate Zac. There is a specific line of code that does the selection. It's with np.argsort(......)[:N]. If you study this in depth you'll see lafoss indeed filters tiles according to their pixels values (a tile with all black pixels would be selected first, because its sum would be 0).
", "rawMarkdown": "I think this is a very good think to investigate Zac. There is a specific line of code that does the selection. It's with np.argsort(......)[:N]. If you study this in depth you'll see lafoss indeed filters tiles according to their pixels values (a tile with all black pixels would be selected first, because its sum would be 0).", "votes": 2 } ] }, { "id": 830271, "postDate": "2020-05-02T13:11:42.213Z", "content": "Congratz on getting that well deserved Kernel Grandmaster @iafoss !
", "rawMarkdown": "Congratz on getting that well deserved Kernel Grandmaster @iafoss !", "votes": 1, "replies": [ { "id": 830273, "postDate": "2020-05-02T13:13:59.683Z", "content": "Thanks so much @theoviel
", "rawMarkdown": "Thanks so much @theoviel ", "votes": 1 } ] }, { "id": 825412, "postDate": "2020-04-29T01:43:48.700Z", "content": "Amazing! Thanks for sharing your idea about the intelligent way of data preprocessing.
", "rawMarkdown": "Amazing! Thanks for sharing your idea about the intelligent way of data preprocessing.", "votes": 1 }, { "id": 825148, "postDate": "2020-04-28T20:14:48.580Z", "content": "This is simply amazing. That's very high quality content. Thank you so much for sharing that kind of stuff !
", "rawMarkdown": "This is simply amazing. That's very high quality content. Thank you so much for sharing that kind of stuff ! ", "votes": 1, "replies": [ { "id": 825157, "postDate": "2020-04-28T20:25:45.747Z", "content": "You are welcome.
", "rawMarkdown": "You are welcome." } ] }, { "id": 825128, "postDate": "2020-04-28T19:53:36.133Z", "content": "Thanks for sharing! As DrHB said we all appropriate your amazing sharing in these comps!
\n\nBut there appears to be an issue with the dataset you provided, there are missing images when trying to run. Did you name any images weirdly?
", "rawMarkdown": "Thanks for sharing! As DrHB said we all appropriate your amazing sharing in these comps!\n\nBut there appears to be an issue with the dataset you provided, there are missing images when trying to run. Did you name any images weirdly?", "votes": 1, "replies": [ { "id": 825137, "postDate": "2020-04-28T20:05:42.703Z", "content": "You are welcome. I think you are referring to images that do not have masks, I saw a few of them, like tens if I remember correctly. Since I'm planning also to explore segmentation based aux, I considered only images with masks when created this dataset.
", "rawMarkdown": "You are welcome. I think you are referring to images that do not have masks, I saw a few of them, like tens if I remember correctly. Since I'm planning also to explore segmentation based aux, I considered only images with masks when created this dataset.", "votes": 1 }, { "id": 829023, "postDate": "2020-05-01T12:53:15.143Z", "content": "Congrats on becoming Notebook GM!!
", "rawMarkdown": "Congrats on becoming Notebook GM!!", "votes": 1 }, { "id": 830546, "postDate": "2020-05-02T17:10:05.570Z", "content": "Thanks so much.
", "rawMarkdown": "Thanks so much.", "votes": 1 } ] }, { "id": 825446, "postDate": "2020-04-29T02:43:20.783Z", "content": "Just a small question: how you decide the tile size and the number of tiles for each image?\nI am wondering if the image is extremely super resolution, then insufficient number of small tiles\nmight lose some information of the original image.
", "rawMarkdown": "Just a small question: how you decide the tile size and the number of tiles for each image?\nI am wondering if the image is extremely super resolution, then insufficient number of small tiles\nmight lose some information of the original image.", "votes": 2, "replies": [ { "id": 825467, "postDate": "2020-04-29T03:06:37.603Z", "content": "You could check the number N' after which the next tiles are nearly empty for most of the images. For example here when I tried to use 16x128x128 tiles, I found that in most cases 3-5 last tiles are almost completely white. So I decided to go 12, but I didn't evaluate the model performance for different setups. Also, make sure that the tiles are not too small especially for intermediate and high resolution images.
", "rawMarkdown": "You could check the number N' after which the next tiles are nearly empty for most of the images. For example here when I tried to use 16x128x128 tiles, I found that in most cases 3-5 last tiles are almost completely white. So I decided to go 12, but I didn't evaluate the model performance for different setups. Also, make sure that the tiles are not too small especially for intermediate and high resolution images.", "votes": 2 }, { "id": 826951, "postDate": "2020-04-30T01:12:06.167Z", "content": "Thanks for illustration. It is really helpful : )
", "rawMarkdown": "Thanks for illustration. It is really helpful : )" } ] }, { "id": 825055, "postDate": "2020-04-28T18:51:33.227Z", "content": "As per usual high quality kernels. Your starter public kernels were extremely useful in Bengali Competition as well =) And well deserved future (very soon) Kernel Grandmaster =)
", "rawMarkdown": "As per usual high quality kernels. Your starter public kernels were extremely useful in Bengali Competition as well =) And well deserved future (very soon) Kernel Grandmaster =) ", "votes": 2, "replies": [ { "id": 825073, "postDate": "2020-04-28T19:02:20.497Z", "content": "Thanks so much. At Bengali competition it was quite surprising that my public starter kernel got to the upper bronze zone at private LB while public score was quite low. I hope it will not happen again, and the final LB would represent the real efforts made by participants.
", "rawMarkdown": "Thanks so much. At Bengali competition it was quite surprising that my public starter kernel got to the upper bronze zone at private LB while public score was quite low. I hope it will not happen again, and the final LB would represent the real efforts made by participants.", "votes": 6 }, { "id": 826651, "postDate": "2020-04-29T19:02:09.710Z", "content": "Your model did good in private , because it generalized well and you built a nice model :)
", "rawMarkdown": "Your model did good in private , because it generalized well and you built a nice model :) ", "votes": 1 } ] }, { "id": 3052294, "postDate": "2024-11-22T08:54:27.383Z", "content": "Great Job! How to reference your method in a research paper?
", "rawMarkdown": "Great Job! How to reference your method in a research paper?" }, { "id": 905831, "postDate": "2020-06-28T21:03:09.827Z", "content": "hi, there @lafoss, can you share a way to make a vanilla submission with AI. thanks.
", "rawMarkdown": "hi, there @lafoss, can you share a way to make a vanilla submission with AI. thanks.", "replies": [ { "id": 905886, "postDate": "2020-06-28T22:56:18.673Z", "content": "Check my public kernels listed above
", "rawMarkdown": "Check my public kernels listed above" }, { "id": 905934, "postDate": "2020-06-29T00:49:14.800Z", "content": "Hi @lafoss thanks for the replay, besides I am learning still how to do kaggle submission, I am using your technique of breaking down the large image into pieces, and passing them on to the model with other techniques.\n~~~\nepoch train_loss valid_loss accuracy error_rate cohen_kappa_score time\n16 1.090601 0.925868 0.981913 0.018087 0.979144 02:58\n~~~\nI want to be able to test the models I have created, but I am having a little bit of a problem understanding what you are doing on your code ( I have learned a lot from you and your postings, and answer to my questions, thanks).
\n\n1.- It looks to me that you are not using fastai predict method, learn.get_preds, but strait PyTorch code, right?\n2.- I was wondering how you will go about and right your Pytorch code with fastai methods, sorry it that I have looked everywhere on the web, and cant find something that will help me.\n3.- when you do this, x = x.cuda() I thought it was best to set the inference images into the CPU not GPU.\n4.- \n #dihedral TTA\n x = torch.stack([x,x.flip(-1),x.flip(-2),x.flip(-1,-2),\n x.transpose(-1,-2),x.transpose(-1,-2).flip(-1),\n x.transpose(-1,-2).flip(-2),x.transpose(-1,-2).flip(-1,-2)],1)\n x = x.view(-1,N,3,sz,sz)
\n\nno idea what you are doing here.. hehehe.... but I guess it does not work with one image, as it meant to be used with many fragments of sz size.
\n\nI have other questions, but I think I can understand the rest if I can get to understand these ones.
\n\nThanks again!
", "rawMarkdown": "Hi @lafoss thanks for the replay, besides I am learning still how to do kaggle submission, I am using your technique of breaking down the large image into pieces, and passing them on to the model with other techniques.\n~~~\nepoch \ttrain_loss \tvalid_loss \taccuracy \terror_rate \tcohen_kappa_score \ttime\n16 1.090601 \t0.925868 \t0.981913 \t0.018087 \t0.979144 \t 02:58\n~~~\nI want to be able to test the models I have created, but I am having a little bit of a problem understanding what you are doing on your code ( I have learned a lot from you and your postings, and answer to my questions, thanks).\n\n1.- It looks to me that you are not using fastai predict method, learn.get_preds, but strait PyTorch code, right?\n2.- I was wondering how you will go about and right your Pytorch code with fastai methods, sorry it that I have looked everywhere on the web, and cant find something that will help me.\n3.- when you do this, x = x.cuda() I thought it was best to set the inference images into the CPU not GPU.\n4.- \n #dihedral TTA\n x = torch.stack([x,x.flip(-1),x.flip(-2),x.flip(-1,-2),\n x.transpose(-1,-2),x.transpose(-1,-2).flip(-1),\n x.transpose(-1,-2).flip(-2),x.transpose(-1,-2).flip(-1,-2)],1)\n x = x.view(-1,N,3,sz,sz)\n\nno idea what you are doing here.. hehehe.... but I guess it does not work with one image, as it meant to be used with many fragments of sz size.\n\nI have other questions, but I think I can understand the rest if I can get to understand these ones.\n\nThanks again!" }, { "id": 905963, "postDate": "2020-06-29T01:59:15.323Z", "content": "1) Yes\n2) fast.ai inference pipeline is quite limited. So, if you want to get prediction for multiple models without reading the data several times, or doing proper TTA, Pytorch is your way to go. If you want to use fast.ai for inference, u can refer to learn.get_preds method, if I remember correctly, but you will need to write your own code.\n3) If u do inference on CPU, it will take really forever. You can try to run CPU kernel (and comment all cuda related stuff) and see how many images per second u can get (change the path to train to do this test interactively).\n4) TTA means Test Time Augmentation. Google it to see further details. I just create a list of augmented images and combine them into a single batch. I recommend u to play with a dummy image and printout the shape after each operation to understand how it works.
", "rawMarkdown": "1) Yes\n2) fast.ai inference pipeline is quite limited. So, if you want to get prediction for multiple models without reading the data several times, or doing proper TTA, Pytorch is your way to go. If you want to use fast.ai for inference, u can refer to learn.get_preds method, if I remember correctly, but you will need to write your own code.\n3) If u do inference on CPU, it will take really forever. You can try to run CPU kernel (and comment all cuda related stuff) and see how many images per second u can get (change the path to train to do this test interactively).\n4) TTA means Test Time Augmentation. Google it to see further details. I just create a list of augmented images and combine them into a single batch. I recommend u to play with a dummy image and printout the shape after each operation to understand how it works." }, { "id": 906635, "postDate": "2020-06-29T13:09:50.143Z", "content": "Hey @lafoss thanks for your response, I appreciate it.\n2.- I see,
\n3.- I see,\n4.a- Will do bro. \n4.b- then when you do \"x = x.view(-1,N,3,sz,sz)\" is when you combine the images into one and then pass the single image into the model with mode(x), right?
Thanks a lot again.
", "rawMarkdown": "Hey @lafoss thanks for your response, I appreciate it.\n2.- I see, \n3.- I see,\n4.a- Will do bro. \n4.b- then when you do \"x = x.view(-1,N,3,sz,sz)\" is when you combine the images into one and then pass the single image into the model with mode(x), right?\n\nThanks a lot again." }, { "id": 906962, "postDate": "2020-06-29T16:30:30.207Z", "content": "As I wrote, it combines augmented images into a single batch.
", "rawMarkdown": "As I wrote, it combines augmented images into a single batch." } ] }, { "id": 899064, "postDate": "2020-06-24T00:11:10.157Z", "content": "Great work, thanks for sharing!\nHowever, is there any resource to better understand the used concat tile pooling technique?\nI'm kinda new in image nets 😩
", "rawMarkdown": "Great work, thanks for sharing!\nHowever, is there any resource to better understand the used concat tile pooling technique?\nI'm kinda new in image nets 😩 ", "replies": [ { "id": 899139, "postDate": "2020-06-24T02:53:14.437Z", "content": "It's a brand new idea, and I haven't seen anything related to it. Though, it's quite simple one, and I wouldn't be surprised if something similar to it already exists.
", "rawMarkdown": "It's a brand new idea, and I haven't seen anything related to it. Though, it's quite simple one, and I wouldn't be surprised if something similar to it already exists.", "votes": 5 }, { "id": 899259, "postDate": "2020-06-24T05:50:51.140Z", "content": "I would recommend maybe to read some review paper on the topic of Multiple Instance Learning. It helped me at least to clarify the idea more or less.
", "rawMarkdown": "I would recommend maybe to read some review paper on the topic of Multiple Instance Learning. It helped me at least to clarify the idea more or less.", "votes": 3 }, { "id": 900491, "postDate": "2020-06-24T20:48:35.653Z", "content": "Thanks, everyone <3
", "rawMarkdown": "Thanks, everyone <3" } ] }, { "id": 873199, "postDate": "2020-06-03T22:25:07.513Z", "content": "1) Almost for all computer vision models the CNN part reduces the res by 32 times, so 128/32=4.\n2) The tiles should be concatenated specially to be an approximation of classification on entire images. The proposed method, ideally, should be the same to one when an entire image is given as an input. However, the issues with different image shape and overhead because of empty white space are solved. \nMeanwhile, channel-wise concatenation, which u mentioned, is based on quite different idea: stacking a conv model based on tiles and NN generating labels for it. Training with just individual tiles is challenging here because labels assigned to entire images are different from one assigned to tiles forming the images. So one could have a net that would generate some pseudo labels in high dim space in end-to-end manner. I didn't check the idea you mentioned, but it may work as well. The thing u should consider in this case, though, is permutation invariance with respect to the order of tiles.
", "rawMarkdown": "1) Almost for all computer vision models the CNN part reduces the res by 32 times, so 128/32=4.\n2) The tiles should be concatenated specially to be an approximation of classification on entire images. The proposed method, ideally, should be the same to one when an entire image is given as an input. However, the issues with different image shape and overhead because of empty white space are solved. \nMeanwhile, channel-wise concatenation, which u mentioned, is based on quite different idea: stacking a conv model based on tiles and NN generating labels for it. Training with just individual tiles is challenging here because labels assigned to entire images are different from one assigned to tiles forming the images. So one could have a net that would generate some pseudo labels in high dim space in end-to-end manner. I didn't check the idea you mentioned, but it may work as well. The thing u should consider in this case, though, is permutation invariance with respect to the order of tiles." }, { "id": 873261, "postDate": "2020-06-04T01:22:34.780Z", "content": "You'll also have to consider the size of the Linear layer after that. If you use a resnet50 your C is probably already 1K or 2K can't remember. So if you use 32 tiles it is a 32K 64K wide... Transforming that into say 512 is a pretty high number of parameters.
", "rawMarkdown": "You'll also have to consider the size of the Linear layer after that. If you use a resnet50 your C is probably already 1K or 2K can't remember. So if you use 32 tiles it is a 32K 64K wide... Transforming that into say 512 is a pretty high number of parameters." } ] }, { "id": 854157, "postDate": "2020-05-19T19:54:49.193Z", "content": "Thanks :)\nI have one question, I am new in this field , so my question might look like a naive question 👀 \nEvery image now is represent in 12 images, I am using keras , how to fit this data into CNN model ? \nCan you provide me a kernel or any reference where I can learn from ?
\n\nThansk :)
", "rawMarkdown": "Thanks :)\nI have one question, I am new in this field , so my question might look like a naive question 👀 \nEvery image now is represent in 12 images, I am using keras , how to fit this data into CNN model ? \nCan you provide me a kernel or any reference where I can learn from ?\n\nThansk :)", "replies": [ { "id": 854212, "postDate": "2020-05-19T21:17:46.053Z", "content": "Unfortunately, I'm not very familiar with keras. I believe that there should be a way to implement it in this framework, but @xiejialun pointed out below that there are some difficulties related to TensorFlow itself, which keras also may inherit.\nI saw one keras public kernel where the tiles are combined into a big single image given to the network. It is not exactly that I do but presumably works similar.
", "rawMarkdown": "Unfortunately, I'm not very familiar with keras. I believe that there should be a way to implement it in this framework, but @xiejialun pointed out below that there are some difficulties related to TensorFlow itself, which keras also may inherit.\nI saw [one keras public kernel](https://www.kaggle.com/vgarshin/panda-keras-baseline) where the tiles are combined into a big single image given to the network. It is not exactly that I do but presumably works similar.", "votes": 2 }, { "id": 854225, "postDate": "2020-05-19T21:52:31.113Z", "content": "Thanks alot :)
", "rawMarkdown": "Thanks alot :)" }, { "id": 854269, "postDate": "2020-05-19T23:23:54.033Z", "content": "@yaheaal You might not able to perform this idea on keras due to the reason I wrote below, but after some experiments I found that tensorflow is okay. In tensorflow, you need to customize the training loop and create two separated model for tiles training. I already shared a notebook(see below link) how to perform this idea in tensorflow, you only need to modify some of your code in keras. Good luck.
\n\nhttps://www.kaggle.com/xiejialun/panda-tiles-training-on-tensorflow-0-7-cv
", "rawMarkdown": "@yaheaal You might not able to perform this idea on keras due to the reason I wrote below, but after some experiments I found that tensorflow is okay. In tensorflow, you need to customize the training loop and create two separated model for tiles training. I already shared a notebook(see below link) how to perform this idea in tensorflow, you only need to modify some of your code in keras. Good luck.\n\nhttps://www.kaggle.com/xiejialun/panda-tiles-training-on-tensorflow-0-7-cv", "votes": 3 }, { "id": 864874, "postDate": "2020-05-28T08:40:15.457Z", "content": "Thank you for helping :), I am really surprised that keras can't preform this idea :/
", "rawMarkdown": "Thank you for helping :), I am really surprised that keras can't preform this idea :/" }, { "id": 865843, "postDate": "2020-05-28T23:43:44.037Z", "content": "What I have found is that you can use a TimeDistributed layer to wrap around your CNN model and then apply the pooling by reshaping the feature map to a grid. However, it is a little inconvenient to implement.
", "rawMarkdown": "What I have found is that you can use a TimeDistributed layer to wrap around your CNN model and then apply the pooling by reshaping the feature map to a grid. However, it is a little inconvenient to implement.", "votes": 2 }, { "id": 866298, "postDate": "2020-05-29T09:21:38.560Z", "content": "@richardxiao03 yea I found this idea here
\n\n\nbottleneck = efn.EfficientNetB1(\n weights='../input/effnetweights/efficientnet-b1_weights_tf_dim_ordering_tf_kernels_autoaugment_notop.h5', \n include_top=False, \n pooling='avg'\n)\nbottleneck = Model(inputs=bottleneck.inputs, outputs=bottleneck.layers[-2].output)\nmodel = Sequential()\nmodel.add(TimeDistributed(bottleneck, input_shape=(SEQ_LEN, IMG_SIZE, IMG_SIZE, 3)))\nmodel.add(TimeDistributed(BatchNormalization()))\nmodel.add(GlobalMaxPooling3D())\nmodel.add(BatchNormalization())\nmodel.add(Dropout(.4))\nmodel.add(Dense(512, activation='Mish'))\nmodel.add(BatchNormalization())\nmodel.add(Dropout(.4))\nmodel.add(Dense(128, activation='Mish'))\nmodel.add(BatchNormalization())\nmodel.add(Dropout(.4))\nmodel.add(Dense(6, activation='softmax'))\n
I think I should share this (works fine for me):
\n\n```\nclass ConvNet(tf.keras.Model):
\n\ndef __init__(self, engine, input_shape, weights):\n super(ConvNet, self).__init__()\n self.engine = engine(\n include_top=False, input_shape=input_shape, weights=weights)\n self.avg_pool2d = tf.keras.layers.GlobalAveragePooling2D()\n self.dropout = tf.keras.layers.Dropout(0.5)\n self.dense_1 = tf.keras.layers.Dense(512)\n self.dense_2 = tf.keras.layers.Dense(1)\n\n@tf.function\ndef call(self, inputs, **kwargs):\n x = self.engine(inputs)\n x = tf.reshape(x, (-1, 16*4, 4, 2048)) # 16 tiles\n x = self.avg_pool2d(x)\n x = self.dropout(x, training=kwargs.get('training', False))\n x = self.dense_1(x)\n x = tf.nn.relu(x)\n return self.dense_2(x)\nmodel = ConvNet(\n engine=Xception, input_shape=(128, 128, 3), weights='imagenet'\n) \n```
", "rawMarkdown": "I think I should share this (works fine for me):\n\n```\nclass ConvNet(tf.keras.Model):\n\n def __init__(self, engine, input_shape, weights):\n super(ConvNet, self).__init__()\n self.engine = engine(\n include_top=False, input_shape=input_shape, weights=weights)\n self.avg_pool2d = tf.keras.layers.GlobalAveragePooling2D()\n self.dropout = tf.keras.layers.Dropout(0.5)\n self.dense_1 = tf.keras.layers.Dense(512)\n self.dense_2 = tf.keras.layers.Dense(1)\n\n @tf.function\n def call(self, inputs, **kwargs):\n x = self.engine(inputs)\n x = tf.reshape(x, (-1, 16*4, 4, 2048)) # 16 tiles\n x = self.avg_pool2d(x)\n x = self.dropout(x, training=kwargs.get('training', False))\n x = self.dense_1(x)\n x = tf.nn.relu(x)\n return self.dense_2(x)\n\nmodel = ConvNet(\n engine=Xception, input_shape=(128, 128, 3), weights='imagenet'\n) \n```", "votes": 5 }, { "id": 899163, "postDate": "2020-06-24T03:33:52.710Z", "content": "Can you please share @akensert if my input is like 16x128x128x3 then how can I pass through this network. It gives me error like dynamic shape error etc. So I have to stick with tiles concatenation..
", "rawMarkdown": "Can you please share @akensert if my input is like 16x128x128x3 then how can I pass through this network. It gives me error like dynamic shape error etc. So I have to stick with tiles concatenation.." }, { "id": 899401, "postDate": "2020-06-24T08:01:17.297Z", "content": "@micheomaano Is it possible to give me some more information? It's hard for me to tell you what to do atm. The above code snippet works fine for me (should work for 16 x 128 x 128 x 3 input. Although the tf.reshape(...) is hardcoded and could cause problems. I suggest you remove the hardcoding by obtaining the shape from the previous layer (output of self.engine) like this:\n\nnum_tiles = 16\n...\n...\nh = x.get_shape()[-3]\nw = x.get_shape()[-2]\nc = x.get_shape()[-1]\nx = tf.reshape(x, (-1, num_tiles*h, w, c))\n
Hi @akensert \nI face the following errors while loading weights with your model class.\nValueError: You are trying to load a weight file containing 3 layers into a model with 1 layers.
", "rawMarkdown": "Hi @akensert \nI face the following errors while loading weights with your model class.\nValueError: You are trying to load a weight file containing 3 layers into a model with 1 layers." }, { "id": 904047, "postDate": "2020-06-27T10:09:01.930Z", "content": "@micheomaano Ok! So you are trying to load finetuned weights into the model? (for example, you trained some weights locally and now want to load them into your model on kaggle?). What you need to do is to build the model model.build(input_shape) before loading the weights. Or pass some dummy data to your model before loading the finetuned weights.
Example 1:\n\nmodel = Model(...)\nmodel.build((None, 128, 128, 3))\nmodel.load_weights('my-finetuned-weights.h5')\n
Example 2:\n\nmodel = Model(...)\ndummy_data = tf.zeros((1, 128, 128, 3), dtype=tf.uint8))\n_ = model(dummy_data)\nmodel.load_weights('my-finetuned-weights.h5')\n
I think both should work!
", "rawMarkdown": "@micheomaano Ok! So you are trying to load finetuned weights into the model? (for example, you trained some weights locally and now want to load them into your model on kaggle?). What you need to do is to build the model `model.build(input_shape)` before loading the weights. Or pass some dummy data to your model before loading the finetuned weights.\n\nExample 1:\n```\nmodel = Model(...)\nmodel.build((None, 128, 128, 3))\nmodel.load_weights('my-finetuned-weights.h5')\n```\n\nExample 2:\n```\nmodel = Model(...)\ndummy_data = tf.zeros((1, 128, 128, 3), dtype=tf.uint8))\n_ = model(dummy_data)\nmodel.load_weights('my-finetuned-weights.h5')\n```\n\nI think both should work!\n" }, { "id": 904073, "postDate": "2020-06-27T10:35:51.370Z", "content": "It worked. Thanks a lot.\nTraining on GPU is too slow.\nI am trying to shift to TPU.\nThanks.
", "rawMarkdown": "It worked. Thanks a lot.\nTraining on GPU is too slow.\nI am trying to shift to TPU.\nThanks.", "votes": 1 }, { "id": 904121, "postDate": "2020-06-27T11:30:07.767Z", "content": "@micheomaano Side note: If you have an RTX graphics card for example, I recommend using mixed precision. That speeds up training significantly (not as fast as TPU, but still much faster). Find more info here. Further, I recommend using a small architecture like EfficientNetB0. You are probably already aware of these two recommendations, but I still put it out there :-)
", "rawMarkdown": "@micheomaano Side note: If you have an RTX graphics card for example, I recommend using mixed precision. That speeds up training significantly (not as fast as TPU, but still much faster). Find more info [here](https://www.tensorflow.org/guide/mixed_precision). Further, I recommend using a small architecture like EfficientNetB0. You are probably already aware of these two recommendations, but I still put it out there :-)" }, { "id": 905144, "postDate": "2020-06-28T09:45:20.553Z", "content": "@akensert xD I don't even have a GPU. \nI am training using TPU.\nConsidering to create a public notebook to train Model with TPU in less than 3 hours with 48x256x256x3. This is Fun....
", "rawMarkdown": "@akensert xD I don't even have a GPU. \nI am training using TPU.\nConsidering to create a public notebook to train Model with TPU in less than 3 hours with 48x256x256x3. This is Fun....", "votes": 1 }, { "id": 905500, "postDate": "2020-06-28T15:43:57.263Z", "content": "@akensert I am facing a problem with tensorflow. There is different loss for model.evaluate and different validation loss while model.fit operation. I passed training=False while evaluation in dropouts and BatchNormalizations. Kind of strange.
", "rawMarkdown": "@akensert I am facing a problem with tensorflow. There is different loss for model.evaluate and different validation loss while model.fit operation. I passed training=False while evaluation in dropouts and BatchNormalizations. Kind of strange. " }, { "id": 905598, "postDate": "2020-06-28T17:07:24.677Z", "content": "@micheomaano I'm not sure exactly how the validation loss is computed when passed to fit (there should be info about this online). Perhaps the validation loss is computed during the training of the epoch, in contrast to being computed after the training epoch is done.
\n\nSide note about training=False: I think there's some keras magic where training=False is automatically set during predicting phase. That said, it's still good/safe to explicitly set training=False like you now did :-)
", "rawMarkdown": "@micheomaano I'm not sure exactly how the validation loss is computed when passed to fit (there should be info about this online). Perhaps the validation loss is computed during the training of the epoch, in contrast to being computed after the training epoch is done.\n\nSide note about training=False: I think there's some keras magic where training=False is automatically set during predicting phase. That said, it's still good/safe to explicitly set training=False like you now did :-)" } ] }, { "id": 844288, "postDate": "2020-05-12T14:43:30.617Z", "content": "Thanks for sharing! A very interesting idea to start exploring
", "rawMarkdown": "\nThanks for sharing! A very interesting idea to start exploring" }, { "id": 836104, "postDate": "2020-05-06T18:00:03.220Z", "content": "Great approach! Which downsampled version are you using? Using the highest resolution version won't work if the size of tile is 128 x 128 as there will be too many tiles having entire tile filled up. Correct me if I'm wrong. Thanks!
", "rawMarkdown": "Great approach! Which downsampled version are you using? Using the highest resolution version won't work if the size of tile is 128 x 128 as there will be too many tiles having entire tile filled up. Correct me if I'm wrong. Thanks!", "replies": [ { "id": 836143, "postDate": "2020-05-06T18:44:32.077Z", "content": "I have used only low and intermediate res tiff layer so far and can say that it is is still working for the later one (but locally).
", "rawMarkdown": "I have used only low and intermediate res tiff layer so far and can say that it is is still working for the later one (but locally)." } ] }, { "id": 831940, "postDate": "2020-05-03T17:41:52.423Z", "content": "nice work
", "rawMarkdown": "nice work\n" }, { "id": 831886, "postDate": "2020-05-03T16:55:22.783Z", "content": "It was very helpful!
", "rawMarkdown": "\nIt was very helpful!" }, { "id": 830544, "postDate": "2020-05-02T17:07:09.170Z", "content": "nice
", "rawMarkdown": "nice" }, { "id": 829230, "postDate": "2020-05-01T15:59:54.663Z", "content": "Good work
", "rawMarkdown": "Good work" }, { "id": 827696, "postDate": "2020-04-30T13:17:46Z", "content": "Such an awesome idea! Too bad it seems quite difficult to implement in tensorflow framework. Another competition makes me want to switch to Pytorch/Fast.ai 😂
", "rawMarkdown": "Such an awesome idea! Too bad it seems quite difficult to implement in tensorflow framework. Another competition makes me want to switch to Pytorch/Fast.ai 😂 ", "replies": [ { "id": 827823, "postDate": "2020-04-30T15:05:50.530Z", "content": "@Xie29 I was also thinking the same.
", "rawMarkdown": "@Xie29 I was also thinking the same.", "votes": 1 }, { "id": 827976, "postDate": "2020-04-30T17:20:22.663Z", "content": "I'm not not very familiar with tensorflow, but the only thing needed to implement my idea is tensor permutation and reshaping, and also a dataloader that combines the images in the corresponding manner.
", "rawMarkdown": "I'm not not very familiar with tensorflow, but the only thing needed to implement my idea is tensor permutation and reshaping, and also a dataloader that combines the images in the corresponding manner.", "votes": 3 }, { "id": 828327, "postDate": "2020-04-30T22:53:58.367Z", "content": "Since the shape of batch_size need to be fixed during training while using tensorflow, so we are not able to reshape the N tile images to bs*N x 128 x 128 x3 at the beginning or reshape back to bs x c x 4*N x4 in the end. But thanks for the great idea anyway, learned a good lesson.
", "rawMarkdown": "Since the shape of batch_size need to be fixed during training while using tensorflow, so we are not able to reshape the N tile images to bs*N x 128 x 128 x3 at the beginning or reshape back to bs x c x 4*N x4 in the end. But thanks for the great idea anyway, learned a good lesson.", "votes": 2 } ] }, { "id": 825138, "postDate": "2020-04-28T20:07:05.367Z", "content": "Wow, great work. This gives me hope that my method will work out well
", "rawMarkdown": "Wow, great work. This gives me hope that my method will work out well" }, { "id": 825116, "postDate": "2020-04-28T19:41:44.357Z", "content": "Upvote
", "rawMarkdown": "Upvote" }, { "id": 829674, "postDate": "2020-05-02T02:15:49.720Z", "rawMarkdown": "", "isDeleted": true }, { "id": 825063, "postDate": "2020-04-28T18:57:02.917Z", "content": "very neat idea, thank you for sharing!
", "rawMarkdown": "very neat idea, thank you for sharing!" }, { "id": 829079, "postDate": "2020-05-01T13:37:28.220Z", "content": "Good Job. Thank you.
", "rawMarkdown": "Good Job. Thank you." } ], "comments": [ { "id": 833079, "author_name": "Iafoss", "author_url": "", "post_date": "2020-05-04T15:30:25.827000", "content": "Just an update on the number N:\n\nN=12 - 0.843 CV\n[[2290 424 106 41 11 1]\n [ 463 1563 455 116 17 2]\n [ 60 396 547 261 67 10]\n [ 26 81 213 397 395 114]\n [ 29 65 128 182 450 391]\n [ 13 31 42 91 270 768]]\n\n\nN=8 - 0.830 CV\n[[2176 519 126 37 12 3]\n [ 474 1512 490 107 30 3]\n [ 70 367 544 264 83 13]\n [ 30 78 206 396 388 128]\n [ 31 74 115 187 424 414]\n [ 24 30 43 82 261 775]]\n\n\nN=16 (smaller bs) - 0.840 CV\n[[2269 442 115 35 11 1]\n [ 435 1541 510 103 25 2]\n [ 59 333 565 284 87 13]\n [ 21 71 210 400 362 162]\n [ 32 64 111 185 407 446]\n [ 19 29 40 95 249 783]]\n\nThese results are obtained for 4-fold CV on 128x128 tiles, while LB is ~0.80.
New candidate at 0.90 LB single fold:\nCV 0.8911
\nEfficientNet-b0 256x256x16 Tiles\nLot of extra computations during inference (TTA and more).
Still using very naive head...
", "votes": 3, "replies": [ { "id": 910132, "author_name": "Salman", "author_url": "", "post_date": "2020-07-01T02:25:22.377000", "content": "Wow.
", "votes": 0, "replies": [] }, { "id": 910133, "author_name": "Salman", "author_url": "", "post_date": "2020-07-01T02:25:45.563000", "content": "Intermediate Resolution or Mixture?
", "votes": 0, "replies": [] } ] }, { "id": 827847, "author_name": "Zac Dannelly", "author_url": "", "post_date": "2020-04-30T15:21:48.127000", "content": "To verify I am looking at the model correctly, you utilize the mask values to determine those tiles with tissue cells in them? Does this then effect the results as the test images as the tiles created are not scaled down for maximizing tissue presence?
", "votes": 3, "replies": [ { "id": 827979, "author_name": "Iafoss", "author_url": "", "post_date": "2020-04-30T17:22:39.877000", "content": "I check how much the pixels are different from the background (255) and sort the tiles based on it, I do not use masks to create tiles.
", "votes": 3, "replies": [] }, { "id": 828610, "author_name": "Benjamin Dubreu", "author_url": "", "post_date": "2020-05-01T07:22:32.500000", "content": "I think this is a very good think to investigate Zac. There is a specific line of code that does the selection. It's with np.argsort(......)[:N]. If you study this in depth you'll see lafoss indeed filters tiles according to their pixels values (a tile with all black pixels would be selected first, because its sum would be 0).
", "votes": 2, "replies": [] } ] }, { "id": 830271, "author_name": "Theo Viel", "author_url": "", "post_date": "2020-05-02T13:11:42.213000", "content": "Congratz on getting that well deserved Kernel Grandmaster @iafoss !
", "votes": 1, "replies": [ { "id": 830273, "author_name": "Iafoss", "author_url": "", "post_date": "2020-05-02T13:13:59.683000", "content": "Thanks so much @theoviel
", "votes": 1, "replies": [] } ] }, { "id": 825412, "author_name": "Wees Jun", "author_url": "", "post_date": "2020-04-29T01:43:48.700000", "content": "Amazing! Thanks for sharing your idea about the intelligent way of data preprocessing.
", "votes": 1, "replies": [] }, { "id": 825148, "author_name": "Benjamin Dubreu", "author_url": "", "post_date": "2020-04-28T20:14:48.580000", "content": "This is simply amazing. That's very high quality content. Thank you so much for sharing that kind of stuff !
", "votes": 1, "replies": [ { "id": 825157, "author_name": "Iafoss", "author_url": "", "post_date": "2020-04-28T20:25:45.747000", "content": "You are welcome.
", "votes": 0, "replies": [] } ] }, { "id": 825128, "author_name": "GreatGameDota", "author_url": "", "post_date": "2020-04-28T19:53:36.133000", "content": "Thanks for sharing! As DrHB said we all appropriate your amazing sharing in these comps!
\n\nBut there appears to be an issue with the dataset you provided, there are missing images when trying to run. Did you name any images weirdly?
", "votes": 1, "replies": [ { "id": 825137, "author_name": "Iafoss", "author_url": "", "post_date": "2020-04-28T20:05:42.703000", "content": "You are welcome. I think you are referring to images that do not have masks, I saw a few of them, like tens if I remember correctly. Since I'm planning also to explore segmentation based aux, I considered only images with masks when created this dataset.
", "votes": 1, "replies": [] }, { "id": 829023, "author_name": "GreatGameDota", "author_url": "", "post_date": "2020-05-01T12:53:15.143000", "content": "Congrats on becoming Notebook GM!!
", "votes": 1, "replies": [] }, { "id": 830546, "author_name": "Iafoss", "author_url": "", "post_date": "2020-05-02T17:10:05.570000", "content": "Thanks so much.
", "votes": 1, "replies": [] } ] }, { "id": 825446, "author_name": "Wees Jun", "author_url": "", "post_date": "2020-04-29T02:43:20.783000", "content": "Just a small question: how you decide the tile size and the number of tiles for each image?\nI am wondering if the image is extremely super resolution, then insufficient number of small tiles\nmight lose some information of the original image.
", "votes": 2, "replies": [ { "id": 825467, "author_name": "Iafoss", "author_url": "", "post_date": "2020-04-29T03:06:37.603000", "content": "You could check the number N' after which the next tiles are nearly empty for most of the images. For example here when I tried to use 16x128x128 tiles, I found that in most cases 3-5 last tiles are almost completely white. So I decided to go 12, but I didn't evaluate the model performance for different setups. Also, make sure that the tiles are not too small especially for intermediate and high resolution images.
", "votes": 2, "replies": [] }, { "id": 826951, "author_name": "Wees Jun", "author_url": "", "post_date": "2020-04-30T01:12:06.167000", "content": "Thanks for illustration. It is really helpful : )
", "votes": 0, "replies": [] } ] }, { "id": 825055, "author_name": "DrHB", "author_url": "", "post_date": "2020-04-28T18:51:33.227000", "content": "As per usual high quality kernels. Your starter public kernels were extremely useful in Bengali Competition as well =) And well deserved future (very soon) Kernel Grandmaster =)
", "votes": 2, "replies": [ { "id": 825073, "author_name": "Iafoss", "author_url": "", "post_date": "2020-04-28T19:02:20.497000", "content": "Thanks so much. At Bengali competition it was quite surprising that my public starter kernel got to the upper bronze zone at private LB while public score was quite low. I hope it will not happen again, and the final LB would represent the real efforts made by participants.
", "votes": 6, "replies": [] }, { "id": 826651, "author_name": "Nirjhar Roy", "author_url": "", "post_date": "2020-04-29T19:02:09.710000", "content": "Your model did good in private , because it generalized well and you built a nice model :)
", "votes": 1, "replies": [] } ] }, { "id": 3052294, "author_name": "Mahmoud Essam", "author_url": "", "post_date": "2024-11-22T08:54:27.383000", "content": "Great Job! How to reference your method in a research paper?
", "votes": 0, "replies": [] }, { "id": 905831, "author_name": "TheStoneMX", "author_url": "", "post_date": "2020-06-28T21:03:09.827000", "content": "hi, there @lafoss, can you share a way to make a vanilla submission with AI. thanks.
", "votes": 0, "replies": [ { "id": 905886, "author_name": "Iafoss", "author_url": "", "post_date": "2020-06-28T22:56:18.673000", "content": "Check my public kernels listed above
", "votes": 0, "replies": [] }, { "id": 905934, "author_name": "TheStoneMX", "author_url": "", "post_date": "2020-06-29T00:49:14.800000", "content": "Hi @lafoss thanks for the replay, besides I am learning still how to do kaggle submission, I am using your technique of breaking down the large image into pieces, and passing them on to the model with other techniques.\n~~~\nepoch train_loss valid_loss accuracy error_rate cohen_kappa_score time\n16 1.090601 0.925868 0.981913 0.018087 0.979144 02:58\n~~~\nI want to be able to test the models I have created, but I am having a little bit of a problem understanding what you are doing on your code ( I have learned a lot from you and your postings, and answer to my questions, thanks).
\n\n1.- It looks to me that you are not using fastai predict method, learn.get_preds, but strait PyTorch code, right?\n2.- I was wondering how you will go about and right your Pytorch code with fastai methods, sorry it that I have looked everywhere on the web, and cant find something that will help me.\n3.- when you do this, x = x.cuda() I thought it was best to set the inference images into the CPU not GPU.\n4.- \n #dihedral TTA\n x = torch.stack([x,x.flip(-1),x.flip(-2),x.flip(-1,-2),\n x.transpose(-1,-2),x.transpose(-1,-2).flip(-1),\n x.transpose(-1,-2).flip(-2),x.transpose(-1,-2).flip(-1,-2)],1)\n x = x.view(-1,N,3,sz,sz)
\n\nno idea what you are doing here.. hehehe.... but I guess it does not work with one image, as it meant to be used with many fragments of sz size.
\n\nI have other questions, but I think I can understand the rest if I can get to understand these ones.
\n\nThanks again!
", "votes": 0, "replies": [] }, { "id": 905963, "author_name": "Iafoss", "author_url": "", "post_date": "2020-06-29T01:59:15.323000", "content": "1) Yes\n2) fast.ai inference pipeline is quite limited. So, if you want to get prediction for multiple models without reading the data several times, or doing proper TTA, Pytorch is your way to go. If you want to use fast.ai for inference, u can refer to learn.get_preds method, if I remember correctly, but you will need to write your own code.\n3) If u do inference on CPU, it will take really forever. You can try to run CPU kernel (and comment all cuda related stuff) and see how many images per second u can get (change the path to train to do this test interactively).\n4) TTA means Test Time Augmentation. Google it to see further details. I just create a list of augmented images and combine them into a single batch. I recommend u to play with a dummy image and printout the shape after each operation to understand how it works.
", "votes": 0, "replies": [] }, { "id": 906635, "author_name": "TheStoneMX", "author_url": "", "post_date": "2020-06-29T13:09:50.143000", "content": "Hey @lafoss thanks for your response, I appreciate it.\n2.- I see,
\n3.- I see,\n4.a- Will do bro. \n4.b- then when you do \"x = x.view(-1,N,3,sz,sz)\" is when you combine the images into one and then pass the single image into the model with mode(x), right?
Thanks a lot again.
", "votes": 0, "replies": [] }, { "id": 906962, "author_name": "Iafoss", "author_url": "", "post_date": "2020-06-29T16:30:30.207000", "content": "As I wrote, it combines augmented images into a single batch.
", "votes": 0, "replies": [] } ] }, { "id": 899064, "author_name": "Fernandes", "author_url": "", "post_date": "2020-06-24T00:11:10.157000", "content": "Great work, thanks for sharing!\nHowever, is there any resource to better understand the used concat tile pooling technique?\nI'm kinda new in image nets 😩
", "votes": 0, "replies": [ { "id": 899139, "author_name": "Iafoss", "author_url": "", "post_date": "2020-06-24T02:53:14.437000", "content": "It's a brand new idea, and I haven't seen anything related to it. Though, it's quite simple one, and I wouldn't be surprised if something similar to it already exists.
", "votes": 5, "replies": [] }, { "id": 899259, "author_name": "A.Demyanchuk", "author_url": "", "post_date": "2020-06-24T05:50:51.140000", "content": "I would recommend maybe to read some review paper on the topic of Multiple Instance Learning. It helped me at least to clarify the idea more or less.
", "votes": 3, "replies": [] }, { "id": 900491, "author_name": "Fernandes", "author_url": "", "post_date": "2020-06-24T20:48:35.653000", "content": "Thanks, everyone <3
", "votes": 0, "replies": [] } ] }, { "id": 873199, "author_name": "Akshay Patel", "author_url": "", "post_date": "2020-06-03T22:25:07.513000", "content": "1) Almost for all computer vision models the CNN part reduces the res by 32 times, so 128/32=4.\n2) The tiles should be concatenated specially to be an approximation of classification on entire images. The proposed method, ideally, should be the same to one when an entire image is given as an input. However, the issues with different image shape and overhead because of empty white space are solved. \nMeanwhile, channel-wise concatenation, which u mentioned, is based on quite different idea: stacking a conv model based on tiles and NN generating labels for it. Training with just individual tiles is challenging here because labels assigned to entire images are different from one assigned to tiles forming the images. So one could have a net that would generate some pseudo labels in high dim space in end-to-end manner. I didn't check the idea you mentioned, but it may work as well. The thing u should consider in this case, though, is permutation invariance with respect to the order of tiles.
", "votes": 0, "replies": [] }, { "id": 873261, "author_name": "Arnaud Roussel", "author_url": "", "post_date": "2020-06-04T01:22:34.780000", "content": "You'll also have to consider the size of the Linear layer after that. If you use a resnet50 your C is probably already 1K or 2K can't remember. So if you use 32 tiles it is a 32K 64K wide... Transforming that into say 512 is a pretty high number of parameters.
", "votes": 0, "replies": [] } ] }, { "id": 854157, "author_name": "Yaheaal", "author_url": "", "post_date": "2020-05-19T19:54:49.193000", "content": "Thanks :)\nI have one question, I am new in this field , so my question might look like a naive question 👀 \nEvery image now is represent in 12 images, I am using keras , how to fit this data into CNN model ? \nCan you provide me a kernel or any reference where I can learn from ?
\n\nThansk :)
", "votes": 0, "replies": [ { "id": 854212, "author_name": "Iafoss", "author_url": "", "post_date": "2020-05-19T21:17:46.053000", "content": "Unfortunately, I'm not very familiar with keras. I believe that there should be a way to implement it in this framework, but @xiejialun pointed out below that there are some difficulties related to TensorFlow itself, which keras also may inherit.\nI saw one keras public kernel where the tiles are combined into a big single image given to the network. It is not exactly that I do but presumably works similar.
", "votes": 2, "replies": [] }, { "id": 854225, "author_name": "Yaheaal", "author_url": "", "post_date": "2020-05-19T21:52:31.113000", "content": "Thanks alot :)
", "votes": 0, "replies": [] }, { "id": 854269, "author_name": "Tsai29", "author_url": "", "post_date": "2020-05-19T23:23:54.033000", "content": "@yaheaal You might not able to perform this idea on keras due to the reason I wrote below, but after some experiments I found that tensorflow is okay. In tensorflow, you need to customize the training loop and create two separated model for tiles training. I already shared a notebook(see below link) how to perform this idea in tensorflow, you only need to modify some of your code in keras. Good luck.
\n\nhttps://www.kaggle.com/xiejialun/panda-tiles-training-on-tensorflow-0-7-cv
", "votes": 3, "replies": [] }, { "id": 864874, "author_name": "Yaheaal", "author_url": "", "post_date": "2020-05-28T08:40:15.457000", "content": "Thank you for helping :), I am really surprised that keras can't preform this idea :/
", "votes": 0, "replies": [] }, { "id": 865843, "author_name": "Richard Xiao", "author_url": "", "post_date": "2020-05-28T23:43:44.037000", "content": "What I have found is that you can use a TimeDistributed layer to wrap around your CNN model and then apply the pooling by reshaping the feature map to a grid. However, it is a little inconvenient to implement.
", "votes": 2, "replies": [] }, { "id": 866298, "author_name": "Yaheaal", "author_url": "", "post_date": "2020-05-29T09:21:38.560000", "content": "@richardxiao03 yea I found this idea here
\n\n\nbottleneck = efn.EfficientNetB1(\n weights='../input/effnetweights/efficientnet-b1_weights_tf_dim_ordering_tf_kernels_autoaugment_notop.h5', \n include_top=False, \n pooling='avg'\n)\nbottleneck = Model(inputs=bottleneck.inputs, outputs=bottleneck.layers[-2].output)\nmodel = Sequential()\nmodel.add(TimeDistributed(bottleneck, input_shape=(SEQ_LEN, IMG_SIZE, IMG_SIZE, 3)))\nmodel.add(TimeDistributed(BatchNormalization()))\nmodel.add(GlobalMaxPooling3D())\nmodel.add(BatchNormalization())\nmodel.add(Dropout(.4))\nmodel.add(Dense(512, activation='Mish'))\nmodel.add(BatchNormalization())\nmodel.add(Dropout(.4))\nmodel.add(Dense(128, activation='Mish'))\nmodel.add(BatchNormalization())\nmodel.add(Dropout(.4))\nmodel.add(Dense(6, activation='softmax'))\n
I think I should share this (works fine for me):
\n\n```\nclass ConvNet(tf.keras.Model):
\n\ndef __init__(self, engine, input_shape, weights):\n super(ConvNet, self).__init__()\n self.engine = engine(\n include_top=False, input_shape=input_shape, weights=weights)\n self.avg_pool2d = tf.keras.layers.GlobalAveragePooling2D()\n self.dropout = tf.keras.layers.Dropout(0.5)\n self.dense_1 = tf.keras.layers.Dense(512)\n self.dense_2 = tf.keras.layers.Dense(1)\n\n@tf.function\ndef call(self, inputs, **kwargs):\n x = self.engine(inputs)\n x = tf.reshape(x, (-1, 16*4, 4, 2048)) # 16 tiles\n x = self.avg_pool2d(x)\n x = self.dropout(x, training=kwargs.get('training', False))\n x = self.dense_1(x)\n x = tf.nn.relu(x)\n return self.dense_2(x)\nmodel = ConvNet(\n engine=Xception, input_shape=(128, 128, 3), weights='imagenet'\n) \n```
", "votes": 5, "replies": [] }, { "id": 899163, "author_name": "Salman", "author_url": "", "post_date": "2020-06-24T03:33:52.710000", "content": "Can you please share @akensert if my input is like 16x128x128x3 then how can I pass through this network. It gives me error like dynamic shape error etc. So I have to stick with tiles concatenation..
", "votes": 0, "replies": [] }, { "id": 899401, "author_name": "Alex", "author_url": "", "post_date": "2020-06-24T08:01:17.297000", "content": "@micheomaano Is it possible to give me some more information? It's hard for me to tell you what to do atm. The above code snippet works fine for me (should work for 16 x 128 x 128 x 3 input. Although the tf.reshape(...) is hardcoded and could cause problems. I suggest you remove the hardcoding by obtaining the shape from the previous layer (output of self.engine) like this:\n\nnum_tiles = 16\n...\n...\nh = x.get_shape()[-3]\nw = x.get_shape()[-2]\nc = x.get_shape()[-1]\nx = tf.reshape(x, (-1, num_tiles*h, w, c))\n
Hi @akensert \nI face the following errors while loading weights with your model class.\nValueError: You are trying to load a weight file containing 3 layers into a model with 1 layers.
", "votes": 0, "replies": [] }, { "id": 904047, "author_name": "Alex", "author_url": "", "post_date": "2020-06-27T10:09:01.930000", "content": "@micheomaano Ok! So you are trying to load finetuned weights into the model? (for example, you trained some weights locally and now want to load them into your model on kaggle?). What you need to do is to build the model model.build(input_shape) before loading the weights. Or pass some dummy data to your model before loading the finetuned weights.
Example 1:\n\nmodel = Model(...)\nmodel.build((None, 128, 128, 3))\nmodel.load_weights('my-finetuned-weights.h5')\n
Example 2:\n\nmodel = Model(...)\ndummy_data = tf.zeros((1, 128, 128, 3), dtype=tf.uint8))\n_ = model(dummy_data)\nmodel.load_weights('my-finetuned-weights.h5')\n
I think both should work!
", "votes": 0, "replies": [] }, { "id": 904073, "author_name": "Salman", "author_url": "", "post_date": "2020-06-27T10:35:51.370000", "content": "It worked. Thanks a lot.\nTraining on GPU is too slow.\nI am trying to shift to TPU.\nThanks.
", "votes": 1, "replies": [] }, { "id": 904121, "author_name": "Alex", "author_url": "", "post_date": "2020-06-27T11:30:07.767000", "content": "@micheomaano Side note: If you have an RTX graphics card for example, I recommend using mixed precision. That speeds up training significantly (not as fast as TPU, but still much faster). Find more info here. Further, I recommend using a small architecture like EfficientNetB0. You are probably already aware of these two recommendations, but I still put it out there :-)
", "votes": 0, "replies": [] }, { "id": 905144, "author_name": "Salman", "author_url": "", "post_date": "2020-06-28T09:45:20.553000", "content": "@akensert xD I don't even have a GPU. \nI am training using TPU.\nConsidering to create a public notebook to train Model with TPU in less than 3 hours with 48x256x256x3. This is Fun....
", "votes": 1, "replies": [] }, { "id": 905500, "author_name": "Salman", "author_url": "", "post_date": "2020-06-28T15:43:57.263000", "content": "@akensert I am facing a problem with tensorflow. There is different loss for model.evaluate and different validation loss while model.fit operation. I passed training=False while evaluation in dropouts and BatchNormalizations. Kind of strange.
", "votes": 0, "replies": [] }, { "id": 905598, "author_name": "Alex", "author_url": "", "post_date": "2020-06-28T17:07:24.677000", "content": "@micheomaano I'm not sure exactly how the validation loss is computed when passed to fit (there should be info about this online). Perhaps the validation loss is computed during the training of the epoch, in contrast to being computed after the training epoch is done.
\n\nSide note about training=False: I think there's some keras magic where training=False is automatically set during predicting phase. That said, it's still good/safe to explicitly set training=False like you now did :-)
", "votes": 0, "replies": [] } ] }, { "id": 844288, "author_name": "Juan Vallalta", "author_url": "", "post_date": "2020-05-12T14:43:30.617000", "content": "Thanks for sharing! A very interesting idea to start exploring
", "votes": 0, "replies": [] }, { "id": 836104, "author_name": "Kaushal Shah", "author_url": "", "post_date": "2020-05-06T18:00:03.220000", "content": "Great approach! Which downsampled version are you using? Using the highest resolution version won't work if the size of tile is 128 x 128 as there will be too many tiles having entire tile filled up. Correct me if I'm wrong. Thanks!
", "votes": 0, "replies": [ { "id": 836143, "author_name": "Iafoss", "author_url": "", "post_date": "2020-05-06T18:44:32.077000", "content": "I have used only low and intermediate res tiff layer so far and can say that it is is still working for the later one (but locally).
", "votes": 0, "replies": [] } ] }, { "id": 831940, "author_name": "Deepak Rajpurohit", "author_url": "", "post_date": "2020-05-03T17:41:52.423000", "content": "nice work
", "votes": 0, "replies": [] }, { "id": 831886, "author_name": "hdate1091", "author_url": "", "post_date": "2020-05-03T16:55:22.783000", "content": "It was very helpful!
", "votes": 0, "replies": [] }, { "id": 830544, "author_name": "Deepak Rajpurohit", "author_url": "", "post_date": "2020-05-02T17:07:09.170000", "content": "nice
", "votes": 0, "replies": [] }, { "id": 829230, "author_name": "Utpal Dutta", "author_url": "", "post_date": "2020-05-01T15:59:54.663000", "content": "Good work
", "votes": 0, "replies": [] }, { "id": 827696, "author_name": "Tsai29", "author_url": "", "post_date": "2020-04-30T13:17:46", "content": "Such an awesome idea! Too bad it seems quite difficult to implement in tensorflow framework. Another competition makes me want to switch to Pytorch/Fast.ai 😂
", "votes": 0, "replies": [ { "id": 827823, "author_name": "Kurian Benoy", "author_url": "", "post_date": "2020-04-30T15:05:50.530000", "content": "@Xie29 I was also thinking the same.
", "votes": 1, "replies": [] }, { "id": 827976, "author_name": "Iafoss", "author_url": "", "post_date": "2020-04-30T17:20:22.663000", "content": "I'm not not very familiar with tensorflow, but the only thing needed to implement my idea is tensor permutation and reshaping, and also a dataloader that combines the images in the corresponding manner.
", "votes": 3, "replies": [] }, { "id": 828327, "author_name": "Tsai29", "author_url": "", "post_date": "2020-04-30T22:53:58.367000", "content": "Since the shape of batch_size need to be fixed during training while using tensorflow, so we are not able to reshape the N tile images to bs*N x 128 x 128 x3 at the beginning or reshape back to bs x c x 4*N x4 in the end. But thanks for the great idea anyway, learned a good lesson.
", "votes": 2, "replies": [] } ] }, { "id": 825138, "author_name": "Matt", "author_url": "", "post_date": "2020-04-28T20:07:05.367000", "content": "Wow, great work. This gives me hope that my method will work out well
", "votes": 0, "replies": [] }, { "id": 825116, "author_name": "Hanjoon Choe", "author_url": "", "post_date": "2020-04-28T19:41:44.357000", "content": "Upvote
", "votes": 0, "replies": [] }, { "id": 829674, "author_name": "", "author_url": "", "post_date": "2020-05-02T02:15:49.720000", "content": "", "votes": 0, "replies": [] }, { "id": 825063, "author_name": "abzaliev", "author_url": "", "post_date": "2020-04-28T18:57:02.917000", "content": "very neat idea, thank you for sharing!
", "votes": 0, "replies": [] }, { "id": 829079, "author_name": "OH SEOK KIM", "author_url": "", "post_date": "2020-05-01T13:37:28.220000", "content": "Good Job. Thank you.
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "824954": "Welcome to Prostate cANcer graDe Assessment (PANDA) Challenge. I have prepared a starter pack based on my idea of concat tile pooling (see the image below and [this kernel](https://www.kaggle.com/iafoss/panda-concat-tile-pooling-starter-0-79-lb)). Specifically, I provide a [data preprocessing kernel](https://www.kaggle.com/iafoss/panda-16x128x128-tiles) creating 16 128x128 tiles for each image selected based on the number of tissue pixels, [model training kernel](https://www.kaggle.com/iafoss/panda-concat-tile-pooling-starter-0-79-lb) with detailed description of the proposed method, and [inference kernel](https://www.kaggle.com/iafoss/panda-concat-tile-pooling-starter-inference), which reached 0.79 public LB. I hope you will enjoy the competition and find these kernels to be useful to start with the challenge.\n\n\n", "833079": "Just an update on the number N:\n```\nN=12 - 0.843 CV\n[[2290 424 106 41 11 1]\n [ 463 1563 455 116 17 2]\n [ 60 396 547 261 67 10]\n [ 26 81 213 397 395 114]\n [ 29 65 128 182 450 391]\n [ 13 31 42 91 270 768]]\n```\n```\nN=8 - 0.830 CV\n[[2176 519 126 37 12 3]\n [ 474 1512 490 107 30 3]\n [ 70 367 544 264 83 13]\n [ 30 78 206 396 388 128]\n [ 31 74 115 187 424 414]\n [ 24 30 43 82 261 775]]\n```\n```\nN=16 (smaller bs) - 0.840 CV\n[[2269 442 115 35 11 1]\n [ 435 1541 510 103 25 2]\n [ 59 333 565 284 87 13]\n [ 21 71 210 400 362 162]\n [ 32 64 111 185 407 446]\n [ 19 29 40 95 249 783]]\n```\nThese results are obtained for 4-fold CV on 128x128 tiles, while LB is ~0.80.", "910115": "New candidate at 0.90 LB single fold:\nCV 0.8911 \nEfficientNet-b0 256x256x16 Tiles\nLot of extra computations during inference (TTA and more).\n\nStill using very naive head...", "827847": "To verify I am looking at the model correctly, you utilize the mask values to determine those tiles with tissue cells in them? Does this then effect the results as the test images as the tiles created are not scaled down for maximizing tissue presence? ", "830271": "Congratz on getting that well deserved Kernel Grandmaster @iafoss !", "825412": "Amazing! Thanks for sharing your idea about the intelligent way of data preprocessing.", "825148": "This is simply amazing. That's very high quality content. Thank you so much for sharing that kind of stuff ! ", "825128": "Thanks for sharing! As DrHB said we all appropriate your amazing sharing in these comps!\n\nBut there appears to be an issue with the dataset you provided, there are missing images when trying to run. Did you name any images weirdly?", "825446": "Just a small question: how you decide the tile size and the number of tiles for each image?\nI am wondering if the image is extremely super resolution, then insufficient number of small tiles\nmight lose some information of the original image.", "825055": "As per usual high quality kernels. Your starter public kernels were extremely useful in Bengali Competition as well =) And well deserved future (very soon) Kernel Grandmaster =) ", "3052294": "Great Job! How to reference your method in a research paper?", "905831": "hi, there @lafoss, can you share a way to make a vanilla submission with AI. thanks.", "899064": "Great work, thanks for sharing!\nHowever, is there any resource to better understand the used concat tile pooling technique?\nI'm kinda new in image nets 😩 ", "873199": "1. what governs the output shape? I'm assuming the 4X4 is the image size after the convolutions\n2. Does it matter how you reshape the output? Wouldn't it be more intuitive if the output was B X C*N X 4 X 4 instead?", "854157": "Thanks :)\nI have one question, I am new in this field , so my question might look like a naive question 👀 \nEvery image now is represent in 12 images, I am using keras , how to fit this data into CNN model ? \nCan you provide me a kernel or any reference where I can learn from ?\n\nThansk :)", "844288": "\nThanks for sharing! A very interesting idea to start exploring", "836104": "Great approach! Which downsampled version are you using? Using the highest resolution version won't work if the size of tile is 128 x 128 as there will be too many tiles having entire tile filled up. Correct me if I'm wrong. Thanks!", "831940": "nice work\n", "831886": "\nIt was very helpful!", "830544": "nice", "829230": "Good work", "827696": "Such an awesome idea! Too bad it seems quite difficult to implement in tensorflow framework. Another competition makes me want to switch to Pytorch/Fast.ai 😂 ", "825138": "Wow, great work. This gives me hope that my method will work out well", "825116": "Upvote", "829674": "", "825063": "very neat idea, thank you for sharing!", "829079": "Good Job. Thank you." } }