Hello everyone,
\nI am a Pytorch Lightning (PL) user and have created a PL baseline. Check corresponding notebook. LB can be improved by increasing the number of iterations, encoder size, image size, etc. Especially if you have the possibility to train locally. My current LB score was achieved with this pipeline.
\nGood luck in the competition.
\nNotebooks with updates:
\n[GR-ICRGW] PL Pipeline Improved
\n[GR-ICRGW] Training with 4 folds
Hello everyone,
\nI am a Pytorch Lightning (PL) user and have created a PL baseline. Check corresponding notebook. LB can be improved by increasing the number of iterations, encoder size, image size, etc. Especially if you have the possibility to train locally. My current LB score was achieved with this pipeline.
\nGood luck in the competition.
\nNotebooks with updates:
\n[GR-ICRGW] PL Pipeline Improved
\n[GR-ICRGW] Training with 4 folds
Hi @egortrushin
\nThank you for great notebook.
I am using your notebook, but when I use \"efficientnet-b0\" I get the following error Do you know any solution?
\nThsnks.
\"It looks like your LightningModule has parameters that were not used in producing the loss returned by training_step. If this is intentional, you must enable the detection of unused parameters in DDP, either by setting the string value strategy='ddp_find_unused_parameters_true' or by setting the flag in the strategy with strategy=DDPStrategy(find_unused_parameters=True)\"
Same here, I couldn't find a solution until now, nothing relevant on PL docs or Google so far.
", "rawMarkdown": "Same here, I couldn't find a solution until now, nothing relevant on PL docs or Google so far. ", "votes": 3 }, { "id": 2314304, "postDate": "2023-06-23T10:33:40.933Z", "content": "Hi. I have looked at this problem a bit and could not find a solution at the moment.
", "rawMarkdown": "Hi. I have looked at this problem a bit and could not find a solution at the moment.", "votes": 4 }, { "id": 2314513, "postDate": "2023-06-23T13:03:41.287Z", "content": "It seems like a problem with ddp+efficientnet. If you use only one cuda device pipeline will work.
", "rawMarkdown": "It seems like a problem with ddp+efficientnet. If you use only one cuda device pipeline will work. ", "votes": 5, "replies": [ { "id": 2314541, "postDate": "2023-06-23T13:18:51.227Z", "content": "Yes Aleksandr, it's just a DDP+eff issue, I wonder if it works with just Pytorch, sometimes PL it's a bit buggy. Apparently, there're some unused parameters on the encoder, and DDP can't handle that and crash, https://github.com/Lightning-AI/lightning/issues/17212.
", "rawMarkdown": "Yes Aleksandr, it's just a DDP+eff issue, I wonder if it works with just Pytorch, sometimes PL it's a bit buggy. Apparently, there're some unused parameters on the encoder, and DDP can't handle that and crash, https://github.com/Lightning-AI/lightning/issues/17212. ", "votes": 6 } ] } ] }, { "id": 2323710, "postDate": "2023-06-30T06:20:37.097Z", "content": "Hi @egortrushin
\nthanks for great notebook, i checked the size of train dataset and find that the label seems not resized as same size as image. would you please explain why label not resize as the image size, thanks advance
img, label = next(iter(data_loader_train))
\nimg.shape, label.shape
(torch.Size([32, 3, 384, 384]), torch.Size([32, 256, 256]))
", "rawMarkdown": "Hi @egortrushin \nthanks for great notebook, i checked the size of train dataset and find that the label seems not resized as same size as image. would you please explain why label not resize as the image size, thanks advance\n\nimg, label = next(iter(data_loader_train))\nimg.shape, label.shape\n\n(torch.Size([32, 3, 384, 384]), torch.Size([32, 256, 256]))", "votes": 1, "replies": [ { "id": 2324038, "postDate": "2023-06-30T10:40:46.887Z", "content": "Hi. In this approach, a model predicts a mask of the size of the resized image, but then this mask is rescaled to 256x256 to be consistent with the 256x256 ground truth mask.
", "rawMarkdown": "Hi. In this approach, a model predicts a mask of the size of the resized image, but then this mask is rescaled to 256x256 to be consistent with the 256x256 ground truth mask.", "votes": 1, "replies": [ { "id": 2324130, "postDate": "2023-06-30T12:04:20.947Z", "content": "I saw the code in training_step, thanks for your reply
\n self != :\n preds = torch(preds, size=, mode=)\nHi @egortrushin
\nThank you for this great notebook! What is the reason for changing the image size from the original 256 to other numbers like 384? Generally speaking, when and why increasing the image resolution may help?
\nThanks!
In the manuscript written by the organizers (see OpenContrails: Benchmarking Contrail Detection on GOES-16 ABI) there is a stable improvement of the results with increasing image size. So it's worth trying to increase the image size.
", "rawMarkdown": "In the manuscript written by the organizers (see [OpenContrails: Benchmarking Contrail Detection on GOES-16 ABI](https://arxiv.org/pdf/2304.02122.pdf)) there is a stable improvement of the results with increasing image size. So it's worth trying to increase the image size.", "votes": 1 } ] }, { "id": 2304169, "postDate": "2023-06-15T18:21:22.543Z", "content": "Hi @egortrushin,
\nThank you for this amazing notebook.
\nHave you tried using TPUs with Pytorch and/or Pytorch Lightning?
\nI want to use TPUs in this competition but I'm unsure if they work well with Pytorch.
\nThanks.
Hi. Yes, I was trying to run Lightning code on TPU. Looks like it is not even possible to install torch-xla on the current Kaggle TPU environment.
", "rawMarkdown": "Hi. Yes, I was trying to run Lightning code on TPU. Looks like it is not even possible to install torch-xla on the current Kaggle TPU environment.", "replies": [ { "id": 2304552, "postDate": "2023-06-16T04:45:05.807Z", "content": "Oh 🙃. What to do then? @egortrushin
", "rawMarkdown": "Oh 🙃. What to do then? @egortrushin ", "votes": 1 }, { "id": 2305345, "postDate": "2023-06-16T16:04:37.953Z", "content": "Pytorch is bad on TPU anyway. It makes more sense to write Tensorflow code for TPU training. We already have some GPU Tensorflow baseline.
", "rawMarkdown": "Pytorch is bad on TPU anyway. It makes more sense to write Tensorflow code for TPU training. We already have some [GPU Tensorflow baseline](https://www.kaggle.com/code/egorfokin/contrails-tensorflow-train-submission-public). " } ] } ] } ], "comments": [ { "id": 2313325, "author_name": "Never$", "author_url": "", "post_date": "2023-06-22T15:30:41.383000", "content": "Hi @egortrushin
\nThank you for great notebook.
I am using your notebook, but when I use \"efficientnet-b0\" I get the following error Do you know any solution?
\nThsnks.
\"It looks like your LightningModule has parameters that were not used in producing the loss returned by training_step. If this is intentional, you must enable the detection of unused parameters in DDP, either by setting the string value strategy='ddp_find_unused_parameters_true' or by setting the flag in the strategy with strategy=DDPStrategy(find_unused_parameters=True)\"
Same here, I couldn't find a solution until now, nothing relevant on PL docs or Google so far.
", "votes": 3, "replies": [] }, { "id": 2314304, "author_name": "Egor Trushin", "author_url": "", "post_date": "2023-06-23T10:33:40.933000", "content": "Hi. I have looked at this problem a bit and could not find a solution at the moment.
", "votes": 4, "replies": [] }, { "id": 2314513, "author_name": "Aleksandr Lavrikov", "author_url": "", "post_date": "2023-06-23T13:03:41.287000", "content": "It seems like a problem with ddp+efficientnet. If you use only one cuda device pipeline will work.
", "votes": 5, "replies": [ { "id": 2314541, "author_name": "Maximiliano Diaz Battan", "author_url": "", "post_date": "2023-06-23T13:18:51.227000", "content": "Yes Aleksandr, it's just a DDP+eff issue, I wonder if it works with just Pytorch, sometimes PL it's a bit buggy. Apparently, there're some unused parameters on the encoder, and DDP can't handle that and crash, https://github.com/Lightning-AI/lightning/issues/17212.
", "votes": 6, "replies": [] } ] } ] }, { "id": 2323710, "author_name": "liuzhangzhen", "author_url": "", "post_date": "2023-06-30T06:20:37.097000", "content": "Hi @egortrushin
\nthanks for great notebook, i checked the size of train dataset and find that the label seems not resized as same size as image. would you please explain why label not resize as the image size, thanks advance
img, label = next(iter(data_loader_train))
\nimg.shape, label.shape
(torch.Size([32, 3, 384, 384]), torch.Size([32, 256, 256]))
", "votes": 1, "replies": [ { "id": 2324038, "author_name": "Egor Trushin", "author_url": "", "post_date": "2023-06-30T10:40:46.887000", "content": "Hi. In this approach, a model predicts a mask of the size of the resized image, but then this mask is rescaled to 256x256 to be consistent with the 256x256 ground truth mask.
", "votes": 1, "replies": [ { "id": 2324130, "author_name": "liuzhangzhen", "author_url": "", "post_date": "2023-06-30T12:04:20.947000", "content": "I saw the code in training_step, thanks for your reply
\n self != :\n preds = torch(preds, size=, mode=)\nHi @egortrushin
\nThank you for this great notebook! What is the reason for changing the image size from the original 256 to other numbers like 384? Generally speaking, when and why increasing the image resolution may help?
\nThanks!
In the manuscript written by the organizers (see OpenContrails: Benchmarking Contrail Detection on GOES-16 ABI) there is a stable improvement of the results with increasing image size. So it's worth trying to increase the image size.
", "votes": 1, "replies": [] } ] }, { "id": 2304169, "author_name": "Shashwat Raman", "author_url": "", "post_date": "2023-06-15T18:21:22.543000", "content": "Hi @egortrushin,
\nThank you for this amazing notebook.
\nHave you tried using TPUs with Pytorch and/or Pytorch Lightning?
\nI want to use TPUs in this competition but I'm unsure if they work well with Pytorch.
\nThanks.
Hi. Yes, I was trying to run Lightning code on TPU. Looks like it is not even possible to install torch-xla on the current Kaggle TPU environment.
", "votes": 0, "replies": [ { "id": 2304552, "author_name": "Shashwat Raman", "author_url": "", "post_date": "2023-06-16T04:45:05.807000", "content": "Oh 🙃. What to do then? @egortrushin
", "votes": 1, "replies": [] }, { "id": 2305345, "author_name": "Egor Trushin", "author_url": "", "post_date": "2023-06-16T16:04:37.953000", "content": "Pytorch is bad on TPU anyway. It makes more sense to write Tensorflow code for TPU training. We already have some GPU Tensorflow baseline.
", "votes": 0, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2299876": "Hello everyone,\n\nI am a [Pytorch Lightning](https://www.pytorchlightning.ai/index.html) (PL) user and have created a PL baseline. Check [corresponding notebook](https://www.kaggle.com/code/egortrushin/gr-icrgw-pytorch-lightning-baseline-unet-resnest). LB can be improved by increasing the number of iterations, encoder size, image size, etc. Especially if you have the possibility to train locally. My current LB score was achieved with this pipeline.\n\nGood luck in the competition.\n\nNotebooks with updates:\n[[GR-ICRGW] PL Pipeline Improved](https://www.kaggle.com/code/egortrushin/gr-icrgw-pl-pipeline-improved)\n[[GR-ICRGW] Training with 4 folds](https://www.kaggle.com/code/egortrushin/gr-icrgw-training-with-4-folds?scriptVersionId=134148499)", "2313325": "Hi @egortrushin \nThank you for great notebook.\n\nI am using your notebook, but when I use \"efficientnet-b0\" I get the following error Do you know any solution?\nThsnks.\n\n\"It looks like your LightningModule has parameters that were not used in producing the loss returned by training_step. If this is intentional, you must enable the detection of unused parameters in DDP, either by setting the string value `strategy='ddp_find_unused_parameters_true'` or by setting the flag in the strategy with `strategy=DDPStrategy(find_unused_parameters=True)`\"", "2323710": "Hi @egortrushin \nthanks for great notebook, i checked the size of train dataset and find that the label seems not resized as same size as image. would you please explain why label not resize as the image size, thanks advance\n\nimg, label = next(iter(data_loader_train))\nimg.shape, label.shape\n\n(torch.Size([32, 3, 384, 384]), torch.Size([32, 256, 256]))", "2320710": "Hi @egortrushin \nThank you for this great notebook! What is the reason for changing the image size from the original 256 to other numbers like 384? Generally speaking, when and why increasing the image resolution may help?\nThanks!", "2304169": "Hi @egortrushin,\nThank you for this amazing notebook.\nHave you tried using TPUs with Pytorch and/or Pytorch Lightning? \nI want to use TPUs in this competition but I'm unsure if they work well with Pytorch.\nThanks." } }