{ "id": 428314, "title": "LB0.658，This is my parameter setting and I would like to discuss it with everyone. I have submitted it dozens of times without any progress, and I am not sure if there is a better way. Any suggestions are helpful. Thank you", "url": "/competitions/google-research-identify-contrails-reduce-global-warming/discussion/428314", "author_name": "kongweihao", "post_date": "2023-08-01T02:07:52.368000", "votes": 14, "comment_count": 13, "views": 0, "content": "

cv 660+ lb 658
\nunet+timm-resnest50d
\nthr0.5
\n5fold 25epoch
\nimage_size 384
\nCosineAnnealingLR(optimizer,T_max=epochs,eta_min=1.0e-6)

", "messages": [ { "id": 2368691, "postDate": "2023-08-01T10:14:36.290Z", "content": "
    \n
  1. Use 512 image size, it has the same performance as 768 and 1024 but much better comparing to 384 in my experiments
    2. \n
  2. Use some better encoder (for example small effnetv2) and experiment with amount of decoder stages you have
    3. \n
  3. Use lower thr - it also helps to increase lb score
    4. \n
  4. In my experiments scheduler not very important so CosineAnnealingLR looks ok
    5. \n
  5. Add tta to your submission if you still do not use it
    6. \n
  6. Also loss selection is important here, pls share which loss are you using now?
    7. \n
", "rawMarkdown": "1. Use 512 image size, it has the same performance as 768 and 1024 but much better comparing to 384 in my experiments\n2. Use some better encoder (for example small effnetv2) and experiment with amount of decoder stages you have\n3. Use lower thr - it also helps to increase lb score\n4. In my experiments scheduler not very important so CosineAnnealingLR looks ok\n5. Add tta to your submission if you still do not use it\n6. Also loss selection is important here, pls share which loss are you using now?", "votes": 15, "replies": [ { "id": 2369744, "postDate": "2023-08-02T02:19:14.663Z", "content": "

Thank you very much for your detailed answer, which has greatly inspired me. This is my Loss function（loss = smp.losses.DiceLoss(mode=\"binary\", smooth=1.0) , and I didn't know that this could also be optimized😂）.

\n

As a beginner, I just found out online what TTA is, which seems very useful. I will try it out immediately. In addition, there may be a rather foolish question, it seems that there is no effnetv2 in SMP, does it refer to either efficientnet-b2 or tim efficiency net-b2?

\n

I will try the methods you mentioned as soon as possible.Thank you again for your answer

", "rawMarkdown": "Thank you very much for your detailed answer, which has greatly inspired me. This is my Loss function（loss = smp.losses.DiceLoss(mode=\"binary\", smooth=1.0) , and I didn't know that this could also be optimized😂）.\n\nAs a beginner, I just found out online what TTA is, which seems very useful. I will try it out immediately. In addition, there may be a rather foolish question, it seems that there is no effnetv2 in SMP, does it refer to either efficientnet-b2 or tim efficiency net-b2? \n\nI will try the methods you mentioned as soon as possible.Thank you again for your answer", "votes": 2, "replies": [ { "id": 2370461, "postDate": "2023-08-02T12:36:14.067Z", "content": "

in SMP you have access to timm general backbones, so use this name tu-tf_efficientnetv2_s.in21k_ft_in1k and weights=True and you will get model with effnetv2 small encoder

", "rawMarkdown": "in SMP you have access to timm general backbones, so use this name tu-tf_efficientnetv2_s.in21k_ft_in1k and weights=True and you will get model with effnetv2 small encoder ", "votes": 6 }, { "id": 2371173, "postDate": "2023-08-03T00:59:24.643Z", "content": "

Thank you. Because of your prompt, I found that this is indeed the case. Then, when using data augmentation, I found that A. Transfer (always_apply=False, p=0.1) seems to be a bit useful but not much. I'm not sure if you have tried it before. If not, I hope it will also be helpful to you

", "rawMarkdown": "Thank you. Because of your prompt, I found that this is indeed the case. Then, when using data augmentation, I found that A. Transfer (always_apply=False, p=0.1) seems to be a bit useful but not much. I'm not sure if you have tried it before. If not, I hope it will also be helpful to you", "votes": 1 } ] }, { "id": 2369780, "postDate": "2023-08-02T02:57:06.050Z", "content": "

By the way, there is another issue that I don't know if you have also encountered. When calculating the validation loss, I often calculate nan values

", "rawMarkdown": "By the way, there is another issue that I don't know if you have also encountered. When calculating the validation loss, I often calculate nan values", "votes": 1, "replies": [ { "id": 2372889, "postDate": "2023-08-04T03:32:29.603Z", "content": "

This discussion post may help with the nan values

", "rawMarkdown": "This [discussion post](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/420980#2370974) may help with the nan values", "votes": 2 }, { "id": 2373239, "postDate": "2023-08-04T07:17:22.477Z", "content": "

Thank you very much. It seems very useful, I am trying it out

", "rawMarkdown": "Thank you very much. It seems very useful, I am trying it out" }, { "id": 2373956, "postDate": "2023-08-04T14:39:46.477Z", "content": "

It works, thanks, good luck with the competition!

", "rawMarkdown": "It works, thanks, good luck with the competition!" } ] }, { "id": 2373750, "postDate": "2023-08-04T12:27:43.960Z", "content": "

Thanks for sharing, currently I only unet++ as the decoder. Is there any other decoders that work better?

", "rawMarkdown": "Thanks for sharing, currently I only unet++ as the decoder. Is there any other decoders that work better?", "replies": [ { "id": 2373948, "postDate": "2023-08-04T14:37:26.117Z", "content": "

You're welcome. Looking through the comments section, it seems that unet is a widely recognized solution. Since I spent a lot of time on other aspects during my first competition, I haven't had time to test other decoders yet. unet++is useful to you, indicating that you have a unique approach. Congratulations.

", "rawMarkdown": "You're welcome. Looking through the comments section, it seems that unet is a widely recognized solution. Since I spent a lot of time on other aspects during my first competition, I haven't had time to test other decoders yet. unet++is useful to you, indicating that you have a unique approach. Congratulations." }, { "id": 2374391, "postDate": "2023-08-04T23:23:50.343Z", "content": "

Unet++ also works but the compute-time to score improvement ratio is so bad I don't use it. You would be better off training more models for ensemble than better indivual Unet++ I reckon.

", "rawMarkdown": "Unet++ also works but the compute-time to score improvement ratio is so bad I don't use it. You would be better off training more models for ensemble than better indivual Unet++ I reckon. " }, { "id": 2374395, "postDate": "2023-08-04T23:39:29.100Z", "content": "

Thanks for your suggestion, let me try other decoders also.

", "rawMarkdown": "Thanks for your suggestion, let me try other decoders also." } ] }, { "id": 2374639, "postDate": "2023-08-05T06:11:19.110Z", "content": "

TTA, I only tried to rotate 180 degrees during the training and testing stages, I found it to be of little use, so I didn't continue. Don't know why

", "rawMarkdown": "TTA, I only tried to rotate 180 degrees during the training and testing stages, I found it to be of little use, so I didn't continue. Don't know why", "votes": 1 } ] }, { "id": 2368072, "postDate": "2023-08-01T02:07:52.370Z", "content": "

cv 660+ lb 658
\nunet+timm-resnest50d
\nthr0.5
\n5fold 25epoch
\nimage_size 384
\nCosineAnnealingLR(optimizer,T_max=epochs,eta_min=1.0e-6)

", "rawMarkdown": "cv 660+ lb 658\nunet+timm-resnest50d\nthr0.5\n5fold 25epoch\nimage_size 384\nCosineAnnealingLR(optimizer,T_max=epochs,eta_min=1.0e-6)", "votes": 13 } ], "comments": [ { "id": 2368691, "author_name": "Kostiantyn Maksymov", "author_url": "", "post_date": "2023-08-01T10:14:36.290000", "content": "
    \n
  1. Use 512 image size, it has the same performance as 768 and 1024 but much better comparing to 384 in my experiments
    2. \n
  2. Use some better encoder (for example small effnetv2) and experiment with amount of decoder stages you have
    3. \n
  3. Use lower thr - it also helps to increase lb score
    4. \n
  4. In my experiments scheduler not very important so CosineAnnealingLR looks ok
    5. \n
  5. Add tta to your submission if you still do not use it
    6. \n
  6. Also loss selection is important here, pls share which loss are you using now?
    7. \n
", "votes": 15, "replies": [ { "id": 2369744, "author_name": "kongweihao", "author_url": "", "post_date": "2023-08-02T02:19:14.663000", "content": "

Thank you very much for your detailed answer, which has greatly inspired me. This is my Loss function（loss = smp.losses.DiceLoss(mode=\"binary\", smooth=1.0) , and I didn't know that this could also be optimized😂）.

\n

As a beginner, I just found out online what TTA is, which seems very useful. I will try it out immediately. In addition, there may be a rather foolish question, it seems that there is no effnetv2 in SMP, does it refer to either efficientnet-b2 or tim efficiency net-b2?

\n

I will try the methods you mentioned as soon as possible.Thank you again for your answer

", "votes": 2, "replies": [ { "id": 2370461, "author_name": "Kostiantyn Maksymov", "author_url": "", "post_date": "2023-08-02T12:36:14.067000", "content": "

in SMP you have access to timm general backbones, so use this name tu-tf_efficientnetv2_s.in21k_ft_in1k and weights=True and you will get model with effnetv2 small encoder

", "votes": 6, "replies": [] }, { "id": 2371173, "author_name": "kongweihao", "author_url": "", "post_date": "2023-08-03T00:59:24.643000", "content": "

Thank you. Because of your prompt, I found that this is indeed the case. Then, when using data augmentation, I found that A. Transfer (always_apply=False, p=0.1) seems to be a bit useful but not much. I'm not sure if you have tried it before. If not, I hope it will also be helpful to you

", "votes": 1, "replies": [] } ] }, { "id": 2369780, "author_name": "kongweihao", "author_url": "", "post_date": "2023-08-02T02:57:06.050000", "content": "

By the way, there is another issue that I don't know if you have also encountered. When calculating the validation loss, I often calculate nan values

", "votes": 1, "replies": [ { "id": 2372889, "author_name": "Ari", "author_url": "", "post_date": "2023-08-04T03:32:29.603000", "content": "

This discussion post may help with the nan values

", "votes": 2, "replies": [] }, { "id": 2373239, "author_name": "kongweihao", "author_url": "", "post_date": "2023-08-04T07:17:22.477000", "content": "

Thank you very much. It seems very useful, I am trying it out

", "votes": 0, "replies": [] }, { "id": 2373956, "author_name": "kongweihao", "author_url": "", "post_date": "2023-08-04T14:39:46.477000", "content": "

It works, thanks, good luck with the competition!

", "votes": 0, "replies": [] } ] }, { "id": 2373750, "author_name": "william.wu", "author_url": "", "post_date": "2023-08-04T12:27:43.960000", "content": "

Thanks for sharing, currently I only unet++ as the decoder. Is there any other decoders that work better?

", "votes": 0, "replies": [ { "id": 2373948, "author_name": "kongweihao", "author_url": "", "post_date": "2023-08-04T14:37:26.117000", "content": "

You're welcome. Looking through the comments section, it seems that unet is a widely recognized solution. Since I spent a lot of time on other aspects during my first competition, I haven't had time to test other decoders yet. unet++is useful to you, indicating that you have a unique approach. Congratulations.

", "votes": 0, "replies": [] }, { "id": 2374391, "author_name": "JEANMPIA", "author_url": "", "post_date": "2023-08-04T23:23:50.343000", "content": "

Unet++ also works but the compute-time to score improvement ratio is so bad I don't use it. You would be better off training more models for ensemble than better indivual Unet++ I reckon.

", "votes": 0, "replies": [] }, { "id": 2374395, "author_name": "william.wu", "author_url": "", "post_date": "2023-08-04T23:39:29.100000", "content": "

Thanks for your suggestion, let me try other decoders also.

", "votes": 0, "replies": [] } ] }, { "id": 2374639, "author_name": "kkkkkkkkk668", "author_url": "", "post_date": "2023-08-05T06:11:19.110000", "content": "

TTA, I only tried to rotate 180 degrees during the training and testing stages, I found it to be of little use, so I didn't continue. Don't know why

", "votes": 1, "replies": [] } ] } ], "raw_markdown_by_id": { "2368691": "1. Use 512 image size, it has the same performance as 768 and 1024 but much better comparing to 384 in my experiments\n2. Use some better encoder (for example small effnetv2) and experiment with amount of decoder stages you have\n3. Use lower thr - it also helps to increase lb score\n4. In my experiments scheduler not very important so CosineAnnealingLR looks ok\n5. Add tta to your submission if you still do not use it\n6. Also loss selection is important here, pls share which loss are you using now?", "2368072": "cv 660+ lb 658\nunet+timm-resnest50d\nthr0.5\n5fold 25epoch\nimage_size 384\nCosineAnnealingLR(optimizer,T_max=epochs,eta_min=1.0e-6)" } }