{ "id": 417565, "title": "a try for ensemble (lb 0.626) (fixed a mistake)", "url": "/competitions/google-research-identify-contrails-reduce-global-warming/discussion/417565", "author_name": "LUPIN11", "post_date": "2023-06-16T08:54:27.138000", "votes": 2, "comment_count": 4, "views": 0, "content": "

I tried to ensemble two U-Nets and got an improvement of 0.08 (0.580+0.618->0.626).
\nThe core function for ensemble:

\n
 ():\n    \n    w1 = ratio_1 * ( / thr_1)\n    w2 = ratio_2 * ( / thr_2)\n    thr =   \n    p = p1*w1 + p2*w2\n    p = (p > thr).astype(np.int32)\n     p\n
\n

One interesting finding is that the lb remains unchanged when ratio_1 and ratio_2 are changed significantly.
\nCode and more detailed experiments are here: Ensemble: U-Net*2 [LB 0.626]
\nMain code and models in this notebook are from Egor Trushin and Shashwat Raman. Thanks to their amazing notebooks, I can try out ensemble here.

\n

Edit:
\nThanks to @imeintanis's patient guidance, I've corrected a serious mistake. Some experiments' result have been updated. I will continue to check my code. So far, the phenomenon of unchanged lb still exists and I still have no idea to explain this.

", "messages": [ { "id": 2304811, "postDate": "2023-06-16T08:54:27.140Z", "content": "

I tried to ensemble two U-Nets and got an improvement of 0.08 (0.580+0.618->0.626).
\nThe core function for ensemble:

\n
 ():\n    \n    w1 = ratio_1 * ( / thr_1)\n    w2 = ratio_2 * ( / thr_2)\n    thr =   \n    p = p1*w1 + p2*w2\n    p = (p > thr).astype(np.int32)\n     p\n
\n

One interesting finding is that the lb remains unchanged when ratio_1 and ratio_2 are changed significantly.
\nCode and more detailed experiments are here: Ensemble: U-Net*2 [LB 0.626]
\nMain code and models in this notebook are from Egor Trushin and Shashwat Raman. Thanks to their amazing notebooks, I can try out ensemble here.

\n

Edit:
\nThanks to @imeintanis's patient guidance, I've corrected a serious mistake. Some experiments' result have been updated. I will continue to check my code. So far, the phenomenon of unchanged lb still exists and I still have no idea to explain this.

", "rawMarkdown": "I tried to ensemble two U-Nets and got an improvement of 0.08 **(0.580+0.618->0.626).** \nThe core function for ensemble:\n```\ndef ensemble(p1, p2):\n \"\"\"\n p1 and p2 are probability maps having a shape of [256, 256] predicted by these 2 U-Nets\n thr_1 and thr_2 are the thresholds used in their orginal notebooks\n ratio_1 and ratio_2 are hyperparameters\n p is the final binary mask produced by ensemble\n \"\"\"\n w1 = ratio_1 * (1 / thr_1)\n w2 = ratio_2 * (1 / thr_2)\n thr = 1 # fixed \n p = p1*w1 + p2*w2\n p = (p > thr).astype(np.int32)\n return p\n```\n**One interesting finding is that the lb remains unchanged when ratio_1 and ratio_2 are changed significantly.**\nCode and more detailed experiments are here: [Ensemble: U-Net*2 [LB 0.626]](https://www.kaggle.com/code/lupin11/ensemble-u-net-2-lb-0-625)\nMain code and models in this notebook are from [Egor Trushin](https://www.kaggle.com/code/egortrushin/gr-icrgw-pytorch-lightning-baseline-unet-resnest) and [Shashwat Raman](https://www.kaggle.com/code/shashwatraman/simple-unet-baseline-infer-lb-0-580). Thanks to their amazing notebooks, I can try out ensemble here.\n\n**Edit:**\nThanks to @imeintanis's patient guidance, I've corrected a serious mistake. Some experiments' result have been updated. I will continue to check my code. So far, the phenomenon of unchanged lb still exists and I still have no idea to explain this.", "votes": 1 }, { "id": 2305285, "postDate": "2023-06-16T15:05:24.737Z", "content": "
\n

One interesting finding is that the lb remains unchanged when ratio_1 and ratio_2 are changed significantly.

\n
\n

I'd suggest to check again your code - particulalry what you are ensembling (Probs1 vs Probs2)

", "rawMarkdown": "> One interesting finding is that the lb remains unchanged when ratio_1 and ratio_2 are changed significantly.\n\nI'd suggest to check again your code - particulalry what you are ensembling (Probs1 vs Probs2) ", "votes": 1, "replies": [ { "id": 2305911, "postDate": "2023-06-17T01:56:13.130Z", "content": "

To be precise, it is like this:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
ratio_1ratio_2lb
0.50.50.625
0.40.60.625
0.450.550.625
0.60.40.585
0.5*0.040.5*250.625
\n

Lb remains 0.625 as long as the weight of the better model is greater than or equals to the weight of another model.
\nSimilar pattern exists when ensembling other models and it's unlikely to be a problem with the code.

", "rawMarkdown": "To be precise, it is like this:\n| ratio_1 | ratio_2 | lb |\n|----------:|----------:|-------:|\n| 0.5 | 0.5 | 0.625 |\n| 0.4 | 0.6 | 0.625 |\n| 0.45 | 0.55 | 0.625 |\n| 0.6 | 0.4 | 0.585 |\n| 0.5*0.04 | 0.5*25 | 0.625 |\n\n\nLb remains 0.625 as long as the weight of the better model is greater than or equals to the weight of another model.\nSimilar pattern exists when ensembling other models and it's unlikely to be a problem with the code.", "replies": [ { "id": 2306351, "postDate": "2023-06-17T08:24:21.173Z", "content": "
\n

and it's unlikely to be a problem with the code.

\n
\n

I insist to look again into your code before you draw any conclusions

\n

I don't speak about the weights - check Probs1, vs Probs2, are both probabilities? I gave you enough hints to find it, now it's up to you

", "rawMarkdown": "> and it's unlikely to be a problem with the code.\n\nI insist to look again into your code before you draw any conclusions \n\nI don't speak about the weights - check Probs1, vs Probs2, are both probabilities? I gave you enough hints to find it, now it's up to you", "votes": 3 }, { "id": 2306445, "postDate": "2023-06-17T09:51:04.690Z", "content": "

Thank you indeed! I've found it. Probs_2 was a binary mask instead of probabilities. I apologize for my stubbornness.

", "rawMarkdown": "Thank you indeed! I've found it. Probs_2 was a binary mask instead of probabilities. I apologize for my stubbornness.", "votes": 1 } ] } ] } ], "comments": [ { "id": 2305285, "author_name": "Ioannis M", "author_url": "", "post_date": "2023-06-16T15:05:24.737000", "content": "
\n

One interesting finding is that the lb remains unchanged when ratio_1 and ratio_2 are changed significantly.

\n
\n

I'd suggest to check again your code - particulalry what you are ensembling (Probs1 vs Probs2)

", "votes": 1, "replies": [ { "id": 2305911, "author_name": "LUPIN11", "author_url": "", "post_date": "2023-06-17T01:56:13.130000", "content": "

To be precise, it is like this:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
ratio_1ratio_2lb
0.50.50.625
0.40.60.625
0.450.550.625
0.60.40.585
0.5*0.040.5*250.625
\n

Lb remains 0.625 as long as the weight of the better model is greater than or equals to the weight of another model.
\nSimilar pattern exists when ensembling other models and it's unlikely to be a problem with the code.

", "votes": 0, "replies": [ { "id": 2306351, "author_name": "Ioannis M", "author_url": "", "post_date": "2023-06-17T08:24:21.173000", "content": "
\n

and it's unlikely to be a problem with the code.

\n
\n

I insist to look again into your code before you draw any conclusions

\n

I don't speak about the weights - check Probs1, vs Probs2, are both probabilities? I gave you enough hints to find it, now it's up to you

", "votes": 3, "replies": [] }, { "id": 2306445, "author_name": "LUPIN11", "author_url": "", "post_date": "2023-06-17T09:51:04.690000", "content": "

Thank you indeed! I've found it. Probs_2 was a binary mask instead of probabilities. I apologize for my stubbornness.

", "votes": 1, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2304811": "I tried to ensemble two U-Nets and got an improvement of 0.08 **(0.580+0.618->0.626).** \nThe core function for ensemble:\n```\ndef ensemble(p1, p2):\n \"\"\"\n p1 and p2 are probability maps having a shape of [256, 256] predicted by these 2 U-Nets\n thr_1 and thr_2 are the thresholds used in their orginal notebooks\n ratio_1 and ratio_2 are hyperparameters\n p is the final binary mask produced by ensemble\n \"\"\"\n w1 = ratio_1 * (1 / thr_1)\n w2 = ratio_2 * (1 / thr_2)\n thr = 1 # fixed \n p = p1*w1 + p2*w2\n p = (p > thr).astype(np.int32)\n return p\n```\n**One interesting finding is that the lb remains unchanged when ratio_1 and ratio_2 are changed significantly.**\nCode and more detailed experiments are here: [Ensemble: U-Net*2 [LB 0.626]](https://www.kaggle.com/code/lupin11/ensemble-u-net-2-lb-0-625)\nMain code and models in this notebook are from [Egor Trushin](https://www.kaggle.com/code/egortrushin/gr-icrgw-pytorch-lightning-baseline-unet-resnest) and [Shashwat Raman](https://www.kaggle.com/code/shashwatraman/simple-unet-baseline-infer-lb-0-580). Thanks to their amazing notebooks, I can try out ensemble here.\n\n**Edit:**\nThanks to @imeintanis's patient guidance, I've corrected a serious mistake. Some experiments' result have been updated. I will continue to check my code. So far, the phenomenon of unchanged lb still exists and I still have no idea to explain this.", "2305285": "> One interesting finding is that the lb remains unchanged when ratio_1 and ratio_2 are changed significantly.\n\nI'd suggest to check again your code - particulalry what you are ensembling (Probs1 vs Probs2) " } }