{ "id": 414782, "title": "Has anyone tried RGB_recipes other than Ash Color Scheme?", "url": "/competitions/google-research-identify-contrails-reduce-global-warming/discussion/414782", "author_name": "Hiroki Narita", "post_date": "2023-06-03T08:40:18.801000", "votes": 6, "comment_count": 8, "views": 0, "content": "

Hello Kaggler people!
\nI am now preparing to participate in this competition.
\nFirst things first, a lot of public notebooks

\n\n

as well as

\n\n

I am planning to build a model using the Ash Color Scheme recommended in
\n(My sincere thanks to all who have made their baselines available!)

\n

Currently, I have not yet participated, but I am considering trying out bands other than Ash Color Scheme after I have completed a set of baselines.
\nThe following is a quote from the \"Visualizing Contrails\" Notebook published by the organizer.

\n

https://www.kaggle.com/code/inversion/visualizing-contrails?scriptVersionId=129621747&cellId=4

\n
\n

Combine bands into a false color image
\n In order to view contrails in GOES, we use the \"ash\" color scheme. This color scheme was originally developed for viewing volcanic ash in the atmosphere but is also useful for viewing thin cirrus, including contrails. In this color scheme, contrails appear in the image as dark blue.

\n

Note that we use a modified version of the ash color scheme here, developed by Kulik et al., which uses slightly different bands and bounds tuned for contrails.

\n

References:

\n

Ash Color Scheme (page 7): https://eumetrain.org/sites/default/files/2020-05/RGB_recipes.pdf

\n
\n

Unless I misinterpreted, the above paper is a recipe for creating arbitrary RGB images from band_{08-16}.npy, and Ash Color Scheme is one of them.

\n

I thought that if we refer to this recipe and ensemble models utilizing different bands, we can refer to more information than using only Ash Color Scheme. However, this is currently an empty theory. Has anyone tried this?

\n

I would appreciate it if you could share!

\n

(I am using deepl translation for this text. Sorry if the text is poorly written.)

", "messages": [ { "id": 2286129, "postDate": "2023-06-03T08:40:18.803Z", "content": "

Hello Kaggler people!
\nI am now preparing to participate in this competition.
\nFirst things first, a lot of public notebooks

\n\n

as well as

\n\n

I am planning to build a model using the Ash Color Scheme recommended in
\n(My sincere thanks to all who have made their baselines available!)

\n

Currently, I have not yet participated, but I am considering trying out bands other than Ash Color Scheme after I have completed a set of baselines.
\nThe following is a quote from the \"Visualizing Contrails\" Notebook published by the organizer.

\n

https://www.kaggle.com/code/inversion/visualizing-contrails?scriptVersionId=129621747&cellId=4

\n
\n

Combine bands into a false color image
\n In order to view contrails in GOES, we use the \"ash\" color scheme. This color scheme was originally developed for viewing volcanic ash in the atmosphere but is also useful for viewing thin cirrus, including contrails. In this color scheme, contrails appear in the image as dark blue.

\n

Note that we use a modified version of the ash color scheme here, developed by Kulik et al., which uses slightly different bands and bounds tuned for contrails.

\n

References:

\n

Ash Color Scheme (page 7): https://eumetrain.org/sites/default/files/2020-05/RGB_recipes.pdf

\n
\n

Unless I misinterpreted, the above paper is a recipe for creating arbitrary RGB images from band_{08-16}.npy, and Ash Color Scheme is one of them.

\n

I thought that if we refer to this recipe and ensemble models utilizing different bands, we can refer to more information than using only Ash Color Scheme. However, this is currently an empty theory. Has anyone tried this?

\n

I would appreciate it if you could share!

\n

(I am using deepl translation for this text. Sorry if the text is poorly written.)

", "rawMarkdown": "Hello Kaggler people!\nI am now preparing to participate in this competition.\nFirst things first, a lot of public notebooks\n\n- https://www.kaggle.com/code/mnokno/getting-started-eda-model-train-submit\n- https://www.kaggle.com/code/phoenix9032/inference-unet-effnetb0-on-ash-v2-baseline\n- https://www.kaggle.com/code/kalyansekhar07/pytorch-deeplabv3-model-with-submission\n- https://www.kaggle.com/code/myso1987/ic2rgw-pytorch-baseline-train-inference\n\nas well as\n\n- https://www.kaggle.com/code/inversion/visualizing-contrails\n\nI am planning to build a model using the Ash Color Scheme recommended in\n(My sincere thanks to all who have made their baselines available!)\n\nCurrently, I have not yet participated, but I am considering trying out bands other than Ash Color Scheme after I have completed a set of baselines.\nThe following is a quote from the \"Visualizing Contrails\" Notebook published by the organizer.\n\nhttps://www.kaggle.com/code/inversion/visualizing-contrails?scriptVersionId=129621747&cellId=4\n>Combine bands into a false color image\n>In order to view contrails in GOES, we use the \"ash\" color scheme. This color scheme was originally developed for viewing volcanic ash in the atmosphere but is also useful for viewing thin cirrus, including contrails. In this color scheme, contrails appear in the image as dark blue.\n>\n>\n>Note that we use a modified version of the ash color scheme here, developed by Kulik et al., which uses slightly different bands and bounds tuned for contrails.\n>\n>References:\n>\n>Ash Color Scheme (page 7): https://eumetrain.org/sites/default/files/2020-05/RGB_recipes.pdf\n\nUnless I misinterpreted, the above paper is a recipe for creating arbitrary RGB images from band_{08-16}.npy, and Ash Color Scheme is one of them.\n\nI thought that if we refer to this recipe and ensemble models utilizing different bands, we can refer to more information than using only Ash Color Scheme. However, this is currently an empty theory. Has anyone tried this?\n\nI would appreciate it if you could share!\n\n\n(I am using deepl translation for this text. Sorry if the text is poorly written.)", "votes": 5 }, { "id": 2294879, "postDate": "2023-06-10T12:16:09.867Z", "content": "

The dataset was human-annotated, and the humans only had access to the ash color scheme. Therefore, I think adding other channels might simply add noise, since humans did not see those other channels while labeling. (Even though adding those might produce more accurate trails, it might not produce more accurate human annotations)

", "rawMarkdown": "The dataset was human-annotated, and the humans only had access to the ash color scheme. Therefore, I think adding other channels might simply add noise, since humans did not see those other channels while labeling. (Even though adding those might produce more accurate trails, it might not produce more accurate human annotations)", "votes": 4, "replies": [ { "id": 2295559, "postDate": "2023-06-11T04:57:12.167Z", "content": "

Thank you.
\nI also realized after this post that humans annotated the ash color scheme by looking at it.
\nhttps://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/414782#2286674

\n

And given that we could only observe the ash color scheme, it certainly seems very likely that it would be noise.
\nWhen I thought about it, I also realized that a very large data set of 400 GB could be greatly reduced.

", "rawMarkdown": "Thank you.\nI also realized after this post that humans annotated the ash color scheme by looking at it.\nhttps://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/414782#2286674\n\nAnd given that we could only observe the ash color scheme, it certainly seems very likely that it would be noise.\nWhen I thought about it, I also realized that a very large data set of 400 GB could be greatly reduced." } ] }, { "id": 2286438, "postDate": "2023-06-03T13:10:13.927Z", "content": "

Hey @koniyaku,

\n
\n

I also tried to train a model on the 6 channels proposed in the preprint, which resulted in poor performance (only around 0.4 in the LB score).

\n
\n

from @janhuebi here
\nI hope this answers your question.

", "rawMarkdown": "Hey @koniyaku, \n> I also tried to train a model on the 6 channels proposed in the preprint, which resulted in poor performance (only around 0.4 in the LB score).\n\nfrom @janhuebi [here](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/413068)\nI hope this answers your question.", "votes": 1, "replies": [ { "id": 2286674, "postDate": "2023-06-03T16:33:17.350Z", "content": "

Thank you for telling us about it.

\n

After I made this post, I learned that the labeling of the datasets offered in this competition is based on the Ash Color Scheme.

\n

Below is a quote from a preprint about the dataset that was written in visualizing-contrails.
\nhttps://arxiv.org/abs/2304.02122

\n
\n

We want to detect contrails throughout both day and night,so we show imagery to human labelers in an \"ash\" false color scheme that combines three longwave GOES-16 brightness temperatures [27].
\n The red, blue and green channels are represented by the 12µm, difference between 12µm and 11µm,and difference between 11µm and 8µm respectively.
\n This color scheme is chosen to help identify contrails by highlighting iceclouds as darker colors.

\n
\n

From the above, it seems that it is necessary to be able to observe the contrails on the image at least as well as in the Ash Color Scheme. We will try to visualize various Color Schemes, and if we can observe the same airplane clouds, we will try to adopt them.

", "rawMarkdown": "Thank you for telling us about it.\n\nAfter I made this post, I learned that the labeling of the datasets offered in this competition is based on the Ash Color Scheme.\n\nBelow is a quote from a preprint about the dataset that was written in [visualizing-contrails](https://www.kaggle.com/code/inversion/visualizing-contrails).\nhttps://arxiv.org/abs/2304.02122\n\n>We want to detect contrails throughout both day and night,so we show imagery to human labelers in an \"ash\" false color scheme that combines three longwave GOES-16 brightness temperatures [27].\n>The red, blue and green channels are represented by the 12µm, difference between 12µm and 11µm,and difference between 11µm and 8µm respectively.\n> This color scheme is chosen to help identify contrails by highlighting iceclouds as darker colors.\n\nFrom the above, it seems that it is necessary to be able to observe the contrails on the image at least as well as in the Ash Color Scheme. We will try to visualize various Color Schemes, and if we can observe the same airplane clouds, we will try to adopt them.\n\n", "votes": 1 } ] }, { "id": 2307261, "postDate": "2023-06-18T04:06:41.727Z", "content": "

My intended strategy is to take some images, plot each IR band, look at them, and use the ones that offer the best contrast between contrail and background.

", "rawMarkdown": "My intended strategy is to take some images, plot each IR band, look at them, and use the ones that offer the best contrast between contrail and background.", "replies": [ { "id": 2307635, "postDate": "2023-06-18T10:48:50.197Z", "content": "

@matthewdehaven
\nThis stategy would work well if the task was to predict contrails the most accurate way possible.
\nHowever, here thats not really the task. We have to predict what the labelers saw as a contrail. Hence, we have to show our models what the labelers saw

", "rawMarkdown": "@matthewdehaven \nThis stategy would work well if the task was to predict contrails the most accurate way possible.\nHowever, here thats not really the task. We have to predict what the labelers saw as a contrail. Hence, we have to show our models what the labelers saw ", "votes": 1, "replies": [ { "id": 2308778, "postDate": "2023-06-19T07:57:58.997Z", "content": "

One disclaimer though. . What a labeler sees may not be the same what Neural Network sees isn't it ? e.g alaska2 conpetition on stegabography where the slight changes in pixel is visible by Neural Network not by plain eye .. Not saying it's the same here. Or that it will work here..but it's a theory ..

", "rawMarkdown": "One disclaimer though. . What a labeler sees may not be the same what Neural Network sees isn't it ? e.g alaska2 conpetition on stegabography where the slight changes in pixel is visible by Neural Network not by plain eye .. Not saying it's the same here. Or that it will work here..but it's a theory ..", "votes": 1 }, { "id": 2309194, "postDate": "2023-06-19T13:02:56.810Z", "content": "

Yeah of course i'm not saying that you shouldn't try anything, but what i'm saying is that a strategy that allows NN to detect contrails better might not be the one to perform better on this comp.

", "rawMarkdown": "Yeah of course i'm not saying that you shouldn't try anything, but what i'm saying is that a strategy that allows NN to detect contrails better might not be the one to perform better on this comp.", "votes": 1 } ] } ] } ], "comments": [ { "id": 2294879, "author_name": "CoreyJamesLevinson", "author_url": "", "post_date": "2023-06-10T12:16:09.867000", "content": "

The dataset was human-annotated, and the humans only had access to the ash color scheme. Therefore, I think adding other channels might simply add noise, since humans did not see those other channels while labeling. (Even though adding those might produce more accurate trails, it might not produce more accurate human annotations)

", "votes": 4, "replies": [ { "id": 2295559, "author_name": "Hiroki Narita", "author_url": "", "post_date": "2023-06-11T04:57:12.167000", "content": "

Thank you.
\nI also realized after this post that humans annotated the ash color scheme by looking at it.
\nhttps://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/414782#2286674

\n

And given that we could only observe the ash color scheme, it certainly seems very likely that it would be noise.
\nWhen I thought about it, I also realized that a very large data set of 400 GB could be greatly reduced.

", "votes": 0, "replies": [] } ] }, { "id": 2286438, "author_name": "JEANMPIA", "author_url": "", "post_date": "2023-06-03T13:10:13.927000", "content": "

Hey @koniyaku,

\n
\n

I also tried to train a model on the 6 channels proposed in the preprint, which resulted in poor performance (only around 0.4 in the LB score).

\n
\n

from @janhuebi here
\nI hope this answers your question.

", "votes": 1, "replies": [ { "id": 2286674, "author_name": "Hiroki Narita", "author_url": "", "post_date": "2023-06-03T16:33:17.350000", "content": "

Thank you for telling us about it.

\n

After I made this post, I learned that the labeling of the datasets offered in this competition is based on the Ash Color Scheme.

\n

Below is a quote from a preprint about the dataset that was written in visualizing-contrails.
\nhttps://arxiv.org/abs/2304.02122

\n
\n

We want to detect contrails throughout both day and night,so we show imagery to human labelers in an \"ash\" false color scheme that combines three longwave GOES-16 brightness temperatures [27].
\n The red, blue and green channels are represented by the 12µm, difference between 12µm and 11µm,and difference between 11µm and 8µm respectively.
\n This color scheme is chosen to help identify contrails by highlighting iceclouds as darker colors.

\n
\n

From the above, it seems that it is necessary to be able to observe the contrails on the image at least as well as in the Ash Color Scheme. We will try to visualize various Color Schemes, and if we can observe the same airplane clouds, we will try to adopt them.

", "votes": 1, "replies": [] } ] }, { "id": 2307261, "author_name": "Matthew DeHaven", "author_url": "", "post_date": "2023-06-18T04:06:41.727000", "content": "

My intended strategy is to take some images, plot each IR band, look at them, and use the ones that offer the best contrast between contrail and background.

", "votes": 0, "replies": [ { "id": 2307635, "author_name": "JEANMPIA", "author_url": "", "post_date": "2023-06-18T10:48:50.197000", "content": "

@matthewdehaven
\nThis stategy would work well if the task was to predict contrails the most accurate way possible.
\nHowever, here thats not really the task. We have to predict what the labelers saw as a contrail. Hence, we have to show our models what the labelers saw

", "votes": 1, "replies": [ { "id": 2308778, "author_name": "Nirjhar Roy", "author_url": "", "post_date": "2023-06-19T07:57:58.997000", "content": "

One disclaimer though. . What a labeler sees may not be the same what Neural Network sees isn't it ? e.g alaska2 conpetition on stegabography where the slight changes in pixel is visible by Neural Network not by plain eye .. Not saying it's the same here. Or that it will work here..but it's a theory ..

", "votes": 1, "replies": [] }, { "id": 2309194, "author_name": "JEANMPIA", "author_url": "", "post_date": "2023-06-19T13:02:56.810000", "content": "

Yeah of course i'm not saying that you shouldn't try anything, but what i'm saying is that a strategy that allows NN to detect contrails better might not be the one to perform better on this comp.

", "votes": 1, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2286129": "Hello Kaggler people!\nI am now preparing to participate in this competition.\nFirst things first, a lot of public notebooks\n\n- https://www.kaggle.com/code/mnokno/getting-started-eda-model-train-submit\n- https://www.kaggle.com/code/phoenix9032/inference-unet-effnetb0-on-ash-v2-baseline\n- https://www.kaggle.com/code/kalyansekhar07/pytorch-deeplabv3-model-with-submission\n- https://www.kaggle.com/code/myso1987/ic2rgw-pytorch-baseline-train-inference\n\nas well as\n\n- https://www.kaggle.com/code/inversion/visualizing-contrails\n\nI am planning to build a model using the Ash Color Scheme recommended in\n(My sincere thanks to all who have made their baselines available!)\n\nCurrently, I have not yet participated, but I am considering trying out bands other than Ash Color Scheme after I have completed a set of baselines.\nThe following is a quote from the \"Visualizing Contrails\" Notebook published by the organizer.\n\nhttps://www.kaggle.com/code/inversion/visualizing-contrails?scriptVersionId=129621747&cellId=4\n>Combine bands into a false color image\n>In order to view contrails in GOES, we use the \"ash\" color scheme. This color scheme was originally developed for viewing volcanic ash in the atmosphere but is also useful for viewing thin cirrus, including contrails. In this color scheme, contrails appear in the image as dark blue.\n>\n>\n>Note that we use a modified version of the ash color scheme here, developed by Kulik et al., which uses slightly different bands and bounds tuned for contrails.\n>\n>References:\n>\n>Ash Color Scheme (page 7): https://eumetrain.org/sites/default/files/2020-05/RGB_recipes.pdf\n\nUnless I misinterpreted, the above paper is a recipe for creating arbitrary RGB images from band_{08-16}.npy, and Ash Color Scheme is one of them.\n\nI thought that if we refer to this recipe and ensemble models utilizing different bands, we can refer to more information than using only Ash Color Scheme. However, this is currently an empty theory. Has anyone tried this?\n\nI would appreciate it if you could share!\n\n\n(I am using deepl translation for this text. Sorry if the text is poorly written.)", "2294879": "The dataset was human-annotated, and the humans only had access to the ash color scheme. Therefore, I think adding other channels might simply add noise, since humans did not see those other channels while labeling. (Even though adding those might produce more accurate trails, it might not produce more accurate human annotations)", "2286438": "Hey @koniyaku, \n> I also tried to train a model on the 6 channels proposed in the preprint, which resulted in poor performance (only around 0.4 in the LB score).\n\nfrom @janhuebi [here](https://www.kaggle.com/competitions/google-research-identify-contrails-reduce-global-warming/discussion/413068)\nI hope this answers your question.", "2307261": "My intended strategy is to take some images, plot each IR band, look at them, and use the ones that offer the best contrast between contrail and background." } }