{ "id": 511274, "title": "Is Tropical Cylones Key to Win this Competition?", "url": "/competitions/leap-atmospheric-physics-ai-climsim/discussion/511274", "author_name": "Bilzard", "post_date": "2024-06-10T05:55:36.932000", "votes": 26, "comment_count": 25, "views": 0, "content": "

Currently, I am searching the path to LB>0.79~0.80, but not found one(s) yet.

\n

However, I noticed some clue: tropical cyclones.

\n

The picture below is MSE of one of my model mapped to location.
\nThe red area is where MSE is high, which means difficult to predict.
\nThe place where my model struggles is biased on Pacific ocean, and April to September.
\nSo as you know, the place where tropical cyclones tend to resolve.

\n

I couldn't find any clever method for utilize this information, so I share this hoping someone will and making this competition more competitive.

\n

Good Luck.

\n

", "messages": [ { "id": 2864354, "postDate": "2024-06-10T05:55:36.933Z", "content": "

Currently, I am searching the path to LB>0.79~0.80, but not found one(s) yet.

\n

However, I noticed some clue: tropical cyclones.

\n

The picture below is MSE of one of my model mapped to location.
\nThe red area is where MSE is high, which means difficult to predict.
\nThe place where my model struggles is biased on Pacific ocean, and April to September.
\nSo as you know, the place where tropical cyclones tend to resolve.

\n

I couldn't find any clever method for utilize this information, so I share this hoping someone will and making this competition more competitive.

\n

Good Luck.

\n

", "rawMarkdown": "Currently, I am searching the path to LB>0.79~0.80, but not found one(s) yet.\n\nHowever, I noticed some clue: *tropical cyclones*.\n\nThe picture below is MSE of one of my model mapped to location.\nThe red area is where MSE is high, which means difficult to predict.\nThe place where my model struggles is biased on Pacific ocean, and April to September.\nSo as you know, the place where tropical cyclones tend to resolve.\n\nI couldn't find any clever method for utilize this information, so I share this hoping someone will and making this competition more competitive.\n\nGood Luck.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4910466%2F60f105df991d392a8546fc3b496dfb10%2Fmse_location.jpeg?generation=1717998482104168&alt=media)", "votes": 26 }, { "id": 2869171, "postDate": "2024-06-12T22:09:14.923Z", "content": "

The phenomenon that you are likely encoutering is something called the Madden-Julian Oscillation. As the name suggests, it has semi-periodic behavior. Hope this helps!

", "rawMarkdown": "The phenomenon that you are likely encoutering is something called the Madden-Julian Oscillation. As the name suggests, it has semi-periodic behavior. Hope this helps!", "votes": 3, "replies": [ { "id": 2869174, "postDate": "2024-06-12T22:28:38.960Z", "content": "

Thank you for sharing domain knowledge. It surely will help.

", "rawMarkdown": "Thank you for sharing domain knowledge. It surely will help.", "votes": 1 } ] }, { "id": 2872749, "postDate": "2024-06-15T03:56:25.340Z", "content": "
\n

The place where my model struggles is biased on Pacific ocean, and April to September.

\n
\n

It might be a silly question: What does \"April to September\" mean? Can we get the time information for each sample?

", "rawMarkdown": ">The place where my model struggles is biased on Pacific ocean, and April to September.\n\nIt might be a silly question: What does \"April to September\" mean? Can we get the time information for each sample?", "votes": 1, "replies": [ { "id": 2872950, "postDate": "2024-06-15T07:32:50.823Z", "content": "

Yes we can, by using the full data set from Hugging Face

", "rawMarkdown": "Yes we can, by using the full data set from Hugging Face", "votes": 1, "replies": [ { "id": 2873080, "postDate": "2024-06-15T10:05:04.313Z", "content": "

Yes, but then you can only get the timestamps for the training dataset, right? So this information would be useful for analysis of your model, gaining insights into what would be missing in the local validation set… but in the end, you are not using timestamp information for the test set. Or am I wrong?

", "rawMarkdown": "Yes, but then you can only get the timestamps for the training dataset, right? So this information would be useful for analysis of your model, gaining insights into what would be missing in the local validation set... but in the end, you are not using timestamp information for the test set. Or am I wrong?", "votes": 1 }, { "id": 2873124, "postDate": "2024-06-15T10:30:47.720Z", "content": "

Well…yes and no. We can get exact timestamp only for train set, but by looking on moving averages of test columns we can get approximate time stamp for test too. Since there is a temporal behaviour of the moving average and the test is not shuffled, only downsampled.

", "rawMarkdown": "Well...yes and no. We can get exact timestamp only for train set, but by looking on moving averages of test columns we can get approximate time stamp for test too. Since there is a temporal behaviour of the moving average and the test is not shuffled, only downsampled. ", "votes": 1 }, { "id": 2875522, "postDate": "2024-06-17T08:02:07.813Z", "content": "

Did you use all of huggingface's data? How much does using all huggingface data improve your cv score?

", "rawMarkdown": "Did you use all of huggingface's data? How much does using all huggingface data improve your cv score?" } ] } ] }, { "id": 2864552, "postDate": "2024-06-10T07:51:25.657Z", "content": "

Are you also inserting the location information as input to the model somehow?

", "rawMarkdown": "Are you also inserting the location information as input to the model somehow?", "votes": 1, "replies": [ { "id": 2864561, "postDate": "2024-06-10T07:54:59.723Z", "content": "

No. However, it’s worth trying (pseudo labeling etc.)

", "rawMarkdown": "No. However, it’s worth trying (pseudo labeling etc.)", "votes": 1 } ] }, { "id": 2864510, "postDate": "2024-06-10T07:13:37.580Z", "content": "

Nice find, thanks for sharing. I suppose this is where domain expertise comes in.

", "rawMarkdown": "Nice find, thanks for sharing. I suppose this is where domain expertise comes in.", "votes": 1, "replies": [ { "id": 2864547, "postDate": "2024-06-10T07:47:50.120Z", "content": "

Maybe. And 1152km x 1152km grid size of low resolution seems pretty challenging to precisely describe physical phenomena with ML models.

", "rawMarkdown": "Maybe. And 1152km x 1152km grid size of low resolution seems pretty challenging to precisely describe physical phenomena with ML models.", "votes": 1 } ] }, { "id": 2864367, "postDate": "2024-06-10T06:07:40.767Z", "content": "

It's a interesting finding. Could you share the code for plotting this image ?

", "rawMarkdown": "It's a interesting finding. Could you share the code for plotting this image ?", "votes": 2, "replies": [ { "id": 2864448, "postDate": "2024-06-10T06:40:23.290Z", "content": "

I omit the technical detail, but we can obtain location information from sample_id, and latitude and longitude are available from original ClimSim dataset.

\n\n

Maybe their demo notebook is helpful
\nhttps://github.com/leap-stc/ClimSim/blob/main/demo_notebooks/water_conservation.ipynb

", "rawMarkdown": "I omit the technical detail, but we can obtain location information from sample_id, and latitude and longitude are available from original ClimSim dataset.\n\n- location_id = sample_id % 384\n\nMaybe their demo notebook is helpful \nhttps://github.com/leap-stc/ClimSim/blob/main/demo_notebooks/water_conservation.ipynb", "votes": 10, "replies": [ { "id": 2866136, "postDate": "2024-06-11T06:47:55.693Z", "content": "

Hi, I am wondering whether location_id = sample_id % 384 has the same meaning for test.csv.

", "rawMarkdown": "Hi, I am wondering whether `location_id = sample_id % 384` has the same meaning for test.csv." }, { "id": 2866179, "postDate": "2024-06-11T07:15:19.323Z", "content": "

@sijunxu I guess no, but you can examine with comparing statistic feature from train and test.

", "rawMarkdown": "@sijunxu I guess no, but you can examine with comparing statistic feature from train and test." }, { "id": 2866319, "postDate": "2024-06-11T08:53:48.923Z", "content": "

Maybe, but probably not since 625000 % 384 != 0. While for train set 10091520 % 384 = 0

", "rawMarkdown": "Maybe, but probably not since 625000 % 384 != 0. While for train set 10091520 % 384 = 0", "votes": 3 }, { "id": 2867718, "postDate": "2024-06-12T03:52:34.050Z", "content": "

I scrambled the sample IDs for the test set. 🙃

", "rawMarkdown": "I scrambled the sample IDs for the test set. 🙃", "votes": 6 }, { "id": 2867762, "postDate": "2024-06-12T04:55:46.483Z", "content": "

@jerrylin96 Yeah, it should be. Thank you for confirmation.

", "rawMarkdown": "@jerrylin96 Yeah, it should be. Thank you for confirmation." }, { "id": 2868420, "postDate": "2024-06-12T11:16:19.167Z", "content": "

but you left \"sample_id\" column unchanged, so test data can be sorted back to its original order. From that label it is easy to deduce location and timestamp of each test sample.

", "rawMarkdown": "but you left \"sample_id\" column unchanged, so test data can be sorted back to its original order. From that label it is easy to deduce location and timestamp of each test sample.", "votes": 10 }, { "id": 2868459, "postDate": "2024-06-12T11:39:37.183Z", "content": "

Hahaha good catch @ymatioun I totally missed it. I guess timestamp and location are back on the table then. Unless the labels themselves are from a scrambled set too.

", "rawMarkdown": "Hahaha good catch @ymatioun I totally missed it. I guess timestamp and location are back on the table then. Unless the labels themselves are from a scrambled set too." }, { "id": 2868466, "postDate": "2024-06-12T11:45:25.830Z", "content": "

I found in testset max(sample_id) + 1 % 384=0, but im not sure how the testset are scrambled.

", "rawMarkdown": "I found in testset `max(sample_id) + 1 % 384=0`, but im not sure how the testset are scrambled.", "votes": 1 }, { "id": 2868486, "postDate": "2024-06-12T12:08:52.060Z", "content": "
\n

but you left \"sample_id\" column unchanged, so test data can be sorted back to its original order. From that label it is easy to deduce location and timestamp of each test sample.

\n
\n

I was hoping I won't have to dive into this, but it seems that there is no choice. Thank you for sharing, big mistake from the host side if true.

", "rawMarkdown": "> but you left \"sample_id\" column unchanged, so test data can be sorted back to its original order. From that label it is easy to deduce location and timestamp of each test sample.\n\nI was hoping I won't have to dive into this, but it seems that there is no choice. Thank you for sharing, big mistake from the host side if true.", "votes": 2 }, { "id": 2868487, "postDate": "2024-06-12T12:09:16.597Z", "content": "

Hmm.., in my understanding, \"scrambled\" means not just shuffle sample ids, but after re-ordering, they are totally random. However, it should be checked…

", "rawMarkdown": "Hmm.., in my understanding, \"scrambled\" means not just shuffle sample ids, but after re-ordering, they are totally random. However, it should be checked...", "votes": 1 }, { "id": 2868521, "postDate": "2024-06-12T12:40:30.953Z", "content": "

@martynoveduard same here…

", "rawMarkdown": "@martynoveduard same here..." } ] } ] }, { "id": 2870726, "postDate": "2024-06-13T18:34:59.423Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 2869171, "author_name": "Anthony Meza", "author_url": "", "post_date": "2024-06-12T22:09:14.923000", "content": "

The phenomenon that you are likely encoutering is something called the Madden-Julian Oscillation. As the name suggests, it has semi-periodic behavior. Hope this helps!

", "votes": 3, "replies": [ { "id": 2869174, "author_name": "Bilzard", "author_url": "", "post_date": "2024-06-12T22:28:38.960000", "content": "

Thank you for sharing domain knowledge. It surely will help.

", "votes": 1, "replies": [] } ] }, { "id": 2872749, "author_name": "Joseph Zhou", "author_url": "", "post_date": "2024-06-15T03:56:25.340000", "content": "
\n

The place where my model struggles is biased on Pacific ocean, and April to September.

\n
\n

It might be a silly question: What does \"April to September\" mean? Can we get the time information for each sample?

", "votes": 1, "replies": [ { "id": 2872950, "author_name": "greySnow", "author_url": "", "post_date": "2024-06-15T07:32:50.823000", "content": "

Yes we can, by using the full data set from Hugging Face

", "votes": 1, "replies": [ { "id": 2873080, "author_name": "Federico Peccia", "author_url": "", "post_date": "2024-06-15T10:05:04.313000", "content": "

Yes, but then you can only get the timestamps for the training dataset, right? So this information would be useful for analysis of your model, gaining insights into what would be missing in the local validation set… but in the end, you are not using timestamp information for the test set. Or am I wrong?

", "votes": 1, "replies": [] }, { "id": 2873124, "author_name": "greySnow", "author_url": "", "post_date": "2024-06-15T10:30:47.720000", "content": "

Well…yes and no. We can get exact timestamp only for train set, but by looking on moving averages of test columns we can get approximate time stamp for test too. Since there is a temporal behaviour of the moving average and the test is not shuffled, only downsampled.

", "votes": 1, "replies": [] }, { "id": 2875522, "author_name": "Zhuoqun Li", "author_url": "", "post_date": "2024-06-17T08:02:07.813000", "content": "

Did you use all of huggingface's data? How much does using all huggingface data improve your cv score?

", "votes": 0, "replies": [] } ] } ] }, { "id": 2864552, "author_name": "Federico Peccia", "author_url": "", "post_date": "2024-06-10T07:51:25.657000", "content": "

Are you also inserting the location information as input to the model somehow?

", "votes": 1, "replies": [ { "id": 2864561, "author_name": "Bilzard", "author_url": "", "post_date": "2024-06-10T07:54:59.723000", "content": "

No. However, it’s worth trying (pseudo labeling etc.)

", "votes": 1, "replies": [] } ] }, { "id": 2864510, "author_name": "sroger", "author_url": "", "post_date": "2024-06-10T07:13:37.580000", "content": "

Nice find, thanks for sharing. I suppose this is where domain expertise comes in.

", "votes": 1, "replies": [ { "id": 2864547, "author_name": "Bilzard", "author_url": "", "post_date": "2024-06-10T07:47:50.120000", "content": "

Maybe. And 1152km x 1152km grid size of low resolution seems pretty challenging to precisely describe physical phenomena with ML models.

", "votes": 1, "replies": [] } ] }, { "id": 2864367, "author_name": "yuanzhe zhou", "author_url": "", "post_date": "2024-06-10T06:07:40.767000", "content": "

It's a interesting finding. Could you share the code for plotting this image ?

", "votes": 2, "replies": [ { "id": 2864448, "author_name": "Bilzard", "author_url": "", "post_date": "2024-06-10T06:40:23.290000", "content": "

I omit the technical detail, but we can obtain location information from sample_id, and latitude and longitude are available from original ClimSim dataset.

\n\n

Maybe their demo notebook is helpful
\nhttps://github.com/leap-stc/ClimSim/blob/main/demo_notebooks/water_conservation.ipynb

", "votes": 10, "replies": [ { "id": 2866136, "author_name": "Sijun Xu", "author_url": "", "post_date": "2024-06-11T06:47:55.693000", "content": "

Hi, I am wondering whether location_id = sample_id % 384 has the same meaning for test.csv.

", "votes": 0, "replies": [] }, { "id": 2866179, "author_name": "Bilzard", "author_url": "", "post_date": "2024-06-11T07:15:19.323000", "content": "

@sijunxu I guess no, but you can examine with comparing statistic feature from train and test.

", "votes": 0, "replies": [] }, { "id": 2866319, "author_name": "greySnow", "author_url": "", "post_date": "2024-06-11T08:53:48.923000", "content": "

Maybe, but probably not since 625000 % 384 != 0. While for train set 10091520 % 384 = 0

", "votes": 3, "replies": [] }, { "id": 2867718, "author_name": "Jerry Lin", "author_url": "", "post_date": "2024-06-12T03:52:34.050000", "content": "

I scrambled the sample IDs for the test set. 🙃

", "votes": 6, "replies": [] }, { "id": 2867762, "author_name": "Bilzard", "author_url": "", "post_date": "2024-06-12T04:55:46.483000", "content": "

@jerrylin96 Yeah, it should be. Thank you for confirmation.

", "votes": 0, "replies": [] }, { "id": 2868420, "author_name": "Youri Matiounine", "author_url": "", "post_date": "2024-06-12T11:16:19.167000", "content": "

but you left \"sample_id\" column unchanged, so test data can be sorted back to its original order. From that label it is easy to deduce location and timestamp of each test sample.

", "votes": 10, "replies": [] }, { "id": 2868459, "author_name": "greySnow", "author_url": "", "post_date": "2024-06-12T11:39:37.183000", "content": "

Hahaha good catch @ymatioun I totally missed it. I guess timestamp and location are back on the table then. Unless the labels themselves are from a scrambled set too.

", "votes": 0, "replies": [] }, { "id": 2868466, "author_name": "Sijun Xu", "author_url": "", "post_date": "2024-06-12T11:45:25.830000", "content": "

I found in testset max(sample_id) + 1 % 384=0, but im not sure how the testset are scrambled.

", "votes": 1, "replies": [] }, { "id": 2868486, "author_name": "slime", "author_url": "", "post_date": "2024-06-12T12:08:52.060000", "content": "
\n

but you left \"sample_id\" column unchanged, so test data can be sorted back to its original order. From that label it is easy to deduce location and timestamp of each test sample.

\n
\n

I was hoping I won't have to dive into this, but it seems that there is no choice. Thank you for sharing, big mistake from the host side if true.

", "votes": 2, "replies": [] }, { "id": 2868487, "author_name": "Bilzard", "author_url": "", "post_date": "2024-06-12T12:09:16.597000", "content": "

Hmm.., in my understanding, \"scrambled\" means not just shuffle sample ids, but after re-ordering, they are totally random. However, it should be checked…

", "votes": 1, "replies": [] }, { "id": 2868521, "author_name": "greySnow", "author_url": "", "post_date": "2024-06-12T12:40:30.953000", "content": "

@martynoveduard same here…

", "votes": 0, "replies": [] } ] } ] }, { "id": 2870726, "author_name": "", "author_url": "", "post_date": "2024-06-13T18:34:59.423000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2864354": "Currently, I am searching the path to LB>0.79~0.80, but not found one(s) yet.\n\nHowever, I noticed some clue: *tropical cyclones*.\n\nThe picture below is MSE of one of my model mapped to location.\nThe red area is where MSE is high, which means difficult to predict.\nThe place where my model struggles is biased on Pacific ocean, and April to September.\nSo as you know, the place where tropical cyclones tend to resolve.\n\nI couldn't find any clever method for utilize this information, so I share this hoping someone will and making this competition more competitive.\n\nGood Luck.\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4910466%2F60f105df991d392a8546fc3b496dfb10%2Fmse_location.jpeg?generation=1717998482104168&alt=media)", "2869171": "The phenomenon that you are likely encoutering is something called the Madden-Julian Oscillation. As the name suggests, it has semi-periodic behavior. Hope this helps!", "2872749": ">The place where my model struggles is biased on Pacific ocean, and April to September.\n\nIt might be a silly question: What does \"April to September\" mean? Can we get the time information for each sample?", "2864552": "Are you also inserting the location information as input to the model somehow?", "2864510": "Nice find, thanks for sharing. I suppose this is where domain expertise comes in.", "2864367": "It's a interesting finding. Could you share the code for plotting this image ?", "2870726": "" } }