The argument that the domain is important is shared in discussions here and here. After seeing these discussions, I considered to apply the conservation laws into the model.
\nBased on this notebook, I created the water conservation loss function.
xarray xr\n torch\n torch nn\n\n () -> torch.tensor:\n \n p = (a * p0).reshape(-, ) + (b.reshape(-, ) * ps.reshape(, -))\n p.diff(dim=).T\n\n\n (nn.Module):\n \n () -> :\n ().__init__()\n self.criterion = nn.MSELoss()\n self.a, self.b, self.p0 = self.load_initial_params()\n self.inv_g = / \n self.delta_t = \n\n () -> [torch.tensor, torch.tensor, torch.tensor]:\n \n initial_conditions = \n xr.open_dataset(initial_conditions) inic:\n a = torch.tensor(inic[].to_numpy())\n b = torch.tensor(inic[].to_numpy())\n p0 = torch.tensor(inic[].to_numpy())\n a, b, p0\n\n () -> :\n self.a = self.a.to(device)\n self.b = self.b.to(device)\n self.p0 = self.p0.to(device)\n\n () -> torch.tensor:\n \n self.apply_device(features.device)\n \n preds[:, :] = preds[:, :] * self.delta_t + features[:, :]\n\n P = preds[:, TARGET_COLS.index()] * \n ps = features[:, FEATURE_COLS.index()] \n delta_p = get_pressure_thickness(self.a, self.b, self.p0, ps)\n\n qv_in = (features[:, :] * delta_p * self.inv_g).(dim=)\n ql_in = (features[:, :] * delta_p * self.inv_g).(dim=)\n qi_in = (features[:, :] * delta_p * self.inv_g).(dim=)\n expected_total_water = (qv_in + ql_in + qi_in) - (P * self.delta_t)\n\n qv_out = (preds[:, :] * delta_p * self.inv_g).(dim=)\n ql_out = (preds[:, :] * delta_p * self.inv_g).(dim=)\n qi_out = (preds[:, :] * delta_p * self.inv_g).(dim=)\n total_water = qv_out + ql_out + qi_out\n\n loss = self.criterion(total_water, expected_total_water)\n loss\nThis loss function is based on the insight that the total moisture content within the grid is conserved. However, this did not contribute to improving the score in my model. Since the loss is decreasing with training, I speculate that the existing model implicitly considers the conservation laws.
\nHas anyone managed to use these conservation law ideas to achieve improvements? or not work?
\nIf there are any mistakes in my implementation, please feel free to point them out!
The argument that the domain is important is shared in discussions here and here. After seeing these discussions, I considered to apply the conservation laws into the model.
\nBased on this notebook, I created the water conservation loss function.
xarray xr\n torch\n torch nn\n\n () -> torch.tensor:\n \n p = (a * p0).reshape(-, ) + (b.reshape(-, ) * ps.reshape(, -))\n p.diff(dim=).T\n\n\n (nn.Module):\n \n () -> :\n ().__init__()\n self.criterion = nn.MSELoss()\n self.a, self.b, self.p0 = self.load_initial_params()\n self.inv_g = / \n self.delta_t = \n\n () -> [torch.tensor, torch.tensor, torch.tensor]:\n \n initial_conditions = \n xr.open_dataset(initial_conditions) inic:\n a = torch.tensor(inic[].to_numpy())\n b = torch.tensor(inic[].to_numpy())\n p0 = torch.tensor(inic[].to_numpy())\n a, b, p0\n\n () -> :\n self.a = self.a.to(device)\n self.b = self.b.to(device)\n self.p0 = self.p0.to(device)\n\n () -> torch.tensor:\n \n self.apply_device(features.device)\n \n preds[:, :] = preds[:, :] * self.delta_t + features[:, :]\n\n P = preds[:, TARGET_COLS.index()] * \n ps = features[:, FEATURE_COLS.index()] \n delta_p = get_pressure_thickness(self.a, self.b, self.p0, ps)\n\n qv_in = (features[:, :] * delta_p * self.inv_g).(dim=)\n ql_in = (features[:, :] * delta_p * self.inv_g).(dim=)\n qi_in = (features[:, :] * delta_p * self.inv_g).(dim=)\n expected_total_water = (qv_in + ql_in + qi_in) - (P * self.delta_t)\n\n qv_out = (preds[:, :] * delta_p * self.inv_g).(dim=)\n ql_out = (preds[:, :] * delta_p * self.inv_g).(dim=)\n qi_out = (preds[:, :] * delta_p * self.inv_g).(dim=)\n total_water = qv_out + ql_out + qi_out\n\n loss = self.criterion(total_water, expected_total_water)\n loss\nThis loss function is based on the insight that the total moisture content within the grid is conserved. However, this did not contribute to improving the score in my model. Since the loss is decreasing with training, I speculate that the existing model implicitly considers the conservation laws.
\nHas anyone managed to use these conservation law ideas to achieve improvements? or not work?
\nIf there are any mistakes in my implementation, please feel free to point them out!
My models seems to learn water convervation well enough with just the MSE loss: https://www.kaggle.com/competitions/leap-atmospheric-physics-ai-climsim/discussion/508272
\nMoreover consider that the water quantity is not exactly conserved since the suspended precipitation is not tracked
", "rawMarkdown": "My models seems to learn water convervation well enough with just the MSE loss: https://www.kaggle.com/competitions/leap-atmospheric-physics-ai-climsim/discussion/508272\n\nMoreover consider that the water quantity is not exactly conserved since the suspended precipitation is not tracked", "votes": 3, "replies": [ { "id": 2897204, "postDate": "2024-06-30T10:38:18.507Z", "content": "Oh, I didn’t check your report.Thanks for sharing!
\n\n\nprecipitation is not tracked
\n
I see, Thanks for your advice.
", "rawMarkdown": "Oh, I didn’t check your report.Thanks for sharing!\n\n> precipitation is not tracked\n\nI see, Thanks for your advice." } ] }, { "id": 2895632, "postDate": "2024-06-29T09:09:46.877Z", "content": "\n\nSince the loss is decreasing with training, I speculate that the existing model implicitly considers the conservation laws.
\n
I'm not so sure about that. WaterConservationLoss can be decreased due to the operation of the basic loss function, but the WaterConservationLoss is still large compared to the ideal solution.
\n\n\nHowever, this did not contribute to improving the score in my model.
\n
I've noticed that conservation laws reduce to equations of the form:
\nWaterConservationLoss = a_1 * ptend_1 + a_2 * ptend_2 + … + a_60 * ptend_60 -> 0.
\nAnd this means that the neural network will tend to zero target values (to a trivial solution).
Thanks! Interesting comment.
\n\n\nWaterConservationLoss is still large compared to the ideal solution.
\n
Does this mean that loss is not close enough to 0 in your experiment?
\n\n\nWaterConservationLoss = a_1 * ptend_1 + a_2 * ptend_2 + … + a_60 * ptend_60 -> 0.
\n
This equation transformation is interesting. I'll try to verify if I get this result too.
", "rawMarkdown": "Thanks! Interesting comment.\n\n>WaterConservationLoss is still large compared to the ideal solution.\n\nDoes this mean that loss is not close enough to 0 in your experiment?\n\n> WaterConservationLoss = a_1 * ptend_1 + a_2 * ptend_2 + … + a_60 * ptend_60 -> 0.\n\nThis equation transformation is interesting. I'll try to verify if I get this result too." } ] }, { "id": 2895004, "postDate": "2024-06-28T20:31:05.127Z", "content": "I think conservation law applied to the loss functions (e.g. physics informed neural networks https://en.wikipedia.org/wiki/Physics-informed_neural_networks) would increase the performance of the model excluding solutions which are not consistent with the underlying physical models. The problem in a single column model is that the lateral convection is ommited and conservation laws cannot be applied in a strict sence. Furthermore I looked in the references of the climSim paper but I could not figure out the exact set of equations they use when generating the data.
", "rawMarkdown": "I think conservation law applied to the loss functions (e.g. physics informed neural networks https://en.wikipedia.org/wiki/Physics-informed_neural_networks) would increase the performance of the model excluding solutions which are not consistent with the underlying physical models. The problem in a single column model is that the lateral convection is ommited and conservation laws cannot be applied in a strict sence. Furthermore I looked in the references of the climSim paper but I could not figure out the exact set of equations they use when generating the data.", "votes": 4, "replies": [ { "id": 2895033, "postDate": "2024-06-28T21:03:58.613Z", "content": "Thanks for your comment!
\n\n\nThe problem in a single column model is that the lateral convection is ommited and conservation laws cannot be applied in a strict sence.
\n
Surely the assumption that moisture content is conserved in the grid may be misaligned given horizontal convection.
", "rawMarkdown": "Thanks for your comment!\n> The problem in a single column model is that the lateral convection is ommited and conservation laws cannot be applied in a strict sence.\n\nSurely the assumption that moisture content is conserved **in the grid** may be misaligned given horizontal convection." } ] }, { "id": 2899379, "postDate": "2024-07-01T16:47:34.617Z", "content": "Thank you for sharing your wonderful insights. As a beginner in competitions, this kind of knowledge is very helpful. I have a question: what equations are used to derive this water conservation?
", "rawMarkdown": "Thank you for sharing your wonderful insights. As a beginner in competitions, this kind of knowledge is very helpful. I have a question: what equations are used to derive this water conservation?", "replies": [ { "id": 2900027, "postDate": "2024-07-02T03:19:20.110Z", "content": "Thanks! I refered the following documentation.
\nhttps://leap-stc.github.io/ClimSim/demo_notebooks/water_conservation.html
Thank you very much. I will refer to it.
", "rawMarkdown": "Thank you very much. I will refer to it." } ] } ] } ], "comments": [ { "id": 2897173, "author_name": "Amedeo Biolatti", "author_url": "", "post_date": "2024-06-30T09:48:52.213000", "content": "My models seems to learn water convervation well enough with just the MSE loss: https://www.kaggle.com/competitions/leap-atmospheric-physics-ai-climsim/discussion/508272
\nMoreover consider that the water quantity is not exactly conserved since the suspended precipitation is not tracked
", "votes": 3, "replies": [ { "id": 2897204, "author_name": "kuto", "author_url": "", "post_date": "2024-06-30T10:38:18.507000", "content": "Oh, I didn’t check your report.Thanks for sharing!
\n\n\nprecipitation is not tracked
\n
I see, Thanks for your advice.
", "votes": 0, "replies": [] } ] }, { "id": 2895632, "author_name": "Urazalinov Baurzhan", "author_url": "", "post_date": "2024-06-29T09:09:46.877000", "content": "\n\nSince the loss is decreasing with training, I speculate that the existing model implicitly considers the conservation laws.
\n
I'm not so sure about that. WaterConservationLoss can be decreased due to the operation of the basic loss function, but the WaterConservationLoss is still large compared to the ideal solution.
\n\n\nHowever, this did not contribute to improving the score in my model.
\n
I've noticed that conservation laws reduce to equations of the form:
\nWaterConservationLoss = a_1 * ptend_1 + a_2 * ptend_2 + … + a_60 * ptend_60 -> 0.
\nAnd this means that the neural network will tend to zero target values (to a trivial solution).
Thanks! Interesting comment.
\n\n\nWaterConservationLoss is still large compared to the ideal solution.
\n
Does this mean that loss is not close enough to 0 in your experiment?
\n\n\nWaterConservationLoss = a_1 * ptend_1 + a_2 * ptend_2 + … + a_60 * ptend_60 -> 0.
\n
This equation transformation is interesting. I'll try to verify if I get this result too.
", "votes": 0, "replies": [] } ] }, { "id": 2895004, "author_name": "HappyKiter", "author_url": "", "post_date": "2024-06-28T20:31:05.127000", "content": "I think conservation law applied to the loss functions (e.g. physics informed neural networks https://en.wikipedia.org/wiki/Physics-informed_neural_networks) would increase the performance of the model excluding solutions which are not consistent with the underlying physical models. The problem in a single column model is that the lateral convection is ommited and conservation laws cannot be applied in a strict sence. Furthermore I looked in the references of the climSim paper but I could not figure out the exact set of equations they use when generating the data.
", "votes": 4, "replies": [ { "id": 2895033, "author_name": "kuto", "author_url": "", "post_date": "2024-06-28T21:03:58.613000", "content": "Thanks for your comment!
\n\n\nThe problem in a single column model is that the lateral convection is ommited and conservation laws cannot be applied in a strict sence.
\n
Surely the assumption that moisture content is conserved in the grid may be misaligned given horizontal convection.
", "votes": 0, "replies": [] } ] }, { "id": 2899379, "author_name": "yasukawatoyoki", "author_url": "", "post_date": "2024-07-01T16:47:34.617000", "content": "Thank you for sharing your wonderful insights. As a beginner in competitions, this kind of knowledge is very helpful. I have a question: what equations are used to derive this water conservation?
", "votes": 0, "replies": [ { "id": 2900027, "author_name": "kuto", "author_url": "", "post_date": "2024-07-02T03:19:20.110000", "content": "Thanks! I refered the following documentation.
\nhttps://leap-stc.github.io/ClimSim/demo_notebooks/water_conservation.html
Thank you very much. I will refer to it.
", "votes": 0, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2894667": "The argument that the domain is important is shared in discussions [here](https://www.kaggle.com/competitions/leap-atmospheric-physics-ai-climsim/discussion/506984) and [here](https://www.kaggle.com/competitions/leap-atmospheric-physics-ai-climsim/discussion/508630). After seeing these discussions, I considered to apply the conservation laws into the model. \nBased on [this notebook](https://leap-stc.github.io/ClimSim/demo_notebooks/water_conservation.html), I created the water conservation loss function.\n\n```python\nimport xarray as xr\nimport torch\nfrom torch import nn\n\ndef get_pressure_thickness(a: torch.tensor, b: torch.tensor, p0: torch.tensor, ps: torch.tensor) -> torch.tensor:\n \"\"\"\n a: (61,)\n b: (61,)\n p0: (1,)\n ps: (n,)\n\n return: (n, 60)\n \"\"\"\n p = (a * p0).reshape(-1, 1) + (b.reshape(-1, 1) * ps.reshape(1, -1))\n return p.diff(dim=0).T\n\n\nclass WaterConservationLoss(nn.Module):\n \"\"\"\n ref: https://leap-stc.github.io/ClimSim/demo_notebooks/water_conservation.html\n \"\"\"\n def __init__(self) -> None:\n super().__init__()\n self.criterion = nn.MSELoss()\n self.a, self.b, self.p0 = self.load_initial_params()\n self.inv_g = 1 / 9.80616\n self.delta_t = 1200\n\n def load_initial_params(self) -> tuple[torch.tensor, torch.tensor, torch.tensor]:\n # ref: https://github.com/leap-stc/ClimSim/blob/main/grid_info/ClimSim_low-res_grid-info.nc\n initial_conditions = \"/home/user/work/climsim/grid_info/ClimSim_low-res_grid-info.nc\"\n with xr.open_dataset(initial_conditions) as inic:\n a = torch.tensor(inic[\"hyai\"].to_numpy())\n b = torch.tensor(inic[\"hybi\"].to_numpy())\n p0 = torch.tensor(inic[\"P0\"].to_numpy())\n return a, b, p0\n\n def apply_device(self, device: torch.device) -> None:\n self.a = self.a.to(device)\n self.b = self.b.to(device)\n self.p0 = self.p0.to(device)\n\n def forward(self, features: torch.tensor, preds: torch.tensor) -> torch.tensor:\n \"\"\"\n features: (n, 556)\n preds: (n, 368)\n \"\"\"\n self.apply_device(features.device)\n # ptend→state\n preds[:, :360] = preds[:, :360] * self.delta_t + features[:, :360]\n\n P = preds[:, TARGET_COLS.index(\"cam_out_PRECC\")] * 1000 # cam_out_PRECC\n ps = features[:, FEATURE_COLS.index(\"state_ps\")] # state_ps(surface pressure)\n delta_p = get_pressure_thickness(self.a, self.b, self.p0, ps)\n\n qv_in = (features[:, 60:120] * delta_p * self.inv_g).sum(dim=1)\n ql_in = (features[:, 120:180] * delta_p * self.inv_g).sum(dim=1)\n qi_in = (features[:, 180:240] * delta_p * self.inv_g).sum(dim=1)\n expected_total_water = (qv_in + ql_in + qi_in) - (P * self.delta_t)\n\n qv_out = (preds[:, 60:120] * delta_p * self.inv_g).sum(dim=1)\n ql_out = (preds[:, 120:180] * delta_p * self.inv_g).sum(dim=1)\n qi_out = (preds[:, 180:240] * delta_p * self.inv_g).sum(dim=1)\n total_water = qv_out + ql_out + qi_out\n\n loss = self.criterion(total_water, expected_total_water)\n return loss\n```\n\nThis loss function is based on the insight that the total moisture content within the grid is conserved. However, this did not contribute to improving the score in my model. Since the loss is decreasing with training, I speculate that the existing model implicitly considers the conservation laws.\n\nHas anyone managed to use these conservation law ideas to achieve improvements? or not work?\nIf there are any mistakes in my implementation, please feel free to point them out! ", "2897173": "My models seems to learn water convervation well enough with just the MSE loss: https://www.kaggle.com/competitions/leap-atmospheric-physics-ai-climsim/discussion/508272\n\nMoreover consider that the water quantity is not exactly conserved since the suspended precipitation is not tracked", "2895632": "> Since the loss is decreasing with training, I speculate that the existing model implicitly considers the conservation laws.\n\nI'm not so sure about that. WaterConservationLoss can be decreased due to the operation of the basic loss function, but the WaterConservationLoss is still large compared to the ideal solution.\n\n>However, this did not contribute to improving the score in my model. \n\nI've noticed that conservation laws reduce to equations of the form: \nWaterConservationLoss = a_1 * ptend_1 + a_2 * ptend_2 + ... + a_60 * ptend_60 -> 0. \nAnd this means that the neural network will tend to zero target values (to a trivial solution).", "2895004": "I think conservation law applied to the loss functions (e.g. physics informed neural networks https://en.wikipedia.org/wiki/Physics-informed_neural_networks) would increase the performance of the model excluding solutions which are not consistent with the underlying physical models. The problem in a single column model is that the lateral convection is ommited and conservation laws cannot be applied in a strict sence. Furthermore I looked in the references of the climSim paper but I could not figure out the exact set of equations they use when generating the data.", "2899379": "Thank you for sharing your wonderful insights. As a beginner in competitions, this kind of knowledge is very helpful. I have a question: what equations are used to derive this water conservation?" } }