Given some input, is it possible to deterministically know the output? Or is this an inverse problem, like inpainting, where it is impossible to know the true output from the input observations alone?
\nAlternatively, does the E3SM-MMF model always exactly resolve to the same output for some particular input? Or is there some uncertainty involved? Like if the simulation has some stochasticity or something. If there is no uncertainty, then simply regressing the output makes more sense. If otherwise, maybe a VAE (and ensembling on different random latents) can make more generalized predictions.
", "messages": [ { "id": 2814159, "postDate": "2024-05-15T07:43:56.903Z", "content": "Given some input, is it possible to deterministically know the output? Or is this an inverse problem, like inpainting, where it is impossible to know the true output from the input observations alone?
\nAlternatively, does the E3SM-MMF model always exactly resolve to the same output for some particular input? Or is there some uncertainty involved? Like if the simulation has some stochasticity or something. If there is no uncertainty, then simply regressing the output makes more sense. If otherwise, maybe a VAE (and ensembling on different random latents) can make more generalized predictions.
", "rawMarkdown": "Given some input, is it possible to deterministically know the output? Or is this an inverse problem, like inpainting, where it is impossible to know the true output from the input observations alone?\n\nAlternatively, does the E3SM-MMF model always exactly resolve to the same output for some particular input? Or is there some uncertainty involved? Like if the simulation has some stochasticity or something. If there is no uncertainty, then simply regressing the output makes more sense. If otherwise, maybe a VAE (and ensembling on different random latents) can make more generalized predictions.", "votes": 7 }, { "id": 2826199, "postDate": "2024-05-20T19:11:46.350Z", "content": "This is a great question! E3SM-MMF is not bit-for-bit reproducible, so the answer to your question is no. Additionally, the Cloud Resolving-Model being emulated also has an internal \"state\" that is not included in the inputs for the ML emulator due to data constraints + previous studies indicating that it may be of secondary importance.
\nhttps://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2019MS001610
", "rawMarkdown": "This is a great question! E3SM-MMF is not bit-for-bit reproducible, so the answer to your question is no. Additionally, the Cloud Resolving-Model being emulated also has an internal \"state\" that is not included in the inputs for the ML emulator due to data constraints + previous studies indicating that it may be of secondary importance.\n\nhttps://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2019MS001610", "votes": 2 }, { "id": 2826177, "postDate": "2024-05-20T18:58:20.837Z", "content": "Great question. If I understand you correctly, you're asking whether the E3SM-MMF (high-resolution, more-physics) model always produces the same values of the target variables (at low resolution), given the same values of the input variables (also at low resolution).
\nBased on my experience of a similar problem in astrophysical simulations, where we also have simulations that model smaller-scale processes to predict larger-scale variables, I would answer \"no\". But I would also say that we probably don't want to model the associated uncertainty, because it's unlikely to be physical.
\nLooking at the details of the E3SM-MMF model, it is governed in part by equations with chaotic solutions - gravity, fluid dynamics etc., similar to astrophysical simulations. The results of E3SM-MMF therefore depend sensitively on the initial conditions: not just on the physical ones we're given, but also on the numerical ones (RNGs, FP roundoff etc.) Unless the system being modeled is so large that this microscopic chaos averages out, the results will have some uncertainty.
\nBecause this dataset involves several different simulations with different configurations, I'd cautiously assume that this numerically-induced variance might be important, and might affect how well our regression models can predict the output, based on the input. I'd be very interested in any studies of this kind of variance in climate simulations…
", "rawMarkdown": "Great question. If I understand you correctly, you're asking whether the E3SM-MMF (high-resolution, more-physics) model always produces the same values of the target variables (at low resolution), given the same values of the input variables (also at low resolution).\n\nBased on my experience of a similar problem in astrophysical simulations, where we also have simulations that model smaller-scale processes to predict larger-scale variables, I would answer \"no\". But I would also say that we probably don't want to model the associated uncertainty, because it's unlikely to be physical.\n\nLooking at the details of the E3SM-MMF model, it is governed in part by equations with chaotic solutions - gravity, fluid dynamics etc., similar to astrophysical simulations. The results of E3SM-MMF therefore depend sensitively on the initial conditions: not just on the physical ones we're given, but also on the numerical ones (RNGs, FP roundoff etc.) Unless the system being modeled is so large that this microscopic chaos averages out, the results will have some uncertainty.\n\nBecause this dataset involves several different simulations with different configurations, I'd cautiously assume that this numerically-induced variance might be important, and might affect how well our regression models can predict the output, based on the input. I'd be very interested in any studies of this kind of variance in climate simulations...", "votes": 2, "replies": [ { "id": 2826186, "postDate": "2024-05-20T19:07:10.783Z", "content": "Hi Sarah! Thanks for expanding the discussion on this! Regarding your interest in studies of this kind, here are a few I'd recommend:
\nhttps://rmets.onlinelibrary.wiley.com/doi/10.1002/qj.49712757202
\n\nhttps://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2021MS002534
\n\n", "rawMarkdown": "Hi Sarah! Thanks for expanding the discussion on this! Regarding your interest in studies of this kind, here are a few I'd recommend:\n\nhttps://rmets.onlinelibrary.wiley.com/doi/10.1002/qj.49712757202\n\nhttps://www.researchgate.net/publication/228799825_Representing_model_uncertainty_in_weather_and_climate_prediction_Annu_Rev_Earth_Planet_Sci_33_163-193\n\nhttps://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2021MS002534\n\n> Great question. If I understand you correctly, you're asking whether the E3SM-MMF (high-resolution, more-physics) model always produces the same values of the target variables (at low resolution), given the same values of the input variables (also at low resolution).\n> \n> Based on my experience of a similar problem in astrophysical simulations, where we also have simulations that model smaller-scale processes to predict larger-scale variables, I would answer \"no\". But I would also say that we probably don't want to model the associated uncertainty, because it's unlikely to be physical.\n> \n> Looking at the details of the E3SM-MMF model, it is governed in part by equations with chaotic solutions - gravity, fluid dynamics etc., similar to astrophysical simulations. The results of E3SM-MMF therefore depend sensitively on the initial conditions: not just on the physical ones we're given, but also on the numerical ones (RNGs, FP roundoff etc.) Unless the system being modeled is so large that this microscopic chaos averages out, the results will have some uncertainty.\n> \n> Because this dataset involves several different simulations with different configurations, I'd cautiously assume that this numerically-induced variance might be important, and might affect how well our regression models can predict the output, based on the input. I'd be very interested in any studies of this kind of variance in climate simulations...\n\n", "votes": 1, "replies": [ { "id": 2826209, "postDate": "2024-05-20T19:16:44.817Z", "content": "Great question. If I understand you correctly, you're asking whether the E3SM-MMF (high-resolution, more-physics) model always produces the same values of the target variables (at low resolution), given the same values of the input variables (also at low resolution).
\nBased on my experience of a similar problem in astrophysical simulations, where we also have simulations that model smaller-scale processes to predict larger-scale variables, I would answer \"no\". But I would also say that we probably don't want to model the associated uncertainty, because it's unlikely to be physical.
\nLooking at the details of the E3SM-MMF model, it is governed in part by equations with chaotic solutions - gravity, fluid dynamics etc., similar to astrophysical simulations. The results of E3SM-MMF therefore depend sensitively on the initial conditions: not just on the physical ones we're given, but also on the numerical ones (RNGs, FP roundoff etc.) Unless the system being modeled is so large that this microscopic chaos averages out, the results will have some uncertainty.
\nBecause this dataset involves several different simulations with different configurations, I'd cautiously assume that this numerically-induced variance might be important, and might affect how well our regression models can predict the output, based on the input. I'd be very interested in any studies of this kind of variance in climate simulations…
\n
Thanks so much! I'll take a look at these.
", "rawMarkdown": "Thanks so much! I'll take a look at these." } ] } ] } ], "comments": [ { "id": 2826199, "author_name": "Jerry Lin", "author_url": "", "post_date": "2024-05-20T19:11:46.350000", "content": "This is a great question! E3SM-MMF is not bit-for-bit reproducible, so the answer to your question is no. Additionally, the Cloud Resolving-Model being emulated also has an internal \"state\" that is not included in the inputs for the ML emulator due to data constraints + previous studies indicating that it may be of secondary importance.
\nhttps://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2019MS001610
", "votes": 2, "replies": [] }, { "id": 2826177, "author_name": "Sarah Jeffreson", "author_url": "", "post_date": "2024-05-20T18:58:20.837000", "content": "Great question. If I understand you correctly, you're asking whether the E3SM-MMF (high-resolution, more-physics) model always produces the same values of the target variables (at low resolution), given the same values of the input variables (also at low resolution).
\nBased on my experience of a similar problem in astrophysical simulations, where we also have simulations that model smaller-scale processes to predict larger-scale variables, I would answer \"no\". But I would also say that we probably don't want to model the associated uncertainty, because it's unlikely to be physical.
\nLooking at the details of the E3SM-MMF model, it is governed in part by equations with chaotic solutions - gravity, fluid dynamics etc., similar to astrophysical simulations. The results of E3SM-MMF therefore depend sensitively on the initial conditions: not just on the physical ones we're given, but also on the numerical ones (RNGs, FP roundoff etc.) Unless the system being modeled is so large that this microscopic chaos averages out, the results will have some uncertainty.
\nBecause this dataset involves several different simulations with different configurations, I'd cautiously assume that this numerically-induced variance might be important, and might affect how well our regression models can predict the output, based on the input. I'd be very interested in any studies of this kind of variance in climate simulations…
", "votes": 2, "replies": [ { "id": 2826186, "author_name": "Jerry Lin", "author_url": "", "post_date": "2024-05-20T19:07:10.783000", "content": "Hi Sarah! Thanks for expanding the discussion on this! Regarding your interest in studies of this kind, here are a few I'd recommend:
\nhttps://rmets.onlinelibrary.wiley.com/doi/10.1002/qj.49712757202
\n\nhttps://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2021MS002534
\n\n", "votes": 1, "replies": [ { "id": 2826209, "author_name": "Sarah Jeffreson", "author_url": "", "post_date": "2024-05-20T19:16:44.817000", "content": "Great question. If I understand you correctly, you're asking whether the E3SM-MMF (high-resolution, more-physics) model always produces the same values of the target variables (at low resolution), given the same values of the input variables (also at low resolution).
\nBased on my experience of a similar problem in astrophysical simulations, where we also have simulations that model smaller-scale processes to predict larger-scale variables, I would answer \"no\". But I would also say that we probably don't want to model the associated uncertainty, because it's unlikely to be physical.
\nLooking at the details of the E3SM-MMF model, it is governed in part by equations with chaotic solutions - gravity, fluid dynamics etc., similar to astrophysical simulations. The results of E3SM-MMF therefore depend sensitively on the initial conditions: not just on the physical ones we're given, but also on the numerical ones (RNGs, FP roundoff etc.) Unless the system being modeled is so large that this microscopic chaos averages out, the results will have some uncertainty.
\nBecause this dataset involves several different simulations with different configurations, I'd cautiously assume that this numerically-induced variance might be important, and might affect how well our regression models can predict the output, based on the input. I'd be very interested in any studies of this kind of variance in climate simulations…
\n
Thanks so much! I'll take a look at these.
", "votes": 0, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2814159": "Given some input, is it possible to deterministically know the output? Or is this an inverse problem, like inpainting, where it is impossible to know the true output from the input observations alone?\n\nAlternatively, does the E3SM-MMF model always exactly resolve to the same output for some particular input? Or is there some uncertainty involved? Like if the simulation has some stochasticity or something. If there is no uncertainty, then simply regressing the output makes more sense. If otherwise, maybe a VAE (and ensembling on different random latents) can make more generalized predictions.", "2826199": "This is a great question! E3SM-MMF is not bit-for-bit reproducible, so the answer to your question is no. Additionally, the Cloud Resolving-Model being emulated also has an internal \"state\" that is not included in the inputs for the ML emulator due to data constraints + previous studies indicating that it may be of secondary importance.\n\nhttps://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2019MS001610", "2826177": "Great question. If I understand you correctly, you're asking whether the E3SM-MMF (high-resolution, more-physics) model always produces the same values of the target variables (at low resolution), given the same values of the input variables (also at low resolution).\n\nBased on my experience of a similar problem in astrophysical simulations, where we also have simulations that model smaller-scale processes to predict larger-scale variables, I would answer \"no\". But I would also say that we probably don't want to model the associated uncertainty, because it's unlikely to be physical.\n\nLooking at the details of the E3SM-MMF model, it is governed in part by equations with chaotic solutions - gravity, fluid dynamics etc., similar to astrophysical simulations. The results of E3SM-MMF therefore depend sensitively on the initial conditions: not just on the physical ones we're given, but also on the numerical ones (RNGs, FP roundoff etc.) Unless the system being modeled is so large that this microscopic chaos averages out, the results will have some uncertainty.\n\nBecause this dataset involves several different simulations with different configurations, I'd cautiously assume that this numerically-induced variance might be important, and might affect how well our regression models can predict the output, based on the input. I'd be very interested in any studies of this kind of variance in climate simulations..." } }