I am interested in other ideas to preprocess the targets before the training procedure. Until now, I have been using just a standard normalization (as proposed in this notebook from @abiolatti ). I also experimented with other clipping values for the std (the original notebook proposed 1e-10), but all other values gave worse results. I also experimented with the PowerTransformer and QuantileTransformer from sklearn, but they also did not worked very well.
\nMy hunch is that there must be a better transformation, that helps the model better learn the outliers in the high-risk target variables.
\nI am interested in your thoughts on this!
\nEDIT1: I am aware of the trick to replace variables q0002_12 to _27. Do you still use them in your model? Or do you just ignore them, and replace them afterwards?
\nEDIT2: of course, the same question applies to the input variables. Are you using something different than a standard normalization?
", "messages": [ { "id": 2895076, "postDate": "2024-06-28T22:51:40.023Z", "content": "Mooers et al. 2021 used range normalization. 😉
\nhttps://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2020MS002385
", "rawMarkdown": "Mooers et al. 2021 used range normalization. 😉\n\nhttps://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2020MS002385", "votes": 1 }, { "id": 2895374, "postDate": "2024-06-29T06:29:03.267Z", "content": "I tried gauss rank, z score, rms, min max and all of them on log scale but rms was the best.
", "rawMarkdown": "I tried gauss rank, z score, rms, min max and all of them on log scale but rms was the best.", "votes": 2 }, { "id": 2889541, "postDate": "2024-06-25T14:48:49.867Z", "content": "I am interested in other ideas to preprocess the targets before the training procedure. Until now, I have been using just a standard normalization (as proposed in this notebook from @abiolatti ). I also experimented with other clipping values for the std (the original notebook proposed 1e-10), but all other values gave worse results. I also experimented with the PowerTransformer and QuantileTransformer from sklearn, but they also did not worked very well.
\nMy hunch is that there must be a better transformation, that helps the model better learn the outliers in the high-risk target variables.
\nI am interested in your thoughts on this!
\nEDIT1: I am aware of the trick to replace variables q0002_12 to _27. Do you still use them in your model? Or do you just ignore them, and replace them afterwards?
\nEDIT2: of course, the same question applies to the input variables. Are you using something different than a standard normalization?
", "rawMarkdown": "I am interested in other ideas to preprocess the targets before the training procedure. Until now, I have been using just a standard normalization (as proposed in [this notebook](https://www.kaggle.com/code/abiolatti/keras-baseline-seq2seq) from @abiolatti ). I also experimented with other clipping values for the std (the original notebook proposed 1e-10), but all other values gave worse results. I also experimented with the PowerTransformer and QuantileTransformer from sklearn, but they also did not worked very well.\n\nMy hunch is that there must be a better transformation, that helps the model better learn the outliers in the high-risk target variables.\n\nI am interested in your thoughts on this!\n\nEDIT1: I am aware of the trick to replace variables q0002_12 to _27. Do you still use them in your model? Or do you just ignore them, and replace them afterwards?\n\nEDIT2: of course, the same question applies to the input variables. Are you using something different than a standard normalization?", "votes": 2 } ], "comments": [ { "id": 2895076, "author_name": "Jerry Lin", "author_url": "", "post_date": "2024-06-28T22:51:40.023000", "content": "Mooers et al. 2021 used range normalization. 😉
\nhttps://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2020MS002385
", "votes": 1, "replies": [] }, { "id": 2895374, "author_name": "Gunes Evitan", "author_url": "", "post_date": "2024-06-29T06:29:03.267000", "content": "I tried gauss rank, z score, rms, min max and all of them on log scale but rms was the best.
", "votes": 2, "replies": [] } ], "raw_markdown_by_id": { "2895076": "Mooers et al. 2021 used range normalization. 😉\n\nhttps://agupubs.onlinelibrary.wiley.com/doi/full/10.1029/2020MS002385", "2895374": "I tried gauss rank, z score, rms, min max and all of them on log scale but rms was the best.", "2889541": "I am interested in other ideas to preprocess the targets before the training procedure. Until now, I have been using just a standard normalization (as proposed in [this notebook](https://www.kaggle.com/code/abiolatti/keras-baseline-seq2seq) from @abiolatti ). I also experimented with other clipping values for the std (the original notebook proposed 1e-10), but all other values gave worse results. I also experimented with the PowerTransformer and QuantileTransformer from sklearn, but they also did not worked very well.\n\nMy hunch is that there must be a better transformation, that helps the model better learn the outliers in the high-risk target variables.\n\nI am interested in your thoughts on this!\n\nEDIT1: I am aware of the trick to replace variables q0002_12 to _27. Do you still use them in your model? Or do you just ignore them, and replace them afterwards?\n\nEDIT2: of course, the same question applies to the input variables. Are you using something different than a standard normalization?" } }