![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F14984949%2Fa0860242fa947c058ce0794fa0494cc4%2FScreenshot%202024-11-01%20200453.png?generation=1730488072791970&alt=media\")
Link to code to train and submit
\nThis year I'm also releasing the full development history: https://github.com/jcottaar/ariel2
\nI'd like to start by thanking the organizers for another great competition. I had a rather frustrating start (having somehow managed to lose all my code from last year), but greatly enjoyed it overall. The Bayesian approach is not used nearly as often as it should be these days, and I'm glad to have the opportunity to show its value here.
My solution is quite similar to last year's; my NeurIPS talk from the associated workshop can be a useful introduction. I'll briefly discuss the differences compared to last year in a reply to this post.
\nTransit analysis is the most common method for detecting and studying exoplanets. A transit occurs when a planet crosses in front of its star relative to Earth, obscuring part of the starlight. By observing how much the light is reduced (the transit depth) as a function of wavelength, we can learn the properties of the planet. But with a tiny planet passing in front of a huge star, this problem has a very low signal to noise ratio. The challenge to us: find the transit depth, including confidence intervals, from synthetic raw spectroscopic signals as the Ariel satellite might see them in the future (For a more extensive overview, see https://www.kaggle.com/competitions/ariel-data-challenge-2025/overview).
\n
This competition is at first glance a retread of a similar competition last year, but the data is far more realistic this time around. I think I figured out some of the changes, but not all of them - as we'll see later on.
\nNote that there are two sensors in play, the single-wavelength FGS sensor and the spectroscopic AIRS sensor. In this writeup I'll focus mainly on the AIRS sensor.
\nMy general approach consists of three steps, as visualized below:
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F14984949%2Fa0b169937d51e43db62a030def487f5c%2FScreenshot%202025-09-29%20212745.png?generation=1759174122624685&alt=media\")
In sections 2, 3, and 4, I'll discuss each of these steps in turn. I'll finish with some closing thoughts in section 5 - mainly discussing why my approach isn't actually working…
\nThe raw data is in counts per pixel; we first need to apply several steps to come to a photon count per pixel. This is explained in a notebook shared by the hosts. I use a time binning of 5 frames for AIRS, and 50 frames for FGS (this puts them approximately in sync). Hot pixel invalidation is disabled. I also added a simple cosmic ray removal.
\nThe opportunity for some real improvement lies in the wavelength binning step, i.e. summing over the dispersion axis. The baseline is simple summing, but there's several effects in the data that make it possible to do better. Most notably, there's the infamous jitter, which we can see most clearly by doing a PCA (with some additional preprocessing) over all AIRS frames in the train data:
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F14984949%2F6c3728e5a47c97f7371dc1f1d5fb8a27%2FScreenshot%202025-09-30%20090624.png?generation=1759216002725805&alt=media\")
The jitter shapes presumably correspond to pointing error and defocus.
\nThe jitter shapes sum to approximately zero over the dispersion axis; this is why simple summing works OK. It also means that any more advanced preprocessing scheme is forced to deal with the jitter, so there's a high barrier to entry. Still, there are several effects in the data that such a scheme can take advantage of:
\nI spent more than half of my time on this competition trying to set up a principled preprocessing flow optimally handling all these effects. Unfortunately, I haven't managed this yet - you'll find in the code several disjointed steps attempting to deal with the above. They also tend to work only in certain configurations and combinations, which I don't yet fully understand. It still leads to an improvement of ~0.01 over simple summing - I'll crack this one next year…
\nNow that we have the photon counts as a function of wavelength and time, we are ready to infer the exoplanet transit depth using Bayesian Inference.
\nBayesian Inference (BI) is a powerful statistical approach, based around defining a prior (a statistical belief about reality) and observations (some form of new information). Using Bayes' law, we then combine these to find the posterior (an updated belief about reality). In our case, this means:
There are two key elements to applying BI: defining the prior (section 3.1), and applying Bayes' law to do the inference (section 3.2).
\nThe signal is decomposed into four key components, as shown visually here:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F14984949%2Ffbb5d6dba1e6836c600b1473cd333121%2FScreenshot%202024-11-01%20153411.png?generation=1730488097922724&alt=media\")
This table shows a detailed description of all prior elements:
\n| Prior element | \nDescription | \nTuning and hyperparameters | \nDegrees of freedom | \n
|---|---|---|---|
| Noise | \nUncorrelated Gaussian per time and wavelength | \nStandard deviations found in preprocessing | \n1350 (FGS) + 317250 (AIRS) | \n
| Star spectrum | \nUncorrelated value per wavelength | \nNot regularized (infinite sigma) | \n283 | \n
| Drift | \nThird order polynomial over time and wavelength | \nNot regularized (infinite sigma) | \n3 (FGS) + 12 (AIRS) | \n
| Transit window | \nFound using batman package, with the following free parameters: - Transit mid-time t0 (separate for FGS and AIRS) - Semi-major axis sma - Period P - Orbital inclination i - Quadratic limb darkening u0 and u1 (including linear dependence on wavelength) - Transit depth Rp^2/Rs^2 (see below) | \nCovariance matrix learned on training set using maximum likelihood estimation (MLE) | \n11 (6 limb darkening, 2 t0, sma, P, i) | \n
| Transit depth: mean | \nSingle value | \nLightly regularized (sigma=0.01) | \n1 | \n
| Transit depth: variation | \nDecomposed into three components below | \nScaling factor determined per planet using MLE (this is the only hyperparameter tuned during inference) | \nN/A | \n
| Transit depth: variation FGS | \nSingle Gaussian value | \nStandard deviation found on training set | \n1 | \n
| Transit depth: variation AIRS | \nGaussian Process over wavelength, sum of two squared-exponential kernels | \nKernel hyperparameters found on training set using maximum likelihood estimation | \n282 | \n
| Transit depth: variation PCA | \nFixed basis functions obtained from PCA analysis | \nPCA shapes and variation magnitudes found on training data (this year there seems to be no train-test population shift for the transit depths) | \n5 | \n
Having defined the prior, finding the posterior is 'just' a matter of applying Bayes' law. The prior is fully Gaussian, which means the inference should just be a single matrix operation. But as usual, there's a snag. In our case, it's that the relationship between the prior parameters and the observation is non-linear (because of the non-linear modeling in batman, and because some prior elements are multiplied rather than added).
\nTo deal with this non-linearity, we linearize the model around the mean of the posterior. This is repeated iteratively (i.e. apply BI -> linearize around the new mean of the posterior -> repeat). Each step we also update the single hyperparameter (the scaling of the transit depth) using gradient descent.
\nThis approach does mean we need a decent starting point, which is handled as follows:
scipy.optimize.minimize. In this step we only consider the mean over wavelengths of the AIRS signal. The prediction of the transit depth and the covariance matrix of its uncertainty can be read out directly in the posterior. The covariance matrix is approximated based on 200 samples; this is mainly a holdover from last year, when there were too many parameters to construct it explicitly.
\nThere are two additional tricks that I apply if the residual of the first two steps (grid search->BFGS) is suspiciously high:
\nThe prediction and uncertainty obtained above can still be improved further with some post-hoc calibration. Note that this is anathema to a proper Bayesian approach - more on that below.
\nWe adapt the mean of the transit depth per sensor based on fitting:
\nThe variation of the transit depth over wavelength is not fudged, i.e. not changed in this step.
\nWe adapt the uncertainty of the transit depth per sensor based on fitting:
\nIn total, the fudging has 12 parameters to fit. These are determined on the full training set, based on whichever values optimize the competition metric.
\nThe fact that fudging helps the score is extremely worrying. And it's not subtle: last year, my solution without fudging could have gotten 1st place; my solution this year without fudging would have ended up around ~20th place. It means there's something missing in the prior, i.e. it does not describe reality (or rather the synthetic data generation process) accurately.
\nI've spent a lot of time trying to find this gap, but haven't gotten much closer. I've come to believe that the issue lies in the transit modeling, but none of the variation I tried (such as different limb darkening models) helped. I'm convinced that finding whatever's going on could boost our scores to well over 0.700.
\nSome more direct evidence of this issue can be found by comparing different transits for the same planet; the plot below shows the ratio of the transit profiles (with some preprocessing including low pass filtering). Nothing in the prior (and nothing else I can come up with) can explain this. See also this discussion and the code.
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F14984949%2F1b6f904fd58b66870aa7febac31ea864%2FScreenshot%202025-09-30%20111015.png?generation=1759223442391305&alt=media\")
\nI hope the organizers are willing to reveal some clues about this, because I don't think we'll figure it out next year either without some help.
Finally, I'll wrap it up with an overview of how much various elements of my solution contribute to the score:
\n| Change to model | \nImpact on private test score | \n
|---|---|
| Simplified loader (host's flow + cosmic ray removal), no jitter correction or background removal | \n-0.013 | \n
| Don't remove first or last few frames of suspicious transits | \n-0.002 | \n
| Don't use Gaussian Process for transit depth, just use PCA Note: this variation does not involve any Gaussian Processes | \n-0.007 | \n
| Don't use PCA for transit depth, just Gaussian Process | \n-0.011 | \n
| Disable all fudging - rely purely on the outcome of the Bayesian predictions | \n-0.152 (Last year: -0.000…) | \n
| Replace all fudging by a single multiplicative factor for the uncertainty | \n-0.017 | \n
| Disable fudging of sigma prediction based on AIRS variation | \n-0.005 | \n
| Disable fudging of mean prediction based on limb darkening parameters | \n-0.005 | \n
| Don't regularize the 11 transit parameters in prior | \n-0.003 | \n
Link to code to train and submit
\nThis year I'm also releasing the full development history: https://github.com/jcottaar/ariel2
\nI'd like to start by thanking the organizers for another great competition. I had a rather frustrating start (having somehow managed to lose all my code from last year), but greatly enjoyed it overall. The Bayesian approach is not used nearly as often as it should be these days, and I'm glad to have the opportunity to show its value here.
My solution is quite similar to last year's; my NeurIPS talk from the associated workshop can be a useful introduction. I'll briefly discuss the differences compared to last year in a reply to this post.
\nTransit analysis is the most common method for detecting and studying exoplanets. A transit occurs when a planet crosses in front of its star relative to Earth, obscuring part of the starlight. By observing how much the light is reduced (the transit depth) as a function of wavelength, we can learn the properties of the planet. But with a tiny planet passing in front of a huge star, this problem has a very low signal to noise ratio. The challenge to us: find the transit depth, including confidence intervals, from synthetic raw spectroscopic signals as the Ariel satellite might see them in the future (For a more extensive overview, see https://www.kaggle.com/competitions/ariel-data-challenge-2025/overview).
\n
This competition is at first glance a retread of a similar competition last year, but the data is far more realistic this time around. I think I figured out some of the changes, but not all of them - as we'll see later on.
\nNote that there are two sensors in play, the single-wavelength FGS sensor and the spectroscopic AIRS sensor. In this writeup I'll focus mainly on the AIRS sensor.
\nMy general approach consists of three steps, as visualized below:
In sections 2, 3, and 4, I'll discuss each of these steps in turn. I'll finish with some closing thoughts in section 5 - mainly discussing why my approach isn't actually working…
\nThe raw data is in counts per pixel; we first need to apply several steps to come to a photon count per pixel. This is explained in a notebook shared by the hosts. I use a time binning of 5 frames for AIRS, and 50 frames for FGS (this puts them approximately in sync). Hot pixel invalidation is disabled. I also added a simple cosmic ray removal.
\nThe opportunity for some real improvement lies in the wavelength binning step, i.e. summing over the dispersion axis. The baseline is simple summing, but there's several effects in the data that make it possible to do better. Most notably, there's the infamous jitter, which we can see most clearly by doing a PCA (with some additional preprocessing) over all AIRS frames in the train data:
The jitter shapes presumably correspond to pointing error and defocus.
\nThe jitter shapes sum to approximately zero over the dispersion axis; this is why simple summing works OK. It also means that any more advanced preprocessing scheme is forced to deal with the jitter, so there's a high barrier to entry. Still, there are several effects in the data that such a scheme can take advantage of:
\nI spent more than half of my time on this competition trying to set up a principled preprocessing flow optimally handling all these effects. Unfortunately, I haven't managed this yet - you'll find in the code several disjointed steps attempting to deal with the above. They also tend to work only in certain configurations and combinations, which I don't yet fully understand. It still leads to an improvement of ~0.01 over simple summing - I'll crack this one next year…
\nNow that we have the photon counts as a function of wavelength and time, we are ready to infer the exoplanet transit depth using Bayesian Inference.
\nBayesian Inference (BI) is a powerful statistical approach, based around defining a prior (a statistical belief about reality) and observations (some form of new information). Using Bayes' law, we then combine these to find the posterior (an updated belief about reality). In our case, this means:
There are two key elements to applying BI: defining the prior (section 3.1), and applying Bayes' law to do the inference (section 3.2).
\nThe signal is decomposed into four key components, as shown visually here:
\n
This table shows a detailed description of all prior elements:
\n| Prior element | \nDescription | \nTuning and hyperparameters | \nDegrees of freedom | \n
|---|---|---|---|
| Noise | \nUncorrelated Gaussian per time and wavelength | \nStandard deviations found in preprocessing | \n1350 (FGS) + 317250 (AIRS) | \n
| Star spectrum | \nUncorrelated value per wavelength | \nNot regularized (infinite sigma) | \n283 | \n
| Drift | \nThird order polynomial over time and wavelength | \nNot regularized (infinite sigma) | \n3 (FGS) + 12 (AIRS) | \n
| Transit window | \nFound using batman package, with the following free parameters: - Transit mid-time t0 (separate for FGS and AIRS) - Semi-major axis sma - Period P - Orbital inclination i - Quadratic limb darkening u0 and u1 (including linear dependence on wavelength) - Transit depth Rp^2/Rs^2 (see below) | \nCovariance matrix learned on training set using maximum likelihood estimation (MLE) | \n11 (6 limb darkening, 2 t0, sma, P, i) | \n
| Transit depth: mean | \nSingle value | \nLightly regularized (sigma=0.01) | \n1 | \n
| Transit depth: variation | \nDecomposed into three components below | \nScaling factor determined per planet using MLE (this is the only hyperparameter tuned during inference) | \nN/A | \n
| Transit depth: variation FGS | \nSingle Gaussian value | \nStandard deviation found on training set | \n1 | \n
| Transit depth: variation AIRS | \nGaussian Process over wavelength, sum of two squared-exponential kernels | \nKernel hyperparameters found on training set using maximum likelihood estimation | \n282 | \n
| Transit depth: variation PCA | \nFixed basis functions obtained from PCA analysis | \nPCA shapes and variation magnitudes found on training data (this year there seems to be no train-test population shift for the transit depths) | \n5 | \n
Having defined the prior, finding the posterior is 'just' a matter of applying Bayes' law. The prior is fully Gaussian, which means the inference should just be a single matrix operation. But as usual, there's a snag. In our case, it's that the relationship between the prior parameters and the observation is non-linear (because of the non-linear modeling in batman, and because some prior elements are multiplied rather than added).
\nTo deal with this non-linearity, we linearize the model around the mean of the posterior. This is repeated iteratively (i.e. apply BI -> linearize around the new mean of the posterior -> repeat). Each step we also update the single hyperparameter (the scaling of the transit depth) using gradient descent.
\nThis approach does mean we need a decent starting point, which is handled as follows:
scipy.optimize.minimize. In this step we only consider the mean over wavelengths of the AIRS signal. The prediction of the transit depth and the covariance matrix of its uncertainty can be read out directly in the posterior. The covariance matrix is approximated based on 200 samples; this is mainly a holdover from last year, when there were too many parameters to construct it explicitly.
\nThere are two additional tricks that I apply if the residual of the first two steps (grid search->BFGS) is suspiciously high:
\nThe prediction and uncertainty obtained above can still be improved further with some post-hoc calibration. Note that this is anathema to a proper Bayesian approach - more on that below.
\nWe adapt the mean of the transit depth per sensor based on fitting:
\nThe variation of the transit depth over wavelength is not fudged, i.e. not changed in this step.
\nWe adapt the uncertainty of the transit depth per sensor based on fitting:
\nIn total, the fudging has 12 parameters to fit. These are determined on the full training set, based on whichever values optimize the competition metric.
\nThe fact that fudging helps the score is extremely worrying. And it's not subtle: last year, my solution without fudging could have gotten 1st place; my solution this year without fudging would have ended up around ~20th place. It means there's something missing in the prior, i.e. it does not describe reality (or rather the synthetic data generation process) accurately.
\nI've spent a lot of time trying to find this gap, but haven't gotten much closer. I've come to believe that the issue lies in the transit modeling, but none of the variation I tried (such as different limb darkening models) helped. I'm convinced that finding whatever's going on could boost our scores to well over 0.700.
\nSome more direct evidence of this issue can be found by comparing different transits for the same planet; the plot below shows the ratio of the transit profiles (with some preprocessing including low pass filtering). Nothing in the prior (and nothing else I can come up with) can explain this. See also this discussion and the code.
\n
\nI hope the organizers are willing to reveal some clues about this, because I don't think we'll figure it out next year either without some help.
Finally, I'll wrap it up with an overview of how much various elements of my solution contribute to the score:
\n| Change to model | \nImpact on private test score | \n
|---|---|
| Simplified loader (host's flow + cosmic ray removal), no jitter correction or background removal | \n-0.013 | \n
| Don't remove first or last few frames of suspicious transits | \n-0.002 | \n
| Don't use Gaussian Process for transit depth, just use PCA Note: this variation does not involve any Gaussian Processes | \n-0.007 | \n
| Don't use PCA for transit depth, just Gaussian Process | \n-0.011 | \n
| Disable all fudging - rely purely on the outcome of the Bayesian predictions | \n-0.152 (Last year: -0.000…) | \n
| Replace all fudging by a single multiplicative factor for the uncertainty | \n-0.017 | \n
| Disable fudging of sigma prediction based on AIRS variation | \n-0.005 | \n
| Disable fudging of mean prediction based on limb darkening parameters | \n-0.005 | \n
| Don't regularize the 11 transit parameters in prior | \n-0.003 | \n
Key differences with my 2024 solution:
\nCongrats!
\nI wonder do you have some high-level insights on why Bayesian without NN work well in 2024 and 2025. As our team tried end-to-end deep neural networks (DNNs) which is extremely vulnerable and still need ensemble with other methods for robustness.
\nMy insight is that
\n1) The actual noise functions is linearly like so that DNNs overfit it.
\n2) The training data is small so that DNNs overfit it.
\n3) The data sample is not infomation-abundent along with outliers so that DNNs overfit it.
Bayesian methods have an edge here because the underlying phsyics are relatively simple, but at the same time challenging for neural networks to identify. You just can't get away with ignoring the domain knowledge - neural networks are not going to learn that here (except perhaps if you provide enough training data so that they can essentially memorize all solutions).
", "rawMarkdown": "Bayesian methods have an edge here because the underlying phsyics are relatively simple, but at the same time challenging for neural networks to identify. You just can't get away with ignoring the domain knowledge - neural networks are not going to learn that here (except perhaps if you provide enough training data so that they can essentially memorize all solutions).", "votes": 2 } ] }, { "id": 3296578, "postDate": "2025-10-01T08:00:16.177Z", "content": "Thanks for the nice summary, @jeroencottaar !
\nIn your model you did not restrict the quadratic u1/u2 LD coefficients, so they could converge to Limb Brightening regime, right?
They are regularized but not constrained. I highly doubt they would have negative values though (that's what you mean with limb brightening right?), that'd be very far outside the training population.
\nI'll run some submission tonight to check this explicitly.
", "rawMarkdown": "They are regularized but not constrained. I highly doubt they would have negative values though (that's what you mean with limb brightening right?), that'd be very far outside the training population. \n\nI'll run some submission tonight to check this explicitly.", "votes": 1, "replies": [ { "id": 3296608, "postDate": "2025-10-01T09:06:58.727Z", "content": "Thanks @jeroencottaar !
\nJust for reference - Figure 1 of Kipping paper illustrates physical constraints on quadratic u1, u2 values.
I checked, and see that I already had constraints imposed on the limb darkening parameters. No negative values occcurred in my winning submission.
", "rawMarkdown": "I checked, and see that I already had constraints imposed on the limb darkening parameters. No negative values occcurred in my winning submission." } ] } ] }, { "id": 3296396, "postDate": "2025-09-30T19:26:04.147Z", "content": "Congrats for the 1st place finish!
\nI'm curious how well your model fits the ingress/egress period.
\nIn my model, I also had to do adjustment to the mean transit depth and I found the likely reason to be the limb darkening model not fitting well near the limbs (as well as lack of observations near the center), which greatly affects the transit depth distortion (measured depth vs (Rp/Rs)2 ) since the stellar flux is heavily dependent on the limb darkening assumption.
", "rawMarkdown": "Congrats for the 1st place finish!\n\nI'm curious how well your model fits the ingress/egress period.\n\nIn my model, I also had to do adjustment to the mean transit depth and I found the likely reason to be the limb darkening model not fitting well near the limbs (as well as lack of observations near the center), which greatly affects the transit depth distortion (measured depth vs (Rp/Rs)2 ) since the stellar flux is heavily dependent on the limb darkening assumption.", "replies": [ { "id": 3296603, "postDate": "2025-10-01T08:53:09.717Z", "content": "The main reason to adjust the mean transit depth is related to how well you do the constant background correction in preprocessing. If you don't do any, you should expect a ~0.6% correction on the AIRS mean. I think my model ends up needing ~0.1%.
\nIn general the ingress/egress period is fitted slightly worse than the rest of the transit (i.e. this range has slightly higher residuals), but not to the degree that it explains the need for fudging in my estimation.
", "rawMarkdown": "The main reason to adjust the mean transit depth is related to how well you do the constant background correction in preprocessing. If you don't do any, you should expect a ~0.6% correction on the AIRS mean. I think my model ends up needing ~0.1%.\n\nIn general the ingress/egress period is fitted slightly worse than the rest of the transit (i.e. this range has slightly higher residuals), but not to the degree that it explains the need for fudging in my estimation.", "replies": [ { "id": 3296849, "postDate": "2025-10-01T18:42:00.300Z", "content": "There is also the issue that different limb darkening models that can fit the observed data similarly well, but lead to different transit depth estimation.
\nWhich means there is a limit in how well we can estimate solely from the observed flux data, without incoporating further stellar properties.
To give an example of how limb darkening assumption can seriously affect transit depth estimation:
\nI fitted both exponential and quadratic on planet 3814020142's light curve.
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F13392195%2F850f9369b2a65c9bdac68d0e4a45b2dd%2F10.png?generation=1759344067867790&alt=media\")
The residuals are similar, and the observed dip in flux is similar. However, there is a serious consequence of the choice of parametric limb darkening modeling. The best-ft quadratic model results in a disk-average intensity of 0.907 while the best-fit exponential results in a disk-average intensity of .884. This is because as you can see from the bottom graph, the exponential describes a much sharper drop in intensity towards the limbs.
\nThis means that choosing quadratic limb darkening results in 2.5% higher center-to-disk average intensity ratio. This leads to 1.8% higher (Rp/Rs)^2 estimation in the model. From my observations, polynomial limb darkening with higher linear coefficient tend to overestimate (Rp/Rs)^2.
", "rawMarkdown": "There is also the issue that different limb darkening models that can fit the observed data similarly well, but lead to different transit depth estimation. \nWhich means there is a limit in how well we can estimate solely from the observed flux data, without incoporating further stellar properties. \n\nTo give an example of how limb darkening assumption can seriously affect transit depth estimation:\nI fitted both exponential and quadratic on planet 3814020142's light curve.\n\n\n\nThe residuals are similar, and the observed dip in flux is similar. However, there is a serious consequence of the choice of parametric limb darkening modeling. The best-ft quadratic model results in a disk-average intensity of 0.907 while the best-fit exponential results in a disk-average intensity of .884. This is because as you can see from the bottom graph, the exponential describes a much sharper drop in intensity towards the limbs.\n\n This means that choosing quadratic limb darkening results in 2.5% higher center-to-disk average intensity ratio. This leads to 1.8% higher (Rp/Rs)^2 estimation in the model. From my observations, polynomial limb darkening with higher linear coefficient tend to overestimate (Rp/Rs)^2." }, { "id": 3296854, "postDate": "2025-10-01T19:00:12.980Z", "content": "Interesting. Perhaps the 'missing element' of my prior that leads to the need for fudging is the use of different limb darkening models for different planets (I couldn't get any improvement by switching all planets to a different model, but perhaps I should have been flexible - would have been a nice opportunity for some further Bayesian tricks).
", "rawMarkdown": "Interesting. Perhaps the 'missing element' of my prior that leads to the need for fudging is the use of different limb darkening models for different planets (I couldn't get any improvement by switching *all* planets to a different model, but perhaps I should have been flexible - would have been a nice opportunity for some further Bayesian tricks)." } ] } ] } ], "comments": [ { "id": 3296154, "author_name": "Jeroen Cottaar", "author_url": "", "post_date": "2025-09-30T09:43:01.380000", "content": "Key differences with my 2024 solution:
\nCongrats!
\nI wonder do you have some high-level insights on why Bayesian without NN work well in 2024 and 2025. As our team tried end-to-end deep neural networks (DNNs) which is extremely vulnerable and still need ensemble with other methods for robustness.
\nMy insight is that
\n1) The actual noise functions is linearly like so that DNNs overfit it.
\n2) The training data is small so that DNNs overfit it.
\n3) The data sample is not infomation-abundent along with outliers so that DNNs overfit it.
Bayesian methods have an edge here because the underlying phsyics are relatively simple, but at the same time challenging for neural networks to identify. You just can't get away with ignoring the domain knowledge - neural networks are not going to learn that here (except perhaps if you provide enough training data so that they can essentially memorize all solutions).
", "votes": 2, "replies": [] } ] }, { "id": 3296578, "author_name": "Oleh Kivernyk", "author_url": "", "post_date": "2025-10-01T08:00:16.177000", "content": "Thanks for the nice summary, @jeroencottaar !
\nIn your model you did not restrict the quadratic u1/u2 LD coefficients, so they could converge to Limb Brightening regime, right?
They are regularized but not constrained. I highly doubt they would have negative values though (that's what you mean with limb brightening right?), that'd be very far outside the training population.
\nI'll run some submission tonight to check this explicitly.
", "votes": 1, "replies": [ { "id": 3296608, "author_name": "Oleh Kivernyk", "author_url": "", "post_date": "2025-10-01T09:06:58.727000", "content": "Thanks @jeroencottaar !
\nJust for reference - Figure 1 of Kipping paper illustrates physical constraints on quadratic u1, u2 values.
I checked, and see that I already had constraints imposed on the limb darkening parameters. No negative values occcurred in my winning submission.
", "votes": 0, "replies": [] } ] } ] }, { "id": 3296396, "author_name": "JungleBeastDS", "author_url": "", "post_date": "2025-09-30T19:26:04.147000", "content": "Congrats for the 1st place finish!
\nI'm curious how well your model fits the ingress/egress period.
\nIn my model, I also had to do adjustment to the mean transit depth and I found the likely reason to be the limb darkening model not fitting well near the limbs (as well as lack of observations near the center), which greatly affects the transit depth distortion (measured depth vs (Rp/Rs)2 ) since the stellar flux is heavily dependent on the limb darkening assumption.
", "votes": 0, "replies": [ { "id": 3296603, "author_name": "Jeroen Cottaar", "author_url": "", "post_date": "2025-10-01T08:53:09.717000", "content": "The main reason to adjust the mean transit depth is related to how well you do the constant background correction in preprocessing. If you don't do any, you should expect a ~0.6% correction on the AIRS mean. I think my model ends up needing ~0.1%.
\nIn general the ingress/egress period is fitted slightly worse than the rest of the transit (i.e. this range has slightly higher residuals), but not to the degree that it explains the need for fudging in my estimation.
", "votes": 0, "replies": [ { "id": 3296849, "author_name": "JungleBeastDS", "author_url": "", "post_date": "2025-10-01T18:42:00.300000", "content": "There is also the issue that different limb darkening models that can fit the observed data similarly well, but lead to different transit depth estimation.
\nWhich means there is a limit in how well we can estimate solely from the observed flux data, without incoporating further stellar properties.
To give an example of how limb darkening assumption can seriously affect transit depth estimation:
\nI fitted both exponential and quadratic on planet 3814020142's light curve.
The residuals are similar, and the observed dip in flux is similar. However, there is a serious consequence of the choice of parametric limb darkening modeling. The best-ft quadratic model results in a disk-average intensity of 0.907 while the best-fit exponential results in a disk-average intensity of .884. This is because as you can see from the bottom graph, the exponential describes a much sharper drop in intensity towards the limbs.
\nThis means that choosing quadratic limb darkening results in 2.5% higher center-to-disk average intensity ratio. This leads to 1.8% higher (Rp/Rs)^2 estimation in the model. From my observations, polynomial limb darkening with higher linear coefficient tend to overestimate (Rp/Rs)^2.
", "votes": 0, "replies": [] }, { "id": 3296854, "author_name": "Jeroen Cottaar", "author_url": "", "post_date": "2025-10-01T19:00:12.980000", "content": "Interesting. Perhaps the 'missing element' of my prior that leads to the need for fudging is the use of different limb darkening models for different planets (I couldn't get any improvement by switching all planets to a different model, but perhaps I should have been flexible - would have been a nice opportunity for some further Bayesian tricks).
", "votes": 0, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "3296150": "[Link to code to train and submit](https://www.kaggle.com/code/jeroencottaar/ariel2025-1st-place-bayesian-exoplanet-analysis)\nThis year I'm also releasing the full development history: [https://github.com/jcottaar/ariel2](https://github.com/jcottaar/ariel2)