{ "id": 609453, "title": "14th private/4th public Solution- Brute Force Posterior Estimation", "url": "/competitions/ariel-data-challenge-2025/discussion/609453", "author_name": "JungleBeastDS", "post_date": "2025-09-26T18:11:34.937000", "votes": 5, "comment_count": 2, "views": 0, "content": "

Brute Force Posterior Estimation of Transit Geometry + Limb Darkening modeling (and also what went wrong)

\n

Summary

\n

A search grid was used to generate a range of possible Transit geometry. Then, a light curve was fit for each point on the grid. Using likelihood of the errors, we can estimate the posterior distribution in a Bayesian manner. The error that dropped my score on the leaderboards was most likely caused by a coarse search grid for partial transiting planets, where the geometry was a lot more sensitive.

\n

The Bayesian approach to modeling transit uncertainties was inspired by last year’s second place winner: I briefly tried Monte Carlo sampling approaches, but search grid was faster (if restricted to a small region around good initial guess), robust, more accurate and could fully utilize the parallel processing of GPUs. However, as I learned more about limb darkening and needed more complex LD models to fit the observations, I ran into severe complexity and runtime issues, so I am not sure if this was the best approach for this competition.
\nMy limb darkening model is unfortunately incomplete and a huge weakness of the model. The final submission used an ensemble of linear limb darkening and 4 parameter limb darkening generated from Atlas searched by Temperature and surface gravity.
\nPre-processing: I used my last year’s processing with the addition of cutting out extreme outliers. I also moved everything to the T4 x2 GPUs.

\n

Transit path calibration

\n

The reason I used a Bayesian framework is that we were given noisy measurements for various geometric properties. And just off initial observation, some of the inclination angle did not seem to fit the data well. Thus, I needed a way to account for the uncertainties in the measurement and how they contribute the final measurement of transit depth.

\n

Initial Transit Detection

\n

A simple algorithm, similar to what were commonly used in previous year’s competitions, was used to estimate the initial guess for the start and end of the Transit zone.

\n

Priors:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
VariablePrior Assumption/Search grid
π/2-inclination anglemeasured inclination angle + uniform(-.2, 0.2) in radians, not including the paths that result in no transit, and extended out if needed. This parameter is quite sensitive, as the gradient of the angle is multiplied by the sma in the transit geometry. Some of the measured angles clearly do not agree with the observations, especially for high impact parameter transits (the partial transits).
Start/End of ingress/egress periodequal weight roughly centered around initial transit detection algorithm), extended if needed
smaThe sma is a lot less sensitive than inclination angle, so was fixed to save compute time.
\n

Model:

\n

The transit path for the light curve was calculated using inclination angle, sma, and start/end of ingress/egress period.
\nLimb Darkening (LD): linear limb darkening law was initially used as it was easy to fit.
\nTo speed up processing, the light blocked by the planet was approximated as follows:
\n\"\"
\n, approximated by
\n\"\"
\n\"\"
\nWhere Ia is the average intensity across the entire stellar disk, calculated by integrating the limb darkening function r є [0,1] and θ є [0,2π]. Iaπ is the normalized out of transit flux. Delta is the transit depth (Planet/Star cross section ratio).
\nFinally, combining the out of transit flux together with gaindrift and adding noise, we model the observations as follows.
\n\"\"

\n

A grid search, optimized on GPU was utilized to estimate the Posterior Distribution. I tried using 32 bit float for speed up, but it doesn’t have enough precision for degree 4 polynomial. I initially assumed Gaussian error and weighted each outcome accordingly, but it did not perform much better than taking the simple average/std of the region around the best fit path, which was faster to compute and less complex (Although, this would inadvertently lead to the error that cost me to drop in the private leaderboard).
\n\"\"
\nFigure 1: Posterior distribution of transit paths fit to observations. Kind of interesting my model predicts two main regions for candidate transit paths, with the region crossing the center having higher probability. The space between many of the lines is due to discretized inclination angle mesh.
\n\"\"Figure 2. This is a partial transit. The reason the posterior distribution is so small is that the partial transiting path has to be quite specific to fit to the observations. My inclination angle mesh was probably not fine enough to model the high sensitivities. And this is most likely what caused an error in private set, when a planet id was not well fitted. There weren’t many partial transiting planets, and so I did not focus on them and let this one slip.
\n\"\"
\nFigure 3. Left: best fit path. Right: inclination angle 2 grid points away (0.02 radians).
\nSpectra
\nAfter the light curve was fit, I took the top 20 paths and used them to fit the individual AIRS wavelengths and averaged the results. The light curve polynomial shape was used to fit the individual wavelengths.
\nLimb Darkening Modeling
\nHowever, I soon noticed severe systematic errors on my transit estimates. While Linear limb darkening does a decent job at modeling the path traversed by the planet since the Planet is only near the limbs during the ingress/egress period, limb darkening near the limbs has higher weight and sensitivity on the disk-average-stellar intensity (I¬a ). And because we need (I¬a ) to calculate the distortion between observed transit depth vs (Rp/Rs)2, the assumption of limb darkening near the limbs leads to systematic fitting errors. As a result, my transit depths were overestimated for higher limb darkening coefficients and underestimated for lower limb darkening coefficients.
\n\"\"
\nFigure 4: Linear limb darkening law limitations, Best fit Linear Law (green) and Observed Limb Darkening vs u (cosine of angle of incidence).

\n

I spent the rest of the competition trying to fit more complex LD shapes using Atlas generated LD models for the Temperature and Specific gravity as a starting point, but added complexity made it very hard to proceed with my search grid method and I needed to overhaul major parts of the model to fit within runtime. I was exploring two different approaches: 1. Fitting 2 parameter exponential limb darkening integrating into my existing method. 2. Estimating the limb darkening first using Atlas models. Then, fitting a 2d-gaussian process across u and wavelength. There is clear covariance between limb darkening of different wavelengths and I wanted to model that, but unfortunately could not find a solution in time.
\nI ran out of time and decided to spend the last week to submit what I had with major simplifications. In the simplified limb darkening model, I only focused on only the spectra. I took out the ingress/egress periods and used a lookup table with 4-parameter coefficients generated from Atlas model to fit the different wavelengths (which unfortunately still did not fit the limbs well). It was then ensembled with the linear model.

\n

Limb Darkening correction:

\n

I used a machine learning method (lgb) to estimate the average stellar intensity for the light curve, based on Rs,Mp/Ms, ip, and linear LDC. Then used it to adjust the predict mean transit values. Not ideal, but a fast way to boost my score at the end. I was aware machine learning methods performed terribly in last year’s competitions with distribution shifts, but it seemed to work on my local CV and the variables I used were mostly geometric (except planet mass, I still don’t understand why this variable result in better performance) and less dependent on chemical composition and other stellar properties.
\nPost-processing: I used this post processing https://www.kaggle.com/competitions/ariel-data-challenge-2024/writeups/c-number-daiwakun-1st-place-solution.

\n

Mean and Sigma estimation:

\n

Mean

\n

Light curve mean + post-processing spectra shape
\nUnlike last year, the presence of limb darkening and the severely low signal-to-noise in individual wavelengths made the absolute transit depth values for the individual wavelengths very unreliable. So I needed to make separate predictions for average transit depth using the light curve and shape using the denoised measurements of the spectra. As such that meant dealing with two different measurements of uncertainty.

\n

Sigma

\n

light curve std (estimated from posterior distribution) + mae of transit depth vs. wavelength curve. So it’s kind of interesting that my std were flat for each wavelength. Maybe I should have sampled post-processed spectra shapes as well.

\n

FSG1

\n

Unfortunately, I did not have the chance to look at FSG1. As it only had around 20% weight and was by no means an easy task, I decided to prioritize improving limb darkening model for AIRS. I just used mean + a higher sigma value.

\n

Second observation

\n

Just repeated the same model and averaged mean and std. I don’t think it helped at all, but certainly caused bugs in my code XD.

\n

What went wrong

\n

The grid that I was using was not fine enough for partial transits. As shown in figure 2, the partial transiting planet has very specific geometry. The dense posterior distribution was not well approximated by the grid and led to a near 0 std prediction. I anticipated something would go wrong with the partial transits and adjusted a very high STD, but forget to set a lower bound for STD. Unfortunately, this was not caught in training or public leaderboard, and cost me quite a bit of points.
\nCompletely zeroing out these transit paths gives me a private score of 5.77, but also not ideal as most of the partial transit are fit pretty well.
\n\"\"
\nedit: With further investigation, it might not be an issue with the grid, but rather my use of linear limb darkening, which especially fits the limbs poorly. If this was the case, I probably needed to deal with these transits separately.

", "messages": [ { "id": 3294788, "postDate": "2025-09-26T18:11:34.937Z", "content": "

Brute Force Posterior Estimation of Transit Geometry + Limb Darkening modeling (and also what went wrong)

\n

Summary

\n

A search grid was used to generate a range of possible Transit geometry. Then, a light curve was fit for each point on the grid. Using likelihood of the errors, we can estimate the posterior distribution in a Bayesian manner. The error that dropped my score on the leaderboards was most likely caused by a coarse search grid for partial transiting planets, where the geometry was a lot more sensitive.

\n

The Bayesian approach to modeling transit uncertainties was inspired by last year’s second place winner: I briefly tried Monte Carlo sampling approaches, but search grid was faster (if restricted to a small region around good initial guess), robust, more accurate and could fully utilize the parallel processing of GPUs. However, as I learned more about limb darkening and needed more complex LD models to fit the observations, I ran into severe complexity and runtime issues, so I am not sure if this was the best approach for this competition.
\nMy limb darkening model is unfortunately incomplete and a huge weakness of the model. The final submission used an ensemble of linear limb darkening and 4 parameter limb darkening generated from Atlas searched by Temperature and surface gravity.
\nPre-processing: I used my last year’s processing with the addition of cutting out extreme outliers. I also moved everything to the T4 x2 GPUs.

\n

Transit path calibration

\n

The reason I used a Bayesian framework is that we were given noisy measurements for various geometric properties. And just off initial observation, some of the inclination angle did not seem to fit the data well. Thus, I needed a way to account for the uncertainties in the measurement and how they contribute the final measurement of transit depth.

\n

Initial Transit Detection

\n

A simple algorithm, similar to what were commonly used in previous year’s competitions, was used to estimate the initial guess for the start and end of the Transit zone.

\n

Priors:

\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n\n
VariablePrior Assumption/Search grid
π/2-inclination anglemeasured inclination angle + uniform(-.2, 0.2) in radians, not including the paths that result in no transit, and extended out if needed. This parameter is quite sensitive, as the gradient of the angle is multiplied by the sma in the transit geometry. Some of the measured angles clearly do not agree with the observations, especially for high impact parameter transits (the partial transits).
Start/End of ingress/egress periodequal weight roughly centered around initial transit detection algorithm), extended if needed
smaThe sma is a lot less sensitive than inclination angle, so was fixed to save compute time.
\n

Model:

\n

The transit path for the light curve was calculated using inclination angle, sma, and start/end of ingress/egress period.
\nLimb Darkening (LD): linear limb darkening law was initially used as it was easy to fit.
\nTo speed up processing, the light blocked by the planet was approximated as follows:
\n\"\"
\n, approximated by
\n\"\"
\n\"\"
\nWhere Ia is the average intensity across the entire stellar disk, calculated by integrating the limb darkening function r є [0,1] and θ є [0,2π]. Iaπ is the normalized out of transit flux. Delta is the transit depth (Planet/Star cross section ratio).
\nFinally, combining the out of transit flux together with gaindrift and adding noise, we model the observations as follows.
\n\"\"

\n

A grid search, optimized on GPU was utilized to estimate the Posterior Distribution. I tried using 32 bit float for speed up, but it doesn’t have enough precision for degree 4 polynomial. I initially assumed Gaussian error and weighted each outcome accordingly, but it did not perform much better than taking the simple average/std of the region around the best fit path, which was faster to compute and less complex (Although, this would inadvertently lead to the error that cost me to drop in the private leaderboard).
\n\"\"
\nFigure 1: Posterior distribution of transit paths fit to observations. Kind of interesting my model predicts two main regions for candidate transit paths, with the region crossing the center having higher probability. The space between many of the lines is due to discretized inclination angle mesh.
\n\"\"Figure 2. This is a partial transit. The reason the posterior distribution is so small is that the partial transiting path has to be quite specific to fit to the observations. My inclination angle mesh was probably not fine enough to model the high sensitivities. And this is most likely what caused an error in private set, when a planet id was not well fitted. There weren’t many partial transiting planets, and so I did not focus on them and let this one slip.
\n\"\"
\nFigure 3. Left: best fit path. Right: inclination angle 2 grid points away (0.02 radians).
\nSpectra
\nAfter the light curve was fit, I took the top 20 paths and used them to fit the individual AIRS wavelengths and averaged the results. The light curve polynomial shape was used to fit the individual wavelengths.
\nLimb Darkening Modeling
\nHowever, I soon noticed severe systematic errors on my transit estimates. While Linear limb darkening does a decent job at modeling the path traversed by the planet since the Planet is only near the limbs during the ingress/egress period, limb darkening near the limbs has higher weight and sensitivity on the disk-average-stellar intensity (I¬a ). And because we need (I¬a ) to calculate the distortion between observed transit depth vs (Rp/Rs)2, the assumption of limb darkening near the limbs leads to systematic fitting errors. As a result, my transit depths were overestimated for higher limb darkening coefficients and underestimated for lower limb darkening coefficients.
\n\"\"
\nFigure 4: Linear limb darkening law limitations, Best fit Linear Law (green) and Observed Limb Darkening vs u (cosine of angle of incidence).

\n

I spent the rest of the competition trying to fit more complex LD shapes using Atlas generated LD models for the Temperature and Specific gravity as a starting point, but added complexity made it very hard to proceed with my search grid method and I needed to overhaul major parts of the model to fit within runtime. I was exploring two different approaches: 1. Fitting 2 parameter exponential limb darkening integrating into my existing method. 2. Estimating the limb darkening first using Atlas models. Then, fitting a 2d-gaussian process across u and wavelength. There is clear covariance between limb darkening of different wavelengths and I wanted to model that, but unfortunately could not find a solution in time.
\nI ran out of time and decided to spend the last week to submit what I had with major simplifications. In the simplified limb darkening model, I only focused on only the spectra. I took out the ingress/egress periods and used a lookup table with 4-parameter coefficients generated from Atlas model to fit the different wavelengths (which unfortunately still did not fit the limbs well). It was then ensembled with the linear model.

\n

Limb Darkening correction:

\n

I used a machine learning method (lgb) to estimate the average stellar intensity for the light curve, based on Rs,Mp/Ms, ip, and linear LDC. Then used it to adjust the predict mean transit values. Not ideal, but a fast way to boost my score at the end. I was aware machine learning methods performed terribly in last year’s competitions with distribution shifts, but it seemed to work on my local CV and the variables I used were mostly geometric (except planet mass, I still don’t understand why this variable result in better performance) and less dependent on chemical composition and other stellar properties.
\nPost-processing: I used this post processing https://www.kaggle.com/competitions/ariel-data-challenge-2024/writeups/c-number-daiwakun-1st-place-solution.

\n

Mean and Sigma estimation:

\n

Mean

\n

Light curve mean + post-processing spectra shape
\nUnlike last year, the presence of limb darkening and the severely low signal-to-noise in individual wavelengths made the absolute transit depth values for the individual wavelengths very unreliable. So I needed to make separate predictions for average transit depth using the light curve and shape using the denoised measurements of the spectra. As such that meant dealing with two different measurements of uncertainty.

\n

Sigma

\n

light curve std (estimated from posterior distribution) + mae of transit depth vs. wavelength curve. So it’s kind of interesting that my std were flat for each wavelength. Maybe I should have sampled post-processed spectra shapes as well.

\n

FSG1

\n

Unfortunately, I did not have the chance to look at FSG1. As it only had around 20% weight and was by no means an easy task, I decided to prioritize improving limb darkening model for AIRS. I just used mean + a higher sigma value.

\n

Second observation

\n

Just repeated the same model and averaged mean and std. I don’t think it helped at all, but certainly caused bugs in my code XD.

\n

What went wrong

\n

The grid that I was using was not fine enough for partial transits. As shown in figure 2, the partial transiting planet has very specific geometry. The dense posterior distribution was not well approximated by the grid and led to a near 0 std prediction. I anticipated something would go wrong with the partial transits and adjusted a very high STD, but forget to set a lower bound for STD. Unfortunately, this was not caught in training or public leaderboard, and cost me quite a bit of points.
\nCompletely zeroing out these transit paths gives me a private score of 5.77, but also not ideal as most of the partial transit are fit pretty well.
\n\"\"
\nedit: With further investigation, it might not be an issue with the grid, but rather my use of linear limb darkening, which especially fits the limbs poorly. If this was the case, I probably needed to deal with these transits separately.

", "rawMarkdown": "## Brute Force Posterior Estimation of Transit Geometry + Limb Darkening modeling (and also what went wrong)\n\n## Summary\nA search grid was used to generate a range of possible Transit geometry. Then, a light curve was fit for each point on the grid. Using likelihood of the errors, we can estimate the posterior distribution in a Bayesian manner. The error that dropped my score on the leaderboards was most likely caused by a coarse search grid for partial transiting planets, where the geometry was a lot more sensitive.\n\nThe Bayesian approach to modeling transit uncertainties was inspired by last year’s [second place winner](https://www.kaggle.com/competitions/ariel-data-challenge-2024/writeups/jeroen-cottaar-2nd-place-solution-pure-bayesian-in.): I briefly tried Monte Carlo sampling approaches, but search grid was faster (if restricted to a small region around good initial guess), robust, more accurate and could fully utilize the parallel processing of GPUs. However, as I learned more about limb darkening and needed more complex LD models to fit the observations, I ran into severe complexity and runtime issues, so I am not sure if this was the best approach for this competition.\nMy limb darkening model is unfortunately incomplete and a huge weakness of the model. The final submission used an ensemble of linear limb darkening and 4 parameter limb darkening generated from Atlas searched by Temperature and surface gravity.\nPre-processing: I used my[ last year’s processing](https://www.kaggle.com/competitions/ariel-data-challenge-2024/writeups/junglebeastds-7th-place-solution) with the addition of cutting out extreme outliers. I also moved everything to the T4 x2 GPUs.\n\n## Transit path calibration\nThe reason I used a Bayesian framework is that we were given noisy measurements for various geometric properties. And just off initial observation, some of the inclination angle did not seem to fit the data well. Thus, I needed a way to account for the uncertainties in the measurement and how they contribute the final measurement of transit depth.\n# Initial Transit Detection\nA simple algorithm, similar to what were commonly used in previous year’s competitions, was used to estimate the initial guess for the start and end of the Transit zone.\n# Priors:\n|Variable | Prior Assumption/Search grid |\n| --- | --- |\n| π/2-inclination angle | measured inclination angle + uniform(-.2, 0.2) in radians, not including the paths that result in no transit, and extended out if needed. This parameter is quite sensitive, as the gradient of the angle is multiplied by the sma in the transit geometry. Some of the measured angles clearly do not agree with the observations, especially for high impact parameter transits (the partial transits).| \n|Start/End of ingress/egress period|equal weight roughly centered around initial transit detection algorithm), extended if needed|\n|sma|The sma is a lot less sensitive than inclination angle, so was fixed to save compute time.|\n\n# Model:\nThe transit path for the light curve was calculated using inclination angle, sma, and start/end of ingress/egress period.\nLimb Darkening (LD): linear limb darkening law was initially used as it was easy to fit.\nTo speed up processing, the light blocked by the planet was approximated as follows: \n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F13392195%2F3efebb1fc49253b67e3f64142a4ea339%2F1.png?generation=1758908302769590&alt=media)\n, approximated by\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F13392195%2Fb95b8702dc5216cae944a1683806c78a%2F2.png?generation=1758908328506489&alt=media)\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F13392195%2Fd8faeacdabd3b05621f5bd0d41ee01f2%2F3.png?generation=1758909821709128&alt=media)\nWhere Ia is the average intensity across the entire stellar disk, calculated by integrating the limb darkening function r є [0,1] and θ є [0,2π]. Iaπ is the normalized out of transit flux. Delta is the transit depth (Planet/Star cross section ratio).\nFinally, combining the out of transit flux together with gaindrift and adding noise, we model the observations as follows.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F13392195%2F6ae38acad6917121b2dd393ad3041130%2F4.png?generation=1758908378236661&alt=media)\n\nA grid search, optimized on GPU was utilized to estimate the Posterior Distribution. I tried using 32 bit float for speed up, but it doesn’t have enough precision for degree 4 polynomial. I initially assumed Gaussian error and weighted each outcome accordingly, but it did not perform much better than taking the simple average/std of the region around the best fit path, which was faster to compute and less complex (Although, this would inadvertently lead to the error that cost me to drop in the private leaderboard).\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F13392195%2Fe7a4a44d985fd2c2fe39bcb9689493b2%2F5.png?generation=1758908423904662&alt=media)\nFigure 1: Posterior distribution of transit paths fit to observations. Kind of interesting my model predicts two main regions for candidate transit paths, with the region crossing the center having higher probability. The space between many of the lines is due to discretized inclination angle mesh.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F13392195%2F1f27d240a4f300ceff237794a1a19c24%2F6.png?generation=1758908450084656&alt=media)Figure 2. This is a partial transit. The reason the posterior distribution is so small is that the partial transiting path has to be quite specific to fit to the observations. My inclination angle mesh was probably not fine enough to model the high sensitivities. And this is most likely what caused an error in private set, when a planet id was not well fitted. There weren’t many partial transiting planets, and so I did not focus on them and let this one slip.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F13392195%2F9c03650b2d61ebd889f479486752c64c%2F7.png?generation=1758908533213509&alt=media)\nFigure 3. Left: best fit path. Right: inclination angle 2 grid points away (0.02 radians).\nSpectra\nAfter the light curve was fit, I took the top 20 paths and used them to fit the individual AIRS wavelengths and averaged the results. The light curve polynomial shape was used to fit the individual wavelengths.\nLimb Darkening Modeling \nHowever, I soon noticed severe systematic errors on my transit estimates. While Linear limb darkening does a decent job at modeling the path traversed by the planet since the Planet is only near the limbs during the ingress/egress period, limb darkening near the limbs has higher weight and sensitivity on the disk-average-stellar intensity (I¬a ). And because we need (I¬a ) to calculate the distortion between observed transit depth vs (Rp/Rs)2, the assumption of limb darkening near the limbs leads to systematic fitting errors. As a result, my transit depths were overestimated for higher limb darkening coefficients and underestimated for lower limb darkening coefficients.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F13392195%2Fe6b5eb4a047119a6e083be28b0b62317%2F8.png?generation=1758908565062537&alt=media)\nFigure 4: Linear limb darkening law limitations, Best fit Linear Law (green) and Observed Limb Darkening vs u (cosine of angle of incidence).\n\nI spent the rest of the competition trying to fit more complex LD shapes using Atlas generated LD models for the Temperature and Specific gravity as a starting point, but added complexity made it very hard to proceed with my search grid method and I needed to overhaul major parts of the model to fit within runtime. I was exploring two different approaches: 1. Fitting 2 parameter exponential limb darkening integrating into my existing method. 2. Estimating the limb darkening first using Atlas models. Then, fitting a 2d-gaussian process across u and wavelength. There is clear covariance between limb darkening of different wavelengths and I wanted to model that, but unfortunately could not find a solution in time.\nI ran out of time and decided to spend the last week to submit what I had with major simplifications. In the simplified limb darkening model, I only focused on only the spectra. I took out the ingress/egress periods and used a lookup table with 4-parameter coefficients generated from Atlas model to fit the different wavelengths (which unfortunately still did not fit the limbs well). It was then ensembled with the linear model.\n# Limb Darkening correction: \nI used a machine learning method (lgb) to estimate the average stellar intensity for the light curve, based on Rs,Mp/Ms, ip, and linear LDC. Then used it to adjust the predict mean transit values. Not ideal, but a fast way to boost my score at the end. I was aware machine learning methods performed terribly in last year’s competitions with distribution shifts, but it seemed to work on my local CV and the variables I used were mostly geometric (except planet mass, I still don’t understand why this variable result in better performance) and less dependent on chemical composition and other stellar properties.\nPost-processing: I used this post processing https://www.kaggle.com/competitions/ariel-data-challenge-2024/writeups/c-number-daiwakun-1st-place-solution.\n## Mean and Sigma estimation:\n# Mean\nLight curve mean + post-processing spectra shape\nUnlike last year, the presence of limb darkening and the severely low signal-to-noise in individual wavelengths made the absolute transit depth values for the individual wavelengths very unreliable. So I needed to make separate predictions for average transit depth using the light curve and shape using the denoised measurements of the spectra. As such that meant dealing with two different measurements of uncertainty.\n# Sigma\nlight curve std (estimated from posterior distribution) + mae of transit depth vs. wavelength curve. So it’s kind of interesting that my std were flat for each wavelength. Maybe I should have sampled post-processed spectra shapes as well.\n# FSG1\nUnfortunately, I did not have the chance to look at FSG1. As it only had around 20% weight and was by no means an easy task, I decided to prioritize improving limb darkening model for AIRS. I just used mean + a higher sigma value.\n# Second observation\nJust repeated the same model and averaged mean and std. I don’t think it helped at all, but certainly caused bugs in my code XD.\n## What went wrong\nThe grid that I was using was not fine enough for partial transits. As shown in figure 2, the partial transiting planet has very specific geometry. The dense posterior distribution was not well approximated by the grid and led to a near 0 std prediction. I anticipated something would go wrong with the partial transits and adjusted a very high STD, but forget to set a lower bound for STD. Unfortunately, this was not caught in training or public leaderboard, and cost me quite a bit of points.\nCompletely zeroing out these transit paths gives me a private score of 5.77, but also not ideal as most of the partial transit are fit pretty well.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F13392195%2Fb662d746f853d7f77ab6ad12d20ca349%2F9.png?generation=1758908599420113&alt=media)\nedit: With further investigation, it might not be an issue with the grid, but rather my use of linear limb darkening, which especially fits the limbs poorly. If this was the case, I probably needed to deal with these transits separately.", "votes": 5 }, { "id": 3294899, "postDate": "2025-09-27T04:28:03.843Z", "content": "

A very competitive solution without FGS1. Congratulations!

", "rawMarkdown": "A very competitive solution without FGS1. Congratulations!" }, { "id": 3294827, "postDate": "2025-09-26T21:27:26.867Z", "content": "

Creative solution! I was also developing a bayesian model using PyMC but had to abandon it due to similar problems with complexity. In hindsight grid search would have been a smart move to try out though, since I was using NUTS.

", "rawMarkdown": "Creative solution! I was also developing a bayesian model using PyMC but had to abandon it due to similar problems with complexity. In hindsight grid search would have been a smart move to try out though, since I was using NUTS." } ], "comments": [ { "id": 3294899, "author_name": "Viji", "author_url": "", "post_date": "2025-09-27T04:28:03.843000", "content": "

A very competitive solution without FGS1. Congratulations!

", "votes": 0, "replies": [] }, { "id": 3294827, "author_name": "Ethan J. Schauder", "author_url": "", "post_date": "2025-09-26T21:27:26.867000", "content": "

Creative solution! I was also developing a bayesian model using PyMC but had to abandon it due to similar problems with complexity. In hindsight grid search would have been a smart move to try out though, since I was using NUTS.

", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "3294788": "## Brute Force Posterior Estimation of Transit Geometry + Limb Darkening modeling (and also what went wrong)\n\n## Summary\nA search grid was used to generate a range of possible Transit geometry. Then, a light curve was fit for each point on the grid. Using likelihood of the errors, we can estimate the posterior distribution in a Bayesian manner. The error that dropped my score on the leaderboards was most likely caused by a coarse search grid for partial transiting planets, where the geometry was a lot more sensitive.\n\nThe Bayesian approach to modeling transit uncertainties was inspired by last year’s [second place winner](https://www.kaggle.com/competitions/ariel-data-challenge-2024/writeups/jeroen-cottaar-2nd-place-solution-pure-bayesian-in.): I briefly tried Monte Carlo sampling approaches, but search grid was faster (if restricted to a small region around good initial guess), robust, more accurate and could fully utilize the parallel processing of GPUs. However, as I learned more about limb darkening and needed more complex LD models to fit the observations, I ran into severe complexity and runtime issues, so I am not sure if this was the best approach for this competition.\nMy limb darkening model is unfortunately incomplete and a huge weakness of the model. The final submission used an ensemble of linear limb darkening and 4 parameter limb darkening generated from Atlas searched by Temperature and surface gravity.\nPre-processing: I used my[ last year’s processing](https://www.kaggle.com/competitions/ariel-data-challenge-2024/writeups/junglebeastds-7th-place-solution) with the addition of cutting out extreme outliers. I also moved everything to the T4 x2 GPUs.\n\n## Transit path calibration\nThe reason I used a Bayesian framework is that we were given noisy measurements for various geometric properties. And just off initial observation, some of the inclination angle did not seem to fit the data well. Thus, I needed a way to account for the uncertainties in the measurement and how they contribute the final measurement of transit depth.\n# Initial Transit Detection\nA simple algorithm, similar to what were commonly used in previous year’s competitions, was used to estimate the initial guess for the start and end of the Transit zone.\n# Priors:\n|Variable | Prior Assumption/Search grid |\n| --- | --- |\n| π/2-inclination angle | measured inclination angle + uniform(-.2, 0.2) in radians, not including the paths that result in no transit, and extended out if needed. This parameter is quite sensitive, as the gradient of the angle is multiplied by the sma in the transit geometry. Some of the measured angles clearly do not agree with the observations, especially for high impact parameter transits (the partial transits).| \n|Start/End of ingress/egress period|equal weight roughly centered around initial transit detection algorithm), extended if needed|\n|sma|The sma is a lot less sensitive than inclination angle, so was fixed to save compute time.|\n\n# Model:\nThe transit path for the light curve was calculated using inclination angle, sma, and start/end of ingress/egress period.\nLimb Darkening (LD): linear limb darkening law was initially used as it was easy to fit.\nTo speed up processing, the light blocked by the planet was approximated as follows: \n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F13392195%2F3efebb1fc49253b67e3f64142a4ea339%2F1.png?generation=1758908302769590&alt=media)\n, approximated by\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F13392195%2Fb95b8702dc5216cae944a1683806c78a%2F2.png?generation=1758908328506489&alt=media)\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F13392195%2Fd8faeacdabd3b05621f5bd0d41ee01f2%2F3.png?generation=1758909821709128&alt=media)\nWhere Ia is the average intensity across the entire stellar disk, calculated by integrating the limb darkening function r є [0,1] and θ є [0,2π]. Iaπ is the normalized out of transit flux. Delta is the transit depth (Planet/Star cross section ratio).\nFinally, combining the out of transit flux together with gaindrift and adding noise, we model the observations as follows.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F13392195%2F6ae38acad6917121b2dd393ad3041130%2F4.png?generation=1758908378236661&alt=media)\n\nA grid search, optimized on GPU was utilized to estimate the Posterior Distribution. I tried using 32 bit float for speed up, but it doesn’t have enough precision for degree 4 polynomial. I initially assumed Gaussian error and weighted each outcome accordingly, but it did not perform much better than taking the simple average/std of the region around the best fit path, which was faster to compute and less complex (Although, this would inadvertently lead to the error that cost me to drop in the private leaderboard).\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F13392195%2Fe7a4a44d985fd2c2fe39bcb9689493b2%2F5.png?generation=1758908423904662&alt=media)\nFigure 1: Posterior distribution of transit paths fit to observations. Kind of interesting my model predicts two main regions for candidate transit paths, with the region crossing the center having higher probability. The space between many of the lines is due to discretized inclination angle mesh.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F13392195%2F1f27d240a4f300ceff237794a1a19c24%2F6.png?generation=1758908450084656&alt=media)Figure 2. This is a partial transit. The reason the posterior distribution is so small is that the partial transiting path has to be quite specific to fit to the observations. My inclination angle mesh was probably not fine enough to model the high sensitivities. And this is most likely what caused an error in private set, when a planet id was not well fitted. There weren’t many partial transiting planets, and so I did not focus on them and let this one slip.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F13392195%2F9c03650b2d61ebd889f479486752c64c%2F7.png?generation=1758908533213509&alt=media)\nFigure 3. Left: best fit path. Right: inclination angle 2 grid points away (0.02 radians).\nSpectra\nAfter the light curve was fit, I took the top 20 paths and used them to fit the individual AIRS wavelengths and averaged the results. The light curve polynomial shape was used to fit the individual wavelengths.\nLimb Darkening Modeling \nHowever, I soon noticed severe systematic errors on my transit estimates. While Linear limb darkening does a decent job at modeling the path traversed by the planet since the Planet is only near the limbs during the ingress/egress period, limb darkening near the limbs has higher weight and sensitivity on the disk-average-stellar intensity (I¬a ). And because we need (I¬a ) to calculate the distortion between observed transit depth vs (Rp/Rs)2, the assumption of limb darkening near the limbs leads to systematic fitting errors. As a result, my transit depths were overestimated for higher limb darkening coefficients and underestimated for lower limb darkening coefficients.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F13392195%2Fe6b5eb4a047119a6e083be28b0b62317%2F8.png?generation=1758908565062537&alt=media)\nFigure 4: Linear limb darkening law limitations, Best fit Linear Law (green) and Observed Limb Darkening vs u (cosine of angle of incidence).\n\nI spent the rest of the competition trying to fit more complex LD shapes using Atlas generated LD models for the Temperature and Specific gravity as a starting point, but added complexity made it very hard to proceed with my search grid method and I needed to overhaul major parts of the model to fit within runtime. I was exploring two different approaches: 1. Fitting 2 parameter exponential limb darkening integrating into my existing method. 2. Estimating the limb darkening first using Atlas models. Then, fitting a 2d-gaussian process across u and wavelength. There is clear covariance between limb darkening of different wavelengths and I wanted to model that, but unfortunately could not find a solution in time.\nI ran out of time and decided to spend the last week to submit what I had with major simplifications. In the simplified limb darkening model, I only focused on only the spectra. I took out the ingress/egress periods and used a lookup table with 4-parameter coefficients generated from Atlas model to fit the different wavelengths (which unfortunately still did not fit the limbs well). It was then ensembled with the linear model.\n# Limb Darkening correction: \nI used a machine learning method (lgb) to estimate the average stellar intensity for the light curve, based on Rs,Mp/Ms, ip, and linear LDC. Then used it to adjust the predict mean transit values. Not ideal, but a fast way to boost my score at the end. I was aware machine learning methods performed terribly in last year’s competitions with distribution shifts, but it seemed to work on my local CV and the variables I used were mostly geometric (except planet mass, I still don’t understand why this variable result in better performance) and less dependent on chemical composition and other stellar properties.\nPost-processing: I used this post processing https://www.kaggle.com/competitions/ariel-data-challenge-2024/writeups/c-number-daiwakun-1st-place-solution.\n## Mean and Sigma estimation:\n# Mean\nLight curve mean + post-processing spectra shape\nUnlike last year, the presence of limb darkening and the severely low signal-to-noise in individual wavelengths made the absolute transit depth values for the individual wavelengths very unreliable. So I needed to make separate predictions for average transit depth using the light curve and shape using the denoised measurements of the spectra. As such that meant dealing with two different measurements of uncertainty.\n# Sigma\nlight curve std (estimated from posterior distribution) + mae of transit depth vs. wavelength curve. So it’s kind of interesting that my std were flat for each wavelength. Maybe I should have sampled post-processed spectra shapes as well.\n# FSG1\nUnfortunately, I did not have the chance to look at FSG1. As it only had around 20% weight and was by no means an easy task, I decided to prioritize improving limb darkening model for AIRS. I just used mean + a higher sigma value.\n# Second observation\nJust repeated the same model and averaged mean and std. I don’t think it helped at all, but certainly caused bugs in my code XD.\n## What went wrong\nThe grid that I was using was not fine enough for partial transits. As shown in figure 2, the partial transiting planet has very specific geometry. The dense posterior distribution was not well approximated by the grid and led to a near 0 std prediction. I anticipated something would go wrong with the partial transits and adjusted a very high STD, but forget to set a lower bound for STD. Unfortunately, this was not caught in training or public leaderboard, and cost me quite a bit of points.\nCompletely zeroing out these transit paths gives me a private score of 5.77, but also not ideal as most of the partial transit are fit pretty well.\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F13392195%2Fb662d746f853d7f77ab6ad12d20ca349%2F9.png?generation=1758908599420113&alt=media)\nedit: With further investigation, it might not be an issue with the grid, but rather my use of linear limb darkening, which especially fits the limbs poorly. If this was the case, I probably needed to deal with these transits separately.", "3294899": "A very competitive solution without FGS1. Congratulations!", "3294827": "Creative solution! I was also developing a bayesian model using PyMC but had to abandon it due to similar problems with complexity. In hindsight grid search would have been a smart move to try out though, since I was using NUTS." } }