First, I would like to extend my deepest gratitude to the competition organizing team and Kaggle staff for hosting this fascinating competition. Having previously worked in experimental sciences including organic chemistry, I am well aware of the challenges in obtaining molecular 3D structural data (ground-truth structures). Considering this, I'm deeply impressed that they were able to hold this Part II competition just less than a year after the Part I competition.\nDuring these approximately two months of trial-and-error work, I gained both meaningful experience and valuable learning.
\nI built \"BPP-Protenix\", a model that integrates Base Pair Probability (BPP) features into an AlphaFold3-style architecture. The model design was inspired by the architecture of RNAPro shared by @theoviel. My final pipeline is an ensemble of TBM, RNAPro, and BPP-Protenix (with plain Protenix as fallback for targets not suitable for BPP calculation). Although I dropped one place in the private LB, after completing the development of BPP-Protenix, I held 1st place on the public LB for the final two weeks of the competition. I guess almost no other participants modified the neural network architecture in this competition, so this appears to be a unique solution.
\nSubmission slot allocation is follows:
\n| slot 1 | \nslot 2 | \nslot 3 | \nslot 4 | \nslot5 | \n
|---|---|---|---|---|
| TBM | \nRNAPro | \nRNAPro | \nBPP-Protenix (with plain Protenix fallback) | \nBPP-Protenix (with plain Protenix fallback) | \n
TBM: I used the notebook published by @kami1976 almost exactly as is. Since it appeared to offer good local scores and seed stability, I decided to borrow this one.
RNAPro: Using the template created by the above TBM as input data, I applied the the pipeline published by @jaejohn.
BPP-Protenix: This is the model that required the most time to develop for this competition. I'll explain it below.
The idea came from the Stanford Ribonanza RNA Folding competition in 2023. @shujun717 (one of the hosts of the Ribonanza competition) had published about RNAdegformer, a transformer predicting mRNA degradation at nucleotide level. In that paper, BPP (Base Pair Probability) matrix was shown to be a very important feature for predicting RNA stability. Inspired by this, almost all top Ribonanza participants added BPP embeddings as bias terms in their Transformers.
\nThe task of RNAdegformer and Ribonanza models was stability prediction, but RNA stability depends a lot on 3D structure. Furthermore, the base pair formation probability information obtained through secondary structure prediction provides proximity constraints on the 3D structure, thereby functioning as an effective inductive bias. Given that, providing BPP information to a 3D structure predictor seemed like a natural extension.
\nThe BPP matrix [N, N] naturally corresponds to the pair representation z [N, N, c] in the Pairformer trunk of AlphaFold3/Protenix architecture — both are pairwise matrices indexed by residue positions. By passing BPP through a simple linear layer (1 → c) and adding the resulting embedding to z_init, we can inject RNA structural knowledge directly into the model.
BPP [N, N]\n → unsqueeze → [N, N, 1]\n → LinearNoBias(1 → 128) → [N, N, 128]\n → z_init += bpp_embedding\nI evaluated BPP quality against experimental 3D coordinates from the training data, checking if residue pairs with high BPP are actually close in 3D space. ViennaRNA and EternaFold, two of the primary RNA secondary structure computation tools, were tested. These secondary structure prediction tools assume RNA monomers as the target of prediction. Additionally, calculating BPP for long sequences during a Kaggle session is difficult in inference. Trherefore, it was considered that the training should also be performed using shorter sequences. The following filter was applied to the approximately 5,000 sequences of Kaggle's official training data:
\nFilter conditions
\nThe BPP quality was evaluated using the 1,532 sequences remaining after applying this filter. Here, a C1'-C1' distance < 12 Å was defined as \"contact.\" However, this judge was only conducted for C1' combinations separated by 4 or more residues (because such residues exist spatially close regardless of base pair formation).
\n| \n | ViennaRNA | \nEternaFold | \n
|---|---|---|
| Contact rate (BPP ≥ 0.7) | \n81.4% | \n96.2% | \n
| Contact rate (BPP ≥ 0.9) | \n86.6% | \n97.4% | \n
| Random baseline* | \n3.6% | \n3.6% | \n
*Random baseline: contact rate when residue pairs are selected randomly (no BPP information).
\nEternaFold BPP was clearly more accurate. Also, it is easily deployable in Kaggle notebooks via the arnie library. So I chose EternaFold for feature engineering.
\nprotenix_base_20250630_v1.0.0| Parameter | \nValue | \n
|---|---|
| diffusion_batch_size | \n32 | \n
| train_crop_size | \n550 | \n
| lr | \n1e-4 | \n
| EMA decay | \n0.995 | \n
| num_steps | \n4960 | \n
The following filters were applied to determine if the target was appropriate for BPP calculation by EternaFold:
\nTargets not passing the filter were predicted by plain Protenix as fallback. Targets exceeding 850 tokens were cropped to 850 residues, with remaining coordinates zero-padded.
\nBefore evaluating the ensemble, I assessed BPP-Protenix alone (all 5 slots filled with Protenix predictions). Inspired by RNAPro's TemplateEmbedder (Protenix v0.5), I tested three integration methods (A,B,C).
\nz_init.| Method | \nArchitecture | \nPublic LB (standalone*) | \n
|---|---|---|
| A (Linear) | \nLinearNoBias(1→128), add BPP-emb to z_init | \n0.340 | \n
| B (+ Binning) | \nMethod A + one_hot(20 bins) → Linear(20→128), add two embs to z_init | \n0.317 | \n
| C (+ BppEmbedder) | \nMethod B + BppEmbedder in recycling loop | \n0.320 | \n
| Plain Protenix (control) | \n- | \n0.309 | \n
*Standalone: all 5 slots filled with BPP-Protenix + plain Protenix fallback
\nThe standalone BPP-Protenix (Method A) scored 0.34 on public LB. This is comparable to TBM-only public notebooks (~0.35), demonstrating that NN-based predictions can match template-based approaches.
\nInteresting finding: allowing protein/DNA partner targets in the BPP filter caused almost no change in public LB score. Possibly few such targets existed in the test set. Worth investigating the performance for RNA/protein assembly with late submissions.
\nConcern: EternaFold does not predict pseudoknots. Could BPP-Protenix is not suitable for potential pseudoknot targets?
\nPost-hoc validation:
\nInterpretation: BPP provides a \"pairing tendency hint\" as continuous values, not a complete secondary structure prediction. Even if pseudoknot base pairs are missed, non-pseudoknot BPP information is still valuable. And maybe, the 48-block Pairformer can absorb BPP inaccuracies.
\nBased on the local validation result, I have decided not to take any special measures specifically targeting pseudoknots.
\n| Configuration | \nPublic LB | \nPrivate LB | \n
|---|---|---|
| TBM 2slot / RNAPro 3slot (baseline) | \n~0.43 | \n— | \n
| TBM 1slot / RNAPro 2slot / Protenix 2slot | \n0.461 | \n0.475 | \n
| TBM 1slot / RNAPro 2slot / BPP-Protenix 2slot | \n0.504 (1st place) | \n0.492 (2nd place) | \n
Thank you to the competition hosts (Rhiju Das and team) for organizing this fascinating competition. Thanks also to the Kaggle community for sharing public notebooks (TBM, RNAPro) that formed the foundation of my ensemble.
", "messages": [ { "id": 3441423, "postDate": "2026-04-14T00:52:33.013Z", "content": "First, I would like to extend my deepest gratitude to the competition organizing team and Kaggle staff for hosting this fascinating competition. Having previously worked in experimental sciences including organic chemistry, I am well aware of the challenges in obtaining molecular 3D structural data (ground-truth structures). Considering this, I'm deeply impressed that they were able to hold this Part II competition just less than a year after the Part I competition.\nDuring these approximately two months of trial-and-error work, I gained both meaningful experience and valuable learning.
\nI built \"BPP-Protenix\", a model that integrates Base Pair Probability (BPP) features into an AlphaFold3-style architecture. The model design was inspired by the architecture of RNAPro shared by @theoviel. My final pipeline is an ensemble of TBM, RNAPro, and BPP-Protenix (with plain Protenix as fallback for targets not suitable for BPP calculation). Although I dropped one place in the private LB, after completing the development of BPP-Protenix, I held 1st place on the public LB for the final two weeks of the competition. I guess almost no other participants modified the neural network architecture in this competition, so this appears to be a unique solution.
\nSubmission slot allocation is follows:
\n| slot 1 | \nslot 2 | \nslot 3 | \nslot 4 | \nslot5 | \n
|---|---|---|---|---|
| TBM | \nRNAPro | \nRNAPro | \nBPP-Protenix (with plain Protenix fallback) | \nBPP-Protenix (with plain Protenix fallback) | \n
TBM: I used the notebook published by @kami1976 almost exactly as is. Since it appeared to offer good local scores and seed stability, I decided to borrow this one.
RNAPro: Using the template created by the above TBM as input data, I applied the the pipeline published by @jaejohn.
BPP-Protenix: This is the model that required the most time to develop for this competition. I'll explain it below.
The idea came from the Stanford Ribonanza RNA Folding competition in 2023. @shujun717 (one of the hosts of the Ribonanza competition) had published about RNAdegformer, a transformer predicting mRNA degradation at nucleotide level. In that paper, BPP (Base Pair Probability) matrix was shown to be a very important feature for predicting RNA stability. Inspired by this, almost all top Ribonanza participants added BPP embeddings as bias terms in their Transformers.
\nThe task of RNAdegformer and Ribonanza models was stability prediction, but RNA stability depends a lot on 3D structure. Furthermore, the base pair formation probability information obtained through secondary structure prediction provides proximity constraints on the 3D structure, thereby functioning as an effective inductive bias. Given that, providing BPP information to a 3D structure predictor seemed like a natural extension.
\nThe BPP matrix [N, N] naturally corresponds to the pair representation z [N, N, c] in the Pairformer trunk of AlphaFold3/Protenix architecture — both are pairwise matrices indexed by residue positions. By passing BPP through a simple linear layer (1 → c) and adding the resulting embedding to z_init, we can inject RNA structural knowledge directly into the model.
BPP [N, N]\n → unsqueeze → [N, N, 1]\n → LinearNoBias(1 → 128) → [N, N, 128]\n → z_init += bpp_embedding\nI evaluated BPP quality against experimental 3D coordinates from the training data, checking if residue pairs with high BPP are actually close in 3D space. ViennaRNA and EternaFold, two of the primary RNA secondary structure computation tools, were tested. These secondary structure prediction tools assume RNA monomers as the target of prediction. Additionally, calculating BPP for long sequences during a Kaggle session is difficult in inference. Trherefore, it was considered that the training should also be performed using shorter sequences. The following filter was applied to the approximately 5,000 sequences of Kaggle's official training data:
\nFilter conditions
\nThe BPP quality was evaluated using the 1,532 sequences remaining after applying this filter. Here, a C1'-C1' distance < 12 Å was defined as \"contact.\" However, this judge was only conducted for C1' combinations separated by 4 or more residues (because such residues exist spatially close regardless of base pair formation).
\n| \n | ViennaRNA | \nEternaFold | \n
|---|---|---|
| Contact rate (BPP ≥ 0.7) | \n81.4% | \n96.2% | \n
| Contact rate (BPP ≥ 0.9) | \n86.6% | \n97.4% | \n
| Random baseline* | \n3.6% | \n3.6% | \n
*Random baseline: contact rate when residue pairs are selected randomly (no BPP information).
\nEternaFold BPP was clearly more accurate. Also, it is easily deployable in Kaggle notebooks via the arnie library. So I chose EternaFold for feature engineering.
\nprotenix_base_20250630_v1.0.0| Parameter | \nValue | \n
|---|---|
| diffusion_batch_size | \n32 | \n
| train_crop_size | \n550 | \n
| lr | \n1e-4 | \n
| EMA decay | \n0.995 | \n
| num_steps | \n4960 | \n
The following filters were applied to determine if the target was appropriate for BPP calculation by EternaFold:
\nTargets not passing the filter were predicted by plain Protenix as fallback. Targets exceeding 850 tokens were cropped to 850 residues, with remaining coordinates zero-padded.
\nBefore evaluating the ensemble, I assessed BPP-Protenix alone (all 5 slots filled with Protenix predictions). Inspired by RNAPro's TemplateEmbedder (Protenix v0.5), I tested three integration methods (A,B,C).
\nz_init.| Method | \nArchitecture | \nPublic LB (standalone*) | \n
|---|---|---|
| A (Linear) | \nLinearNoBias(1→128), add BPP-emb to z_init | \n0.340 | \n
| B (+ Binning) | \nMethod A + one_hot(20 bins) → Linear(20→128), add two embs to z_init | \n0.317 | \n
| C (+ BppEmbedder) | \nMethod B + BppEmbedder in recycling loop | \n0.320 | \n
| Plain Protenix (control) | \n- | \n0.309 | \n
*Standalone: all 5 slots filled with BPP-Protenix + plain Protenix fallback
\nThe standalone BPP-Protenix (Method A) scored 0.34 on public LB. This is comparable to TBM-only public notebooks (~0.35), demonstrating that NN-based predictions can match template-based approaches.
\nInteresting finding: allowing protein/DNA partner targets in the BPP filter caused almost no change in public LB score. Possibly few such targets existed in the test set. Worth investigating the performance for RNA/protein assembly with late submissions.
\nConcern: EternaFold does not predict pseudoknots. Could BPP-Protenix is not suitable for potential pseudoknot targets?
\nPost-hoc validation:
\nInterpretation: BPP provides a \"pairing tendency hint\" as continuous values, not a complete secondary structure prediction. Even if pseudoknot base pairs are missed, non-pseudoknot BPP information is still valuable. And maybe, the 48-block Pairformer can absorb BPP inaccuracies.
\nBased on the local validation result, I have decided not to take any special measures specifically targeting pseudoknots.
\n| Configuration | \nPublic LB | \nPrivate LB | \n
|---|---|---|
| TBM 2slot / RNAPro 3slot (baseline) | \n~0.43 | \n— | \n
| TBM 1slot / RNAPro 2slot / Protenix 2slot | \n0.461 | \n0.475 | \n
| TBM 1slot / RNAPro 2slot / BPP-Protenix 2slot | \n0.504 (1st place) | \n0.492 (2nd place) | \n
Thank you to the competition hosts (Rhiju Das and team) for organizing this fascinating competition. Thanks also to the Kaggle community for sharing public notebooks (TBM, RNAPro) that formed the foundation of my ensemble.
", "rawMarkdown": "First, I would like to extend my deepest gratitude to the competition organizing team and Kaggle staff for hosting this fascinating competition. Having previously worked in experimental sciences including organic chemistry, I am well aware of the challenges in obtaining molecular 3D structural data (ground-truth structures). Considering this, I'm deeply impressed that they were able to hold this Part II competition just less than a year after the [Part I competition](https://www.kaggle.com/competitions/stanford-rna-3d-folding).\nDuring these approximately two months of trial-and-error work, I gained both meaningful experience and valuable learning.\n\n## Summary of My Solution\n\nI built \"**BPP-Protenix**\", a model that integrates Base Pair Probability (BPP) features into an AlphaFold3-style architecture. The model design was inspired by the architecture of [RNAPro](https://www.kaggle.com/competitions/stanford-rna-3d-folding-2/discussion/668412) shared by @theoviel. My final pipeline is an ensemble of TBM, RNAPro, and BPP-Protenix (with plain Protenix as fallback for targets not suitable for BPP calculation). Although I dropped one place in the private LB, after completing the development of BPP-Protenix, I held 1st place on the public LB for the final two weeks of the competition. I guess almost no other participants modified the neural network architecture in this competition, so this appears to be a unique solution.\n\n**Submission slot allocation is follows**:\n\n| slot 1 | slot 2 | slot 3 | slot 4 | slot5 |\n| :----: | :----: | :----: | :-------------------------------------------: | :-------------------------------------------: |\n| TBM | RNAPro | RNAPro | **BPP-Protenix**