{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"name":"python","version":"3.12.12","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"kaggle":{"accelerator":"nvidiaTeslaT4","dataSources":[{"sourceId":118765,"databundleVersionId":15231210,"sourceType":"competition"}],"dockerImageVersionId":31236,"isInternetEnabled":false,"language":"python","sourceType":"notebook","isGpuEnabled":true}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# From Zero to Submission: Simple 1D-CNN with Optuna\n\n👋 **Hello!**\nThis notebook presents an **end-to-end workflow** for the *Stanford RNA 3D Folding* competition, covering everything from raw data processing to generating a valid `submission.csv`.\n\n---\n\n### 📌 What’s inside this notebook?\n\nThe goal of this notebook is to provide a **clear and practical baseline** using a **Simple 1D Convolutional Neural Network (1D-CNN)** for predicting RNA 3D coordinates.\nInstead of relying on complex architectures, the focus is on building a **clean, reproducible pipeline** that is easy to understand and extend.\n\nThis notebook includes:\n\n* **Feature engineering** from sequence and metadata\n* **Residue-level data preparation**\n* **RNA base encoding** and feature normalization\n* A **1D-CNN model** for learning local sequence patterns\n* **Hyperparameter tuning with Optuna**\n* **Inference and submission generation**\n\nThis serves as a **strong baseline** before experimenting with more advanced models.\n\n---\n\n### 🎯 Objectives\n\n* Build a **complete pipeline** from training to submission\n* Provide a **readable and reproducible baseline**\n* Emphasize **simplicity and correctness** over model complexity\n\n---\n\n### 👍 If you find this notebook useful…\n\nPlease consider giving it an **upvote** so it can reach more participants.\nGood luck with the competition!","metadata":{"_uuid":"656231dc-c562-4f5b-b14e-69812fd9af1d","_cell_guid":"780e85d7-a3c4-4ba7-bd32-7e64185fe4fa","trusted":true,"collapsed":false,"jupyter":{"outputs_hidden":false}}},{"cell_type":"markdown","source":"\n---\n\n# 🧪 1. Feature Engineering: Decoding the Biological Context\n\nMachine learning models need numbers, not raw biological strings. In this section, we define a suite of **stateless utility functions** to extract rich structural signals from the raw metadata.\n\nOur goal is to capture the **chemical environment** that dictates how an RNA molecule folds.\n\n### A. 🧬 Parsing the Context (`parse_all_seq`)\n\nRNA rarely folds in isolation. It often interacts with proteins or other RNA chains.\n\n* **What we do:** We assume the `all_sequences` field contains the full biological context. We separate the target RNA from its protein partners.\n* **Why it matters:** A protein \"hugging\" an RNA molecule radically changes its 3D shape. Knowing the length and count of these partners gives the model clues about the global structure.\n\n### B. ⚡ The Invisible Architects: Ligands & Metals (`extract_ligand_features`)\n\nRNA is negatively charged, and it needs positively charged ions (like Magnesium or Zinc) to fold into complex shapes.\n\n* **What we do:** We mine the **SMILES** strings to detect the presence of specific metal ions () and measure the complexity of other ligands.\n* **Why it matters:** Metals act as \"molecular glue.\" Without knowing if  is present, the model might fail to predict tight turns or specific folding motifs.\n\n### C. 📐 Stoichiometry & Scale (`parse_stoich`)\n\n* **What we do:** We parse the copy-number of each chain in the complex.\n* **Why it matters:** Is this RNA floating alone (monomer) or is it part of a massive ribosome-like machine (multimer)? The `is_multimer` flag is a strong signal for structural complexity.\n\n### D. 🧲 Chemical Magnetism (`rna_count_features`)\n\n* **What we do:** We calculate the fraction of specific amino acids (like Arginine, Lysine, Histidine) in the protein partners.\n* **Why it matters:** These amino acids are positively charged and naturally attract the negatively charged RNA backbone. A high fraction of them suggests a strong protein-RNA binding interface, which constrains the RNA's 3D coordinates.\n\n### E. 🔄 The Grand Expansion (`prepare_for_transformer`)\n\n**Crucial Step for CNNs:**\nRaw data comes as **1 row per sequence**. However, our target is to predict coordinates for **every single nucleotide**.\n\n* **The Transformation:** We \"explode\" the dataset so that each row represents a single residue (nucleotide).\n* **The Enrichment:** Each nucleotide inherits the global metadata (ligands, stoichiometry) of its parent sequence, plus its own local information (residue character `A, C, G, U` and relative position).\n\n> **Note:** These functions are modular and will be applied to both **Train** and **Test** sets in the next step to ensure consistent feature distribution.","metadata":{}},{"cell_type":"code","source":"# =========================\n# IMPORT\n# =========================\nimport re\nimport pandas as pd\nfrom collections import Counter\nimport pandas as pd \n\ndf = pd.read_csv('/kaggle/input/stanford-rna-3d-folding-2/train_sequences.csv')\nlabel = pd.read_csv('/kaggle/input/stanford-rna-3d-folding-2/train_labels.csv')\n\n# =========================\n# KONSTANTA\n# =========================\nRNA_INTERACTING = \"RKHSTNQYFW\"\n\nMETAL_ATOMS_SMILES = {\"Mg\", \"Zn\", \"Ca\", \"K\", \"Na\", \"Mn\", \"Co\"}\nMETAL_ATOMS_IDS = {\"MG\", \"ZN\", \"CA\", \"K\", \"NA\", \"MN\", \"CO\"}\nNUCLEOTIDE_LIKE = {\"ATP\", \"GTP\", \"CTP\", \"UTP\", \"TTP\", \"ADP\", \"GDP\"}\n\n\n# =========================\n# PARSE SEQUENCE (RNA & PROTEIN)\n# =========================\ndef parse_all_seq(text):\n    result = {\n        \"rna_sequence\": None,\n        \"protein_sequence\": None,\n        \"rna_length\": 0,\n        \"protein_length\": 0,\n        \"n_rna_chains\": 0,\n        \"n_protein_chains\": 0,\n        \"has_modified_rna\": 0,\n    }\n\n    if pd.isna(text):\n        return pd.Series(result)\n\n    entries = text.split(\">\")\n\n    for entry in entries:\n        if not entry.strip():\n            continue\n\n        lines = entry.splitlines()\n        header = lines[0]\n        seq = \"\".join(lines[1:]).upper()\n\n        # RNA\n        if \"RNA\" in header:\n            clean_rna = re.sub(r\"[^AUGC]\", \"\", seq)\n            if clean_rna:\n                result[\"rna_sequence\"] = clean_rna\n                result[\"rna_length\"] += len(clean_rna)\n                result[\"n_rna_chains\"] += 1\n\n            if \"(\" in header and \"*\" in header:\n                result[\"has_modified_rna\"] = 1\n\n        # Protein\n        else:\n            clean_protein = re.sub(r\"[^ACDEFGHIKLMNPQRSTVWY]\", \"\", seq)\n            if clean_protein:\n                result[\"protein_sequence\"] = clean_protein\n                result[\"protein_length\"] += len(clean_protein)\n                result[\"n_protein_chains\"] += 1\n\n    return pd.Series(result)\n\n\n# =========================\n# LIGAND FEATURES (SMILES)\n# =========================\ndef extract_ligand_features(smiles):\n    if pd.isna(smiles):\n        return {\n            \"n_ligands\": 0,\n            \"has_metal\": 0,\n            \"metal_count\": 0,\n            \"metal_types\": \"none\",\n            \"ligand_complexity\": 0,\n            \"is_nucleotide_like\": 0\n        }\n\n    parts = [s.strip() for s in smiles.split(\";\") if s.strip()]\n    metal_set = set()\n    is_nucleotide_like = 0\n\n    for p in parts:\n        for m in METAL_ATOMS_SMILES:\n            if m in p:\n                metal_set.add(m)\n\n        if \"P\" in p and \"(\" in p:\n            is_nucleotide_like = 1\n\n    return {\n        \"n_ligands\": len(parts),\n        \"has_metal\": int(len(metal_set) > 0),\n        \"metal_count\": len(metal_set),\n        \"metal_types\": (\n            f\"{list(metal_set)[0]}_only\"\n            if len(metal_set) == 1\n            else \"multi_metal\" if len(metal_set) > 1 else \"none\"\n        ),\n        \"ligand_complexity\": len(smiles),\n        \"is_nucleotide_like\": is_nucleotide_like\n    }\n\n\n# =========================\n# STOICHIOMETRY\n# =========================\ndef parse_stoich(s):\n    if pd.isna(s):\n        return {\n            \"n_entities\": 0,\n            \"total_copies\": 0,\n            \"max_copy\": 0,\n            \"min_copy\": 0,\n            \"mean_copy\": 0,\n            \"is_multimer\": 0\n        }\n\n    counts = []\n    for p in s.split(\";\"):\n        try:\n            counts.append(int(p.split(\":\")[1]))\n        except:\n            continue\n\n    if not counts:\n        return {\n            \"n_entities\": 0,\n            \"total_copies\": 0,\n            \"max_copy\": 0,\n            \"min_copy\": 0,\n            \"mean_copy\": 0,\n            \"is_multimer\": 0\n        }\n\n    return {\n        \"n_entities\": len(counts),\n        \"total_copies\": sum(counts),\n        \"max_copy\": max(counts),\n        \"min_copy\": min(counts),\n        \"mean_copy\": sum(counts) / len(counts),\n        \"is_multimer\": int(sum(counts) > 1)\n    }\n\n\n# =========================\n# LIGAND IDS FEATURES\n# =========================\ndef ligand_ids_features(ids):\n    if pd.isna(ids):\n        return {\n            \"n_ligands\": 0,\n            \"has_metal\": 0,\n            \"metal_count\": 0,\n            \"metal_types\": \"none\",\n            \"has_nucleotide\": 0\n        }\n\n    ligs = [s.strip().upper() for s in ids.split(\";\") if s.strip()]\n    metals = [l for l in ligs if l in METAL_ATOMS_IDS]\n\n    return {\n        \"n_ligands\": len(ligs),\n        \"has_metal\": int(len(metals) > 0),\n        \"metal_count\": len(metals),\n        \"metal_types\": (\n            f\"{metals[0]}_only\"\n            if len(set(metals)) == 1\n            else \"multi_metal\" if metals else \"none\"\n        ),\n        \"has_nucleotide\": int(any(l in NUCLEOTIDE_LIKE for l in ligs))\n    }\n\n\n# =========================\n# RNA INTERACTING FEATURES\n# =========================\ndef rna_count_features(seq):\n    if pd.isna(seq):\n        return {\"rna_count\": 0, \"rna_fraction\": 0}\n\n    c = Counter(seq.upper())\n    total = sum(c.get(aa, 0) for aa in RNA_INTERACTING)\n    length = len(seq)\n\n    return {\n        \"rna_count\": total,\n        \"rna_fraction\": total / length if length > 0 else 0\n    }\n\n\ndef rna_residue_features(seq):\n    if pd.isna(seq):\n        return {f\"frac_{aa}\": 0 for aa in RNA_INTERACTING}\n\n    c = Counter(seq.upper())\n    length = len(seq)\n\n    return {f\"frac_{aa}\": c.get(aa, 0) / length for aa in RNA_INTERACTING}\n    \n\n# =========================\n# PREPARE DATA FOR TRANSFORMER\n# =========================\ndef prepare_for_transformer(df):\n    \"\"\"\n    Expand satu baris (1 sequence) menjadi banyak node (1 residu / posisi).\n    Setiap node membawa:\n    - informasi posisi (resid, rel_pos)\n    - karakter residu\n    - seluruh fitur metadata (numeric & categorical)\n    \"\"\"\n\n    # Kolom metadata (selain id, sequence, cutoff)\n    meta_columns = [\n        col for col in df.columns\n        if col not in {\"target_id\", \"sequence\", \"temporal_cutoff\"}\n    ]\n\n    all_nodes = []\n\n    for _, row in df.iterrows():\n        seq = row[\"sequence\"]\n        target_id = row[\"target_id\"]\n\n        # Ambil metadata baris ini sekali saja\n        meta_values = row[meta_columns].to_dict()\n        seq_len = len(seq)\n\n        for idx, base in enumerate(seq, start=1):\n            node = {\n                \"target_id\": target_id,\n                \"resid\": idx,\n                \"base_char\": base,           \n                \"rel_pos\": idx / seq_len     \n            }\n\n            node.update(meta_values)\n            all_nodes.append(node)\n\n    return pd.DataFrame(all_nodes)","metadata":{"_uuid":"4ed0a86b-ef6b-4793-bc2b-37a9f04bdfea","_cell_guid":"86d8af32-9351-4da3-821c-efd5cd9b712b","trusted":true,"collapsed":false,"execution":{"iopub.status.busy":"2026-01-13T12:00:18.674504Z","iopub.execute_input":"2026-01-13T12:00:18.674755Z","iopub.status.idle":"2026-01-13T12:00:18.697421Z","shell.execute_reply.started":"2026-01-13T12:00:18.674734Z","shell.execute_reply":"2026-01-13T12:00:18.69656Z"},"jupyter":{"outputs_hidden":false}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n\n## 🏗️ Main Feature Engineering Pipeline\n\nThis central function orchestrates the entire preprocessing workflow, converting raw sequences into a **residue-level dataset** ready for the 1D-CNN.\n\n**Key Operations:**\n\n* **Parses & Extracts:** Integrates sequence, stoichiometry, and ligand data using the utility functions defined above.\n* **Expands Dimensions:** Converts data from \"one-row-per-sequence\" to **\"one-row-per-residue\"** for granular prediction.\n* **Prepares Targets:** Automatically merges 3D coordinate labels (`train_labels.csv`) during the training phase.\n\n> **⚗️ Metal Grouping Strategy:**\n> To handle data sparsity, we group specific metal ions into coarse chemical categories (e.g., **Alkali/Alkaline Earth** vs. **Transition Metals**). This helps the model learn generalized chemical interactions rather than overfitting to rare individual elements.","metadata":{}},{"cell_type":"code","source":"def feature_engineering(\n    df_raw,\n):\n    \"\"\"\n    df_raw   : dataframe mentah (train / test)\n    \"\"\"\n\n    df = df_raw.copy()\n\n    # =========================\n    # SEQUENCE PARSING\n    # =========================\n    parsed_all_seq = df[\"all_sequences\"].apply(parse_all_seq)\n    df[\"protein_sequence\"] = parsed_all_seq[\"protein_sequence\"]\n\n    # =========================\n    # LIGAND / STOICHIOMETRY\n    # =========================\n    lig_features = df[\"ligand_SMILES\"].apply(\n        extract_ligand_features\n    ).apply(pd.Series)\n\n    stoich_features = df[\"stoichiometry\"].apply(\n        parse_stoich\n    ).apply(pd.Series)\n\n    lig_id_features = df[\"ligand_ids\"].apply(\n        ligand_ids_features\n    ).apply(pd.Series)\n\n    # =========================\n    # RNA FEATURES\n    # =========================\n    rna_features = df[\"protein_sequence\"].apply(\n        rna_residue_features\n    ).apply(pd.Series)\n\n    rna_total_features = df[\"protein_sequence\"].apply(\n        rna_count_features\n    ).apply(pd.Series)\n\n    # =========================\n    # CONCAT SEMUA\n    # =========================\n    df = pd.concat(\n        [\n            df,\n            parsed_all_seq,\n            lig_features,\n            stoich_features,\n            lig_id_features,\n            rna_features,\n            rna_total_features,\n        ],\n        axis=1\n    )\n\n    # =========================\n    # DROP RAW COLUMNS\n    # =========================\n    df.drop(\n        columns=[\n            \"description\",\n            \"stoichiometry\",\n            \"all_sequences\",\n            \"ligand_ids\",\n            \"ligand_SMILES\",\n            \"rna_sequence\",\n            \"protein_sequence\",\n            \"temporal_cutoff\",\n        ],\n        inplace=True,\n        errors=\"ignore\"\n    )\n\n    # =========================\n    # EXPAND PER RESIDUE\n    # =========================\n    df_expanded = prepare_for_transformer(df)\n\n    # =========================\n    # METAL GROUPING\n    # =========================\n    alkali = {'NA_only', 'K_only', 'CA_only', 'MG_only'}\n    transition = {'ZN_only', 'MN_only', 'CO_only'}\n\n    df_expanded['metal_alkali'] = (\n        df_expanded['metal_types'].isin(alkali).astype(int)\n    )\n    df_expanded['metal_transition'] = (\n        df_expanded['metal_types'].isin(transition).astype(int)\n    )\n\n    df_expanded.drop(\n        columns='metal_types',\n        inplace=True,\n        errors=\"ignore\"\n    )\n    \n    df_expanded['ID'] = df_expanded['target_id'] + '_' + df_expanded['resid'].astype(str)\n    df_merged = df_expanded.merge(\n    label[['ID', 'x_1','y_1','z_1']],\n    on='ID',\n    how='left'  \n    )\n\n    return df_merged","metadata":{"_uuid":"911963f3-f440-4b75-8135-7670d749c9b8","_cell_guid":"a2dd1099-f78f-4b14-ac19-5c10b5157725","trusted":true,"collapsed":false,"execution":{"iopub.status.busy":"2026-01-13T12:00:18.699089Z","iopub.execute_input":"2026-01-13T12:00:18.699477Z","iopub.status.idle":"2026-01-13T12:00:18.71556Z","shell.execute_reply.started":"2026-01-13T12:00:18.699457Z","shell.execute_reply":"2026-01-13T12:00:18.714771Z"},"jupyter":{"outputs_hidden":false}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"df_train = feature_engineering(\n    df\n)\ndf_train.dropna(inplace=True)","metadata":{"_uuid":"e8438ad6-1113-46a2-81d5-84d7d6298802","_cell_guid":"efb02118-9e5f-4d90-94ff-930a1fe65369","trusted":true,"collapsed":false,"execution":{"iopub.status.busy":"2026-01-13T12:00:18.716484Z","iopub.execute_input":"2026-01-13T12:00:18.716733Z","iopub.status.idle":"2026-01-13T12:01:58.092794Z","shell.execute_reply.started":"2026-01-13T12:00:18.716702Z","shell.execute_reply":"2026-01-13T12:01:58.092134Z"},"jupyter":{"outputs_hidden":false}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n\n# 🔢 2. Residue Encoding (RNA Tokens)\n\nRNA sequences are converted from characters into **numerical tokens** to make them readable for the neural network.\n\n**Mapping Logic:**\n\n* `A → 0`, `C → 1`, `G → 2`, `U → 3`\n* **Unknown/Missing:** `UNK_ID → 4`\n\nThis tokenization preserves the linear order of the RNA while allowing the model to learn specific \"base identity\" features through an Embedding layer.\n\n","metadata":{}},{"cell_type":"code","source":"import torch\nimport torch.nn as nn\nimport optuna\nimport random\nimport numpy as np\nfrom sklearn.preprocessing import StandardScaler\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")","metadata":{"_uuid":"f7805b58-eecd-4da3-a1e5-02ad4ae3d437","_cell_guid":"b53d370c-c914-453b-ae30-650b9938ad39","trusted":true,"collapsed":false,"execution":{"iopub.status.busy":"2026-01-13T12:03:37.268535Z","iopub.execute_input":"2026-01-13T12:03:37.269326Z","iopub.status.idle":"2026-01-13T12:03:37.27376Z","shell.execute_reply.started":"2026-01-13T12:03:37.269297Z","shell.execute_reply":"2026-01-13T12:03:37.273051Z"},"jupyter":{"outputs_hidden":false}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"base_to_id = {'A':0,'C':1,'G':2,'U':3}\nUNK_ID = 4\n\ndf_train['base_id'] = (\n    df_train['base_char']\n    .map(base_to_id)\n    .fillna(UNK_ID)\n    .astype(int)\n)","metadata":{"_uuid":"188465f1-99a2-41be-929e-1ab327b7d0b7","_cell_guid":"ad338ca9-2cfe-4d53-beaa-4bb4112e54f0","trusted":true,"collapsed":false,"execution":{"iopub.status.busy":"2026-01-13T12:05:32.554609Z","iopub.execute_input":"2026-01-13T12:05:32.555321Z","iopub.status.idle":"2026-01-13T12:05:32.971376Z","shell.execute_reply.started":"2026-01-13T12:05:32.555287Z","shell.execute_reply":"2026-01-13T12:05:32.970243Z"},"jupyter":{"outputs_hidden":false}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n\n# ⚖️ 3. Feature Normalization (Metadata Scaling)\n\nOur metadata features (like SMILES length or stoichiometry) come in various scales. We use **Standardization** to level the playing field.\n\n**The Process:**\n\n* **Exclude:** Non-numerical columns (IDs, Coordinates).\n* **Standardize:** Each feature is scaled to **Zero Mean** and **Unit Variance** using `StandardScaler`.\n\n**Why it matters:** Standardization prevents features with large values from \"overpowering\" others, ensuring stable gradients and faster model convergence.","metadata":{}},{"cell_type":"code","source":"exclude_cols = [\n    'target_id','resid','base_char',\n    'x_1','y_1','z_1','base_id','ID'\n]\n\nmeta_columns = [c for c in df_train.columns if c not in exclude_cols]\nscaler = StandardScaler()\ndf_train[meta_columns] = scaler.fit_transform(df_train[meta_columns])","metadata":{"_uuid":"04166ce1-85e0-435b-b91e-6d1bd146795c","_cell_guid":"97437fd8-8b7f-405c-b895-0fdb0776eaf9","trusted":true,"collapsed":false,"execution":{"iopub.status.busy":"2026-01-13T12:06:07.35058Z","iopub.execute_input":"2026-01-13T12:06:07.351247Z","iopub.status.idle":"2026-01-13T12:06:12.960184Z","shell.execute_reply.started":"2026-01-13T12:06:07.351217Z","shell.execute_reply":"2026-01-13T12:06:12.959545Z"},"jupyter":{"outputs_hidden":false}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"\n---\n\n# 📦 4. Sequence-wise Tensor Construction\n\nThe final step is to package our tabular data into **PyTorch Tensors**. We group residues by their `target_id` to maintain structural integrity.\n\n**How it's structured:**\n\n* **Sorting:** Residues are ordered by `resid` to ensure the 1D-CNN sees the sequence in its natural biological order.\n* **Input Tensors:** Base tokens are stored as `LongTensor`, while metadata features are stored as `FloatTensor`.\n* **Target Tensors:** The  coordinates of the **C1'** atom are stored as our learning targets.\n\nThis creates a dictionary-based lookup (`inputs_per_rna` and `labels_per_rna`) that allows the model to handle RNA molecules of different lengths efficiently.\n","metadata":{}},{"cell_type":"code","source":"inputs_per_rna = {}\nlabels_per_rna = {}\n\nfor tid, group in df_train.groupby(\"target_id\"):\n    group = group.sort_values(\"resid\")\n\n    inputs_per_rna[tid] = {\n        \"base_id\": torch.LongTensor(group['base_id'].values),\n        \"meta\": torch.FloatTensor(group[meta_columns].values)\n    }\n\n    labels_per_rna[tid] = torch.FloatTensor(\n        group[['x_1','y_1','z_1']].values\n    )","metadata":{"_uuid":"e8f5d460-2860-452a-8a49-7552c004bea6","_cell_guid":"d6016224-4ac2-42b1-a800-520acbb28710","trusted":true,"collapsed":false,"execution":{"iopub.status.busy":"2026-01-13T12:06:16.814271Z","iopub.execute_input":"2026-01-13T12:06:16.814832Z","iopub.status.idle":"2026-01-13T12:06:31.174248Z","shell.execute_reply.started":"2026-01-13T12:06:16.814803Z","shell.execute_reply":"2026-01-13T12:06:31.173615Z"},"jupyter":{"outputs_hidden":false}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n\n# 🧠 5. Model Architecture: Simple 1D-CNN\n\nOur model is designed to be a **lightweight and efficient baseline**. It processes RNA sequences at the residue level by fusing two types of information: **Biological Identity** (RNA tokens) and **Environmental Context** (Metadata).\n\n### 🛠️ Key Design Choices:\n\n* **Learnable Embeddings**: RNA bases () are mapped into a high-dimensional space where the model can learn the \"hidden\" chemical relationships between them.\n* **Feature Fusion**: Global metadata is concatenated directly with the residue embeddings, providing the network with essential context (like ligand presence) at every step.\n* **1D Convolutional Layers**: These layers slide across the sequence, acting as a \"local scanner\" to capture structural motifs and patterns between neighboring nucleotides.\n* **Independent Coordinate Regression**: The final layer predicts the 3D coordinates () for each residue, focusing on local structural fidelity.\n\n> **Why 1D-CNN?**\n> While Transformers are powerful for long-range interactions, 1D-CNNs are excellent at capturing **local geometry**. Given the complexity of RNA folding, a 1D-CNN provides a stable and fast starting point for sequence-to-structure tasks.\n","metadata":{}},{"cell_type":"code","source":"class RNACNN(nn.Module):\n    def __init__(self, meta_dim, emb_dim, hidden, n_layers, kernel):\n        super().__init__()\n\n        self.embedding = nn.Embedding(5, emb_dim)\n\n        layers = []\n        in_ch = emb_dim + meta_dim\n\n        for _ in range(n_layers):\n            layers += [\n                nn.Conv1d(in_ch, hidden, kernel, padding=kernel//2),\n                nn.ReLU()\n            ]\n            in_ch = hidden\n\n        self.conv = nn.Sequential(*layers)\n        self.head = nn.Linear(hidden, 3)\n\n    def forward(self, base_id, meta):\n        emb = self.embedding(base_id)      # (L, emb)\n        x = torch.cat([emb, meta], dim=1)  # (L, F)\n        x = x.T.unsqueeze(0)               # (1, F, L)\n\n        x = self.conv(x)\n        x = x.squeeze(0).T                 # (L, hidden)\n\n        return self.head(x)","metadata":{"_uuid":"2b1fb546-e973-45f8-b2f9-2d9f301763f0","_cell_guid":"7e419388-b3ef-49c9-ac9f-420a94c1b58d","trusted":true,"collapsed":false,"execution":{"iopub.status.busy":"2026-01-13T12:06:31.175766Z","iopub.execute_input":"2026-01-13T12:06:31.176128Z","iopub.status.idle":"2026-01-13T12:06:31.183363Z","shell.execute_reply.started":"2026-01-13T12:06:31.176081Z","shell.execute_reply":"2026-01-13T12:06:31.182743Z"},"jupyter":{"outputs_hidden":false}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n\n# ⚖️ 6. Training and Validation Strategy\n\nTraining is performed at the **RNA-sequence level** rather than per residue.\n\n**How it works:**\n\n* **Full Sequence Pass:** All residues are processed in a single forward pass.\n* **Loss Function:** Mean Squared Error (MSE) measures coordinate accuracy.\n* **Optimizer:** Adam is used for efficient weight updates.\n\n**Validation:**\nPerformed on **held-out RNA structures** to evaluate how well the model generalizes to unseen sequences.","metadata":{}},{"cell_type":"code","source":"def train_and_eval(train_ids, val_ids, params):\n    model = RNACNN(\n        meta_dim=len(meta_columns),\n        emb_dim=params['emb_dim'],\n        hidden=params['hidden'],\n        n_layers=params['n_layers'],\n        kernel=params['kernel']\n    ).to(device)\n\n    optim = torch.optim.Adam(model.parameters(), lr=params['lr'])\n\n    # ---- TRAIN ----\n    for _ in range(10):\n        model.train()\n        for tid in train_ids:\n            x = inputs_per_rna[tid]\n            y = labels_per_rna[tid].to(device)\n\n            pred = model(\n                x['base_id'].to(device),\n                x['meta'].to(device)\n            )\n\n            loss = ((pred - y)**2).mean()\n\n            optim.zero_grad()\n            loss.backward()\n            optim.step()\n\n    # ---- VALID ----\n    model.eval()\n    total = 0\n    with torch.no_grad():\n        for tid in val_ids:\n            x = inputs_per_rna[tid]\n            y = labels_per_rna[tid].to(device)\n\n            pred = model(\n                x['base_id'].to(device),\n                x['meta'].to(device)\n            )\n\n            total += ((pred - y)**2).mean().item()\n\n    return total / len(val_ids)","metadata":{"_uuid":"41bb734b-e8c3-4b71-8266-3060dcfe52e0","_cell_guid":"203f65b5-9b0d-4797-acf4-52c2cd928c26","trusted":true,"collapsed":false,"execution":{"iopub.status.busy":"2026-01-13T12:06:31.184261Z","iopub.execute_input":"2026-01-13T12:06:31.184492Z","iopub.status.idle":"2026-01-13T12:06:31.195356Z","shell.execute_reply.started":"2026-01-13T12:06:31.184469Z","shell.execute_reply":"2026-01-13T12:06:31.194558Z"},"jupyter":{"outputs_hidden":false}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n\n# 🧪 7. Hyperparameter Optimization with Optuna\n\nTo maximize performance, we use **Optuna** to automatically navigate the complex search space of model configurations.\n\n**Optimized Parameters:**\n\n* **Architecture:** Embedding dimensions, number of convolution layers, hidden channel sizes, and kernel sizes.\n* **Optimization:** The ideal Learning Rate for the Adam optimizer.\n\n**The Workflow:**\nEach trial trains a model variant and evaluates its **Validation MSE**. Optuna intelligently prunes poor trials and focuses on the most promising parameter combinations to minimize our total error.","metadata":{}},{"cell_type":"code","source":"def objective(trial):\n    params = {\n        \"emb_dim\": trial.suggest_categorical(\"emb_dim\", [8,16,32]),\n        \"hidden\": trial.suggest_int(\"hidden\", 32, 128, step=32),\n        \"n_layers\": trial.suggest_int(\"n_layers\", 1, 4),\n        \"kernel\": trial.suggest_categorical(\"kernel\", [3,5,7]),\n        \"lr\": trial.suggest_loguniform(\"lr\", 1e-4, 3e-3),\n    }\n\n    ids = list(inputs_per_rna.keys())\n    random.shuffle(ids)\n\n    split = int(0.8 * len(ids))\n    train_ids = ids[:split]\n    val_ids   = ids[split:]\n\n    return train_and_eval(train_ids, val_ids, params)","metadata":{"_uuid":"49b4b9c0-869d-4e63-b00d-c0a75791d7c0","_cell_guid":"95849547-90e7-4ca8-82b2-e87cb266aa2c","trusted":true,"collapsed":false,"execution":{"iopub.status.busy":"2026-01-13T12:06:31.197159Z","iopub.execute_input":"2026-01-13T12:06:31.197496Z","iopub.status.idle":"2026-01-13T12:06:31.208569Z","shell.execute_reply.started":"2026-01-13T12:06:31.197472Z","shell.execute_reply":"2026-01-13T12:06:31.207954Z"},"jupyter":{"outputs_hidden":false}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"study = optuna.create_study(direction=\"minimize\")\nstudy.optimize(objective, n_trials=30)\n\nprint(study.best_value)\nprint(study.best_params)","metadata":{"_uuid":"39fd757d-19ca-4436-a775-ea160dae85d3","_cell_guid":"47e55c93-2358-41b1-8177-4ca4d3fd02bd","trusted":true,"collapsed":false,"execution":{"iopub.status.busy":"2026-01-13T12:06:31.209576Z","iopub.execute_input":"2026-01-13T12:06:31.209948Z","iopub.status.idle":"2026-01-13T13:08:19.353167Z","shell.execute_reply.started":"2026-01-13T12:06:31.209924Z","shell.execute_reply":"2026-01-13T13:08:19.352244Z"},"jupyter":{"outputs_hidden":false}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n\n# 🏗️ 8. Final Model Training\n\nWith the optimal hyperparameters identified by Optuna, we perform the **final training pass**.\n\n**The Final Run:**\n\n* **Complete Training:** The model is retrained on the full dataset to leverage all available structural information.\n* **Best Config:** We apply the \"winning\" architecture (embedding size, kernel size, etc.) found during the optimization phase.\n\nThe result is a fully-tuned **production-ready model** used for generating our final submission.\n","metadata":{}},{"cell_type":"code","source":"best = study.best_params\n\nmodel = RNACNN(\n    meta_dim=len(meta_columns),\n    emb_dim=best['emb_dim'],\n    hidden=best['hidden'],\n    n_layers=best['n_layers'],\n    kernel=best['kernel']\n).to(device)\n\noptim = torch.optim.Adam(model.parameters(), lr=best['lr'])\n\nfor epoch in range(30):\n    model.train()\n    for tid in inputs_per_rna.keys():\n        x = inputs_per_rna[tid]\n        y = labels_per_rna[tid].to(device)\n\n        pred = model(\n            x['base_id'].to(device),\n            x['meta'].to(device)\n        )\n\n        loss = ((pred - y)**2).mean()\n\n        optim.zero_grad()\n        loss.backward()\n        optim.step()\n\n    print(f\"epoch {epoch}\")","metadata":{"_uuid":"2696a00c-df29-457e-bf55-6764b5362145","_cell_guid":"c97bfc56-acc6-42b8-985d-e68ce7b4c6f6","trusted":true,"collapsed":false,"execution":{"iopub.status.busy":"2026-01-13T13:08:19.354686Z","iopub.execute_input":"2026-01-13T13:08:19.355133Z","iopub.status.idle":"2026-01-13T13:16:19.39924Z","shell.execute_reply.started":"2026-01-13T13:08:19.355107Z","shell.execute_reply":"2026-01-13T13:16:19.398423Z"},"jupyter":{"outputs_hidden":false}},"outputs":[],"execution_count":null},{"cell_type":"code","source":"model.eval()\n\ntotal_loss = 0.0\nn = 0\n\nwith torch.no_grad():\n    for tid in inputs_per_rna.keys():\n        x = inputs_per_rna[tid]\n        y = labels_per_rna[tid].to(device)\n\n        pred = model(\n            x['base_id'].to(device),\n            x['meta'].to(device)\n        )\n\n        loss = ((pred - y) ** 2).mean()\n        total_loss += loss.item()\n        n += 1\n\nprint(\"Final Train MSE:\", total_loss / n)","metadata":{"_uuid":"dc98da25-b960-4cee-b94d-cafc68f2ee1c","_cell_guid":"a1e1f301-bd27-4be7-b392-d99d9dcdc97f","trusted":true,"collapsed":false,"execution":{"iopub.status.busy":"2026-01-13T14:46:29.977681Z","iopub.execute_input":"2026-01-13T14:46:29.97801Z","iopub.status.idle":"2026-01-13T14:46:34.487878Z","shell.execute_reply.started":"2026-01-13T14:46:29.977966Z","shell.execute_reply":"2026-01-13T14:46:34.487161Z"},"jupyter":{"outputs_hidden":false}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"---\n\n# 🚀 9. Inference and Submission: Multi-Pass Strategy\n\nThis final section handles the test data processing and generates the submission file.\n\n**The \"Best-of-5\" Strategy:**\nSince the competition evaluates the best of 5 submitted structures (TM-score), submitting 5 identical predictions is a wasted opportunity.\n\n**Our Approach:**\n\n1. **Standard Preprocessing:** Apply the same feature engineering and scaling to the test set.\n2. **Multi-Pass Inference:** We run the model to predict the base coordinates ().\n3. **Structural Variation:** We generate 5 distinct structures by applying a tiny, progressive **coordinate offset** (`0.02`) to each pass.\n* *Pass 1:* Raw prediction.\n* *Pass 2-5:* Prediction + small shift.\n\nThis simple heuristic creates a \"cloud\" of potential structures, increasing the probability that one of them closely matches the experimental ground truth.","metadata":{}},{"cell_type":"code","source":"import pandas as pd\nimport torch\nfrom tqdm import tqdm\n\n# =========================\n# LOAD & FEATURE ENGINEERING\n# =========================\ndf_test = pd.read_csv('/kaggle/input/stanford-rna-3d-folding-2/test_sequences.csv')\ndf_test = feature_engineering(df_test)\n\ndevice = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\nmodel.to(device).eval()\n\n# =========================\n# PREPROCESSING\n# =========================\ndf_test['base_id'] = (\n    df_test['base_char']\n    .map(base_to_id)\n    .fillna(UNK_ID)\n    .astype(int)\n)\n\ndf_test[meta_columns] = scaler.transform(df_test[meta_columns])\n\ntest_inputs = {}\ntest_meta = {}\n\nfor tid, group in tqdm(df_test.groupby(\"target_id\")):\n    group = group.sort_values(\"resid\")\n\n    test_inputs[tid] = {\n        \"base_id\": torch.LongTensor(group['base_id'].values),\n        \"meta\": torch.FloatTensor(group[meta_columns].values)\n    }\n\n    test_meta[tid] = {\n        \"resid\": group['resid'].values,\n        \"resname\": group['base_char'].values\n    }\n\n# =========================\n# INFERENCE\n# =========================\nrows = []\n\nmodel.eval()\n\nwith torch.no_grad():\n    for tid, x in tqdm(test_inputs.items()):\n        preds_5 = []\n        \n        for k in range(1, 6):\n            p = model(x['base_id'].to(device), x['meta'].to(device)).cpu().numpy()\n            \n            offset = (k - 1) * 0.02 \n            preds_5.append(p + offset)\n\n        resids = test_meta[tid]['resid']\n        resnames = test_meta[tid]['resname']\n\n        for i in range(len(resids)):\n            row = {\n                'ID': f\"{tid}_{resids[i]}\",\n                'resname': resnames[i],\n                'resid': resids[i]\n            }\n            \n            for k in range(1, 6):\n                x_val, y_val, z_val = preds_5[k-1][i]\n                row[f'x_{k}'] = float(x_val)\n                row[f'y_{k}'] = float(y_val)\n                row[f'z_{k}'] = float(z_val)\n            \n            rows.append(row)\n\n# =========================\n# SUBMISSION\n# =========================\ndf_submission = pd.DataFrame(rows)\n\ncols = ['ID', 'resname', 'resid']\nfor k in range(1, 6):\n    cols += [f'x_{k}', f'y_{k}', f'z_{k}']\n\ndf_submission = df_submission[cols]\ndf_submission.to_csv(\"submission.csv\", index=False)\n\nprint(df_submission.head())\nprint(\"✅ submission.csv siap\")","metadata":{"_uuid":"2bc63cd8-abae-441b-b72b-75f39e24f7a6","_cell_guid":"10fac4cf-455c-4166-9be2-ae9f00e8a2f1","trusted":true,"collapsed":false,"execution":{"iopub.status.busy":"2026-01-13T15:00:35.475538Z","iopub.execute_input":"2026-01-13T15:00:35.475856Z","iopub.status.idle":"2026-01-13T15:00:43.185288Z","shell.execute_reply.started":"2026-01-13T15:00:35.47583Z","shell.execute_reply":"2026-01-13T15:00:43.184518Z"},"jupyter":{"outputs_hidden":false}},"outputs":[],"execution_count":null},{"cell_type":"markdown","source":"## 🏁 Conclusion \n\nWe have successfully built an end-to-end pipeline for RNA 3D folding, taking the raw sequences all the way to a valid submission file.\n\n**Summary of our workflow:**\n\n* **Robust Pipeline:** A unified feature engineering system for both Train and Test.\n* **Simple Baseline:** A lightweight 1D-CNN that efficiently learns local structural motifs.\n* **Optimization:** Automatic hyperparameter tuning using Optuna to find the best configuration.\n* **Smart Inference:** A \"Best-of-5\" strategy using coordinate offsets to maximize the TM-score.\n\n\n### 🤝 Support & Feedback\n\nIf you found this starter notebook helpful or learned something new, **please consider upvoting!** ⬆️\n\nYour support keeps me motivated to share more improvements and experiments. If you have any questions or ideas, feel free to drop a comment below.\n\n**Good luck and Happy Kaggling!** 🚀","metadata":{}}]}