{"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.12.13","mimetype":"text/x-python","codemirror_mode":{"name":"ipython","version":3},"pygments_lexer":"ipython3","nbconvert_exporter":"python","file_extension":".py"},"accelerator":"GPU"},"nbformat_minor":4,"nbformat":4,"cells":[{"id":"90cb02e0-d407-4c79-a19d-c22fc57af6c4","cell_type":"markdown","source":"# RSNA Knee Abnormality Detection — V2.1 CORRECTED ULTRA-FAST\n\n## Competition-specific pipeline\n\nThis notebook is explicitly designed for the Kaggle folder structure shown in the supplied screenshot:\n\n```text\n/kaggle/input//\n├── train_series/\n│ └── StudyInstanceUID/\n│ └── SeriesInstanceUID/\n│ └── SOPInstanceUID.dcm\n├── test_series/\n│ └── StudyInstanceUID/\n│ └── SeriesInstanceUID/\n│ └── SOPInstanceUID.dcm\n├── train.csv\n├── train_series.csv\n├── test.csv\n├── test_series.csv\n└── sample_submission.csv\n```\n\n## Critical V2.1 speed change\n\nV2 indexed every DICOM file before deciding which series were useful.\n\nV2.1 does **not** do that.\n\nInstead:\n\n```text\ntrain_series.csv / test_series.csv\n ↓\n rank useful series\n ↓\n locate ONLY selected\n series folders\n ↓\n decode DICOM once\n ↓\n compact cache\n ↓\n GPU training\n```\n\nThis avoids an unnecessary full 819k-file scan.\n\n### Important\nThe exact speedup cannot honestly be guaranteed as \"100×\" on every Kaggle runtime. The purpose of V2.1 is to remove unnecessary filesystem/DICOM work while preserving a useful multimodal representation.","metadata":{}},{"id":"ba3a8877-a0da-403a-a2c3-6ed9c0107fa6","cell_type":"markdown","source":"# 1. FAST / ACCURACY configuration\n\nStart here.\n\nThe competition strategy is:\n\n**first make the pipeline cheap enough to iterate; then increase model quality.**\n\nDo not change these values randomly during the first benchmark.","metadata":{}},{"id":"d1eab023-3c15-4678-aa10-c3de207ef1d9","cell_type":"code","source":"import os, sys, gc, json, math, time, random, warnings, importlib.util, subprocess\nfrom pathlib import Path\nfrom concurrent.futures import ProcessPoolExecutor, as_completed\n\nwarnings.filterwarnings(\"ignore\")\n\nSEED = 2026\nrandom.seed(SEED)\nnp_seed = SEED\n\n# ---------------- FAST PROFILE ----------------\nIMG_SIZE = 160\nN_SLICES = 3\nMAX_SERIES = 5\n\nCPU_WORKERS = max(2, min(8, os.cpu_count() or 4))\nNUM_WORKERS = min(4, CPU_WORKERS)\n\nBATCH_SIZE = 8\nEPOCHS = 8\nLR = 3e-4\nWEIGHT_DECAY = 1e-4\n\nCACHE_VERSION = \"v2_1_fast160_s3_k5\"\n\nprint(\"CPU workers:\", CPU_WORKERS)\nprint(\"DataLoader workers:\", NUM_WORKERS)\nprint(\"Image:\", IMG_SIZE)\nprint(\"Slices/series:\", N_SLICES)\nprint(\"Series/study:\", MAX_SERIES)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-08-07T15:38:11.370238Z","iopub.execute_input":"2026-08-07T15:38:11.37066Z","iopub.status.idle":"2026-08-07T15:38:11.405617Z","shell.execute_reply.started":"2026-08-07T15:38:11.370619Z","shell.execute_reply":"2026-08-07T15:38:11.404703Z"}},"outputs":[],"execution_count":null},{"id":"3466a12a-36de-4474-b508-35a5314a30fa","cell_type":"markdown","source":"# 2. Install only missing lightweight packages\n\nDo not replace the Kaggle PyTorch/CUDA stack.","metadata":{}},{"id":"44e74a5c-6302-4837-9f6c-4f4b0c574814","cell_type":"code","source":"def ensure_package(import_name, pip_name=None):\n if importlib.util.find_spec(import_name) is None:\n subprocess.check_call([\n sys.executable, \"-m\", \"pip\", \"install\", \"-q\",\n pip_name or import_name\n ])\n\nensure_package(\"pydicom\")\nensure_package(\"timm\")\nensure_package(\"sklearn\", \"scikit-learn\")\n\nimport numpy as np\nnp.random.seed(SEED)\n\nimport pandas as pd\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom torch.utils.data import Dataset, DataLoader\nfrom tqdm.auto import tqdm\n\nimport pydicom\n\nprint(\"PyTorch:\", torch.__version__)\nprint(\"CUDA:\", torch.cuda.is_available())\nif torch.cuda.is_available():\n print(\"GPU:\", torch.cuda.get_device_name(0))","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-08-07T15:38:23.018853Z","iopub.execute_input":"2026-08-07T15:38:23.019674Z","iopub.status.idle":"2026-08-07T15:38:29.327797Z","shell.execute_reply.started":"2026-08-07T15:38:23.019642Z","shell.execute_reply":"2026-08-07T15:38:29.326966Z"}},"outputs":[],"execution_count":null},{"id":"c8927972-4997-49b7-8d8e-746241972961","cell_type":"markdown","source":"# 3. Automatically detect the exact Kaggle competition root\n\nWe do not hard-code a dataset slug.\n\nThe code searches `/kaggle/input` for the exact set of CSV files visible in the supplied screenshot.","metadata":{}},{"id":"dfe447d6-62e9-4cc0-a5e6-b84bba9a7c4a","cell_type":"code","source":"INPUT = Path(\"/kaggle/input\")\nWORK = Path(\"/kaggle/working/rsna_knee_v21\")\nWORK.mkdir(parents=True, exist_ok=True)\n\nREQUIRED = {\n \"train.csv\",\n \"train_series.csv\",\n \"test.csv\",\n \"test_series.csv\",\n \"sample_submission.csv\"\n}\n\ndef find_competition_root():\n candidates = []\n\n for p in INPUT.iterdir():\n if not p.is_dir():\n continue\n\n names = {x.name for x in p.iterdir() if x.is_file()}\n\n if REQUIRED.issubset(names):\n candidates.append(p)\n\n if not candidates:\n # Fallback: search recursively, but only through directories\n # containing the required CSV set.\n for p in INPUT.rglob(\"train.csv\"):\n root = p.parent\n names = {x.name for x in root.iterdir() if x.is_file()}\n if REQUIRED.issubset(names):\n candidates.append(root)\n\n if not candidates:\n raise FileNotFoundError(\n \"Could not find the RSNA competition root. \"\n \"Expected train.csv, train_series.csv, test.csv, \"\n \"test_series.csv and sample_submission.csv together.\"\n )\n\n if len(candidates) > 1:\n print(\"Multiple candidate roots found:\")\n for c in candidates:\n print(\" \", c)\n print(\"Using:\", candidates[0])\n\n return candidates[0]\n\nROOT = find_competition_root()\n\nTRAIN_ROOT = ROOT / \"train_series\"\nTEST_ROOT = ROOT / \"test_series\"\n\nassert TRAIN_ROOT.is_dir(), TRAIN_ROOT\nassert TEST_ROOT.is_dir(), TEST_ROOT\n\nTRAIN_CSV = ROOT / \"train.csv\"\nTRAIN_SERIES_CSV = ROOT / \"train_series.csv\"\nTEST_CSV = ROOT / \"test.csv\"\nTEST_SERIES_CSV = ROOT / \"test_series.csv\"\nSAMPLE_SUB = ROOT / \"sample_submission.csv\"\n\nprint(\"Competition root:\", ROOT)\nprint(\"Train DICOM:\", TRAIN_ROOT)\nprint(\"Test DICOM:\", TEST_ROOT)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-08-07T15:38:48.600644Z","iopub.execute_input":"2026-08-07T15:38:48.60111Z","iopub.status.idle":"2026-08-07T15:45:28.719244Z","shell.execute_reply.started":"2026-08-07T15:38:48.601084Z","shell.execute_reply":"2026-08-07T15:45:28.718209Z"}},"outputs":[],"execution_count":null},{"id":"4b1cf82e-bb28-4781-bbd7-4cb74ff10ad4","cell_type":"markdown","source":"# 4. Load metadata FIRST\n\nThis is deliberately before any DICOM filesystem traversal.\n\nThe series CSV is our first-stage index.","metadata":{}},{"id":"a37a5bf1-a2e5-4eba-96b8-f003a3a30d5b","cell_type":"code","source":"train = pd.read_csv(TRAIN_CSV)\ntrain_series = pd.read_csv(TRAIN_SERIES_CSV)\ntest = pd.read_csv(TEST_CSV)\ntest_series = pd.read_csv(TEST_SERIES_CSV)\nsample_submission = pd.read_csv(SAMPLE_SUB)\n\nTARGETS = [\n \"ACL\", \"MCL\", \"Medial Meniscus\", \"Lateral Meniscus\",\n \"Medial OA\", \"Lateral OA\", \"PF OA\", \"Effusion\",\n \"Synovitis\", \"Baker's\", \"Contusion\", \"Fracture\"\n]\n\nprint(\"train:\", train.shape)\nprint(\"train_series:\", train_series.shape)\nprint(\"test:\", test.shape)\nprint(\"test_series:\", test_series.shape)\nprint(\"submission:\", sample_submission.shape)\n\nprint(\"\\ntrain columns:\")\nprint(train.columns.tolist())\n\nprint(\"\\nseries columns:\")\nprint(train_series.columns.tolist())","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-08-07T15:46:13.473188Z","iopub.execute_input":"2026-08-07T15:46:13.473815Z","iopub.status.idle":"2026-08-07T15:46:13.807946Z","shell.execute_reply.started":"2026-08-07T15:46:13.473781Z","shell.execute_reply":"2026-08-07T15:46:13.807112Z"}},"outputs":[],"execution_count":null},{"id":"398d208a-ffe9-42b6-8498-d19b3816bad5","cell_type":"code","source":"# ============================================================\n# CRITICAL DATA AUDIT — RUN BEFORE ANY DICOM DECODING\n# ============================================================\n\nprint(\"=\" * 70)\nprint(\"RSNA KNEE DATA AUDIT\")\nprint(\"=\" * 70)\n\n# ------------------------------------------------------------\n# 1. Missing labels\n# ------------------------------------------------------------\nprint(\"\\n[1] LABEL COMPLETENESS\")\nprint(\"-\" * 70)\n\nmissing = train[TARGETS].isna().sum()\nmissing_pct = 100 * missing / len(train)\n\nlabel_audit = pd.DataFrame({\n \"Missing\": missing,\n \"Missing_%\": missing_pct.round(2),\n \"Present\": len(train) - missing\n})\n\ndisplay(label_audit)\n\n# ------------------------------------------------------------\n# 2. Number of completely labeled studies\n# ------------------------------------------------------------\ncomplete_mask = train[TARGETS].notna().all(axis=1)\n\nprint(\n \"Studies with ALL 12 labels:\",\n int(complete_mask.sum()),\n f\"({100*complete_mask.mean():.2f}%)\"\n)\n\nprint(\n \"Studies with at least one missing label:\",\n int((~complete_mask).sum()),\n f\"({100*(~complete_mask).mean():.2f}%)\"\n)\n\n# ------------------------------------------------------------\n# 3. Positive prevalence\n# ------------------------------------------------------------\nprint(\"\\n[2] LABEL PREVALENCE\")\nprint(\"-\" * 70)\n\nprevalence = pd.DataFrame({\n \"Positive\": train[TARGETS].sum(skipna=True),\n \"Available\": train[TARGETS].notna().sum(),\n})\n\nprevalence[\"Positive_%\"] = (\n 100 * prevalence[\"Positive\"]\n / prevalence[\"Available\"]\n)\n\ndisplay(\n prevalence.sort_values(\n \"Positive_%\",\n ascending=False\n ).round(2)\n)\n\n# ------------------------------------------------------------\n# 4. Report availability\n# ------------------------------------------------------------\nprint(\"\\n[3] RADIOLOGY REPORTS\")\nprint(\"-\" * 70)\n\nreport_available = (\n train[\"Report\"]\n .fillna(\"\")\n .astype(str)\n .str.strip()\n .ne(\"\")\n)\n\nprint(\n \"Reports available:\",\n int(report_available.sum()),\n f\"({100*report_available.mean():.2f}%)\"\n)\n\nreport_lengths = (\n train[\"Report\"]\n .fillna(\"\")\n .astype(str)\n .str.len()\n)\n\nprint(\n \"Median report length:\",\n int(report_lengths.median())\n)\n\nprint(\n \"Mean report length:\",\n round(report_lengths.mean(), 1)\n)\n\n# ------------------------------------------------------------\n# 5. Series distribution\n# ------------------------------------------------------------\nprint(\"\\n[4] SERIES DISTRIBUTION\")\nprint(\"-\" * 70)\n\nseries_per_study = (\n train_series\n .groupby(\"StudyInstanceUID\")\n .size()\n)\n\nprint(\n \"Studies represented in train_series:\",\n len(series_per_study)\n)\n\nprint(\n \"Mean series/study:\",\n round(series_per_study.mean(), 2)\n)\n\nprint(\n \"Median series/study:\",\n int(series_per_study.median())\n)\n\nprint(\n \"Min series/study:\",\n int(series_per_study.min())\n)\n\nprint(\n \"Max series/study:\",\n int(series_per_study.max())\n)\n\ndisplay(\n series_per_study\n .describe()\n .to_frame(\"Series_per_Study\")\n)\n\n# ------------------------------------------------------------\n# 6. MRI metadata distribution\n# ------------------------------------------------------------\nprint(\"\\n[5] MRI SERIES METADATA\")\nprint(\"-\" * 70)\n\nprint(\"\\nAnatomical plane:\")\ndisplay(\n train_series[\"Anatomical_Plane\"]\n .value_counts(dropna=False)\n)\n\nprint(\"\\nFluid sensitive:\")\ndisplay(\n train_series[\"Fluid_Sensitive\"]\n .value_counts(dropna=False)\n)\n\nprint(\"\\nFat suppression:\")\ndisplay(\n train_series[\"Fat_Suppression\"]\n .value_counts(dropna=False)\n)\n\n# ------------------------------------------------------------\n# 7. Duplicate study IDs\n# ------------------------------------------------------------\nprint(\"\\n[6] INTEGRITY CHECK\")\nprint(\"-\" * 70)\n\nprint(\n \"Duplicate StudyInstanceUID rows in train.csv:\",\n int(train[\"StudyInstanceUID\"].duplicated().sum())\n)\n\nprint(\n \"Duplicate StudyInstanceUID rows in train_series.csv:\",\n int(\n train_series[\n \"StudyInstanceUID\"\n ].duplicated().sum()\n )\n)\n\nprint(\n \"Duplicate SeriesInstanceUID:\",\n int(\n train_series[\n \"SeriesInstanceUID\"\n ].duplicated().sum()\n )\n)\n\n# ------------------------------------------------------------\n# 8. Train/test ID overlap\n# ------------------------------------------------------------\noverlap = set(\n train[\"StudyInstanceUID\"].astype(str)\n) & set(\n test[\"StudyInstanceUID\"].astype(str)\n)\n\nprint(\n \"\\nTrain/test StudyInstanceUID overlap:\",\n len(overlap)\n)\n\nassert len(overlap) == 0\n\nprint(\"\\n\" + \"=\" * 70)\nprint(\"AUDIT COMPLETE — DO NOT START DICOM DECODING YET\")\nprint(\"=\" * 70)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-08-07T15:48:10.39873Z","iopub.execute_input":"2026-08-07T15:48:10.399234Z","iopub.status.idle":"2026-08-07T15:48:10.481996Z","shell.execute_reply.started":"2026-08-07T15:48:10.399197Z","shell.execute_reply":"2026-08-07T15:48:10.48133Z"}},"outputs":[],"execution_count":null},{"id":"cc4d6793-91be-4197-bc38-497558722e5d","cell_type":"markdown","source":"# 5. Rank series without touching DICOM\n\nThe supplied metadata gives us:\n\n- Fluid_Sensitive\n- Fat_Suppression\n- Anatomical_Plane\n\nWe prioritize fluid-sensitive and fat-suppressed sequences while retaining plane diversity.\n\n### Why plane diversity matters\n\nWe do not want five nearly identical sagittal sequences.\n\nThe selector therefore uses a greedy diversity rule.","metadata":{}},{"id":"6d96a3fc-6ccc-43d9-bad7-b49b784dd5c2","cell_type":"code","source":"def prepare_series_metadata(df):\n x = df.copy()\n\n x[\"Fluid_Sensitive\"] = (\n pd.to_numeric(\n x[\"Fluid_Sensitive\"], errors=\"coerce\"\n ).fillna(0).astype(int)\n )\n\n x[\"Fat_Suppression\"] = (\n pd.to_numeric(\n x[\"Fat_Suppression\"], errors=\"coerce\"\n ).fillna(0).astype(int)\n )\n\n x[\"Anatomical_Plane\"] = (\n x[\"Anatomical_Plane\"]\n .fillna(\"Unknown\")\n .astype(str)\n )\n\n return x\n\nseries_meta = prepare_series_metadata(train_series)\ntest_series_meta = prepare_series_metadata(test_series)\n\nplane_bonus = {\n \"Sagittal\": 3.0,\n \"Coronal\": 2.0,\n \"Axial\": 1.0,\n \"Unknown\": 0.0\n}\n\ndef base_priority(r):\n return (\n 5.0 * r[\"Fluid_Sensitive\"]\n + 3.0 * r[\"Fat_Suppression\"]\n + plane_bonus.get(r[\"Anatomical_Plane\"], 0.0)\n )\n\nseries_meta[\"base_priority\"] = series_meta.apply(\n base_priority, axis=1\n)\ntest_series_meta[\"base_priority\"] = test_series_meta.apply(\n base_priority, axis=1\n)\n\ndef select_diverse_series(df, max_series=MAX_SERIES):\n selected = {}\n\n for sid, g in df.groupby(\"StudyInstanceUID\"):\n g = g.copy()\n g[\"priority\"] = g[\"base_priority\"]\n\n chosen = []\n\n # First pass: one high-priority sequence from each plane.\n for plane in [\"Sagittal\", \"Coronal\", \"Axial\"]:\n q = g[g[\"Anatomical_Plane\"] == plane]\n if len(q):\n row = q.sort_values(\n [\"priority\", \"Fat_Suppression\", \"Fluid_Sensitive\"],\n ascending=False\n ).iloc[0]\n chosen.append(row)\n\n # Second pass: highest remaining priority.\n used = {\n x[\"SeriesInstanceUID\"] for x in chosen\n }\n\n remaining = g[\n ~g[\"SeriesInstanceUID\"].isin(used)\n ].sort_values(\n [\"priority\", \"Fat_Suppression\", \"Fluid_Sensitive\"],\n ascending=False\n )\n\n for _, row in remaining.iterrows():\n if len(chosen) >= max_series:\n break\n chosen.append(row)\n\n selected[sid] = pd.DataFrame(chosen)\n\n return selected\n\nselected_train = select_diverse_series(series_meta)\nselected_test = select_diverse_series(test_series_meta)\n\nprint(\n \"Training studies with selected series:\",\n len(selected_train)\n)\nprint(\n \"Test studies with selected series:\",\n len(selected_test)\n)\n\nprint(\"\\nExample selection:\")\ndisplay(\n selected_train[\n next(iter(selected_train))\n ][[\n \"SeriesInstanceUID\",\n \"Fluid_Sensitive\",\n \"Fat_Suppression\",\n \"Anatomical_Plane\"\n ]]\n)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-08-07T15:46:54.459453Z","iopub.execute_input":"2026-08-07T15:46:54.45993Z","iopub.status.idle":"2026-08-07T15:47:24.941094Z","shell.execute_reply.started":"2026-08-07T15:46:54.459897Z","shell.execute_reply":"2026-08-07T15:47:24.940393Z"}},"outputs":[],"execution_count":null},{"id":"21e5a716-a6ed-4753-947b-eade85e43bc5","cell_type":"code","source":"# ============================================================================\n# RSNA KNEE V3.2\n# REPORT TEACHER — ROBUST CALIBRATION + FORENSICS + CONFIDENCE\n# ============================================================================\n#\n# PURPOSE\n# -------\n# 1. Automatically locate the training dataframe.\n# 2. Automatically locate the REPORT_* probability CSV.\n# 3. Correctly map:\n# ACL -> REPORT_ACL\n# MCL -> REPORT_MCL\n# ...\n# Baker's -> REPORT_Baker's\n# 4. Verify all 12 gold targets.\n# 5. Evaluate only the 58 gold-labelled studies.\n# 6. Find optimal thresholds.\n# 7. Generate confidence categories.\n# 8. Save thresholds, gold predictions and full-study predictions.\n#\n# IMPORTANT\n# ---------\n# This is a REPORT-TEACHER calibration stage.\n# It is NOT an MRI-image model.\n#\n# The very high V3.1.2 scores must NOT automatically be interpreted as\n# clinical/generalization performance because the gold set is only 58 studies\n# and the probabilities are derived from reports.\n# ============================================================================\n\nimport os\nimport re\nimport json\nimport warnings\nfrom pathlib import Path\n\nimport numpy as np\nimport pandas as pd\n\nfrom sklearn.metrics import (\n roc_auc_score,\n average_precision_score,\n f1_score,\n accuracy_score,\n precision_score,\n recall_score,\n confusion_matrix,\n)\n\nwarnings.filterwarnings(\"ignore\")\n\n# ============================================================================\n# 1. CONFIGURATION\n# ============================================================================\n\nSEED = 42\nnp.random.seed(SEED)\n\nTARGETS = [\n \"ACL\",\n \"MCL\",\n \"Medial Meniscus\",\n \"Lateral Meniscus\",\n \"Medial OA\",\n \"Lateral OA\",\n \"PF OA\",\n \"Effusion\",\n \"Synovitis\",\n \"Baker's\",\n \"Contusion\",\n \"Fracture\",\n]\n\nID_COL = \"StudyInstanceUID\"\n\n# Threshold search grid.\n# We deliberately search from 0.05 to 0.95 rather than assuming 0.50.\nTHRESHOLDS = np.round(np.arange(0.05, 0.951, 0.01), 2)\n\nINPUT_ROOT = Path(\"/kaggle/input\")\nWORK_ROOT = Path(\"/kaggle/working\")\n\nprint(\"=\" * 78)\nprint(\"RSNA KNEE V3.2 — REPORT TEACHER\")\nprint(\"=\" * 78)\nprint(\"Targets:\", len(TARGETS))\nprint(\"Working directory:\", WORK_ROOT)\nprint()\n\n\n# ============================================================================\n# 2. FIND FILES ROBUSTLY\n# ============================================================================\n\ndef all_csv_files():\n files = []\n\n for root in [INPUT_ROOT, WORK_ROOT]:\n if root.exists():\n try:\n files.extend(root.rglob(\"*.csv\"))\n except Exception:\n pass\n\n # Remove duplicates\n unique = {}\n for p in files:\n try:\n unique[str(p.resolve())] = p\n except Exception:\n unique[str(p)] = p\n\n return list(unique.values())\n\n\nCSV_FILES = all_csv_files()\n\nprint(\"CSV files discovered:\", len(CSV_FILES))\n\n\n# ============================================================================\n# 3. IDENTIFY TRAINING DATAFRAME\n# ============================================================================\n\ndef score_train_file(path):\n \"\"\"\n Score a CSV as a likely training dataframe.\n We expect:\n StudyInstanceUID\n 12 target columns\n possibly Report\n \"\"\"\n try:\n cols = list(pd.read_csv(path, nrows=0).columns)\n except Exception:\n return -1\n\n score = 0\n\n if ID_COL in cols:\n score += 20\n\n for t in TARGETS:\n if t in cols:\n score += 5\n\n if \"Report\" in cols:\n score += 10\n\n # Avoid probability files.\n report_prob_count = sum(\n str(c).startswith(\"REPORT_\") for c in cols\n )\n\n score -= report_prob_count * 10\n\n return score\n\n\ntrain_candidates = []\n\nfor p in CSV_FILES:\n s = score_train_file(p)\n if s >= 40:\n train_candidates.append((s, p))\n\n\nif not train_candidates:\n raise FileNotFoundError(\n \"\\nCould not automatically identify the training CSV.\\n\"\n \"Expected a CSV containing StudyInstanceUID and the 12 target columns:\\n\"\n + \", \".join(TARGETS)\n )\n\n\ntrain_candidates.sort(key=lambda x: x[0], reverse=True)\n\nTRAIN_PATH = train_candidates[0][1]\n\nprint()\nprint(\"TRAIN CSV:\")\nprint(TRAIN_PATH)\n\n\n# ============================================================================\n# 4. LOAD TRAINING DATA\n# ============================================================================\n\ntrain = pd.read_csv(TRAIN_PATH)\n\nprint()\nprint(\"Train shape:\", train.shape)\n\nmissing_targets = [\n t for t in TARGETS\n if t not in train.columns\n]\n\nif missing_targets:\n raise ValueError(\n \"Missing target columns in training CSV:\\n\"\n + str(missing_targets)\n )\n\nif ID_COL not in train.columns:\n raise ValueError(\n f\"Required ID column '{ID_COL}' not found.\"\n )\n\nprint(\"✓ All 12 original target columns found.\")\n\n\n# ============================================================================\n# 5. IDENTIFY REPORT PROBABILITY FILE\n# ============================================================================\n\nEXPECTED_PROB_COLUMNS = [\n f\"REPORT_{t}\" for t in TARGETS\n]\n\ndef score_probability_file(path):\n try:\n cols = list(pd.read_csv(path, nrows=0).columns)\n except Exception:\n return -1\n\n score = 0\n\n if ID_COL in cols:\n score += 20\n\n for c in EXPECTED_PROB_COLUMNS:\n if c in cols:\n score += 10\n\n # Strong bonus for exact 13-column structure.\n if len(cols) == 13:\n score += 10\n\n return score\n\n\nprob_candidates = []\n\nfor p in CSV_FILES:\n s = score_probability_file(p)\n if s >= 50:\n prob_candidates.append((s, p))\n\n\nif not prob_candidates:\n raise FileNotFoundError(\n \"\\nCould not find the REPORT probability CSV.\\n\"\n \"Expected columns such as:\\n\"\n + \"\\n\".join(EXPECTED_PROB_COLUMNS)\n )\n\n\nprob_candidates.sort(key=lambda x: x[0], reverse=True)\n\nPROB_PATH = prob_candidates[0][1]\n\nprint()\nprint(\"PROBABILITY CSV:\")\nprint(PROB_PATH)\n\n\n# ============================================================================\n# 6. LOAD PROBABILITIES\n# ============================================================================\n\nprob = pd.read_csv(PROB_PATH)\n\nprint()\nprint(\"Probability shape:\", prob.shape)\n\nif ID_COL not in prob.columns:\n raise ValueError(\n f\"'{ID_COL}' missing from probability CSV.\"\n )\n\nmissing_probs = [\n c for c in EXPECTED_PROB_COLUMNS\n if c not in prob.columns\n]\n\nif missing_probs:\n raise ValueError(\n \"\\nMissing REPORT probability columns:\\n\"\n + \"\\n\".join(missing_probs)\n )\n\nprint(\"✓ All 12 REPORT_* probability columns found.\")\n\n\n# ============================================================================\n# 7. CHECK PROBABILITY RANGES\n# ============================================================================\n\nprint()\nprint(\"Probability range check:\")\n\nfor target in TARGETS:\n\n col = f\"REPORT_{target}\"\n\n values = pd.to_numeric(\n prob[col],\n errors=\"coerce\"\n )\n\n n_nan = values.isna().sum()\n\n min_v = values.min()\n max_v = values.max()\n\n print(\n f\"{target:20s} \"\n f\"min={min_v:.4f} \"\n f\"max={max_v:.4f} \"\n f\"NaN={n_nan}\"\n )\n\n if n_nan > 0:\n raise ValueError(\n f\"{col} contains {n_nan} non-numeric/NaN values.\"\n )\n\n if min_v < 0 or max_v > 1:\n raise ValueError(\n f\"{col} contains probabilities outside [0,1].\"\n )\n\n\n# ============================================================================\n# 8. REMOVE DUPLICATE STUDY IDs\n# ============================================================================\n\nprint()\nprint(\"Duplicate ID check:\")\n\ntrain_dup = train[ID_COL].duplicated().sum()\nprob_dup = prob[ID_COL].duplicated().sum()\n\nprint(\"Train duplicate StudyInstanceUID:\", train_dup)\nprint(\"Probability duplicate StudyInstanceUID:\", prob_dup)\n\nif train_dup:\n print(\"WARNING: training data contains duplicate study IDs.\")\n\nif prob_dup:\n print(\"WARNING: probability data contains duplicate study IDs.\")\n\n\n# ============================================================================\n# 9. MERGE\n# ============================================================================\n\n# Keep one probability row per study for safe calibration.\nprob_unique = prob.drop_duplicates(\n subset=[ID_COL],\n keep=\"first\"\n).copy()\n\ndf = train.merge(\n prob_unique[\n [ID_COL] + EXPECTED_PROB_COLUMNS\n ],\n on=ID_COL,\n how=\"left\",\n validate=\"one_to_one\"\n)\n\nprint()\nprint(\"Merged shape:\", df.shape)\n\nmissing_probability_rows = df[EXPECTED_PROB_COLUMNS].isna().any(axis=1).sum()\n\nprint(\n \"Studies without report probabilities:\",\n missing_probability_rows\n)\n\n\n# ============================================================================\n# 10. CONVERT GOLD LABELS TO NUMERIC\n# ============================================================================\n\nfor target in TARGETS:\n df[target] = pd.to_numeric(\n df[target],\n errors=\"coerce\"\n )\n\n\n# ============================================================================\n# 11. IDENTIFY GOLD-LABELLED STUDIES\n# ============================================================================\n\ngold_mask = df[TARGETS].notna().all(axis=1)\n\ngold = df.loc[gold_mask].copy()\n\nprint()\nprint(\"=\" * 78)\nprint(\"GOLD DATASET\")\nprint(\"=\" * 78)\n\nprint(\"Gold-labelled studies:\", len(gold))\n\nif len(gold) != 58:\n print(\n \"WARNING: Expected 58 gold studies according to the previous audit, \"\n f\"but detected {len(gold)}.\"\n )\nelse:\n print(\"✓ Gold study count matches expected audit: 58\")\n\n\n# ============================================================================\n# 12. GOLD LABEL DISTRIBUTION\n# ============================================================================\n\nprint()\nprint(\"=\" * 78)\nprint(\"GOLD LABEL DISTRIBUTION\")\nprint(\"=\" * 78)\n\nfor target in TARGETS:\n\n pos = int((gold[target] == 1).sum())\n neg = int((gold[target] == 0).sum())\n\n print(\n f\"{target:20s} \"\n f\"positive={pos:2d} \"\n f\"negative={neg:2d}\"\n )\n\n\n# ============================================================================\n# 13. THRESHOLD OPTIMIZATION\n# ============================================================================\n\ndef safe_metric(metric_function, y_true, y_pred):\n try:\n return float(metric_function(y_true, y_pred))\n except Exception:\n return np.nan\n\n\nresults = []\nthreshold_dict = {}\n\nprint()\nprint(\"=\" * 78)\nprint(\"V3.2 OPTIMAL THRESHOLD SEARCH\")\nprint(\"=\" * 78)\n\nfor target in TARGETS:\n\n prob_col = f\"REPORT_{target}\"\n\n sub = gold[\n [target, prob_col]\n ].dropna()\n\n y = sub[target].astype(int).values\n p = sub[prob_col].astype(float).values\n\n # AUC / AP\n if len(np.unique(y)) == 2:\n auc = roc_auc_score(y, p)\n ap = average_precision_score(y, p)\n else:\n auc = np.nan\n ap = np.nan\n\n best = None\n\n for threshold in THRESHOLDS:\n\n pred = (p >= threshold).astype(int)\n\n f1 = f1_score(\n y,\n pred,\n zero_division=0\n )\n\n acc = accuracy_score(\n y,\n pred\n )\n\n precision = precision_score(\n y,\n pred,\n zero_division=0\n )\n\n recall = recall_score(\n y,\n pred,\n zero_division=0\n )\n\n # Primary optimization = F1.\n # Secondary = accuracy.\n key = (\n f1,\n acc,\n precision\n )\n\n if best is None or key > best[\"key\"]:\n best = {\n \"key\": key,\n \"threshold\": threshold,\n \"f1\": f1,\n \"accuracy\": acc,\n \"precision\": precision,\n \"recall\": recall,\n }\n\n threshold_dict[target] = best[\"threshold\"]\n\n results.append({\n \"Target\": target,\n \"Positive\": int((y == 1).sum()),\n \"Negative\": int((y == 0).sum()),\n \"AUC\": auc,\n \"AP\": ap,\n \"Optimal_Threshold\": best[\"threshold\"],\n \"F1\": best[\"f1\"],\n \"Accuracy\": best[\"accuracy\"],\n \"Precision\": best[\"precision\"],\n \"Recall\": best[\"recall\"],\n })\n\n print(\n f\"{target:20s} \"\n f\"AUC={auc:.4f} \"\n f\"AP={ap:.4f} \"\n f\"TH={best['threshold']:.2f} \"\n f\"F1={best['f1']:.4f} \"\n f\"ACC={best['accuracy']:.4f}\"\n )\n\n\nresults_df = pd.DataFrame(results)\n\n\n# ============================================================================\n# 14. SUMMARY\n# ============================================================================\n\nprint()\nprint(\"=\" * 78)\nprint(\"V3.2 CALIBRATION RESULTS\")\nprint(\"=\" * 78)\n\ndisplay(results_df)\n\n\nprint()\nprint(\"=\" * 78)\nprint(\"MEAN PERFORMANCE — GOLD SET\")\nprint(\"=\" * 78)\n\nmean_auc = results_df[\"AUC\"].mean()\nmean_ap = results_df[\"AP\"].mean()\nmean_f1 = results_df[\"F1\"].mean()\nmean_acc = results_df[\"Accuracy\"].mean()\n\nprint(f\"Mean AUC : {mean_auc:.6f}\")\nprint(f\"Mean AP : {mean_ap:.6f}\")\nprint(f\"Mean F1 : {mean_f1:.6f}\")\nprint(f\"Mean Accuracy : {mean_acc:.6f}\")\n\n\n# ============================================================================\n# 15. GENERATE GOLD PREDICTIONS\n# ============================================================================\n\ngold_predictions = gold[\n [ID_COL]\n].copy()\n\nfor target in TARGETS:\n\n prob_col = f\"REPORT_{target}\"\n\n threshold = threshold_dict[target]\n\n gold_predictions[f\"{target}_PROB\"] = gold[\n prob_col\n ].values\n\n gold_predictions[f\"{target}_PRED\"] = (\n gold[prob_col].values >= threshold\n ).astype(int)\n\n gold_predictions[f\"{target}_TRUE\"] = (\n gold[target].values.astype(int)\n )\n\n\n# ============================================================================\n# 16. CONFUSION MATRICES\n# ============================================================================\n\nprint()\nprint(\"=\" * 78)\nprint(\"GOLD CONFUSION MATRICES\")\nprint(\"=\" * 78)\n\nconfusion_rows = []\n\nfor target in TARGETS:\n\n y_true = gold[target].astype(int).values\n\n p = gold[\n f\"REPORT_{target}\"\n ].astype(float).values\n\n threshold = threshold_dict[target]\n\n y_pred = (\n p >= threshold\n ).astype(int)\n\n tn, fp, fn, tp = confusion_matrix(\n y_true,\n y_pred,\n labels=[0, 1]\n ).ravel()\n\n confusion_rows.append({\n \"Target\": target,\n \"TN\": tn,\n \"FP\": fp,\n \"FN\": fn,\n \"TP\": tp,\n })\n\nconfusion_df = pd.DataFrame(confusion_rows)\n\ndisplay(confusion_df)\n\n\n# ============================================================================\n# 17. FULL DATASET PREDICTIONS\n# ============================================================================\n\nprint()\nprint(\"=\" * 78)\nprint(\"GENERATING FULL-STUDY REPORT TEACHER PREDICTIONS\")\nprint(\"=\" * 78)\n\nfull_predictions = df[\n [ID_COL]\n].copy()\n\nfor target in TARGETS:\n\n prob_col = f\"REPORT_{target}\"\n\n threshold = threshold_dict[target]\n\n p = pd.to_numeric(\n df[prob_col],\n errors=\"coerce\"\n )\n\n full_predictions[f\"{target}_PROB\"] = p\n\n full_predictions[f\"{target}_PRED\"] = (\n p >= threshold\n ).astype(\"Int64\")\n\n\n# ============================================================================\n# 18. V3.2 CONFIDENCE SYSTEM\n# ============================================================================\n#\n# IMPORTANT:\n# We do NOT use the previous arbitrary:\n#\n# POS >= 0.90\n# NEG <= 0.10\n#\n# because the report probabilities are clustered around ~0.2-0.6.\n#\n# Instead, confidence is based on distance from the calibrated threshold.\n#\n# Very close to threshold -> UNCERTAIN\n# Moderate distance -> MODERATE\n# Large distance -> HIGH\n#\n# This is much more meaningful for this teacher.\n# ============================================================================\n\ndef confidence_from_threshold(probability, threshold):\n\n if pd.isna(probability):\n return \"MISSING\"\n\n distance = abs(float(probability) - float(threshold))\n\n if distance < 0.05:\n return \"UNCERTAIN\"\n\n elif distance < 0.15:\n return \"MODERATE\"\n\n else:\n return \"HIGH\"\n\n\nfor target in TARGETS:\n\n prob_col = f\"{target}_PROB\"\n\n full_predictions[\n f\"{target}_CONFIDENCE\"\n ] = full_predictions[\n prob_col\n ].apply(\n lambda x, t=threshold_dict[target]:\n confidence_from_threshold(x, t)\n )\n\n\n# ============================================================================\n# 19. CONFIDENCE SUMMARY\n# ============================================================================\n\nprint()\nprint(\"=\" * 78)\nprint(\"V3.2 CONFIDENCE SUMMARY\")\nprint(\"=\" * 78)\n\nconfidence_rows = []\n\nfor target in TARGETS:\n\n c = full_predictions[\n f\"{target}_CONFIDENCE\"\n ]\n\n confidence_rows.append({\n \"Target\": target,\n \"HIGH\": int((c == \"HIGH\").sum()),\n \"MODERATE\": int((c == \"MODERATE\").sum()),\n \"UNCERTAIN\": int((c == \"UNCERTAIN\").sum()),\n \"MISSING\": int((c == \"MISSING\").sum()),\n })\n\nconfidence_df = pd.DataFrame(confidence_rows)\n\ndisplay(confidence_df)\n\n\n# ============================================================================\n# 20. THRESHOLD DICTIONARY\n# ============================================================================\n\nprint()\nprint(\"=\" * 78)\nprint(\"V3.2 THRESHOLD DICTIONARY\")\nprint(\"=\" * 78)\n\nfor target in TARGETS:\n print(\n f\" {target!r}: \"\n f\"{threshold_dict[target]:.2f},\"\n )\n\n\n# ============================================================================\n# 21. SAVE ALL OUTPUTS\n# ============================================================================\n\nOUT = WORK_ROOT\n\nthreshold_path = (\n OUT / \"rsna_report_teacher_thresholds_v3_2.csv\"\n)\n\ngold_metrics_path = (\n OUT / \"rsna_report_teacher_gold_metrics_v3_2.csv\"\n)\n\ngold_predictions_path = (\n OUT / \"rsna_report_teacher_gold_predictions_v3_2.csv\"\n)\n\nconfusion_path = (\n OUT / \"rsna_report_teacher_confusion_v3_2.csv\"\n)\n\nconfidence_path = (\n OUT / \"rsna_report_teacher_confidence_v3_2.csv\"\n)\n\nfull_predictions_path = (\n OUT / \"rsna_report_teacher_full_predictions_v3_2.csv\"\n)\n\nthreshold_json_path = (\n OUT / \"rsna_report_teacher_thresholds_v3_2.json\"\n)\n\n\n# Save threshold table\nresults_df.to_csv(\n threshold_path,\n index=False\n)\n\n# Save metrics\nresults_df.to_csv(\n gold_metrics_path,\n index=False\n)\n\n# Save gold predictions\ngold_predictions.to_csv(\n gold_predictions_path,\n index=False\n)\n\n# Save confusion matrices\nconfusion_df.to_csv(\n confusion_path,\n index=False\n)\n\n# Save confidence summary\nconfidence_df.to_csv(\n confidence_path,\n index=False\n)\n\n# Save full-study predictions\nfull_predictions.to_csv(\n full_predictions_path,\n index=False\n)\n\n# Save JSON dictionary\nwith open(\n threshold_json_path,\n \"w\",\n encoding=\"utf-8\"\n) as f:\n\n json.dump(\n threshold_dict,\n f,\n indent=4\n )\n\n\n# ============================================================================\n# 22. FINAL AUDIT\n# ============================================================================\n\nprint()\nprint(\"=\" * 78)\nprint(\"V3.2 OUTPUT FILES\")\nprint(\"=\" * 78)\n\nfor p in [\n threshold_path,\n gold_metrics_path,\n gold_predictions_path,\n confusion_path,\n confidence_path,\n full_predictions_path,\n threshold_json_path,\n]:\n\n print(\"✓\", p)\n\nprint()\nprint(\"=\" * 78)\nprint(\"V3.2 FINAL AUDIT\")\nprint(\"=\" * 78)\n\nprint(\"Train studies:\", len(train))\nprint(\"Probability studies:\", len(prob_unique))\nprint(\"Merged studies:\", len(df))\nprint(\"Gold studies:\", len(gold))\n\nprint()\nprint(\"Mean AUC:\", round(mean_auc, 6))\nprint(\"Mean AP :\", round(mean_ap, 6))\nprint(\"Mean F1 :\", round(mean_f1, 6))\nprint(\"Mean ACC:\", round(mean_acc, 6))\n\nprint()\nprint(\"✓ All 12 report teachers processed.\")\nprint(\"✓ REPORT_* probability names handled correctly.\")\nprint(\"✓ Gold-study calibration completed.\")\nprint(\"✓ Thresholds saved.\")\nprint(\"✓ Gold predictions saved.\")\nprint(\"✓ Full-study predictions saved.\")\nprint(\"✓ Confidence categories saved.\")\n\nprint()\nprint(\"=\" * 78)\nprint(\"RSNA KNEE V3.2 — COMPLETE\")\nprint(\"=\" * 78)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-08-07T16:38:20.061444Z","iopub.execute_input":"2026-08-07T16:38:20.061776Z","iopub.status.idle":"2026-08-07T16:41:08.193708Z","shell.execute_reply.started":"2026-08-07T16:38:20.06175Z","shell.execute_reply":"2026-08-07T16:41:08.192961Z"}},"outputs":[],"execution_count":null},{"id":"4c65a8a4-98a7-44c0-8fbe-3bdf9b506a66","cell_type":"code","source":"# =============================================================================\n# RSNA KNEE V3.3 — REPORT TEACHER LEAKAGE AUDIT\n# =============================================================================\n# Purpose:\n# Investigate the suspicious V3.2 near-perfect performance.\n#\n# IMPORTANT:\n# This cell does NOT train a new model.\n# It audits the existing V3.2 probability and prediction files.\n# =============================================================================\n\nimport os\nimport json\nimport glob\nimport warnings\nimport numpy as np\nimport pandas as pd\n\nfrom sklearn.metrics import (\n roc_auc_score,\n average_precision_score,\n accuracy_score,\n f1_score,\n precision_score,\n recall_score,\n confusion_matrix,\n)\n\nwarnings.filterwarnings(\"ignore\")\n\nprint(\"=\" * 78)\nprint(\"RSNA KNEE V3.3 — REPORT TEACHER LEAKAGE AUDIT\")\nprint(\"=\" * 78)\n\n# -------------------------------------------------------------------------\n# CONFIGURATION\n# -------------------------------------------------------------------------\n\nWORK = \"/kaggle/working\"\n\nTRAIN_CSV = \"/kaggle/input/competitions/rsna-knee-abnormality-detection/train.csv\"\n\nPROB_CSV = os.path.join(\n WORK,\n \"rsna_report_probabilities_v3.csv\"\n)\n\nTHRESHOLD_CSV = os.path.join(\n WORK,\n \"rsna_report_teacher_thresholds_v3_2.csv\"\n)\n\nGOLD_PRED_CSV = os.path.join(\n WORK,\n \"rsna_report_teacher_gold_predictions_v3_2.csv\"\n)\n\nFULL_PRED_CSV = os.path.join(\n WORK,\n \"rsna_report_teacher_full_predictions_v3_2.csv\"\n)\n\nTARGETS = [\n \"ACL\",\n \"MCL\",\n \"Medial Meniscus\",\n \"Lateral Meniscus\",\n \"Medial OA\",\n \"Lateral OA\",\n \"PF OA\",\n \"Effusion\",\n \"Synovitis\",\n \"Baker's\",\n \"Contusion\",\n \"Fracture\",\n]\n\n# -------------------------------------------------------------------------\n# FILE CHECK\n# -------------------------------------------------------------------------\n\nrequired_files = {\n \"TRAIN\": TRAIN_CSV,\n \"PROBABILITY\": PROB_CSV,\n \"THRESHOLD\": THRESHOLD_CSV,\n \"GOLD_PREDICTIONS\": GOLD_PRED_CSV,\n \"FULL_PREDICTIONS\": FULL_PRED_CSV,\n}\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"FILE AUDIT\")\nprint(\"=\" * 78)\n\nfor name, path in required_files.items():\n exists = os.path.exists(path)\n print(f\"{name:<20} {'✓ FOUND' if exists else '✗ MISSING'}\")\n if exists:\n print(\" \", path)\n\nmissing = [name for name, path in required_files.items()\n if not os.path.exists(path)]\n\nif missing:\n raise FileNotFoundError(\n f\"\\nMissing required files: {missing}\"\n )\n\n# -------------------------------------------------------------------------\n# LOAD DATA\n# -------------------------------------------------------------------------\n\ntrain = pd.read_csv(TRAIN_CSV)\nprob = pd.read_csv(PROB_CSV)\nthresholds = pd.read_csv(THRESHOLD_CSV)\n\nprint(\"\\nTrain shape:\", train.shape)\nprint(\"Probability shape:\", prob.shape)\nprint(\"Threshold shape:\", thresholds.shape)\n\n# -------------------------------------------------------------------------\n# TARGET COLUMN DETECTION\n# -------------------------------------------------------------------------\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"TARGET COLUMN AUDIT\")\nprint(\"=\" * 78)\n\nmissing_targets = [t for t in TARGETS if t not in train.columns]\n\nif missing_targets:\n print(\"✗ Missing target columns:\", missing_targets)\nelse:\n print(\"✓ All 12 original target columns found.\")\n\n# -------------------------------------------------------------------------\n# PROBABILITY COLUMN AUDIT\n# -------------------------------------------------------------------------\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"PROBABILITY COLUMN AUDIT\")\nprint(\"=\" * 78)\n\nprob_cols = {}\n\nfor target in TARGETS:\n expected = f\"REPORT_{target}\"\n\n if expected in prob.columns:\n prob_cols[target] = expected\n print(f\"{target:<20} -> {expected}\")\n else:\n print(f\"{target:<20} -> MISSING\")\n\nif len(prob_cols) != len(TARGETS):\n raise ValueError(\n \"Not all REPORT_* probability columns were found.\"\n )\n\n# -------------------------------------------------------------------------\n# ID AUDIT\n# -------------------------------------------------------------------------\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"STUDY ID AUDIT\")\nprint(\"=\" * 78)\n\nif \"StudyInstanceUID\" not in train.columns:\n raise ValueError(\"StudyInstanceUID missing from train.csv\")\n\nif \"StudyInstanceUID\" not in prob.columns:\n raise ValueError(\"StudyInstanceUID missing from probability CSV\")\n\ntrain_ids = train[\"StudyInstanceUID\"].astype(str)\nprob_ids = prob[\"StudyInstanceUID\"].astype(str)\n\nprint(\"Train IDs:\", len(train_ids))\nprint(\"Probability IDs:\", len(prob_ids))\n\nprint(\n \"Train duplicate IDs:\",\n train_ids.duplicated().sum()\n)\n\nprint(\n \"Probability duplicate IDs:\",\n prob_ids.duplicated().sum()\n)\n\nprint(\n \"IDs exactly identical:\",\n train_ids.equals(prob_ids)\n)\n\n# -------------------------------------------------------------------------\n# MERGE\n# -------------------------------------------------------------------------\n\ndf = train.merge(\n prob,\n on=\"StudyInstanceUID\",\n how=\"left\",\n validate=\"one_to_one\",\n suffixes=(\"\", \"_PROB\")\n)\n\nprint(\"\\nMerged shape:\", df.shape)\n\nmissing_prob_rows = df[\n df[list(prob_cols.values())].isna().any(axis=1)\n]\n\nprint(\n \"Studies missing probability:\",\n len(missing_prob_rows)\n)\n\n# -------------------------------------------------------------------------\n# GOLD DATASET\n# -------------------------------------------------------------------------\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"GOLD DATASET AUDIT\")\nprint(\"=\" * 78)\n\n# A study is considered gold if at least one target has a usable\n# binary label AND the dataset's gold labels match the expected\n# manually audited 58-study subset.\n\n# Determine rows with completely usable binary targets.\ngold_mask = pd.Series(True, index=df.index)\n\nfor target in TARGETS:\n vals = pd.to_numeric(\n df[target],\n errors=\"coerce\"\n )\n\n gold_mask &= vals.isin([0, 1])\n\ngold = df.loc[gold_mask].copy()\n\nprint(\"Rows with complete binary labels:\", len(gold))\n\n# If your original audit says 58, flag deviation.\nif len(gold) == 58:\n print(\"✓ Gold count = 58\")\nelse:\n print(\n \"⚠ Gold count differs from expected 58:\",\n len(gold)\n )\n\n# -------------------------------------------------------------------------\n# DIRECT CORRELATION / SEPARABILITY TEST\n# -------------------------------------------------------------------------\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"DIRECT PROBABILITY VS GOLD LABEL AUDIT\")\nprint(\"=\" * 78)\n\naudit_rows = []\n\nfor target in TARGETS:\n\n y = pd.to_numeric(\n gold[target],\n errors=\"coerce\"\n )\n\n p = pd.to_numeric(\n gold[prob_cols[target]],\n errors=\"coerce\"\n )\n\n valid = y.notna() & p.notna()\n\n y = y[valid].astype(int)\n p = p[valid].astype(float)\n\n if y.nunique() < 2:\n auc = np.nan\n ap = np.nan\n else:\n auc = roc_auc_score(y, p)\n ap = average_precision_score(y, p)\n\n pos = p[y == 1]\n neg = p[y == 0]\n\n min_pos = pos.min() if len(pos) else np.nan\n max_neg = neg.max() if len(neg) else np.nan\n\n perfect_separation = (\n len(pos) > 0 and\n len(neg) > 0 and\n min_pos > max_neg\n )\n\n audit_rows.append({\n \"Target\": target,\n \"Positive\": int((y == 1).sum()),\n \"Negative\": int((y == 0).sum()),\n \"AUC\": auc,\n \"AP\": ap,\n \"Min_Positive_Probability\": min_pos,\n \"Max_Negative_Probability\": max_neg,\n \"Perfect_Separation\": perfect_separation,\n })\n\naudit = pd.DataFrame(audit_rows)\n\nprint(audit.to_string(index=False))\n\n# -------------------------------------------------------------------------\n# PROBABILITY DISTRIBUTION AUDIT\n# -------------------------------------------------------------------------\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"PROBABILITY DISTRIBUTION BY GOLD LABEL\")\nprint(\"=\" * 78)\n\nfor target in TARGETS:\n\n pcol = prob_cols[target]\n\n y = pd.to_numeric(\n gold[target],\n errors=\"coerce\"\n )\n\n p = pd.to_numeric(\n gold[pcol],\n errors=\"coerce\"\n )\n\n pos = p[y == 1]\n neg = p[y == 0]\n\n print(f\"\\n{target}\")\n\n print(\n f\" Positive: \"\n f\"n={len(pos):2d} \"\n f\"mean={pos.mean():.4f} \"\n f\"std={pos.std():.4f} \"\n f\"min={pos.min():.4f} \"\n f\"max={pos.max():.4f}\"\n )\n\n print(\n f\" Negative: \"\n f\"n={len(neg):2d} \"\n f\"mean={neg.mean():.4f} \"\n f\"std={neg.std():.4f} \"\n f\"min={neg.min():.4f} \"\n f\"max={neg.max():.4f}\"\n )\n\n# -------------------------------------------------------------------------\n# TARGET-PROBABILITY CORRELATION\n# -------------------------------------------------------------------------\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"LABEL ↔ PROBABILITY CORRELATION\")\nprint(\"=\" * 78)\n\ncorr_rows = []\n\nfor target in TARGETS:\n\n y = pd.to_numeric(\n gold[target],\n errors=\"coerce\"\n )\n\n p = pd.to_numeric(\n gold[prob_cols[target]],\n errors=\"coerce\"\n )\n\n valid = y.notna() & p.notna()\n\n if valid.sum() > 2:\n corr = np.corrcoef(\n y[valid].astype(float),\n p[valid].astype(float)\n )[0, 1]\n else:\n corr = np.nan\n\n corr_rows.append({\n \"Target\": target,\n \"Correlation\": corr\n })\n\ncorr_df = pd.DataFrame(corr_rows)\n\nprint(corr_df.to_string(index=False))\n\n# -------------------------------------------------------------------------\n# THRESHOLD AUDIT\n# -------------------------------------------------------------------------\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"THRESHOLD AUDIT\")\nprint(\"=\" * 78)\n\nprint(thresholds.to_string(index=False))\n\n# -------------------------------------------------------------------------\n# CRITICAL TEST:\n# ARE THRESHOLDS OPTIMIZED ON THE SAME GOLD SET?\n# -------------------------------------------------------------------------\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"CRITICAL EVALUATION DESIGN AUDIT\")\nprint(\"=\" * 78)\n\nprint(\n \"\"\"\nThe V3.2 thresholds were selected using the 58 gold-labelled\nstudies and then evaluated on those same 58 studies.\n\nTherefore:\n\n threshold selection data == evaluation data\n\nThis is NOT an independent test.\n\nThe reported V3.2 F1/Accuracy therefore cannot be treated as\nunbiased generalization performance.\n\"\"\"\n)\n\n# -------------------------------------------------------------------------\n# CHECK FOR PERFECT SEPARATION\n# -------------------------------------------------------------------------\n\nperfect_targets = audit.loc[\n audit[\"Perfect_Separation\"] == True,\n \"Target\"\n].tolist()\n\nprint(\n \"Targets with mathematically perfect probability separation:\"\n)\n\nif perfect_targets:\n for x in perfect_targets:\n print(\" ⚠\", x)\nelse:\n print(\" None\")\n\nprint(\n f\"\\nPerfectly separated targets: \"\n f\"{len(perfect_targets)}/{len(TARGETS)}\"\n)\n\n# -------------------------------------------------------------------------\n# CHECK WHETHER PROBABILITY VALUES LOOK LIKE SOFT LABELS\n# -------------------------------------------------------------------------\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"SOFT-LABEL / PROBABILITY SIGNATURE AUDIT\")\nprint(\"=\" * 78)\n\nsignature_rows = []\n\nfor target in TARGETS:\n\n p = pd.to_numeric(\n gold[prob_cols[target]],\n errors=\"coerce\"\n )\n\n y = pd.to_numeric(\n gold[target],\n errors=\"coerce\"\n )\n\n pos = p[y == 1]\n neg = p[y == 0]\n\n if len(pos) and len(neg):\n\n midpoint = (\n pos.min() + neg.max()\n ) / 2.0\n\n separation_margin = (\n pos.min() - neg.max()\n )\n\n else:\n midpoint = np.nan\n separation_margin = np.nan\n\n signature_rows.append({\n \"Target\": target,\n \"MinPositive\": pos.min(),\n \"MaxNegative\": neg.max(),\n \"SeparationMargin\": separation_margin,\n \"SuggestedMidpoint\": midpoint,\n })\n\nsignature_df = pd.DataFrame(signature_rows)\n\nprint(signature_df.to_string(index=False))\n\n# -------------------------------------------------------------------------\n# SAVE AUDIT FILES\n# -------------------------------------------------------------------------\n\naudit_path = os.path.join(\n WORK,\n \"rsna_report_teacher_v3_3_leakage_audit.csv\"\n)\n\ncorr_path = os.path.join(\n WORK,\n \"rsna_report_teacher_v3_3_label_probability_correlation.csv\"\n)\n\nsignature_path = os.path.join(\n WORK,\n \"rsna_report_teacher_v3_3_probability_signature.csv\"\n)\n\naudit.to_csv(audit_path, index=False)\ncorr_df.to_csv(corr_path, index=False)\nsignature_df.to_csv(signature_path, index=False)\n\n# -------------------------------------------------------------------------\n# FINAL DIAGNOSTIC\n# -------------------------------------------------------------------------\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"V3.3 LEAKAGE AUDIT — FINAL DIAGNOSTIC\")\nprint(\"=\" * 78)\n\nif len(perfect_targets) >= 8:\n verdict = \"HIGH SUSPICION\"\nelif len(perfect_targets) >= 4:\n verdict = \"MODERATE SUSPICION\"\nelse:\n verdict = \"LOWER SUSPICION\"\n\nprint(\"Diagnostic level:\", verdict)\n\nprint(\n \"\"\"\nIMPORTANT:\n\nThis audit does not accuse the pipeline of leakage merely because\nperformance is high.\n\nIt identifies whether the evaluation design is capable of producing\nartificially high performance.\n\nThe two biggest issues currently requiring investigation are:\n\n1. Thresholds were optimized on the same 58 gold studies used for\n evaluation.\n\n2. The REPORT_* probabilities almost perfectly separate the gold\n labels despite V3.0 showing much weaker performance.\n\nThe next step after this cell is to trace HOW rsna_report_probabilities_v3.csv\nwas generated.\n\nDo not report V3.2's 0.9999 AUC as final model performance until that\ngeneration process has been audited.\n\"\"\"\n)\n\nprint(\"\\nSaved:\")\nprint(\"✓\", audit_path)\nprint(\"✓\", corr_path)\nprint(\"✓\", signature_path)\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"RSNA KNEE V3.3 — LEAKAGE AUDIT COMPLETE\")\nprint(\"=\" * 78)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-08-07T16:43:38.919885Z","iopub.execute_input":"2026-08-07T16:43:38.920287Z","iopub.status.idle":"2026-08-07T16:43:39.185597Z","shell.execute_reply.started":"2026-08-07T16:43:38.920257Z","shell.execute_reply":"2026-08-07T16:43:39.184812Z"}},"outputs":[],"execution_count":null},{"id":"5599e17f-ca25-4c56-9ae0-b7aaf552e140","cell_type":"code","source":"# =============================================================================\n# RSNA KNEE V3.4 — REPORT TEACHER FORENSIC / PROVENANCE AUDIT\n# =============================================================================\n#\n# PURPOSE\n# -------\n# V3.3 discovered:\n# 1. Thresholds were optimized on the same 58 gold studies.\n# 2. REPORT_* probabilities almost perfectly separate gold labels.\n#\n# V3.4 investigates HOW the probability file was generated.\n#\n# THIS CELL:\n# - Does NOT train a model\n# - Does NOT modify source data\n# - Does NOT optimize thresholds\n# - Does NOT overwrite existing files\n# - Produces forensic audit files only\n#\n# =============================================================================\n\nimport os\nimport re\nimport json\nimport hashlib\nimport warnings\nfrom pathlib import Path\n\nimport numpy as np\nimport pandas as pd\n\nfrom sklearn.metrics import (\n roc_auc_score,\n average_precision_score,\n mutual_info_score,\n)\nfrom sklearn.feature_selection import mutual_info_classif\n\nwarnings.filterwarnings(\"ignore\")\n\n# -----------------------------------------------------------------------------\n# CONFIGURATION\n# -----------------------------------------------------------------------------\n\nWORK = Path(\"/kaggle/working\")\n\nTRAIN_PATH = Path(\n \"/kaggle/input/competitions/rsna-knee-abnormality-detection/train.csv\"\n)\n\nPROB_PATH = WORK / \"rsna_report_probabilities_v3.csv\"\n\nTHRESHOLD_PATH = WORK / \"rsna_report_teacher_thresholds_v3_2.csv\"\n\nTARGETS = [\n \"ACL\",\n \"MCL\",\n \"Medial Meniscus\",\n \"Lateral Meniscus\",\n \"Medial OA\",\n \"Lateral OA\",\n \"PF OA\",\n \"Effusion\",\n \"Synovitis\",\n \"Baker's\",\n \"Contusion\",\n \"Fracture\",\n]\n\nUID = \"StudyInstanceUID\"\n\n# Output prefix\nPREFIX = WORK / \"rsna_report_teacher_v3_4\"\n\nprint(\"=\" * 78)\nprint(\"RSNA KNEE V3.4 — REPORT TEACHER FORENSIC / PROVENANCE AUDIT\")\nprint(\"=\" * 78)\n\n# =============================================================================\n# 1. BASIC FILE AUDIT\n# =============================================================================\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"1. FILE AUDIT\")\nprint(\"=\" * 78)\n\nrequired = {\n \"TRAIN\": TRAIN_PATH,\n \"PROBABILITY\": PROB_PATH,\n \"THRESHOLD\": THRESHOLD_PATH,\n}\n\nfor name, path in required.items():\n if path.exists():\n print(f\"{name:<15} ✓ FOUND\")\n print(f\" {path}\")\n print(f\" Size: {path.stat().st_size:,} bytes\")\n else:\n print(f\"{name:<15} ✗ MISSING\")\n print(f\" {path}\")\n\nif not TRAIN_PATH.exists():\n raise FileNotFoundError(f\"Training file not found: {TRAIN_PATH}\")\n\nif not PROB_PATH.exists():\n raise FileNotFoundError(f\"Probability file not found: {PROB_PATH}\")\n\n# =============================================================================\n# 2. LOAD DATA\n# =============================================================================\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"2. LOAD DATA\")\nprint(\"=\" * 78)\n\ntrain = pd.read_csv(TRAIN_PATH)\nprob = pd.read_csv(PROB_PATH)\n\nprint(\"Train shape :\", train.shape)\nprint(\"Probability shape :\", prob.shape)\n\n# =============================================================================\n# 3. TARGET COLUMN AUDIT\n# =============================================================================\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"3. TARGET COLUMN AUDIT\")\nprint(\"=\" * 78)\n\nmissing_targets = [t for t in TARGETS if t not in train.columns]\n\nif missing_targets:\n raise ValueError(f\"Missing target columns: {missing_targets}\")\n\nprint(\"✓ All 12 original target columns found.\")\n\n# =============================================================================\n# 4. PROBABILITY COLUMN AUDIT\n# =============================================================================\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"4. PROBABILITY COLUMN AUDIT\")\nprint(\"=\" * 78)\n\nprob_map = {}\n\nfor target in TARGETS:\n expected = f\"REPORT_{target}\"\n\n if expected in prob.columns:\n prob_map[target] = expected\n print(f\"{target:<20} -> {expected}\")\n else:\n print(f\"{target:<20} -> NOT FOUND\")\n\nmissing_prob = [t for t in TARGETS if t not in prob_map]\n\nif missing_prob:\n raise ValueError(\n f\"Missing probability columns: {missing_prob}\"\n )\n\nprint(\"\\n✓ All REPORT_* probability columns found.\")\n\n# =============================================================================\n# 5. UID FORENSICS\n# =============================================================================\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"5. STUDY UID FORENSICS\")\nprint(\"=\" * 78)\n\nif UID not in train.columns:\n raise ValueError(\"StudyInstanceUID missing from train.\")\n\nif UID not in prob.columns:\n raise ValueError(\"StudyInstanceUID missing from probability file.\")\n\ntrain_ids = train[UID].astype(str)\nprob_ids = prob[UID].astype(str)\n\nprint(\"Train IDs :\", len(train_ids))\nprint(\"Probability IDs :\", len(prob_ids))\n\nprint(\"Train duplicates:\",\n train_ids.duplicated().sum())\n\nprint(\"Prob duplicates :\",\n prob_ids.duplicated().sum())\n\ntrain_set = set(train_ids)\nprob_set = set(prob_ids)\n\nprint(\"Missing in probability:\",\n len(train_set - prob_set))\n\nprint(\"Extra in probability:\",\n len(prob_set - train_set))\n\nsame_ids = list(train_ids) == list(prob_ids)\n\nprint(\"IDs same order :\", same_ids)\nprint(\"IDs same set :\", train_set == prob_set)\n\n# =============================================================================\n# 6. MERGE — PRESERVE ORIGINAL TRAIN DATA\n# =============================================================================\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"6. FORENSIC MERGE\")\nprint(\"=\" * 78)\n\nprob_small = prob[[UID] + list(prob_map.values())].copy()\n\ndf = train.merge(\n prob_small,\n on=UID,\n how=\"left\",\n validate=\"one_to_one\",\n)\n\nprint(\"Merged shape:\", df.shape)\n\nfor target, pcol in prob_map.items():\n missing = df[pcol].isna().sum()\n print(f\"{target:<20} missing probabilities = {missing}\")\n\n# =============================================================================\n# 7. GOLD DATASET\n# =============================================================================\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"7. GOLD DATASET AUDIT\")\nprint(\"=\" * 78)\n\ngold_mask = train[TARGETS].notna().all(axis=1)\n\nfor target in TARGETS:\n gold_mask &= train[target].isin([0, 1])\n\ngold = df.loc[gold_mask].copy()\n\nprint(\"Gold-labelled studies:\", len(gold))\n\nif len(gold) != 58:\n print(\n f\"⚠ WARNING: expected 58 gold studies but found {len(gold)}\"\n )\nelse:\n print(\"✓ Gold count = 58\")\n\n# =============================================================================\n# 8. FILE HASHES\n# =============================================================================\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"8. FILE HASH FORENSICS\")\nprint(\"=\" * 78)\n\ndef sha256_file(path, chunk_size=1024 * 1024):\n h = hashlib.sha256()\n with open(path, \"rb\") as f:\n while True:\n chunk = f.read(chunk_size)\n if not chunk:\n break\n h.update(chunk)\n return h.hexdigest()\n\nhash_rows = []\n\nfor label, path in required.items():\n if path.exists():\n hash_rows.append({\n \"File\": label,\n \"Path\": str(path),\n \"Bytes\": path.stat().st_size,\n \"SHA256\": sha256_file(path),\n \"Modified\": pd.Timestamp.fromtimestamp(\n path.stat().st_mtime\n ),\n })\n\nhash_df = pd.DataFrame(hash_rows)\n\nprint(hash_df.to_string(index=False))\n\nhash_df.to_csv(\n f\"{PREFIX}_file_hashes.csv\",\n index=False\n)\n\n# =============================================================================\n# 9. PROBABILITY RANGE / DATA TYPE FORENSICS\n# =============================================================================\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"9. PROBABILITY RANGE FORENSICS\")\nprint(\"=\" * 78)\n\nrange_rows = []\n\nfor target in TARGETS:\n\n pcol = prob_map[target]\n\n x = pd.to_numeric(\n df[pcol],\n errors=\"coerce\"\n )\n\n range_rows.append({\n \"Target\": target,\n \"dtype\": str(df[pcol].dtype),\n \"min\": x.min(),\n \"max\": x.max(),\n \"mean\": x.mean(),\n \"std\": x.std(),\n \"NaN\": x.isna().sum(),\n \"Below_0\": (x < 0).sum(),\n \"Above_1\": (x > 1).sum(),\n \"Unique\": x.nunique(),\n })\n\nrange_df = pd.DataFrame(range_rows)\n\nprint(range_df.to_string(index=False))\n\nrange_df.to_csv(\n f\"{PREFIX}_probability_ranges.csv\",\n index=False\n)\n\n# =============================================================================\n# 10. DIRECT LABEL ↔ PROBABILITY FORENSICS\n# =============================================================================\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"10. DIRECT LABEL ↔ PROBABILITY FORENSICS\")\nprint(\"=\" * 78)\n\ndirect_rows = []\n\nfor target in TARGETS:\n\n pcol = prob_map[target]\n\n g = gold[[target, pcol]].copy()\n\n y = pd.to_numeric(\n g[target],\n errors=\"coerce\"\n ).astype(int)\n\n p = pd.to_numeric(\n g[pcol],\n errors=\"coerce\"\n )\n\n valid = y.notna() & p.notna()\n\n y = y[valid].to_numpy()\n p = p[valid].to_numpy()\n\n auc = roc_auc_score(y, p)\n ap = average_precision_score(y, p)\n\n pos = p[y == 1]\n neg = p[y == 0]\n\n min_pos = float(np.min(pos))\n max_neg = float(np.max(neg))\n\n margin = min_pos - max_neg\n\n direct_rows.append({\n \"Target\": target,\n \"N\": len(y),\n \"Positive\": int((y == 1).sum()),\n \"Negative\": int((y == 0).sum()),\n \"AUC\": auc,\n \"AP\": ap,\n \"Min_Positive\": min_pos,\n \"Max_Negative\": max_neg,\n \"Separation_Margin\": margin,\n \"Perfect_Separation\": margin > 0,\n })\n\ndirect_df = pd.DataFrame(direct_rows)\n\nprint(\n direct_df.to_string(\n index=False,\n float_format=lambda x: f\"{x:.6f}\"\n )\n)\n\ndirect_df.to_csv(\n f\"{PREFIX}_direct_label_probability.csv\",\n index=False\n)\n\n# =============================================================================\n# 11. LABEL CORRELATION\n# =============================================================================\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"11. LABEL ↔ PROBABILITY CORRELATION\")\nprint(\"=\" * 78)\n\ncorr_rows = []\n\nfor target in TARGETS:\n\n x = pd.to_numeric(\n gold[target],\n errors=\"coerce\"\n )\n\n p = pd.to_numeric(\n gold[prob_map[target]],\n errors=\"coerce\"\n )\n\n valid = x.notna() & p.notna()\n\n corr = np.corrcoef(\n x[valid],\n p[valid]\n )[0, 1]\n\n corr_rows.append({\n \"Target\": target,\n \"Correlation\": corr,\n })\n\ncorr_df = pd.DataFrame(corr_rows)\n\nprint(corr_df.to_string(index=False))\n\ncorr_df.to_csv(\n f\"{PREFIX}_label_probability_correlation.csv\",\n index=False\n)\n\n# =============================================================================\n# 12. MUTUAL INFORMATION\n# =============================================================================\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"12. MUTUAL INFORMATION FORENSICS\")\nprint(\"=\" * 78)\n\nmi_rows = []\n\nfor target in TARGETS:\n\n y = gold[target].astype(int).to_numpy()\n\n p = gold[prob_map[target]].astype(float).to_numpy()\n\n try:\n mi = mutual_info_classif(\n p.reshape(-1, 1),\n y,\n random_state=42\n )[0]\n except Exception:\n mi = np.nan\n\n mi_rows.append({\n \"Target\": target,\n \"Mutual_Information\": mi,\n })\n\nmi_df = pd.DataFrame(mi_rows)\n\nprint(mi_df.to_string(index=False))\n\nmi_df.to_csv(\n f\"{PREFIX}_mutual_information.csv\",\n index=False\n)\n\n# =============================================================================\n# 13. TEST WHETHER PROBABILITY IS ALMOST A DETERMINISTIC LABEL TRANSFORM\n# =============================================================================\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"13. PROBABILITY ↔ LABEL DETERMINISM TEST\")\nprint(\"=\" * 78)\n\ndet_rows = []\n\nfor target in TARGETS:\n\n y = gold[target].astype(int)\n p = gold[prob_map[target]].astype(float)\n\n pos = p[y == 1]\n neg = p[y == 0]\n\n pos_mean = pos.mean()\n neg_mean = neg.mean()\n\n pooled_std = np.sqrt(\n (\n ((len(pos) - 1) * pos.var())\n +\n ((len(neg) - 1) * neg.var())\n )\n /\n max(len(pos) + len(neg) - 2, 1)\n )\n\n standardized_gap = (\n (pos_mean - neg_mean) / pooled_std\n if pooled_std > 0\n else np.nan\n )\n\n # A simple midpoint classifier.\n midpoint = (pos_mean + neg_mean) / 2\n\n pred = (p >= midpoint).astype(int)\n\n accuracy = np.mean(pred == y)\n\n det_rows.append({\n \"Target\": target,\n \"Positive_Mean\": pos_mean,\n \"Negative_Mean\": neg_mean,\n \"Mean_Gap\": pos_mean - neg_mean,\n \"Pooled_STD\": pooled_std,\n \"Standardized_Gap\": standardized_gap,\n \"Midpoint\": midpoint,\n \"Midpoint_Accuracy\": accuracy,\n })\n\ndet_df = pd.DataFrame(det_rows)\n\nprint(\n det_df.to_string(\n index=False,\n float_format=lambda x: f\"{x:.6f}\"\n )\n)\n\ndet_df.to_csv(\n f\"{PREFIX}_determinism_test.csv\",\n index=False\n)\n\n# =============================================================================\n# 14. RANK / ORDER FORENSICS\n# =============================================================================\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"14. GOLD PROBABILITY ORDER FORENSICS\")\nprint(\"=\" * 78)\n\nrank_rows = []\n\nfor target in TARGETS:\n\n g = gold[[UID, target, prob_map[target]]].copy()\n\n g[\"prob\"] = pd.to_numeric(\n g[prob_map[target]],\n errors=\"coerce\"\n )\n\n g[\"label\"] = pd.to_numeric(\n g[target],\n errors=\"coerce\"\n )\n\n g = g.dropna(\n subset=[\"prob\", \"label\"]\n )\n\n g[\"rank\"] = g[\"prob\"].rank(\n method=\"average\"\n )\n\n positive_rank_mean = g.loc[\n g[\"label\"] == 1,\n \"rank\"\n ].mean()\n\n negative_rank_mean = g.loc[\n g[\"label\"] == 0,\n \"rank\"\n ].mean()\n\n rank_rows.append({\n \"Target\": target,\n \"Positive_Rank_Mean\": positive_rank_mean,\n \"Negative_Rank_Mean\": negative_rank_mean,\n \"Rank_Gap\": positive_rank_mean - negative_rank_mean,\n })\n\nrank_df = pd.DataFrame(rank_rows)\n\nprint(rank_df.to_string(index=False))\n\nrank_df.to_csv(\n f\"{PREFIX}_rank_forensics.csv\",\n index=False\n)\n\n# =============================================================================\n# 15. CHECK WHETHER GOLD LABELS ARE ENCODED BY UID ORDER\n# =============================================================================\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"15. UID / ROW-ORDER FORENSICS\")\nprint(\"=\" * 78)\n\ngold_order = gold.copy()\n\ngold_order[\"_row_position\"] = np.arange(len(gold_order))\n\nuid_numeric_like = pd.to_numeric(\n gold_order[UID],\n errors=\"coerce\"\n)\n\nprint(\n \"Numeric UID conversion:\",\n uid_numeric_like.notna().sum(),\n \"/\",\n len(gold_order)\n)\n\nuid_rows = []\n\nfor target in TARGETS:\n\n y = gold_order[target].astype(int).to_numpy()\n\n positions = gold_order[\"_row_position\"].to_numpy()\n\n if len(np.unique(y)) == 2:\n\n corr = np.corrcoef(\n positions,\n y\n )[0, 1]\n\n else:\n corr = np.nan\n\n uid_rows.append({\n \"Target\": target,\n \"RowPosition_Label_Correlation\": corr,\n })\n\nuid_df = pd.DataFrame(uid_rows)\n\nprint(uid_df.to_string(index=False))\n\nuid_df.to_csv(\n f\"{PREFIX}_uid_row_order_forensics.csv\",\n index=False\n)\n\n# =============================================================================\n# 16. SEARCH FOR OTHER DATA FILES IN /KAGGLE/WORKING\n# =============================================================================\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"16. WORKING-DIRECTORY DATA INVENTORY\")\nprint(\"=\" * 78)\n\nextensions = {\n \".csv\",\n \".json\",\n \".jsonl\",\n \".parquet\",\n \".pkl\",\n \".pickle\",\n \".npy\",\n \".npz\",\n \".xlsx\",\n \".txt\",\n \".py\",\n \".ipynb\",\n}\n\ninventory = []\n\nfor path in WORK.rglob(\"*\"):\n\n if not path.is_file():\n continue\n\n if path.name.startswith(\"rsna_report_teacher_v3_4\"):\n continue\n\n if path.suffix.lower() not in extensions:\n continue\n\n try:\n size = path.stat().st_size\n modified = pd.Timestamp.fromtimestamp(\n path.stat().st_mtime\n )\n except Exception:\n size = np.nan\n modified = pd.NaT\n\n inventory.append({\n \"Path\": str(path),\n \"Extension\": path.suffix.lower(),\n \"Size\": size,\n \"Modified\": modified,\n })\n\ninventory_df = pd.DataFrame(inventory)\n\nif len(inventory_df):\n print(\n inventory_df.sort_values(\n \"Size\",\n ascending=False\n ).to_string(index=False)\n )\nelse:\n print(\"No additional recognized files found.\")\n\ninventory_df.to_csv(\n f\"{PREFIX}_working_directory_inventory.csv\",\n index=False\n)\n\n# =============================================================================\n# 17. SEARCH CODE / NOTEBOOKS FOR REPORT PROBABILITY GENERATION\n# =============================================================================\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"17. CODE / NOTEBOOK PROVENANCE SEARCH\")\nprint(\"=\" * 78)\n\ncode_files = []\n\nfor path in WORK.rglob(\"*\"):\n\n if not path.is_file():\n continue\n\n if path.suffix.lower() not in {\".py\", \".ipynb\", \".txt\"}:\n continue\n\n code_files.append(path)\n\nsearch_terms = [\n \"rsna_report_probabilities\",\n \"REPORT_ACL\",\n \"REPORT_MCL\",\n \"REPORT_Medial\",\n \"REPORT_Lateral\",\n \"REPORT_PF\",\n \"REPORT_Effusion\",\n \"REPORT_Synovitis\",\n \"REPORT_Baker\",\n \"REPORT_Contusion\",\n \"REPORT_Fracture\",\n \"report_prob\",\n \"report_probability\",\n \"probabilities\",\n \"sigmoid\",\n \"expit\",\n \"logistic\",\n \"train_test_split\",\n \"gold\",\n \"weak\",\n \"weak_label\",\n \"pseudo\",\n \"teacher\",\n]\n\ncode_hits = []\n\nfor path in code_files:\n\n try:\n text = path.read_text(\n encoding=\"utf-8\",\n errors=\"ignore\"\n )\n except Exception:\n continue\n\n lower = text.lower()\n\n hits = []\n\n for term in search_terms:\n if term.lower() in lower:\n hits.append(term)\n\n if hits:\n\n code_hits.append({\n \"File\": str(path),\n \"Matched_Terms\": \"; \".join(sorted(set(hits))),\n \"Number_of_Matches\": len(hits),\n })\n\n print(\"\\nFILE:\", path)\n print(\"MATCHES:\", \", \".join(sorted(set(hits))))\n\nif not code_hits:\n print(\n \"\\n⚠ No probability-generation source code was found \"\n \"inside /kaggle/working.\"\n )\n\ncode_hits_df = pd.DataFrame(code_hits)\n\ncode_hits_df.to_csv(\n f\"{PREFIX}_code_provenance_hits.csv\",\n index=False\n)\n\n# =============================================================================\n# 18. EXTRACT RELEVANT CODE LINES\n# =============================================================================\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"18. RELEVANT CODE-LINE FORENSICS\")\nprint(\"=\" * 78)\n\ncode_line_rows = []\n\nfor path in code_files:\n\n try:\n lines = path.read_text(\n encoding=\"utf-8\",\n errors=\"ignore\"\n ).splitlines()\n except Exception:\n continue\n\n for i, line in enumerate(lines, start=1):\n\n low = line.lower()\n\n if any(term.lower() in low for term in search_terms):\n\n code_line_rows.append({\n \"File\": str(path),\n \"Line\": i,\n \"Code\": line[:1000],\n })\n\ncode_lines_df = pd.DataFrame(code_line_rows)\n\nif len(code_lines_df):\n\n print(\n code_lines_df.head(300).to_string(\n index=False\n )\n )\n\nelse:\n print(\"No relevant code lines detected.\")\n\ncode_lines_df.to_csv(\n f\"{PREFIX}_relevant_code_lines.csv\",\n index=False\n)\n\n# =============================================================================\n# 19. SEARCH FOR LABEL-LIKE COLUMNS IN ALL CSV FILES\n# =============================================================================\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"19. OTHER CSV LABEL-SOURCE FORENSICS\")\nprint(\"=\" * 78)\n\ncsv_files = list(WORK.rglob(\"*.csv\"))\n\ncsv_schema_rows = []\n\nfor path in csv_files:\n\n try:\n tmp = pd.read_csv(\n path,\n nrows=5\n )\n except Exception:\n continue\n\n columns = list(tmp.columns)\n\n label_like = []\n\n for col in columns:\n\n low = str(col).lower()\n\n if any(\n keyword in low\n for keyword in [\n \"label\",\n \"target\",\n \"acl\",\n \"mcl\",\n \"meniscus\",\n \"oa\",\n \"effusion\",\n \"synovitis\",\n \"baker\",\n \"contusion\",\n \"fracture\",\n \"report\",\n \"diagnosis\",\n ]\n ):\n label_like.append(str(col))\n\n csv_schema_rows.append({\n \"File\": str(path),\n \"Columns\": \" | \".join(map(str, columns)),\n \"Label_Like_Columns\":\n \" | \".join(label_like),\n })\n\ncsv_schema_df = pd.DataFrame(csv_schema_rows)\n\nif len(csv_schema_df):\n\n print(\n csv_schema_df.to_string(index=False)\n )\n\ncsv_schema_df.to_csv(\n f\"{PREFIX}_csv_schema_forensics.csv\",\n index=False\n)\n\n# =============================================================================\n# 20. CHECK WHETHER PROBABILITIES ARE COPIES / TRANSFORMS OF TARGETS\n# =============================================================================\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"20. LABEL TRANSFORMATION FORENSICS\")\nprint(\"=\" * 78)\n\ntransform_rows = []\n\nfor target in TARGETS:\n\n y = gold[target].astype(float).to_numpy()\n\n p = gold[prob_map[target]].astype(float).to_numpy()\n\n # Candidate transformations.\n candidates = {\n \"label\": y,\n \"1-label\": 1.0 - y,\n \"0.5+0.25*label\": 0.5 + 0.25 * y,\n \"0.25+0.50*label\": 0.25 + 0.50 * y,\n \"0.3+0.4*label\": 0.3 + 0.4 * y,\n \"0.2+0.5*label\": 0.2 + 0.5 * y,\n \"0.25+0.5*label\": 0.25 + 0.5 * y,\n }\n\n best_name = None\n best_mae = np.inf\n\n for name, pred in candidates.items():\n\n mae = np.mean(\n np.abs(\n p - pred\n )\n )\n\n if mae < best_mae:\n best_mae = mae\n best_name = name\n\n transform_rows.append({\n \"Target\": target,\n \"Best_Simple_Transform\": best_name,\n \"MAE\": best_mae,\n })\n\ntransform_df = pd.DataFrame(transform_rows)\n\nprint(\n transform_df.to_string(\n index=False,\n float_format=lambda x: f\"{x:.6f}\"\n )\n)\n\ntransform_df.to_csv(\n f\"{PREFIX}_label_transform_forensics.csv\",\n index=False\n)\n\n# =============================================================================\n# 21. CHECK MONOTONICITY OF PROBABILITY WITH GOLD LABEL\n# =============================================================================\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"21. MONOTONICITY FORENSICS\")\nprint(\"=\" * 78)\n\nmono_rows = []\n\nfor target in TARGETS:\n\n g = gold[[target, prob_map[target]]].copy()\n\n g[target] = pd.to_numeric(\n g[target],\n errors=\"coerce\"\n )\n\n g[prob_map[target]] = pd.to_numeric(\n g[prob_map[target]],\n errors=\"coerce\"\n )\n\n g = g.dropna()\n\n pos = g.loc[\n g[target] == 1,\n prob_map[target]\n ]\n\n neg = g.loc[\n g[target] == 0,\n prob_map[target]\n ]\n\n # Count all positive-negative pairs.\n pair_total = len(pos) * len(neg)\n\n if pair_total > 0:\n\n correctly_ordered = sum(\n float(a) > float(b)\n for a in pos\n for b in neg\n )\n\n tied = sum(\n float(a) == float(b)\n for a in pos\n for b in neg\n )\n\n ordering_rate = (\n (correctly_ordered + 0.5 * tied)\n / pair_total\n )\n\n else:\n ordering_rate = np.nan\n\n mono_rows.append({\n \"Target\": target,\n \"Positive_N\": len(pos),\n \"Negative_N\": len(neg),\n \"Pair_Count\": pair_total,\n \"Positive_Above_Negative_Rate\": ordering_rate,\n })\n\nmono_df = pd.DataFrame(mono_rows)\n\nprint(mono_df.to_string(index=False))\n\nmono_df.to_csv(\n f\"{PREFIX}_monotonicity_forensics.csv\",\n index=False\n)\n\n# =============================================================================\n# 22. ROW-LEVEL GOLD FORENSICS\n# =============================================================================\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"22. GOLD ROW-LEVEL FORENSIC SNAPSHOT\")\nprint(\"=\" * 78)\n\nsnapshot_cols = [UID]\n\nfor target in TARGETS:\n snapshot_cols.extend([\n target,\n prob_map[target],\n ])\n\ngold_snapshot = gold[snapshot_cols].copy()\n\ngold_snapshot.to_csv(\n f\"{PREFIX}_gold_row_level_snapshot.csv\",\n index=False\n)\n\nprint(\n gold_snapshot.head(10).to_string(\n index=False\n )\n)\n\n# =============================================================================\n# 23. DETECT IDENTICAL PROBABILITY VECTORS\n# =============================================================================\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"23. PROBABILITY VECTOR DUPLICATE FORENSICS\")\nprint(\"=\" * 78)\n\nprob_cols = [\n prob_map[t]\n for t in TARGETS\n]\n\nprob_matrix = df[prob_cols].astype(float)\n\nrounded = prob_matrix.round(8)\n\nduplicate_vector_count = (\n rounded.duplicated(keep=False).sum()\n)\n\nunique_vectors = (\n rounded.drop_duplicates().shape[0]\n)\n\nprint(\n \"Total probability vectors:\",\n len(rounded)\n)\n\nprint(\n \"Unique probability vectors:\",\n unique_vectors\n)\n\nprint(\n \"Rows belonging to duplicated vectors:\",\n duplicate_vector_count\n)\n\n# =============================================================================\n# 24. GOLD-ONLY DUPLICATE VECTOR CHECK\n# =============================================================================\n\ngold_prob = gold[prob_cols].astype(float).round(8)\n\ngold_duplicate_vectors = (\n gold_prob.duplicated(keep=False).sum()\n)\n\nprint(\n \"Gold rows in duplicated probability vectors:\",\n gold_duplicate_vectors\n)\n\n# =============================================================================\n# 25. CHECK WHETHER LABELS CAN BE PREDICTED FROM PROBABILITY ALONE\n# =============================================================================\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"25. PROBABILITY-ONLY LABEL RECONSTRUCTION TEST\")\nprint(\"=\" * 78)\n\nreconstruction_rows = []\n\nfor target in TARGETS:\n\n x = gold[prob_map[target]].astype(float).to_numpy()\n y = gold[target].astype(int).to_numpy()\n\n # Search a grid WITHOUT using the previously saved V3.2 threshold.\n thresholds = np.linspace(\n np.min(x),\n np.max(x),\n 1001\n )\n\n best_acc = -1\n best_th = None\n\n for th in thresholds:\n\n pred = (\n x >= th\n ).astype(int)\n\n acc = np.mean(\n pred == y\n )\n\n if acc > best_acc:\n\n best_acc = acc\n best_th = th\n\n reconstruction_rows.append({\n \"Target\": target,\n \"Best_Reconstruction_Accuracy\": best_acc,\n \"Best_Reconstruction_Threshold\": best_th,\n })\n\nrecon_df = pd.DataFrame(\n reconstruction_rows\n)\n\nprint(\n recon_df.to_string(\n index=False,\n float_format=lambda x: f\"{x:.6f}\"\n )\n)\n\nrecon_df.to_csv(\n f\"{PREFIX}_probability_label_reconstruction.csv\",\n index=False\n)\n\n# =============================================================================\n# 26. CRITICAL TEST:\n# COMPARE GOLD LABELS AGAINST PROBABILITY FILE CREATION ORDER\n# =============================================================================\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"26. CREATION-ORDER / ALIGNMENT FORENSICS\")\nprint(\"=\" * 78)\n\nalignment = pd.DataFrame({\n \"UID_train\": train[UID].astype(str),\n \"UID_prob\": prob[UID].astype(str),\n})\n\nalignment[\"same\"] = (\n alignment[\"UID_train\"]\n ==\n alignment[\"UID_prob\"]\n)\n\nprint(\n \"Exact row-by-row UID alignment:\",\n alignment[\"same\"].all()\n)\n\n# Test whether probability rows have been reordered in a way\n# that would become suspiciously aligned after sorting.\nsorted_train = train.sort_values(UID).reset_index(drop=True)\nsorted_prob = prob.sort_values(UID).reset_index(drop=True)\n\nsorted_same = (\n sorted_train[UID].astype(str).to_numpy()\n ==\n sorted_prob[UID].astype(str).to_numpy()\n).all()\n\nprint(\n \"Same UID set after sorting:\",\n sorted_same\n)\n\n# =============================================================================\n# 27. LOOK FOR GOLD LABEL INFORMATION IN PROBABILITY FILE METADATA\n# =============================================================================\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"27. PROBABILITY FILE SCHEMA / METADATA FORENSICS\")\nprint(\"=\" * 78)\n\nprint(\"\\nProbability columns:\")\n\nfor i, col in enumerate(prob.columns):\n\n print(\n f\"{i:3d}: {repr(col)} \"\n f\"dtype={prob[col].dtype}\"\n )\n\nprint(\"\\nProbability file memory usage:\")\nprint(\n prob.memory_usage(\n deep=True\n ).to_string()\n)\n\n# =============================================================================\n# 28. SUSPICION SCORING\n# =============================================================================\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"28. FORENSIC RISK SCORING\")\nprint(\"=\" * 78)\n\nrisk_rows = []\n\nfor target in TARGETS:\n\n row = direct_df[\n direct_df[\"Target\"] == target\n ].iloc[0]\n\n auc = row[\"AUC\"]\n margin = row[\"Separation_Margin\"]\n\n corr_row = corr_df[\n corr_df[\"Target\"] == target\n ].iloc[0]\n\n corr = corr_row[\"Correlation\"]\n\n recon_row = recon_df[\n recon_df[\"Target\"] == target\n ].iloc[0]\n\n recon_acc = recon_row[\n \"Best_Reconstruction_Accuracy\"\n ]\n\n risk = 0\n reasons = []\n\n if auc >= 0.99:\n risk += 2\n reasons.append(\"AUC>=0.99\")\n\n if abs(corr) >= 0.90:\n risk += 2\n reasons.append(\"|correlation|>=0.90\")\n\n if margin > 0:\n risk += 2\n reasons.append(\"perfect gold separation\")\n\n if recon_acc >= 0.98:\n risk += 2\n reasons.append(\"probability reconstructs gold labels\")\n\n if risk >= 6:\n level = \"HIGH\"\n elif risk >= 3:\n level = \"MODERATE\"\n else:\n level = \"LOW\"\n\n risk_rows.append({\n \"Target\": target,\n \"Risk_Score\": risk,\n \"Risk_Level\": level,\n \"Reasons\": \"; \".join(reasons),\n })\n\nrisk_df = pd.DataFrame(risk_rows)\n\nprint(\n risk_df.to_string(index=False)\n)\n\nrisk_df.to_csv(\n f\"{PREFIX}_risk_score.csv\",\n index=False\n)\n\n# =============================================================================\n# 29. OVERALL DIAGNOSTIC\n# =============================================================================\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"29. OVERALL FORENSIC DIAGNOSTIC\")\nprint(\"=\" * 78)\n\nmean_auc = direct_df[\"AUC\"].mean()\nmean_corr = corr_df[\"Correlation\"].mean()\nperfect_count = int(\n direct_df[\"Perfect_Separation\"].sum()\n)\nhigh_risk_count = int(\n (risk_df[\"Risk_Level\"] == \"HIGH\").sum()\n)\n\nprint(f\"Mean gold AUC : {mean_auc:.6f}\")\nprint(f\"Mean label-probability corr. : {mean_corr:.6f}\")\nprint(f\"Perfectly separated targets : {perfect_count}/12\")\nprint(f\"High-risk targets : {high_risk_count}/12\")\n\nprint(\"\\nInterpretation:\")\n\nif mean_auc >= 0.99 and perfect_count >= 8:\n\n print(\n \"\"\"\n⚠ HIGH FORENSIC SUSPICION\n\nThe REPORT_* probability file contains extremely strong information\nabout the 58 gold labels.\n\nThis does NOT by itself prove data leakage.\n\nHowever, the evidence is strong enough that the probability-generation\npipeline MUST be inspected before using the 0.9999 AUC as model\ngeneralization performance.\n\"\"\"\n )\n\nelif mean_auc >= 0.90:\n\n print(\n \"\"\"\n⚠ MODERATE/HIGH SUSPICION\n\nThe probability file has unusually strong discrimination on the gold\nsubset. The generation mechanism should be traced.\n\"\"\"\n )\n\nelse:\n\n print(\n \"\"\"\n✓ No extreme gold-set probability separation detected.\n\nThe probability file still requires normal provenance verification,\nbut the current forensic evidence does not show extreme separation.\n\"\"\"\n )\n\n# =============================================================================\n# 30. SPECIFIC QUESTIONS THE FORENSIC AUDIT ANSWERS\n# =============================================================================\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"30. FORENSIC QUESTIONS\")\nprint(\"=\" * 78)\n\nquestions = [\n\n (\n \"Q1\",\n \"Are train and probability UIDs aligned?\",\n same_ids\n ),\n\n (\n \"Q2\",\n \"Are there duplicate train UIDs?\",\n train_ids.duplicated().sum() == 0\n ),\n\n (\n \"Q3\",\n \"Are there duplicate probability UIDs?\",\n prob_ids.duplicated().sum() == 0\n ),\n\n (\n \"Q4\",\n \"Are all 58 gold studies represented?\",\n len(gold) == 58\n ),\n\n (\n \"Q5\",\n \"Does the probability file contain NaN values?\",\n prob[prob_cols].isna().sum().sum() == 0\n ),\n\n (\n \"Q6\",\n \"Do all probabilities lie in [0,1]?\",\n (\n (prob[prob_cols] >= 0)\n &\n (prob[prob_cols] <= 1)\n ).all().all()\n ),\n\n (\n \"Q7\",\n \"Are most targets perfectly separated on gold?\",\n perfect_count >= 8\n ),\n\n (\n \"Q8\",\n \"Is the mean AUC >= 0.99?\",\n mean_auc >= 0.99\n ),\n\n (\n \"Q9\",\n \"Is label-probability correlation unusually high?\",\n mean_corr >= 0.90\n ),\n]\n\nfor q, text, result in questions:\n\n symbol = \"✓\" if result else \"⚠\"\n\n print(\n f\"{symbol} {q}: {text} -> {result}\"\n )\n\n# =============================================================================\n# 31. SAVE MASTER FORENSIC REPORT\n# =============================================================================\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"31. SAVING MASTER FORENSIC REPORT\")\nprint(\"=\" * 78)\n\nmaster = {\n \"version\": \"V3.4\",\n \"train_shape\": list(train.shape),\n \"probability_shape\": list(prob.shape),\n \"gold_studies\": int(len(gold)),\n \"mean_auc_gold\": float(mean_auc),\n \"mean_label_probability_correlation\": float(mean_corr),\n \"perfectly_separated_targets\": perfect_count,\n \"high_risk_targets\": high_risk_count,\n \"uid_exact_alignment\": bool(same_ids),\n \"train_duplicate_uids\": int(\n train_ids.duplicated().sum()\n ),\n \"probability_duplicate_uids\": int(\n prob_ids.duplicated().sum()\n ),\n \"missing_probability_rows\": int(\n df[prob_cols].isna().any(axis=1).sum()\n ),\n \"probability_values_in_0_1\": bool(\n (\n (prob[prob_cols] >= 0)\n &\n (prob[prob_cols] <= 1)\n ).all().all()\n ),\n \"diagnostic_level\": (\n \"HIGH SUSPICION\"\n if mean_auc >= 0.99 and perfect_count >= 8\n else\n \"MODERATE/HIGH SUSPICION\"\n if mean_auc >= 0.90\n else\n \"NO EXTREME SEPARATION DETECTED\"\n ),\n \"important_limitation\":\n \"This audit cannot prove leakage without tracing the \"\n \"actual probability-generation code/data lineage.\",\n}\n\nmaster_path = (\n WORK /\n \"rsna_report_teacher_v3_4_master_forensic_report.json\"\n)\n\nwith open(master_path, \"w\", encoding=\"utf-8\") as f:\n\n json.dump(\n master,\n f,\n indent=2\n )\n\nprint(f\"✓ {master_path}\")\n\n# =============================================================================\n# 32. OUTPUT FILE LIST\n# =============================================================================\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"32. V3.4 OUTPUT FILES\")\nprint(\"=\" * 78)\n\ncreated = sorted(\n WORK.glob(\"rsna_report_teacher_v3_4*\")\n)\n\nfor path in created:\n\n if path.is_file():\n\n print(\n f\"✓ {path}\"\n )\n\n# =============================================================================\n# 33. FINAL CONCLUSION\n# =============================================================================\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"RSNA KNEE V3.4 — FORENSIC AUDIT COMPLETE\")\nprint(\"=\" * 78)\n\nprint(\n \"\"\"\nNEXT DECISION:\n\nIf V3.4 again confirms the extreme gold-label separation, DO NOT\ntrain another calibration model yet.\n\nThe next investigation should inspect the exact code that created:\n\n rsna_report_probabilities_v3.csv\n\nIn particular, look for code that:\n\n • reads the original report/target labels\n • merges labels with StudyInstanceUID\n • creates REPORT_* columns\n • converts labels into probabilities\n • uses gold labels during probability generation\n • fits a model on all 4407 studies before evaluation\n • uses the 58 gold studies during teacher construction\n • creates probabilities from the same labels later used for testing\n • performs any post-processing using the target columns\n\nOnly after that provenance investigation can we decide whether the\n0.9999 AUC represents genuine model performance or label leakage /\nevaluation contamination.\n\"\"\"\n)\n\nprint(\"\\n\" + \"=\" * 78)\nprint(\"END OF V3.4\")\nprint(\"=\" * 78)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-08-07T16:46:53.146235Z","iopub.execute_input":"2026-08-07T16:46:53.147092Z","iopub.status.idle":"2026-08-07T16:46:54.116004Z","shell.execute_reply.started":"2026-08-07T16:46:53.147066Z","shell.execute_reply":"2026-08-07T16:46:54.115217Z"},"collapsed":true,"jupyter":{"outputs_hidden":true}},"outputs":[],"execution_count":null},{"id":"b3d328b6-83b1-48bb-b76f-3c7a062ef2e2","cell_type":"markdown","source":"# 6. Locate selected series ONLY\n\nThis is the replacement for the slow full-DICOM indexing used in V2.\n\nWe do **not** walk every DICOM file.\n\nFor each selected series we directly access:\n\n```text\ntrain_series / StudyInstanceUID / SeriesInstanceUID\n```\n\nand then list the DICOM files only inside that selected directory.","metadata":{}},{"id":"a62f5d24-6c86-4705-910d-062928c64120","cell_type":"code","source":"def locate_selected_series(selected_dict, root):\n records = []\n\n missing = 0\n\n for sid, g in tqdm(\n selected_dict.items(),\n desc=f\"Locate selected series in {root.name}\"\n ):\n study_dir = root / str(sid)\n\n for _, r in g.iterrows():\n suid = str(r[\"SeriesInstanceUID\"])\n series_dir = study_dir / suid\n\n if not series_dir.is_dir():\n missing += 1\n continue\n\n # This touches only selected series directories.\n paths = [\n str(p) for p in series_dir.iterdir()\n if p.is_file() and p.suffix.lower() == \".dcm\"\n ]\n\n records.append({\n \"StudyInstanceUID\": str(sid),\n \"SeriesInstanceUID\": suid,\n \"series_dir\": str(series_dir),\n \"paths\": paths,\n \"Fluid_Sensitive\": int(r[\"Fluid_Sensitive\"]),\n \"Fat_Suppression\": int(r[\"Fat_Suppression\"]),\n \"Anatomical_Plane\": str(r[\"Anatomical_Plane\"]),\n \"base_priority\": float(r[\"base_priority\"]),\n \"n_slices\": len(paths)\n })\n\n out = pd.DataFrame(records)\n\n print(\"Selected series located:\", len(out))\n print(\"Missing selected series:\", missing)\n print(\"DICOM files actually touched:\", int(out[\"n_slices\"].sum()))\n\n return out\n\ntrain_selected = locate_selected_series(\n selected_train, TRAIN_ROOT\n)\n\ntest_selected = locate_selected_series(\n selected_test, TEST_ROOT\n)\n\ntrain_selected.to_pickle(\n WORK / \"train_selected_series.pkl\"\n)\ntest_selected.to_pickle(\n WORK / \"test_selected_series.pkl\"\n)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-08-07T15:48:58.230282Z","iopub.execute_input":"2026-08-07T15:48:58.230755Z","iopub.status.idle":"2026-08-07T15:58:01.907881Z","shell.execute_reply.started":"2026-08-07T15:48:58.230724Z","shell.execute_reply":"2026-08-07T15:58:01.905372Z"}},"outputs":[],"execution_count":null},{"id":"db1dd16f-5619-428d-8090-cb936fe76b32","cell_type":"markdown","source":"# 7. Compare filesystem work against the old approach\n\nThis is an important diagnostic.\n\nV2 would enumerate all DICOM files.\n\nV2.1 enumerates only selected series.","metadata":{}},{"id":"c179cb1a-c731-4931-904b-b35f3ed57a7b","cell_type":"code","source":"# Metadata-level estimate of reduction.\n# Exact full-tree count is intentionally NOT computed because\n# computing it would recreate the expensive operation we removed.\n\nprint(\"V2.1 selected train DICOM files:\",\n int(train_selected.n_slices.sum()))\n\nprint(\"V2.1 selected test DICOM files:\",\n int(test_selected.n_slices.sum()))\n\nprint(\n \"\\nThe notebook deliberately does not count all 819k files \"\n \"because doing so would defeat the optimization.\"\n)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-08-07T15:58:10.897737Z","iopub.execute_input":"2026-08-07T15:58:10.898603Z","iopub.status.idle":"2026-08-07T15:58:10.904495Z","shell.execute_reply.started":"2026-08-07T15:58:10.89857Z","shell.execute_reply":"2026-08-07T15:58:10.903645Z"}},"outputs":[],"execution_count":null},{"id":"1a21e23a-d53d-4c11-90a1-36ef58095a0e","cell_type":"markdown","source":"# 8. Fast DICOM decoder\n\nOnly selected series reach this function.\n\nThe decoder:\n- reads required pixel data\n- applies RescaleSlope/Intercept\n- sorts slices\n- percentile-normalizes\n- samples only 3 representative central/uniform positions\n- resizes immediately\n- stores float16\n\nNo full 3D volume is retained.","metadata":{}},{"id":"71d4fc9c-511f-4105-8edf-6de34f06b172","cell_type":"code","source":"def normalize_fast(x):\n x = np.nan_to_num(\n x.astype(np.float32),\n nan=0.0,\n posinf=0.0,\n neginf=0.0\n )\n\n lo, hi = np.percentile(x, [1, 99])\n\n if hi <= lo:\n lo = float(x.min())\n hi = float(x.max())\n\n x = np.clip(x, lo, hi)\n return (x - lo) / (hi - lo + 1e-6)\n\ndef resize_numpy(x, size):\n t = torch.from_numpy(\n x.astype(np.float32)\n )[None, None]\n\n t = F.interpolate(\n t,\n size=(size, size),\n mode=\"bilinear\",\n align_corners=False\n )\n\n return t[0,0].numpy().astype(np.float16)\n\ndef decode_series_job(args):\n sid, suid, paths, out_file = args\n\n if os.path.exists(out_file):\n return out_file\n\n slices = []\n\n for p in paths:\n try:\n ds = pydicom.dcmread(\n p,\n force=True,\n specific_tags=[\n \"PixelData\",\n \"RescaleSlope\",\n \"RescaleIntercept\",\n \"ImagePositionPatient\",\n \"InstanceNumber\"\n ]\n )\n\n arr = ds.pixel_array.astype(np.float32)\n\n slope = float(\n getattr(ds, \"RescaleSlope\", 1.0)\n )\n intercept = float(\n getattr(ds, \"RescaleIntercept\", 0.0)\n )\n\n arr = arr * slope + intercept\n\n if hasattr(ds, \"ImagePositionPatient\"):\n order_value = float(\n ds.ImagePositionPatient[-1]\n )\n else:\n order_value = float(\n getattr(ds, \"InstanceNumber\", len(slices))\n )\n\n slices.append(\n (order_value, arr)\n )\n\n except Exception:\n continue\n\n if not slices:\n np.savez(\n out_file,\n images=np.zeros(\n (N_SLICES,3,IMG_SIZE,IMG_SIZE),\n dtype=np.float16\n )\n )\n return out_file\n\n slices.sort(key=lambda z: z[0])\n\n vol = np.stack(\n [z[1] for z in slices]\n )\n\n vol = normalize_fast(vol)\n\n if len(vol) <= N_SLICES:\n centers = np.arange(len(vol))\n else:\n centers = np.linspace(\n 0,\n len(vol)-1,\n N_SLICES\n ).round().astype(int)\n\n result = []\n\n for c in centers:\n ids = [\n max(0, min(len(vol)-1, c-1)),\n int(c),\n max(0, min(len(vol)-1, c+1))\n ]\n\n channels = [\n resize_numpy(vol[i], IMG_SIZE)\n for i in ids\n ]\n\n rgb = np.stack(\n channels,\n axis=0\n )\n\n # Per-channel standardization.\n mean = rgb.mean(\n axis=(1,2),\n keepdims=True\n )\n std = rgb.std(\n axis=(1,2),\n keepdims=True\n ) + 1e-5\n\n rgb = (rgb - mean) / std\n\n result.append(\n rgb.astype(np.float16)\n )\n\n np.savez(\n out_file,\n images=np.stack(result)\n )\n\n return out_file","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-08-07T15:58:27.247107Z","iopub.execute_input":"2026-08-07T15:58:27.247469Z","iopub.status.idle":"2026-08-07T15:58:27.262691Z","shell.execute_reply.started":"2026-08-07T15:58:27.247445Z","shell.execute_reply":"2026-08-07T15:58:27.261884Z"}},"outputs":[],"execution_count":null},{"id":"9dea57d0-32a0-4017-8be5-6ce1aec6fd00","cell_type":"code","source":"# ============================================================\n# RSNA KNEE V3.0 — VALIDATED REPORT TEACHER\n# ============================================================\n# Purpose:\n# 1. Use all 4,407 reports\n# 2. Learn report -> 12 abnormality probabilities\n# 3. Validate ONLY on the 58 completely labelled studies\n# 4. Avoid naive keyword pseudo-labeling\n# 5. Produce probabilities for the 4,349 weakly-labelled studies\n#\n# NO DICOM DECODING IN THIS CELL\n# ============================================================\n\nimport os\nimport re\nimport gc\nimport numpy as np\nimport pandas as pd\n\nfrom sklearn.feature_extraction.text import TfidfVectorizer\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.pipeline import Pipeline\nfrom sklearn.model_selection import StratifiedKFold, cross_val_predict\nfrom sklearn.metrics import (\n roc_auc_score,\n average_precision_score,\n f1_score,\n accuracy_score\n)\n\nROOT = \"/kaggle/input/competitions/rsna-knee-abnormality-detection\"\n\nTRAIN_CSV = os.path.join(ROOT, \"train.csv\")\n\nTARGETS = [\n \"ACL\",\n \"MCL\",\n \"Medial Meniscus\",\n \"Lateral Meniscus\",\n \"Medial OA\",\n \"Lateral OA\",\n \"PF OA\",\n \"Effusion\",\n \"Synovitis\",\n \"Baker's\",\n \"Contusion\",\n \"Fracture\"\n]\n\nprint(\"=\" * 70)\nprint(\"RSNA KNEE V3.0 — VALIDATED REPORT TEACHER\")\nprint(\"=\" * 70)\n\ndf = pd.read_csv(TRAIN_CSV)\n\nprint(\"Train shape:\", df.shape)\n\n# ------------------------------------------------------------\n# Identify fully labelled gold studies\n# ------------------------------------------------------------\n\ngold_mask = df[TARGETS].notna().all(axis=1)\n\ngold = df.loc[gold_mask].copy()\nweak = df.loc[~gold_mask].copy()\n\nprint(\"Gold studies:\", len(gold))\nprint(\"Weak studies:\", len(weak))\n\nassert len(gold) == 58, (\n f\"Expected 58 fully labelled studies, found {len(gold)}\"\n)\n\n# ------------------------------------------------------------\n# Text normalization\n# ------------------------------------------------------------\n\ndef normalize_report(text):\n\n if pd.isna(text):\n return \"\"\n\n text = str(text)\n\n # Lowercase\n text = text.lower()\n\n # Normalize common punctuation\n text = text.replace(\"\\n\", \" \")\n text = text.replace(\"\\r\", \" \")\n text = text.replace(\"\\t\", \" \")\n\n # Preserve medically useful characters\n text = re.sub(r\"\\s+\", \" \", text)\n\n return text.strip()\n\n\ndf[\"Report_clean\"] = df[\"Report\"].map(normalize_report)\n\ngold[\"Report_clean\"] = gold[\"Report\"].map(normalize_report)\nweak[\"Report_clean\"] = weak[\"Report\"].map(normalize_report)\n\n# ------------------------------------------------------------\n# IMPORTANT:\n# We use word + character ngrams.\n#\n# Character ngrams help with:\n# multilingual spelling\n# medical abbreviations\n# ACL/LCA/LCA-related variants\n# morphological variants\n# ------------------------------------------------------------\n\nword_vectorizer = TfidfVectorizer(\n analyzer=\"word\",\n ngram_range=(1, 3),\n min_df=1,\n max_df=0.98,\n sublinear_tf=True,\n max_features=120000\n)\n\nchar_vectorizer = TfidfVectorizer(\n analyzer=\"char\",\n ngram_range=(3, 5),\n min_df=1,\n max_features=150000,\n sublinear_tf=True\n)\n\n# ------------------------------------------------------------\n# We cannot blindly use one classifier because some targets\n# are rare.\n#\n# Therefore use balanced logistic regression.\n# ------------------------------------------------------------\n\nresults = []\n\ngold_text = gold[\"Report_clean\"].values\n\nprint(\"\\nTraining report models...\")\nprint(\"-\" * 70)\n\nfor target in TARGETS:\n\n y = gold[target].astype(int).values\n\n positives = int(y.sum())\n negatives = int(len(y) - positives)\n\n print(\n f\"{target:20s} \"\n f\"positive={positives:2d} \"\n f\"negative={negatives:2d}\"\n )\n\n # --------------------------------------------------------\n # Word model\n # --------------------------------------------------------\n\n word_pipe = Pipeline([\n (\n \"tfidf\",\n TfidfVectorizer(\n analyzer=\"word\",\n ngram_range=(1, 3),\n min_df=1,\n sublinear_tf=True,\n max_features=100000\n )\n ),\n (\n \"clf\",\n LogisticRegression(\n C=2.0,\n class_weight=\"balanced\",\n max_iter=2000,\n solver=\"liblinear\"\n )\n )\n ])\n\n # --------------------------------------------------------\n # Character model\n # --------------------------------------------------------\n\n char_pipe = Pipeline([\n (\n \"tfidf\",\n TfidfVectorizer(\n analyzer=\"char\",\n ngram_range=(3, 5),\n min_df=1,\n sublinear_tf=True,\n max_features=120000\n )\n ),\n (\n \"clf\",\n LogisticRegression(\n C=2.0,\n class_weight=\"balanced\",\n max_iter=2000,\n solver=\"liblinear\"\n )\n )\n ])\n\n # --------------------------------------------------------\n # Leave-one-out style prediction.\n #\n # 58 samples are extremely small.\n # We therefore use stratified CV where possible.\n # --------------------------------------------------------\n\n min_class = min(positives, negatives)\n\n if min_class >= 3:\n\n n_splits = min(5, min_class)\n\n cv = StratifiedKFold(\n n_splits=n_splits,\n shuffle=True,\n random_state=42\n )\n\n try:\n\n word_prob = cross_val_predict(\n word_pipe,\n gold_text,\n y,\n cv=cv,\n method=\"predict_proba\",\n n_jobs=-1\n )[:, 1]\n\n char_prob = cross_val_predict(\n char_pipe,\n gold_text,\n y,\n cv=cv,\n method=\"predict_proba\",\n n_jobs=-1\n )[:, 1]\n\n # Ensemble\n prob = 0.60 * word_prob + 0.40 * char_prob\n\n auc = roc_auc_score(y, prob)\n ap = average_precision_score(y, prob)\n\n pred = (prob >= 0.5).astype(int)\n\n f1 = f1_score(\n y,\n pred,\n zero_division=0\n )\n\n acc = accuracy_score(y, pred)\n\n print(\n f\" AUC={auc:.4f} \"\n f\"AP={ap:.4f} \"\n f\"F1={f1:.4f} \"\n f\"ACC={acc:.4f}\"\n )\n\n results.append({\n \"Target\": target,\n \"Positive\": positives,\n \"AUC\": auc,\n \"AP\": ap,\n \"F1\": f1,\n \"Accuracy\": acc\n })\n\n except Exception as e:\n\n print(\" CV failed:\", repr(e))\n\n results.append({\n \"Target\": target,\n \"Positive\": positives,\n \"AUC\": np.nan,\n \"AP\": np.nan,\n \"F1\": np.nan,\n \"Accuracy\": np.nan\n })\n\n else:\n\n print(\" Too few samples for reliable CV\")\n\n results.append({\n \"Target\": target,\n \"Positive\": positives,\n \"AUC\": np.nan,\n \"AP\": np.nan,\n \"F1\": np.nan,\n \"Accuracy\": np.nan\n })\n\n del word_pipe\n del char_pipe\n\n gc.collect()\n\n\n# ------------------------------------------------------------\n# Results\n# ------------------------------------------------------------\n\nresults_df = pd.DataFrame(results)\n\nprint(\"\\n\")\nprint(\"=\" * 70)\nprint(\"REPORT TEACHER VALIDATION\")\nprint(\"=\" * 70)\n\ndisplay(\n results_df.style.format({\n \"AUC\": \"{:.4f}\",\n \"AP\": \"{:.4f}\",\n \"F1\": \"{:.4f}\",\n \"Accuracy\": \"{:.4f}\"\n })\n)\n\nprint(\"\\nMean AUC:\",\n np.nanmean(results_df[\"AUC\"]))\n\nprint(\"Mean AP:\",\n np.nanmean(results_df[\"AP\"]))\n\n# ------------------------------------------------------------\n# TRAIN FINAL REPORT MODELS ON ALL 58 GOLD STUDIES\n# ------------------------------------------------------------\n\nprint(\"\\n\")\nprint(\"=\" * 70)\nprint(\"TRAINING FINAL REPORT TEACHERS\")\nprint(\"=\" * 70)\n\nreport_models = {}\n\nfor target in TARGETS:\n\n y = gold[target].astype(int).values\n\n word_pipe = Pipeline([\n (\n \"tfidf\",\n TfidfVectorizer(\n analyzer=\"word\",\n ngram_range=(1, 3),\n min_df=1,\n sublinear_tf=True,\n max_features=120000\n )\n ),\n (\n \"clf\",\n LogisticRegression(\n C=2.0,\n class_weight=\"balanced\",\n max_iter=2500,\n solver=\"liblinear\"\n )\n )\n ])\n\n char_pipe = Pipeline([\n (\n \"tfidf\",\n TfidfVectorizer(\n analyzer=\"char\",\n ngram_range=(3, 5),\n min_df=1,\n sublinear_tf=True,\n max_features=150000\n )\n ),\n (\n \"clf\",\n LogisticRegression(\n C=2.0,\n class_weight=\"balanced\",\n max_iter=2500,\n solver=\"liblinear\"\n )\n )\n ])\n\n word_pipe.fit(gold_text, y)\n char_pipe.fit(gold_text, y)\n\n report_models[target] = {\n \"word\": word_pipe,\n \"char\": char_pipe\n }\n\n print(\"✓\", target)\n\n\n# ------------------------------------------------------------\n# Predict ALL 4,407 reports\n# ------------------------------------------------------------\n\nprint(\"\\nGenerating report probabilities...\")\n\nall_text = df[\"Report_clean\"].values\n\nreport_probabilities = np.zeros(\n (len(df), len(TARGETS)),\n dtype=np.float32\n)\n\nfor j, target in enumerate(TARGETS):\n\n wm = report_models[target][\"word\"]\n cm = report_models[target][\"char\"]\n\n p_word = wm.predict_proba(all_text)[:, 1]\n p_char = cm.predict_proba(all_text)[:, 1]\n\n report_probabilities[:, j] = (\n 0.60 * p_word +\n 0.40 * p_char\n )\n\n# ------------------------------------------------------------\n# Save report probabilities\n# ------------------------------------------------------------\n\nprob_cols = [\n f\"REPORT_{t}\"\n for t in TARGETS\n]\n\nreport_df = pd.DataFrame(\n report_probabilities,\n columns=prob_cols\n)\n\nreport_df.insert(\n 0,\n \"StudyInstanceUID\",\n df[\"StudyInstanceUID\"].values\n)\n\nREPORT_CACHE = \"/kaggle/working/rsna_report_probabilities_v3.csv\"\n\nreport_df.to_csv(\n REPORT_CACHE,\n index=False\n)\n\nprint(\"\\nSaved:\")\nprint(REPORT_CACHE)\n\n# ------------------------------------------------------------\n# Confidence analysis\n# ------------------------------------------------------------\n\nprint(\"\\n\")\nprint(\"=\" * 70)\nprint(\"REPORT TEACHER CONFIDENCE\")\nprint(\"=\" * 70)\n\nfor target in TARGETS:\n\n p = report_df[f\"REPORT_{target}\"]\n\n high_pos = int((p >= 0.90).sum())\n high_neg = int((p <= 0.10).sum())\n uncertain = int(\n ((p > 0.10) & (p < 0.90)).sum()\n )\n\n print(\n f\"{target:20s} \"\n f\"POS>=0.90: {high_pos:4d} | \"\n f\"NEG<=0.10: {high_neg:4d} | \"\n f\"UNCERTAIN: {uncertain:4d}\"\n )\n\nprint(\"\\n\")\nprint(\"=\" * 70)\nprint(\"V3 REPORT TEACHER COMPLETE\")\nprint(\"=\" * 70)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-08-07T16:20:39.713642Z","iopub.execute_input":"2026-08-07T16:20:39.714354Z","iopub.status.idle":"2026-08-07T16:22:55.663051Z","shell.execute_reply.started":"2026-08-07T16:20:39.714309Z","shell.execute_reply":"2026-08-07T16:22:55.661985Z"}},"outputs":[],"execution_count":null},{"id":"fdd6f04b-4aa6-406c-9e21-2cbace44c144","cell_type":"code","source":"# ================================================================\n# V3.1.1 — INSPECT ACTUAL PROBABILITY FILE\n# ================================================================\n\nimport pandas as pd\n\nPROB_CSV = \"/kaggle/working/rsna_report_probabilities_v3.csv\"\n\nprobs = pd.read_csv(PROB_CSV)\n\nprint(\"=\" * 75)\nprint(\"RSNA KNEE — PROBABILITY FILE FORENSICS\")\nprint(\"=\" * 75)\n\nprint(\"\\nShape:\")\nprint(probs.shape)\n\nprint(\"\\nActual columns:\")\nfor i, col in enumerate(probs.columns):\n print(f\"{i:2d} : {repr(col)}\")\n\nprint(\"\\nFirst 5 rows:\")\ndisplay(probs.head())\n\nprint(\"\\nData types:\")\ndisplay(probs.dtypes)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-08-07T16:26:54.711626Z","iopub.execute_input":"2026-08-07T16:26:54.712201Z","iopub.status.idle":"2026-08-07T16:26:54.751742Z","shell.execute_reply.started":"2026-08-07T16:26:54.712121Z","shell.execute_reply":"2026-08-07T16:26:54.750907Z"}},"outputs":[],"execution_count":null},{"id":"a534e74c-5496-4c9d-82af-ba5f7cf25daa","cell_type":"code","source":"# ================================================================\n# RSNA KNEE V3.1.2 — REPORT TEACHER CALIBRATION\n# EXACT MATCH TO YOUR ACTUAL FILE STRUCTURE\n# ================================================================\n\nimport os\nimport numpy as np\nimport pandas as pd\n\nfrom sklearn.metrics import (\n roc_auc_score,\n average_precision_score,\n f1_score,\n accuracy_score,\n precision_score,\n recall_score\n)\n\nprint(\"=\" * 75)\nprint(\"RSNA KNEE V3.1.2 — REPORT TEACHER CALIBRATION\")\nprint(\"=\" * 75)\n\n# ================================================================\n# 1. PATHS\n# ================================================================\n\nROOT = \"/kaggle/input/competitions/rsna-knee-abnormality-detection\"\n\nTRAIN_CSV = os.path.join(ROOT, \"train.csv\")\n\nPROB_CSV = (\n \"/kaggle/working/\"\n \"rsna_report_probabilities_v3.csv\"\n)\n\n# ================================================================\n# 2. TARGETS\n# ================================================================\n\nTARGETS = [\n \"ACL\",\n \"MCL\",\n \"Medial Meniscus\",\n \"Lateral Meniscus\",\n \"Medial OA\",\n \"Lateral OA\",\n \"PF OA\",\n \"Effusion\",\n \"Synovitis\",\n \"Baker's\",\n \"Contusion\",\n \"Fracture\"\n]\n\n# ================================================================\n# 3. LOAD\n# ================================================================\n\ntrain = pd.read_csv(TRAIN_CSV)\nprobs = pd.read_csv(PROB_CSV)\n\nprint(\"\\nTrain:\", train.shape)\nprint(\"Probabilities:\", probs.shape)\n\n# ================================================================\n# 4. VERIFY TRAIN LABELS\n# ================================================================\n\nmissing_train_targets = [\n t for t in TARGETS\n if t not in train.columns\n]\n\nif missing_train_targets:\n raise ValueError(\n f\"Missing target columns: {missing_train_targets}\"\n )\n\nprint(\"\\n✓ All 12 target columns found.\")\n\n# ================================================================\n# 5. VERIFY REPORT PROBABILITY COLUMNS\n# ================================================================\n\nREPORT_COLS = {\n target: f\"REPORT_{target}\"\n for target in TARGETS\n}\n\nprint(\"\\nProbability columns:\")\n\nfor target in TARGETS:\n\n col = REPORT_COLS[target]\n\n if col not in probs.columns:\n raise ValueError(\n f\"Missing probability column: {col}\"\n )\n\n print(f\"{target:20s} -> {col}\")\n\nprint(\"\\n✓ All 12 REPORT_* probability columns found.\")\n\n# ================================================================\n# 6. ALIGN USING StudyInstanceUID\n# ================================================================\n\nif \"StudyInstanceUID\" not in train.columns:\n raise ValueError(\n \"StudyInstanceUID missing from train.csv\"\n )\n\nif \"StudyInstanceUID\" not in probs.columns:\n raise ValueError(\n \"StudyInstanceUID missing from probability file\"\n )\n\n# Only keep required columns\ntrain_small = train[\n [\"StudyInstanceUID\"] + TARGETS\n].copy()\n\nprob_small = probs[\n [\"StudyInstanceUID\"] +\n [REPORT_COLS[t] for t in TARGETS]\n].copy()\n\n# Merge\ndf = train_small.merge(\n prob_small,\n on=\"StudyInstanceUID\",\n how=\"inner\"\n)\n\nprint(\"\\nMerged:\", df.shape)\n\n# ================================================================\n# 7. VERIFY GOLD-LABELLED STUDIES\n# ================================================================\n\n# A study is gold-labelled when ALL 12 labels exist\ngold_mask = df[TARGETS].notna().all(axis=1)\n\ngold = df.loc[gold_mask].copy()\n\nprint(\"\\nGold-labelled studies:\", len(gold))\nprint(\n \"Expected from audit: 58\"\n)\n\nif len(gold) != 58:\n print(\n \"WARNING: number of gold studies differs from previous audit.\"\n )\n\n# ================================================================\n# 8. CHECK GOLD LABEL DISTRIBUTION\n# ================================================================\n\nprint(\"\\n\" + \"=\" * 75)\nprint(\"GOLD LABEL DISTRIBUTION\")\nprint(\"=\" * 75)\n\nfor target in TARGETS:\n\n y = pd.to_numeric(\n gold[target],\n errors=\"coerce\"\n )\n\n print(\n f\"{target:20s} \"\n f\"positive={int(y.sum()):2d} \"\n f\"negative={int((1-y).sum()):2d}\"\n )\n\n# ================================================================\n# 9. OPTIMAL THRESHOLD SEARCH\n# ================================================================\n\nprint(\"\\n\" + \"=\" * 75)\nprint(\"OPTIMAL THRESHOLD SEARCH\")\nprint(\"=\" * 75)\n\nTHRESHOLDS = np.arange(\n 0.05,\n 0.951,\n 0.01\n)\n\nresults = []\n\nfor target in TARGETS:\n\n # ------------------------------------------------------------\n # Ground truth\n # ------------------------------------------------------------\n\n y = pd.to_numeric(\n gold[target],\n errors=\"coerce\"\n ).values\n\n # ------------------------------------------------------------\n # Report teacher probability\n # ------------------------------------------------------------\n\n p = pd.to_numeric(\n gold[REPORT_COLS[target]],\n errors=\"coerce\"\n ).values\n\n # ------------------------------------------------------------\n # Remove invalid values\n # ------------------------------------------------------------\n\n valid = (\n np.isfinite(y) &\n np.isfinite(p)\n )\n\n y = y[valid].astype(int)\n p = p[valid].astype(float)\n\n if len(y) == 0:\n print(\n f\"{target:20s} NO VALID DATA\"\n )\n continue\n\n if len(np.unique(y)) < 2:\n print(\n f\"{target:20s} ONLY ONE CLASS\"\n )\n continue\n\n # ------------------------------------------------------------\n # AUC\n # ------------------------------------------------------------\n\n auc = roc_auc_score(\n y,\n p\n )\n\n # ------------------------------------------------------------\n # Average Precision\n # ------------------------------------------------------------\n\n ap = average_precision_score(\n y,\n p\n )\n\n # ------------------------------------------------------------\n # Find best F1 threshold\n # ------------------------------------------------------------\n\n best = None\n\n for threshold in THRESHOLDS:\n\n pred = (\n p >= threshold\n ).astype(int)\n\n f1 = f1_score(\n y,\n pred,\n zero_division=0\n )\n\n acc = accuracy_score(\n y,\n pred\n )\n\n precision = precision_score(\n y,\n pred,\n zero_division=0\n )\n\n recall = recall_score(\n y,\n pred,\n zero_division=0\n )\n\n candidate = {\n \"threshold\": threshold,\n \"f1\": f1,\n \"accuracy\": acc,\n \"precision\": precision,\n \"recall\": recall\n }\n\n if (\n best is None or\n candidate[\"f1\"] > best[\"f1\"]\n ):\n best = candidate\n\n # ------------------------------------------------------------\n # Store\n # ------------------------------------------------------------\n\n results.append({\n\n \"Target\": target,\n\n \"Positive\": int(y.sum()),\n\n \"Negative\": int(len(y) - y.sum()),\n\n \"AUC\": auc,\n\n \"AP\": ap,\n\n \"Optimal_Threshold\": best[\"threshold\"],\n\n \"F1\": best[\"f1\"],\n\n \"Accuracy\": best[\"accuracy\"],\n\n \"Precision\": best[\"precision\"],\n\n \"Recall\": best[\"recall\"]\n\n })\n\n print(\n f\"{target:20s} \"\n f\"AUC={auc:.4f} \"\n f\"AP={ap:.4f} \"\n f\"TH={best['threshold']:.2f} \"\n f\"F1={best['f1']:.4f} \"\n f\"ACC={best['accuracy']:.4f}\"\n )\n\n# ================================================================\n# 10. RESULTS TABLE\n# ================================================================\n\nresults_df = pd.DataFrame(\n results\n)\n\nprint(\"\\n\" + \"=\" * 75)\nprint(\"CALIBRATION RESULTS\")\nprint(\"=\" * 75)\n\ndisplay(results_df)\n\n# ================================================================\n# 11. MEAN METRICS\n# ================================================================\n\nif len(results_df) > 0:\n\n print(\"\\n\" + \"=\" * 75)\n print(\"MEAN PERFORMANCE\")\n print(\"=\" * 75)\n\n print(\n f\"Mean AUC : \"\n f\"{results_df['AUC'].mean():.4f}\"\n )\n\n print(\n f\"Mean AP : \"\n f\"{results_df['AP'].mean():.4f}\"\n )\n\n print(\n f\"Mean F1 : \"\n f\"{results_df['F1'].mean():.4f}\"\n )\n\n print(\n f\"Mean Accuracy : \"\n f\"{results_df['Accuracy'].mean():.4f}\"\n )\n\n# ================================================================\n# 12. SAVE THRESHOLDS\n# ================================================================\n\nTHRESHOLD_FILE = (\n \"/kaggle/working/\"\n \"rsna_report_teacher_thresholds_v3_1.csv\"\n)\n\nresults_df.to_csv(\n THRESHOLD_FILE,\n index=False\n)\n\nprint(\n \"\\n✓ Saved:\"\n)\nprint(THRESHOLD_FILE)\n\n# ================================================================\n# 13. CREATE SIMPLE THRESHOLD DICTIONARY\n# ================================================================\n\noptimal_thresholds = dict(\n zip(\n results_df[\"Target\"],\n results_df[\"Optimal_Threshold\"]\n )\n)\n\nprint(\"\\nOptimal threshold dictionary:\")\n\nfor target, threshold in optimal_thresholds.items():\n\n print(\n f\" {target!r}: {threshold:.2f},\"\n )\n\n# ================================================================\n# 14. END\n# ================================================================\n\nprint(\"\\n\" + \"=\" * 75)\nprint(\"V3.1.2 CALIBRATION COMPLETE\")\nprint(\"=\" * 75)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-08-07T16:28:05.278114Z","iopub.execute_input":"2026-08-07T16:28:05.278681Z","iopub.status.idle":"2026-08-07T16:28:11.932225Z","shell.execute_reply.started":"2026-08-07T16:28:05.278635Z","shell.execute_reply":"2026-08-07T16:28:11.931531Z"}},"outputs":[],"execution_count":null},{"id":"8950c551-3f2f-4fc8-be17-e18af276111a","cell_type":"code","source":"# ================================================================\n# RSNA KNEE V3.1.2 — REPORT TEACHER CALIBRATION\n# EXACT MATCH TO YOUR ACTUAL FILE STRUCTURE\n# ================================================================\n\nimport os\nimport numpy as np\nimport pandas as pd\n\nfrom sklearn.metrics import (\n roc_auc_score,\n average_precision_score,\n f1_score,\n accuracy_score,\n precision_score,\n recall_score\n)\n\nprint(\"=\" * 75)\nprint(\"RSNA KNEE V3.1.2 — REPORT TEACHER CALIBRATION\")\nprint(\"=\" * 75)\n\n# ================================================================\n# 1. PATHS\n# ================================================================\n\nROOT = \"/kaggle/input/competitions/rsna-knee-abnormality-detection\"\n\nTRAIN_CSV = os.path.join(ROOT, \"train.csv\")\n\nPROB_CSV = (\n \"/kaggle/working/\"\n \"rsna_report_probabilities_v3.csv\"\n)\n\n# ================================================================\n# 2. TARGETS\n# ================================================================\n\nTARGETS = [\n \"ACL\",\n \"MCL\",\n \"Medial Meniscus\",\n \"Lateral Meniscus\",\n \"Medial OA\",\n \"Lateral OA\",\n \"PF OA\",\n \"Effusion\",\n \"Synovitis\",\n \"Baker's\",\n \"Contusion\",\n \"Fracture\"\n]\n\n# ================================================================\n# 3. LOAD\n# ================================================================\n\ntrain = pd.read_csv(TRAIN_CSV)\nprobs = pd.read_csv(PROB_CSV)\n\nprint(\"\\nTrain:\", train.shape)\nprint(\"Probabilities:\", probs.shape)\n\n# ================================================================\n# 4. VERIFY TRAIN LABELS\n# ================================================================\n\nmissing_train_targets = [\n t for t in TARGETS\n if t not in train.columns\n]\n\nif missing_train_targets:\n raise ValueError(\n f\"Missing target columns: {missing_train_targets}\"\n )\n\nprint(\"\\n✓ All 12 target columns found.\")\n\n# ================================================================\n# 5. VERIFY REPORT PROBABILITY COLUMNS\n# ================================================================\n\nREPORT_COLS = {\n target: f\"REPORT_{target}\"\n for target in TARGETS\n}\n\nprint(\"\\nProbability columns:\")\n\nfor target in TARGETS:\n\n col = REPORT_COLS[target]\n\n if col not in probs.columns:\n raise ValueError(\n f\"Missing probability column: {col}\"\n )\n\n print(f\"{target:20s} -> {col}\")\n\nprint(\"\\n✓ All 12 REPORT_* probability columns found.\")\n\n# ================================================================\n# 6. ALIGN USING StudyInstanceUID\n# ================================================================\n\nif \"StudyInstanceUID\" not in train.columns:\n raise ValueError(\n \"StudyInstanceUID missing from train.csv\"\n )\n\nif \"StudyInstanceUID\" not in probs.columns:\n raise ValueError(\n \"StudyInstanceUID missing from probability file\"\n )\n\n# Only keep required columns\ntrain_small = train[\n [\"StudyInstanceUID\"] + TARGETS\n].copy()\n\nprob_small = probs[\n [\"StudyInstanceUID\"] +\n [REPORT_COLS[t] for t in TARGETS]\n].copy()\n\n# Merge\ndf = train_small.merge(\n prob_small,\n on=\"StudyInstanceUID\",\n how=\"inner\"\n)\n\nprint(\"\\nMerged:\", df.shape)\n\n# ================================================================\n# 7. VERIFY GOLD-LABELLED STUDIES\n# ================================================================\n\n# A study is gold-labelled when ALL 12 labels exist\ngold_mask = df[TARGETS].notna().all(axis=1)\n\ngold = df.loc[gold_mask].copy()\n\nprint(\"\\nGold-labelled studies:\", len(gold))\nprint(\n \"Expected from audit: 58\"\n)\n\nif len(gold) != 58:\n print(\n \"WARNING: number of gold studies differs from previous audit.\"\n )\n\n# ================================================================\n# 8. CHECK GOLD LABEL DISTRIBUTION\n# ================================================================\n\nprint(\"\\n\" + \"=\" * 75)\nprint(\"GOLD LABEL DISTRIBUTION\")\nprint(\"=\" * 75)\n\nfor target in TARGETS:\n\n y = pd.to_numeric(\n gold[target],\n errors=\"coerce\"\n )\n\n print(\n f\"{target:20s} \"\n f\"positive={int(y.sum()):2d} \"\n f\"negative={int((1-y).sum()):2d}\"\n )\n\n# ================================================================\n# 9. OPTIMAL THRESHOLD SEARCH\n# ================================================================\n\nprint(\"\\n\" + \"=\" * 75)\nprint(\"OPTIMAL THRESHOLD SEARCH\")\nprint(\"=\" * 75)\n\nTHRESHOLDS = np.arange(\n 0.05,\n 0.951,\n 0.01\n)\n\nresults = []\n\nfor target in TARGETS:\n\n # ------------------------------------------------------------\n # Ground truth\n # ------------------------------------------------------------\n\n y = pd.to_numeric(\n gold[target],\n errors=\"coerce\"\n ).values\n\n # ------------------------------------------------------------\n # Report teacher probability\n # ------------------------------------------------------------\n\n p = pd.to_numeric(\n gold[REPORT_COLS[target]],\n errors=\"coerce\"\n ).values\n\n # ------------------------------------------------------------\n # Remove invalid values\n # ------------------------------------------------------------\n\n valid = (\n np.isfinite(y) &\n np.isfinite(p)\n )\n\n y = y[valid].astype(int)\n p = p[valid].astype(float)\n\n if len(y) == 0:\n print(\n f\"{target:20s} NO VALID DATA\"\n )\n continue\n\n if len(np.unique(y)) < 2:\n print(\n f\"{target:20s} ONLY ONE CLASS\"\n )\n continue\n\n # ------------------------------------------------------------\n # AUC\n # ------------------------------------------------------------\n\n auc = roc_auc_score(\n y,\n p\n )\n\n # ------------------------------------------------------------\n # Average Precision\n # ------------------------------------------------------------\n\n ap = average_precision_score(\n y,\n p\n )\n\n # ------------------------------------------------------------\n # Find best F1 threshold\n # ------------------------------------------------------------\n\n best = None\n\n for threshold in THRESHOLDS:\n\n pred = (\n p >= threshold\n ).astype(int)\n\n f1 = f1_score(\n y,\n pred,\n zero_division=0\n )\n\n acc = accuracy_score(\n y,\n pred\n )\n\n precision = precision_score(\n y,\n pred,\n zero_division=0\n )\n\n recall = recall_score(\n y,\n pred,\n zero_division=0\n )\n\n candidate = {\n \"threshold\": threshold,\n \"f1\": f1,\n \"accuracy\": acc,\n \"precision\": precision,\n \"recall\": recall\n }\n\n if (\n best is None or\n candidate[\"f1\"] > best[\"f1\"]\n ):\n best = candidate\n\n # ------------------------------------------------------------\n # Store\n # ------------------------------------------------------------\n\n results.append({\n\n \"Target\": target,\n\n \"Positive\": int(y.sum()),\n\n \"Negative\": int(len(y) - y.sum()),\n\n \"AUC\": auc,\n\n \"AP\": ap,\n\n \"Optimal_Threshold\": best[\"threshold\"],\n\n \"F1\": best[\"f1\"],\n\n \"Accuracy\": best[\"accuracy\"],\n\n \"Precision\": best[\"precision\"],\n\n \"Recall\": best[\"recall\"]\n\n })\n\n print(\n f\"{target:20s} \"\n f\"AUC={auc:.4f} \"\n f\"AP={ap:.4f} \"\n f\"TH={best['threshold']:.2f} \"\n f\"F1={best['f1']:.4f} \"\n f\"ACC={best['accuracy']:.4f}\"\n )\n\n# ================================================================\n# 10. RESULTS TABLE\n# ================================================================\n\nresults_df = pd.DataFrame(\n results\n)\n\nprint(\"\\n\" + \"=\" * 75)\nprint(\"CALIBRATION RESULTS\")\nprint(\"=\" * 75)\n\ndisplay(results_df)\n\n# ================================================================\n# 11. MEAN METRICS\n# ================================================================\n\nif len(results_df) > 0:\n\n print(\"\\n\" + \"=\" * 75)\n print(\"MEAN PERFORMANCE\")\n print(\"=\" * 75)\n\n print(\n f\"Mean AUC : \"\n f\"{results_df['AUC'].mean():.4f}\"\n )\n\n print(\n f\"Mean AP : \"\n f\"{results_df['AP'].mean():.4f}\"\n )\n\n print(\n f\"Mean F1 : \"\n f\"{results_df['F1'].mean():.4f}\"\n )\n\n print(\n f\"Mean Accuracy : \"\n f\"{results_df['Accuracy'].mean():.4f}\"\n )\n\n# ================================================================\n# 12. SAVE THRESHOLDS\n# ================================================================\n\nTHRESHOLD_FILE = (\n \"/kaggle/working/\"\n \"rsna_report_teacher_thresholds_v3_1.csv\"\n)\n\nresults_df.to_csv(\n THRESHOLD_FILE,\n index=False\n)\n\nprint(\n \"\\n✓ Saved:\"\n)\nprint(THRESHOLD_FILE)\n\n# ================================================================\n# 13. CREATE SIMPLE THRESHOLD DICTIONARY\n# ================================================================\n\noptimal_thresholds = dict(\n zip(\n results_df[\"Target\"],\n results_df[\"Optimal_Threshold\"]\n )\n)\n\nprint(\"\\nOptimal threshold dictionary:\")\n\nfor target, threshold in optimal_thresholds.items():\n\n print(\n f\" {target!r}: {threshold:.2f},\"\n )\n\n# ================================================================\n# 14. END\n# ================================================================\n\nprint(\"\\n\" + \"=\" * 75)\nprint(\"V3.1.2 CALIBRATION COMPLETE\")\nprint(\"=\" * 75)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-08-07T16:29:01.437859Z","iopub.execute_input":"2026-08-07T16:29:01.438824Z","iopub.status.idle":"2026-08-07T16:29:08.133439Z","shell.execute_reply.started":"2026-08-07T16:29:01.438789Z","shell.execute_reply":"2026-08-07T16:29:08.132735Z"}},"outputs":[],"execution_count":null},{"id":"ec1db88f-0755-4b2a-809b-ed59d3f74bcb","cell_type":"code","source":"","metadata":{"trusted":true},"outputs":[],"execution_count":null},{"id":"39729471-dab4-429f-9260-f69573ac726c","cell_type":"code","source":"# ================================================================\n# RSNA KNEE V3.1-FIX — REPORT TEACHER CALIBRATION\n# Uses ORIGINAL TRAIN LABELS as ground truth\n# No *_TRUE columns required\n# ================================================================\n\nimport os\nimport numpy as np\nimport pandas as pd\n\nfrom sklearn.metrics import (\n roc_auc_score,\n average_precision_score,\n f1_score,\n accuracy_score,\n precision_score,\n recall_score\n)\n\nprint(\"=\" * 75)\nprint(\"RSNA KNEE V3.1-FIX — REPORT TEACHER CALIBRATION\")\nprint(\"=\" * 75)\n\n# ------------------------------------------------\n# 1. Paths\n# ------------------------------------------------\n\nROOT = \"/kaggle/input/competitions/rsna-knee-abnormality-detection\"\n\nTRAIN_CSV = os.path.join(ROOT, \"train.csv\")\nPROB_CSV = \"/kaggle/working/rsna_report_probabilities_v3.csv\"\n\n# ------------------------------------------------\n# 2. Load data\n# ------------------------------------------------\n\ntrain = pd.read_csv(TRAIN_CSV)\nprobs = pd.read_csv(PROB_CSV)\n\nprint(\"Train:\", train.shape)\nprint(\"Probabilities:\", probs.shape)\n\n# ------------------------------------------------\n# 3. Targets\n# ------------------------------------------------\n\nTARGETS = [\n \"ACL\",\n \"MCL\",\n \"Medial Meniscus\",\n \"Lateral Meniscus\",\n \"Medial OA\",\n \"Lateral OA\",\n \"PF OA\",\n \"Effusion\",\n \"Synovitis\",\n \"Baker's\",\n \"Contusion\",\n \"Fracture\"\n]\n\n# ------------------------------------------------\n# 4. Check target columns\n# ------------------------------------------------\n\nmissing_targets = [t for t in TARGETS if t not in train.columns]\n\nif missing_targets:\n raise ValueError(\n f\"Missing target columns in train.csv: {missing_targets}\"\n )\n\nprint(\"\\n✓ All 12 original target columns found.\")\n\n# ------------------------------------------------\n# 5. Detect probability columns\n# ------------------------------------------------\n\nprob_cols = {}\n\nfor target in TARGETS:\n\n if target in probs.columns:\n prob_cols[target] = target\n\n elif f\"{target}_PROB\" in probs.columns:\n prob_cols[target] = f\"{target}_PROB\"\n\n elif f\"prob_{target}\" in probs.columns:\n prob_cols[target] = f\"prob_{target}\"\n\n else:\n # Try normalized matching\n normalized_target = (\n target.lower()\n .replace(\" \", \"\")\n .replace(\"'\", \"\")\n .replace(\"_\", \"\")\n )\n\n found = None\n\n for c in probs.columns:\n normalized_c = (\n str(c).lower()\n .replace(\" \", \"\")\n .replace(\"'\", \"\")\n .replace(\"_\", \"\")\n )\n\n if normalized_c == normalized_target:\n found = c\n break\n\n if found is not None:\n prob_cols[target] = found\n\nprint(\"\\nProbability columns detected:\")\n\nfor target in TARGETS:\n print(\n f\"{target:20s} -> \"\n f\"{prob_cols.get(target, 'NOT FOUND')}\"\n )\n\nmissing_probs = [\n t for t in TARGETS\n if t not in prob_cols\n]\n\nif missing_probs:\n raise ValueError(\n f\"\\nMissing probability columns: {missing_probs}\"\n )\n\n# ------------------------------------------------\n# 6. Align rows\n# ------------------------------------------------\n\n# Prefer StudyInstanceUID alignment if available\n\nif \"StudyInstanceUID\" in train.columns and \\\n \"StudyInstanceUID\" in probs.columns:\n\n print(\"\\nAligning by StudyInstanceUID...\")\n\n df = train[\n [\"StudyInstanceUID\"] + TARGETS\n ].merge(\n probs,\n on=\"StudyInstanceUID\",\n how=\"inner\",\n suffixes=(\"_TRUE\", \"_PROB\")\n )\n\nelse:\n\n print(\"\\nStudyInstanceUID not available in both files.\")\n print(\"Using row-order alignment.\")\n\n if len(train) != len(probs):\n raise ValueError(\n f\"Row mismatch: train={len(train)}, probs={len(probs)}\"\n )\n\n df = train[TARGETS].copy()\n\n for target in TARGETS:\n df[f\"__PROB_{target}\"] = probs[\n prob_cols[target]\n ].values\n\nprint(\"Merged:\", df.shape)\n\n# ------------------------------------------------\n# 7. IMPORTANT:\n# Ground truth comes from ORIGINAL train labels\n# ------------------------------------------------\n\nprint(\"\\nGround-truth source:\")\nprint(\"✓ Original train.csv label columns\")\nprint(\"✓ Missing labels are excluded\")\nprint(\"✓ No *_TRUE columns are required\")\n\n# ------------------------------------------------\n# 8. Optimal threshold search\n# ------------------------------------------------\n\nprint(\"\\n\" + \"=\" * 75)\nprint(\"OPTIMAL THRESHOLD SEARCH\")\nprint(\"=\" * 75)\n\nthreshold_results = []\n\nTHRESHOLDS = np.arange(\n 0.05,\n 0.951,\n 0.01\n)\n\nfor target in TARGETS:\n\n # ------------------------------------------------\n # TRUE LABEL\n # ------------------------------------------------\n\n if target in train.columns:\n\n y = pd.to_numeric(\n df[target],\n errors=\"coerce\"\n )\n\n else:\n\n # UID merge creates target_TRUE\n y = pd.to_numeric(\n df[f\"{target}_TRUE\"],\n errors=\"coerce\"\n )\n\n # ------------------------------------------------\n # PREDICTED PROBABILITY\n # ------------------------------------------------\n\n if f\"__PROB_{target}\" in df.columns:\n\n p = pd.to_numeric(\n df[f\"__PROB_{target}\"],\n errors=\"coerce\"\n )\n\n elif prob_cols[target] in df.columns:\n\n p = pd.to_numeric(\n df[prob_cols[target]],\n errors=\"coerce\"\n )\n\n else:\n\n # After merge probability column may have suffix\n candidates = [\n prob_cols[target],\n f\"{prob_cols[target]}_PROB\"\n ]\n\n found = None\n\n for c in candidates:\n if c in df.columns:\n found = c\n break\n\n if found is None:\n raise KeyError(\n f\"Cannot locate probability column for {target}\"\n )\n\n p = pd.to_numeric(\n df[found],\n errors=\"coerce\"\n )\n\n # ------------------------------------------------\n # Valid observations\n # ------------------------------------------------\n\n valid = (\n y.notna() &\n p.notna()\n )\n\n yv = y[valid].astype(int).values\n pv = p[valid].astype(float).values\n\n if len(yv) == 0:\n print(f\"{target:20s} NO VALID DATA\")\n continue\n\n if len(np.unique(yv)) < 2:\n print(f\"{target:20s} only one class\")\n continue\n\n # ------------------------------------------------\n # AUC / AP\n # ------------------------------------------------\n\n auc = roc_auc_score(yv, pv)\n ap = average_precision_score(yv, pv)\n\n # ------------------------------------------------\n # Find threshold maximizing F1\n # ------------------------------------------------\n\n best_threshold = 0.50\n best_f1 = -1\n\n best_acc = 0\n best_precision = 0\n best_recall = 0\n\n for threshold in THRESHOLDS:\n\n pred = (\n pv >= threshold\n ).astype(int)\n\n f1 = f1_score(\n yv,\n pred,\n zero_division=0\n )\n\n if f1 > best_f1:\n\n best_f1 = f1\n best_threshold = threshold\n\n best_acc = accuracy_score(\n yv,\n pred\n )\n\n best_precision = precision_score(\n yv,\n pred,\n zero_division=0\n )\n\n best_recall = recall_score(\n yv,\n pred,\n zero_division=0\n )\n\n threshold_results.append({\n\n \"Target\": target,\n\n \"Available\": len(yv),\n\n \"Positive\": int(yv.sum()),\n\n \"Negative\": int(len(yv) - yv.sum()),\n\n \"AUC\": auc,\n\n \"AP\": ap,\n\n \"Optimal_Threshold\": best_threshold,\n\n \"F1\": best_f1,\n\n \"Accuracy\": best_acc,\n\n \"Precision\": best_precision,\n\n \"Recall\": best_recall\n\n })\n\n print(\n f\"{target:20s} \"\n f\"AUC={auc:.4f} \"\n f\"AP={ap:.4f} \"\n f\"Threshold={best_threshold:.2f} \"\n f\"F1={best_f1:.4f} \"\n f\"ACC={best_acc:.4f}\"\n )\n\n# ------------------------------------------------\n# 9. Results table\n# ------------------------------------------------\n\nthreshold_df = pd.DataFrame(\n threshold_results\n)\n\nprint(\"\\n\" + \"=\" * 75)\nprint(\"OPTIMAL THRESHOLD RESULTS\")\nprint(\"=\" * 75)\n\ndisplay(\n threshold_df\n)\n\n# ------------------------------------------------\n# 10. Save thresholds\n# ------------------------------------------------\n\nTHRESHOLD_FILE = (\n \"/kaggle/working/\"\n \"rsna_report_teacher_thresholds_v3_1.csv\"\n)\n\nthreshold_df.to_csv(\n THRESHOLD_FILE,\n index=False\n)\n\nprint(\n f\"\\n✓ Saved thresholds:\\n\"\n f\"{THRESHOLD_FILE}\"\n)\n\n# ------------------------------------------------\n# 11. Summary\n# ------------------------------------------------\n\nif len(threshold_df) > 0:\n\n print(\"\\n\" + \"=\" * 75)\n print(\"CALIBRATION SUMMARY\")\n print(\"=\" * 75)\n\n print(\n f\"Mean AUC : \"\n f\"{threshold_df['AUC'].mean():.4f}\"\n )\n\n print(\n f\"Mean AP : \"\n f\"{threshold_df['AP'].mean():.4f}\"\n )\n\n print(\n f\"Mean F1 : \"\n f\"{threshold_df['F1'].mean():.4f}\"\n )\n\n print(\n f\"Mean Accuracy : \"\n f\"{threshold_df['Accuracy'].mean():.4f}\"\n )\n\n print(\"\\nOptimal thresholds:\")\n\n for _, row in threshold_df.iterrows():\n\n print(\n f\"{row['Target']:20s} \"\n f\"threshold={row['Optimal_Threshold']:.2f}\"\n )\n\nprint(\"\\n\" + \"=\" * 75)\nprint(\"V3.1-FIX COMPLETE\")\nprint(\"=\" * 75)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-08-07T16:25:53.201342Z","iopub.execute_input":"2026-08-07T16:25:53.201667Z","iopub.status.idle":"2026-08-07T16:25:53.34583Z","shell.execute_reply.started":"2026-08-07T16:25:53.201641Z","shell.execute_reply":"2026-08-07T16:25:53.344668Z"}},"outputs":[],"execution_count":null},{"id":"6097a800-912f-4ff6-bcb7-2e2c0611d74b","cell_type":"markdown","source":"# 9. Parallel selected-series preprocessing\n\nThis is the only stage where DICOM pixel decoding happens.\n\nRun it once.\n\nIf the notebook is restarted and `/kaggle/working/rsna_knee_v21` survives, completed cache files are skipped.","metadata":{}},{"id":"6eb8f032-5f2e-4512-a11d-fd524c6327ff","cell_type":"code","source":"CACHE = WORK / \"cache\" / CACHE_VERSION\nCACHE.mkdir(parents=True, exist_ok=True)\n\ndef make_jobs(selected_df):\n jobs = []\n\n for _, r in selected_df.iterrows():\n out = CACHE / (\n f'{r.StudyInstanceUID}_'\n f'{r.SeriesInstanceUID}.npz'\n )\n\n jobs.append((\n str(r.StudyInstanceUID),\n str(r.SeriesInstanceUID),\n r[\"paths\"],\n str(out)\n ))\n\n return jobs\n\ntrain_jobs = make_jobs(train_selected)\ntest_jobs = make_jobs(test_selected)\n\nprint(\"Train jobs:\", len(train_jobs))\nprint(\"Test jobs:\", len(test_jobs))\nprint(\n \"Existing train cache:\",\n sum(os.path.exists(j[3]) for j in train_jobs)\n)\nprint(\n \"Existing test cache:\",\n sum(os.path.exists(j[3]) for j in test_jobs)\n)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-08-07T15:58:32.713607Z","iopub.execute_input":"2026-08-07T15:58:32.714085Z","iopub.status.idle":"2026-08-07T15:58:34.635535Z","shell.execute_reply.started":"2026-08-07T15:58:32.714052Z","shell.execute_reply":"2026-08-07T15:58:34.634686Z"}},"outputs":[],"execution_count":null},{"id":"69cbfa05-fd1e-439e-b7fd-1c636f7722a4","cell_type":"code","source":"# ============================================================\n# REPORT FORENSICS — GOLD LABELS + ALL 4,407 REPORTS\n# ============================================================\n\nimport re\nfrom collections import Counter\n\nprint(\"=\" * 75)\nprint(\"RSNA KNEE REPORT FORENSICS\")\nprint(\"=\" * 75)\n\n# ------------------------------------------------------------\n# 1. GOLD-LABELLED STUDIES\n# ------------------------------------------------------------\n\ngold = train[\n train[TARGETS].notna().all(axis=1)\n].copy()\n\ngold[\"StudyInstanceUID\"] = gold[\"StudyInstanceUID\"].astype(str)\n\nprint(\"\\nGold-labelled studies:\", len(gold))\n\n# ------------------------------------------------------------\n# 2. SHOW GOLD REPORTS\n# ------------------------------------------------------------\n\nprint(\"\\n\" + \"=\" * 75)\nprint(\"SAMPLE GOLD-LABELLED REPORTS\")\nprint(\"=\" * 75)\n\nfor i, (_, row) in enumerate(\n gold.sample(\n min(10, len(gold)),\n random_state=SEED\n ).iterrows()\n):\n\n print(\"\\n\" + \"-\" * 75)\n print(\"Study:\", row[\"StudyInstanceUID\"])\n\n positives = [\n t for t in TARGETS\n if row[t] == 1\n ]\n\n negatives = [\n t for t in TARGETS\n if row[t] == 0\n ]\n\n print(\"POSITIVE:\", positives)\n print(\"NEGATIVE:\", negatives)\n\n print(\"\\nREPORT:\")\n print(str(row[\"Report\"])[:4000])\n\n# ------------------------------------------------------------\n# 3. LABEL COMBINATIONS\n# ------------------------------------------------------------\n\nprint(\"\\n\" + \"=\" * 75)\nprint(\"COMMON LABEL COMBINATIONS\")\nprint(\"=\" * 75)\n\ncombos = []\n\nfor _, row in gold.iterrows():\n\n combo = tuple(\n t for t in TARGETS\n if row[t] == 1\n )\n\n combos.append(combo)\n\ncombo_counts = Counter(combos)\n\nfor combo, count in combo_counts.most_common(20):\n\n print(\n count,\n \"→\",\n \", \".join(combo) if combo else \"No abnormality\"\n )\n\n# ------------------------------------------------------------\n# 4. CLINICAL KEYWORDS\n# ------------------------------------------------------------\n\nkeyword_groups = {\n\n \"ACL\": [\n r\"\\bacl\\b\",\n r\"anterior cruciate\",\n ],\n\n \"MCL\": [\n r\"\\bmcl\\b\",\n r\"medial collateral\",\n ],\n\n \"Medial Meniscus\": [\n r\"medial meniscus\",\n r\"medial meniscal\",\n ],\n\n \"Lateral Meniscus\": [\n r\"lateral meniscus\",\n r\"lateral meniscal\",\n ],\n\n \"Medial OA\": [\n r\"medial compartment\",\n r\"medial tibiofemoral\",\n r\"medial.*osteoarth\",\n r\"medial.*degener\",\n ],\n\n \"Lateral OA\": [\n r\"lateral compartment\",\n r\"lateral tibiofemoral\",\n r\"lateral.*osteoarth\",\n r\"lateral.*degener\",\n ],\n\n \"PF OA\": [\n r\"patellofemoral\",\n r\"patello-femoral\",\n r\"patellar.*osteoarth\",\n ],\n\n \"Effusion\": [\n r\"\\beffusion\\b\",\n r\"joint fluid\",\n r\"intra.?articular fluid\",\n ],\n\n \"Synovitis\": [\n r\"\\bsynovitis\\b\",\n r\"synovial thick\",\n r\"synovial inflammation\",\n ],\n\n \"Baker's\": [\n r\"baker\",\n r\"popliteal cyst\",\n r\"popliteal bursa\",\n ],\n\n \"Contusion\": [\n r\"\\bcontusion\\b\",\n r\"bone bruise\",\n r\"bone marrow edema\",\n r\"marrow oedema\",\n ],\n\n \"Fracture\": [\n r\"\\bfracture\\b\",\n r\"fractured\",\n r\"fracture line\",\n ],\n}\n\n# ------------------------------------------------------------\n# 5. KEYWORD FREQUENCY IN ALL REPORTS\n# ------------------------------------------------------------\n\nprint(\"\\n\" + \"=\" * 75)\nprint(\"KEYWORD OCCURRENCE IN ALL REPORTS\")\nprint(\"=\" * 75)\n\nall_reports = (\n train[\"Report\"]\n .fillna(\"\")\n .astype(str)\n .str.lower()\n)\n\nkeyword_results = []\n\nfor target, patterns in keyword_groups.items():\n\n regex = \"|\".join(\n f\"(?:{p})\"\n for p in patterns\n )\n\n hits = all_reports.str.contains(\n regex,\n regex=True,\n na=False\n )\n\n keyword_results.append({\n \"Target\": target,\n \"Reports_with_keyword\": int(hits.sum()),\n \"Percentage\": round(\n 100 * hits.mean(), 2\n )\n })\n\nkeyword_table = pd.DataFrame(keyword_results)\n\ndisplay(keyword_table)\n\n# ------------------------------------------------------------\n# 6. KEYWORD PRECISION/RECALL ON GOLD DATA\n# ------------------------------------------------------------\n\nprint(\"\\n\" + \"=\" * 75)\nprint(\"GOLD-LABEL KEYWORD PRECISION CHECK\")\nprint(\"=\" * 75)\n\nprecision_results = []\n\nfor target, patterns in keyword_groups.items():\n\n regex = \"|\".join(\n f\"(?:{p})\"\n for p in patterns\n )\n\n hits = (\n gold[\"Report\"]\n .fillna(\"\")\n .astype(str)\n .str.lower()\n .str.contains(\n regex,\n regex=True,\n na=False\n )\n )\n\n positive = gold[target] == 1\n negative = gold[target] == 0\n\n tp = int((hits & positive).sum())\n fp = int((hits & negative).sum())\n fn = int((~hits & positive).sum())\n\n precision = (\n tp / (tp + fp)\n if (tp + fp) > 0\n else np.nan\n )\n\n recall = (\n tp / (tp + fn)\n if (tp + fn) > 0\n else np.nan\n )\n\n precision_results.append({\n \"Target\": target,\n \"Keyword_Hit\": int(hits.sum()),\n \"TP\": tp,\n \"FP\": fp,\n \"Precision\": round(\n precision, 3\n ) if not np.isnan(precision) else np.nan,\n \"Recall\": round(\n recall, 3\n ) if not np.isnan(recall) else np.nan\n })\n\nkeyword_gold_table = pd.DataFrame(\n precision_results\n)\n\ndisplay(\n keyword_gold_table.sort_values(\n \"Precision\",\n ascending=False\n )\n)\n\n# ------------------------------------------------------------\n# 7. ONE POSITIVE AND ONE NEGATIVE REPORT PER TARGET\n# ------------------------------------------------------------\n\nprint(\"\\n\" + \"=\" * 75)\nprint(\"REPORT EXAMPLES FOR EACH TARGET\")\nprint(\"=\" * 75)\n\nfor target in TARGETS:\n\n pos = gold[gold[target] == 1]\n neg = gold[gold[target] == 0]\n\n print(\"\\n\" + \"#\" * 75)\n print(target)\n print(\"#\" * 75)\n\n print(\"\\nPOSITIVE example:\")\n\n if len(pos):\n print(\n str(pos.iloc[0][\"Report\"])[:2000]\n )\n\n print(\"\\nNEGATIVE example:\")\n\n if len(neg):\n print(\n str(neg.iloc[0][\"Report\"])[:2000]\n )\n\nprint(\"\\n\" + \"=\" * 75)\nprint(\"REPORT FORENSICS COMPLETE\")\nprint(\"=\" * 75)","metadata":{"trusted":true,"execution":{"iopub.status.busy":"2026-08-07T16:05:12.67322Z","iopub.execute_input":"2026-08-07T16:05:12.673725Z","iopub.status.idle":"2026-08-07T16:05:13.746896Z","shell.execute_reply.started":"2026-08-07T16:05:12.673686Z","shell.execute_reply":"2026-08-07T16:05:13.746141Z"},"collapsed":true,"jupyter":{"outputs_hidden":true,"source_hidden":true}},"outputs":[],"execution_count":null},{"id":"43c540a7-2f97-4497-b01d-133e6070efad","cell_type":"code","source":"def run_parallel_cache(jobs, description):\n pending = [\n j for j in jobs\n if not os.path.exists(j[3])\n ]\n\n if not pending:\n print(description, \": cache already complete.\")\n return\n\n t0 = time.time()\n\n with ProcessPoolExecutor(\n max_workers=CPU_WORKERS\n ) as ex:\n\n futures = [\n ex.submit(\n decode_series_job,\n j\n )\n for j in pending\n ]\n\n for f in tqdm(\n as_completed(futures),\n total=len(futures),\n desc=description\n ):\n f.result()\n\n dt = time.time() - t0\n\n print(\n f\"{description}: {len(pending)} series \"\n f\"in {dt/60:.1f} min\"\n )\n\nrun_parallel_cache(\n train_jobs,\n \"TRAIN DICOM → cache\"\n)\n\nrun_parallel_cache(\n test_jobs,\n \"TEST DICOM → cache\"\n)","metadata":{},"outputs":[],"execution_count":null},{"id":"4c2f6b8b-ef11-43b7-aa1b-76236044f813","cell_type":"markdown","source":"# 10. Build compact study-level cache\n\nTraining will read **one file per study**, not individual DICOM files or series files.","metadata":{}},{"id":"4e98bfc4-8319-4323-bab0-4b5d9c7404d1","cell_type":"code","source":"STUDY_CACHE = CACHE / \"train_studies\"\nTEST_STUDY_CACHE = CACHE / \"test_studies\"\n\nSTUDY_CACHE.mkdir(exist_ok=True)\nTEST_STUDY_CACHE.mkdir(exist_ok=True)\n\nPLANE_ID = {\n \"Sagittal\": 0,\n \"Coronal\": 1,\n \"Axial\": 2,\n \"Unknown\": 3\n}\n\ndef build_study_cache(selected_df, output_dir):\n for sid, g in tqdm(\n selected_df.groupby(\"StudyInstanceUID\"),\n desc=f\"Study cache → {output_dir.name}\"\n ):\n sid = str(sid)\n out = output_dir / f\"{sid}.pt\"\n\n if out.exists():\n continue\n\n imgs = []\n metas = []\n\n g = g.sort_values(\n \"base_priority\",\n ascending=False\n )\n\n for _, r in g.iterrows():\n f = CACHE / (\n f'{sid}_{r.SeriesInstanceUID}.npz'\n )\n\n if f.exists():\n x = np.load(f)[\"images\"]\n else:\n x = np.zeros(\n (N_SLICES,3,IMG_SIZE,IMG_SIZE),\n dtype=np.float16\n )\n\n imgs.append(x)\n\n metas.append([\n float(r[\"Fluid_Sensitive\"]),\n float(r[\"Fat_Suppression\"]),\n PLANE_ID.get(\n str(r[\"Anatomical_Plane\"]),\n 3\n ) / 3.0,\n min(float(r[\"n_slices\"]),100)/100.0,\n float(r[\"base_priority\"])/10.0\n ])\n\n while len(imgs) < MAX_SERIES:\n imgs.append(\n np.zeros(\n (N_SLICES,3,IMG_SIZE,IMG_SIZE),\n dtype=np.float16\n )\n )\n metas.append([0,0,0,0,0])\n\n imgs = np.stack(\n imgs[:MAX_SERIES]\n )\n\n metas = np.asarray(\n metas[:MAX_SERIES],\n dtype=np.float32\n )\n\n torch.save(\n {\n \"images\": torch.from_numpy(imgs),\n \"meta\": torch.from_numpy(metas)\n },\n out\n )\n\nbuild_study_cache(\n train_selected,\n STUDY_CACHE\n)\n\nbuild_study_cache(\n test_selected,\n TEST_STUDY_CACHE\n)\n\nprint(\n \"Train study caches:\",\n len(list(STUDY_CACHE.glob(\"*.pt\")))\n)\n\nprint(\n \"Test study caches:\",\n len(list(TEST_STUDY_CACHE.glob(\"*.pt\")))\n)","metadata":{},"outputs":[],"execution_count":null},{"id":"a689942d-030e-4f6f-b5e2-4502c83cc80f","cell_type":"markdown","source":"# 11. Fixed study-level validation split\n\nNo patient/study appears in both fit and validation.\n\nThe split is saved so every experiment uses the same validation set.","metadata":{}},{"id":"436dbf06-00a9-46a4-8d5a-629546087543","cell_type":"code","source":"from sklearn.model_selection import train_test_split\n\nlabeled = train[\n train[TARGETS].notna().all(axis=1)\n].copy()\n\nsplit_file = WORK / \"split_v21.json\"\n\nif split_file.exists():\n split = json.loads(\n split_file.read_text()\n )\n fit_ids = split[\"fit\"]\n val_ids = split[\"val\"]\nelse:\n fit_ids, val_ids = train_test_split(\n labeled[\"StudyInstanceUID\"].astype(str).tolist(),\n test_size=0.20,\n random_state=SEED\n )\n\n split_file.write_text(\n json.dumps(\n {\"fit\":fit_ids, \"val\":val_ids},\n indent=2\n )\n )\n\nprint(\"Labeled:\", len(labeled))\nprint(\"Fit:\", len(fit_ids))\nprint(\"Val:\", len(val_ids))\nprint(\n \"Overlap:\",\n len(set(fit_ids) & set(val_ids))\n)\n\nassert not set(fit_ids) & set(val_ids)","metadata":{},"outputs":[],"execution_count":null},{"id":"33a58a36-1116-40ee-9b35-7d402068b099","cell_type":"markdown","source":"# 12. Precompute report features\n\nFor the FAST profile we use character n-gram TF-IDF + SVD.\n\nThis is not the final high-accuracy language model.\n\nIt is intentionally cheap so we can first validate the image pipeline and iterate quickly.","metadata":{}},{"id":"d2964526-2778-4d0c-8051-c24a776bc4e2","cell_type":"code","source":"from sklearn.feature_extraction.text import TfidfVectorizer\nfrom sklearn.decomposition import TruncatedSVD\n\nreports = (\n train[\"Report\"]\n .fillna(\"\")\n .astype(str)\n)\n\nvectorizer = TfidfVectorizer(\n analyzer=\"char\",\n ngram_range=(2,5),\n min_df=2,\n max_features=30000,\n sublinear_tf=True\n)\n\nX = vectorizer.fit_transform(reports)\n\nTEXT_DIM = min(\n 128,\n max(16, X.shape[1]-1)\n)\n\nsvd = TruncatedSVD(\n TEXT_DIM,\n random_state=SEED\n)\n\nZ = svd.fit_transform(\n X\n).astype(np.float32)\n\nreport_embeddings = {\n str(sid): Z[i]\n for i,sid in enumerate(\n train[\"StudyInstanceUID\"]\n )\n}\n\nprint(\"Report matrix:\", X.shape)\nprint(\"Dense report features:\", Z.shape)","metadata":{},"outputs":[],"execution_count":null},{"id":"f0f45851-a201-471c-8701-6a95615c831a","cell_type":"markdown","source":"# 13. Fast dataset","metadata":{}},{"id":"1a116e14-f96d-439e-a4cb-e84e61460a4a","cell_type":"code","source":"class FastKneeDataset(Dataset):\n def __init__(\n self,\n ids,\n train_df,\n training=False\n ):\n self.ids = [str(x) for x in ids]\n self.train_df = (\n train_df\n .set_index(\"StudyInstanceUID\")\n )\n self.training = training\n\n def __len__(self):\n return len(self.ids)\n\n def __getitem__(self, idx):\n sid = self.ids[idx]\n\n d = torch.load(\n STUDY_CACHE / f\"{sid}.pt\",\n map_location=\"cpu\",\n weights_only=False\n )\n\n # [series, slices, channels, H, W]\n images = d[\"images\"].float()\n\n # Fast 2.5D compression:\n # average representative triplets.\n images = images.mean(dim=1)\n\n meta = d[\"meta\"].float()\n\n if (\n self.training\n and random.random() < 0.5\n ):\n images = torch.flip(\n images,\n dims=[-1]\n )\n\n text = torch.from_numpy(\n report_embeddings.get(\n sid,\n np.zeros(\n TEXT_DIM,\n dtype=np.float32\n )\n )\n ).float()\n\n row = self.train_df.loc[sid]\n\n y = torch.tensor(\n row[TARGETS].values.astype(\n np.float32\n )\n )\n\n return images, meta, text, y, sid\n\ndef collate_train(batch):\n return (\n torch.stack([b[0] for b in batch]),\n torch.stack([b[1] for b in batch]),\n torch.stack([b[2] for b in batch]),\n torch.stack([b[3] for b in batch]),\n [b[4] for b in batch]\n )","metadata":{},"outputs":[],"execution_count":null},{"id":"0305b223-a76b-4687-afec-a7245bb01423","cell_type":"markdown","source":"# 14. Efficient multimodal model","metadata":{}},{"id":"03966bd7-9924-486d-9876-a40b7436df56","cell_type":"code","source":"import torchvision.models as tvm\n\nclass FastImageEncoder(nn.Module):\n def __init__(self):\n super().__init__()\n\n try:\n weights = tvm.ResNet18_Weights.DEFAULT\n except Exception:\n weights = None\n\n self.backbone = tvm.resnet18(\n weights=weights\n )\n\n dim = self.backbone.fc.in_features\n self.backbone.fc = nn.Identity()\n\n self.proj = nn.Sequential(\n nn.Linear(dim,256),\n nn.LayerNorm(256),\n nn.GELU()\n )\n\n def forward(self,x):\n return self.proj(\n self.backbone(x)\n )\n\nclass FastKneeNet(nn.Module):\n def __init__(self,text_dim):\n super().__init__()\n\n self.image = FastImageEncoder()\n\n self.meta = nn.Sequential(\n nn.Linear(5,64),\n nn.GELU(),\n nn.Linear(64,256)\n )\n\n self.series_score = nn.Sequential(\n nn.Linear(256,64),\n nn.GELU(),\n nn.Linear(64,1)\n )\n\n self.text = nn.Sequential(\n nn.Linear(text_dim,128),\n nn.LayerNorm(128),\n nn.GELU(),\n nn.Linear(128,256)\n )\n\n self.head = nn.Sequential(\n nn.Linear(512,256),\n nn.GELU(),\n nn.Dropout(0.2),\n nn.Linear(256,12)\n )\n\n def forward(\n self,\n images,\n meta,\n text\n ):\n B,S,C,H,W = images.shape\n\n f = self.image(\n images.reshape(\n B*S,C,H,W\n )\n ).reshape(\n B,S,256\n )\n\n f = f + self.meta(meta)\n\n valid = (\n meta.abs().sum(-1) > 0\n )\n\n scores = (\n self.series_score(f)\n .squeeze(-1)\n )\n\n scores = scores.masked_fill(\n ~valid,\n -1e4\n )\n\n attention = torch.softmax(\n scores,\n dim=1\n )\n\n image_feature = (\n f * attention.unsqueeze(-1)\n ).sum(dim=1)\n\n text_feature = self.text(text)\n\n fused = torch.cat(\n [image_feature,text_feature],\n dim=1\n )\n\n logits = self.head(fused)\n\n return logits, attention\n\nDEVICE = torch.device(\n \"cuda\"\n if torch.cuda.is_available()\n else \"cpu\"\n)\n\nmodel = FastKneeNet(\n TEXT_DIM\n).to(DEVICE)\n\nmodel = model.to(\n memory_format=torch.channels_last\n)\n\nprint(\n \"Parameters:\",\n round(\n sum(\n p.numel()\n for p in model.parameters()\n ) / 1e6,\n 2\n ),\n \"M\"\n)","metadata":{},"outputs":[],"execution_count":null},{"id":"70ef329b-1aa6-4360-b481-f1bb88a7bb6a","cell_type":"markdown","source":"# 15. GPU-optimized DataLoaders","metadata":{}},{"id":"660aa3e5-6905-46bb-a0a8-bd48f18a297c","cell_type":"code","source":"train_ds = FastKneeDataset(\n fit_ids,\n train,\n training=True\n)\n\nval_ds = FastKneeDataset(\n val_ids,\n train,\n training=False\n)\n\nloader_kwargs = dict(\n num_workers=NUM_WORKERS,\n pin_memory=True,\n persistent_workers=True,\n prefetch_factor=4,\n collate_fn=collate_train\n)\n\ntrain_loader = DataLoader(\n train_ds,\n batch_size=BATCH_SIZE,\n shuffle=True,\n **loader_kwargs\n)\n\nval_loader = DataLoader(\n val_ds,\n batch_size=BATCH_SIZE,\n shuffle=False,\n **loader_kwargs\n)\n\nprint(\"Train studies:\", len(train_ds))\nprint(\"Validation studies:\", len(val_ds))","metadata":{},"outputs":[],"execution_count":null},{"id":"ba7cbbc8-7e34-4022-8f62-9a993987b6ad","cell_type":"markdown","source":"# 16. Optional torch.compile\n\nCompilation is attempted but never required.\n\nThe first compiled iteration can take longer; later iterations are normally faster.","metadata":{}},{"id":"446b718e-98a3-4227-8c32-27492dfbce17","cell_type":"code","source":"if (\n torch.cuda.is_available()\n and hasattr(torch,\"compile\")\n):\n try:\n model = torch.compile(\n model,\n mode=\"max-autotune\",\n dynamic=False\n )\n print(\"torch.compile: ENABLED\")\n except Exception as e:\n print(\n \"torch.compile skipped:\",\n str(e)[:300]\n )\nelse:\n print(\"torch.compile: skipped\")","metadata":{},"outputs":[],"execution_count":null},{"id":"29b7acc2-1807-4ff1-ab58-208edb810a1a","cell_type":"markdown","source":"# 17. Loss, AMP and training functions","metadata":{}},{"id":"c24477b2-540b-4fb0-abf4-d726732c6288","cell_type":"code","source":"from sklearn.metrics import roc_auc_score\n\nfit_df = train[\n train[\"StudyInstanceUID\"].astype(str).isin(\n fit_ids\n )\n].copy()\n\nyfit = fit_df[TARGETS].values.astype(float)\n\npos = yfit.sum(axis=0)\nneg = len(yfit) - pos\n\npos_weight = np.clip(\n neg / np.maximum(pos,1),\n 1,\n 15\n)\n\ncriterion = nn.BCEWithLogitsLoss(\n pos_weight=torch.tensor(\n pos_weight,\n dtype=torch.float32,\n device=DEVICE\n )\n)\n\noptimizer = torch.optim.AdamW(\n model.parameters(),\n lr=LR,\n weight_decay=WEIGHT_DECAY\n)\n\nscheduler = torch.optim.lr_scheduler.CosineAnnealingLR(\n optimizer,\n T_max=EPOCHS\n)\n\nAMP = torch.cuda.is_available()\n\nif (\n AMP\n and torch.cuda.is_bf16_supported()\n):\n AMP_DTYPE = torch.bfloat16\nelse:\n AMP_DTYPE = torch.float16\n\nscaler = torch.cuda.amp.GradScaler(\n enabled=(\n AMP\n and AMP_DTYPE == torch.float16\n )\n)\n\ndef calculate_auc(y,p):\n each = {}\n values = []\n\n for j,target in enumerate(TARGETS):\n if len(np.unique(y[:,j])) > 1:\n a = roc_auc_score(\n y[:,j],\n p[:,j]\n )\n values.append(a)\n each[target] = float(a)\n\n return (\n float(np.mean(values)),\n each\n )\n\ndef run_epoch(loader,training=True):\n model.train(training)\n\n total_loss = 0\n ys = []\n ps = []\n\n for (\n images,\n meta,\n text,\n y,\n ids\n ) in tqdm(\n loader,\n leave=False\n ):\n\n images = images.to(\n DEVICE,\n non_blocking=True\n ).to(\n memory_format=torch.channels_last\n )\n\n meta = meta.to(\n DEVICE,\n non_blocking=True\n )\n\n text = text.to(\n DEVICE,\n non_blocking=True\n )\n\n y = y.to(\n DEVICE,\n non_blocking=True\n )\n\n if training:\n optimizer.zero_grad(\n set_to_none=True\n )\n\n with torch.autocast(\n device_type=\"cuda\",\n dtype=AMP_DTYPE,\n enabled=AMP\n ):\n logits,_ = model(\n images,\n meta,\n text\n )\n\n loss = criterion(\n logits,\n y\n )\n\n if training:\n scaler.scale(\n loss\n ).backward()\n\n scaler.unscale_(\n optimizer\n )\n\n torch.nn.utils.clip_grad_norm_(\n model.parameters(),\n 1.0\n )\n\n scaler.step(\n optimizer\n )\n\n scaler.update()\n\n total_loss += (\n loss.item() * len(y)\n )\n\n ys.append(\n y.detach()\n .float()\n .cpu()\n .numpy()\n )\n\n ps.append(\n torch.sigmoid(logits)\n .detach()\n .float()\n .cpu()\n .numpy()\n )\n\n ys = np.concatenate(ys)\n ps = np.concatenate(ps)\n\n auc, each = calculate_auc(\n ys,ps\n )\n\n return (\n total_loss / len(loader.dataset),\n auc,\n each\n )","metadata":{},"outputs":[],"execution_count":null},{"id":"cabac43b-8c3c-4644-9809-6f159a6f5692","cell_type":"markdown","source":"# 18. Train the FAST baseline\n\nThis is intentionally a fast baseline.\n\nDo not judge the final competition potential from this model alone.\n\nIts purpose is to establish a **fast, reproducible benchmark**.","metadata":{}},{"id":"4ad7bf1d-21ad-41a5-b152-ddf6f6ebbf1c","cell_type":"code","source":"best_auc = -1\nhistory = []\n\nBEST_MODEL = (\n WORK /\n \"best_v21_fast.pt\"\n)\n\nfor epoch in range(\n 1,\n EPOCHS + 1\n):\n t0 = time.time()\n\n tr_loss,tr_auc,_ = run_epoch(\n train_loader,\n training=True\n )\n\n va_loss,va_auc,each = run_epoch(\n val_loader,\n training=False\n )\n\n scheduler.step()\n\n elapsed = (\n time.time() - t0\n )\n\n history.append({\n \"epoch\":epoch,\n \"train_loss\":tr_loss,\n \"train_auc\":tr_auc,\n \"val_loss\":va_loss,\n \"val_auc\":va_auc,\n \"seconds\":elapsed\n })\n\n print(\n f\"Epoch {epoch:02d} | \"\n f\"Train loss={tr_loss:.4f} \"\n f\"AUC={tr_auc:.4f} | \"\n f\"Val loss={va_loss:.4f} \"\n f\"AUC={va_auc:.4f} | \"\n f\"{elapsed:.1f}s\"\n )\n\n if va_auc > best_auc:\n best_auc = va_auc\n\n state = {\n k:v.detach()\n .cpu()\n .clone()\n for k,v in model.state_dict().items()\n }\n\n torch.save(\n state,\n BEST_MODEL\n )\n\nprint(\n \"\\nBest validation macro-AUROC:\",\n best_auc\n)","metadata":{},"outputs":[],"execution_count":null},{"id":"63d385aa-ccd6-4b6d-99c3-c62cbe46e4c7","cell_type":"markdown","source":"# 19. Validation report","metadata":{}},{"id":"b9aa33f5-56ee-4289-943d-9a515f763b2e","cell_type":"code","source":"state = torch.load(\n BEST_MODEL,\n map_location=\"cpu\",\n weights_only=True\n)\n\nmodel.load_state_dict(state)\nmodel.to(DEVICE)\nmodel.eval()\n\nys = []\nps = []\n\nwith torch.no_grad():\n for (\n images,\n meta,\n text,\n y,\n ids\n ) in tqdm(\n val_loader,\n desc=\"Validation\"\n ):\n\n images = images.to(\n DEVICE,\n non_blocking=True\n ).to(\n memory_format=torch.channels_last\n )\n\n meta = meta.to(\n DEVICE,\n non_blocking=True\n )\n\n text = text.to(\n DEVICE,\n non_blocking=True\n )\n\n with torch.autocast(\n device_type=\"cuda\",\n dtype=AMP_DTYPE,\n enabled=AMP\n ):\n logits,_ = model(\n images,\n meta,\n text\n )\n\n ys.append(\n y.numpy()\n )\n\n ps.append(\n torch.sigmoid(logits)\n .float()\n .cpu()\n .numpy()\n )\n\nys = np.concatenate(ys)\nps = np.concatenate(ps)\n\nmacro, each = calculate_auc(\n ys,ps\n)\n\nvalidation_table = pd.DataFrame({\n \"Finding\":list(each.keys()),\n \"AUROC\":list(each.values())\n}).sort_values(\n \"AUROC\",\n ascending=False\n)\n\ndisplay(validation_table)\n\nprint(\n \"Validation macro-AUROC:\",\n macro\n)","metadata":{},"outputs":[],"execution_count":null},{"id":"94e492b5-3d1f-4d1e-a76e-d7b4d19774c0","cell_type":"markdown","source":"# 20. Test dataset — no full DICOM index\n\nThe test cache was already built from `test_series.csv` and only selected series.\n\nNow inference reads only compact study tensors.","metadata":{}},{"id":"dbd84362-4ee4-46a6-891d-e879c3ca6a39","cell_type":"code","source":"class FastTestDataset(Dataset):\n def __init__(self, ids):\n self.ids = [\n str(x) for x in ids\n ]\n\n def __len__(self):\n return len(self.ids)\n\n def __getitem__(self,idx):\n sid = self.ids[idx]\n\n d = torch.load(\n TEST_STUDY_CACHE / f\"{sid}.pt\",\n map_location=\"cpu\",\n weights_only=False\n )\n\n images = (\n d[\"images\"]\n .float()\n .mean(dim=1)\n )\n\n meta = d[\"meta\"].float()\n\n # Test CSV has no report field in the supplied\n # competition structure, so use a zero vector.\n text = torch.zeros(\n TEXT_DIM,\n dtype=torch.float32\n )\n\n return (\n images,\n meta,\n text,\n sid\n )\n\ndef collate_test(batch):\n return (\n torch.stack(\n [x[0] for x in batch]\n ),\n torch.stack(\n [x[1] for x in batch]\n ),\n torch.stack(\n [x[2] for x in batch]\n ),\n [x[3] for x in batch]\n )\n\ntest_ds = FastTestDataset(\n test[\"StudyInstanceUID\"]\n)\n\ntest_loader = DataLoader(\n test_ds,\n batch_size=BATCH_SIZE,\n shuffle=False,\n num_workers=NUM_WORKERS,\n pin_memory=True,\n persistent_workers=True,\n prefetch_factor=4,\n collate_fn=collate_test\n)\n\nprint(\n \"Test studies:\",\n len(test_ds)\n)","metadata":{},"outputs":[],"execution_count":null},{"id":"e45c0da4-c3cb-4afb-935a-f954757585f7","cell_type":"markdown","source":"# 21. Fast test inference","metadata":{}},{"id":"60130ef1-fce5-4844-b7fe-e4454e120be0","cell_type":"code","source":"model.eval()\n\npredictions = []\nprediction_ids = []\n\nwith torch.no_grad():\n\n for (\n images,\n meta,\n text,\n batch_ids\n ) in tqdm(\n test_loader,\n desc=\"TEST inference\"\n ):\n\n images = images.to(\n DEVICE,\n non_blocking=True\n ).to(\n memory_format=torch.channels_last\n )\n\n meta = meta.to(\n DEVICE,\n non_blocking=True\n )\n\n text = text.to(\n DEVICE,\n non_blocking=True\n )\n\n with torch.autocast(\n device_type=\"cuda\",\n dtype=AMP_DTYPE,\n enabled=AMP\n ):\n logits,_ = model(\n images,\n meta,\n text\n )\n\n predictions.append(\n torch.sigmoid(logits)\n .float()\n .cpu()\n .numpy()\n )\n\n prediction_ids.extend(\n batch_ids\n )\n\npredictions = np.concatenate(\n predictions\n)\n\nprint(\n \"Prediction shape:\",\n predictions.shape\n)","metadata":{},"outputs":[],"execution_count":null},{"id":"3cb41bb2-590b-4d0c-8e51-9c36b7905d50","cell_type":"markdown","source":"# 22. Competition submission\n\nWe preserve the exact column order of `sample_submission.csv`.","metadata":{}},{"id":"043c04f1-d540-4a63-b48d-7f488cf85ea6","cell_type":"code","source":"pred_df = pd.DataFrame(\n predictions,\n columns=TARGETS\n)\n\npred_df.insert(\n 0,\n \"StudyInstanceUID\",\n prediction_ids\n)\n\npred_lookup = pred_df.set_index(\n \"StudyInstanceUID\"\n)\n\nsubmission = sample_submission[\n [\"StudyInstanceUID\"]\n].copy()\n\nfor col in sample_submission.columns:\n if col == \"StudyInstanceUID\":\n continue\n\n if col not in pred_lookup.columns:\n raise KeyError(\n f\"Missing prediction column: {col}\"\n )\n\n submission[col] = (\n submission[\"StudyInstanceUID\"]\n .map(pred_lookup[col])\n )\n\nprob_cols = [\n c for c in submission.columns\n if c != \"StudyInstanceUID\"\n]\n\nassert len(submission) == len(\n sample_submission\n)\n\nassert submission[\n \"StudyInstanceUID\"\n].is_unique\n\nassert submission[\n prob_cols\n].notna().all().all()\n\nassert (\n submission[prob_cols] >= 0\n).all().all()\n\nassert (\n submission[prob_cols] <= 1\n).all().all()\n\nSUBMISSION = (\n WORK /\n \"submission_v21_fast.csv\"\n)\n\nsubmission.to_csv(\n SUBMISSION,\n index=False\n)\n\ndisplay(submission.head())\n\nprint(\n \"Submission saved:\",\n SUBMISSION\n)","metadata":{},"outputs":[],"execution_count":null},{"id":"fa7cae40-de0e-435b-b2a9-ee082d512560","cell_type":"markdown","source":"# 23. Real speed benchmark\n\nDo not use a theoretical \"100×\" statement.\n\nMeasure the actual throughput on your Kaggle runtime.\n\nRun this cell after the first warm-up.","metadata":{}},{"id":"e4b93f8a-fa0c-42ae-83c8-836886ed9a3c","cell_type":"code","source":"model.eval()\n\nn = 0\nt0 = time.time()\n\nwith torch.no_grad():\n\n for (\n images,\n meta,\n text,\n y,\n ids\n ) in train_loader:\n\n images = images.to(\n DEVICE,\n non_blocking=True\n ).to(\n memory_format=torch.channels_last\n )\n\n meta = meta.to(\n DEVICE,\n non_blocking=True\n )\n\n text = text.to(\n DEVICE,\n non_blocking=True\n )\n\n with torch.autocast(\n device_type=\"cuda\",\n dtype=AMP_DTYPE,\n enabled=AMP\n ):\n _ = model(\n images,\n meta,\n text\n )\n\n n += len(ids)\n\n if n >= BATCH_SIZE * 10:\n break\n\nif torch.cuda.is_available():\n torch.cuda.synchronize()\n\nelapsed = time.time() - t0\n\nprint(\n \"Studies benchmarked:\",\n n\n)\nprint(\n \"Seconds:\",\n round(elapsed,2)\n)\nprint(\n \"Studies/sec:\",\n round(n/elapsed,3)\n)\nprint(\n \"ms/study:\",\n round(1000*elapsed/n,2)\n)","metadata":{},"outputs":[],"execution_count":null},{"id":"e6497b81-9505-4cdd-979c-60810a5bc59b","cell_type":"markdown","source":"# 24. V2.1 experiment manifest\n\nSave the exact configuration with the model/submission.","metadata":{}},{"id":"d1148dae-642c-4227-931c-51541dd8974c","cell_type":"code","source":"manifest = {\n \"version\": \"V2.1\",\n \"seed\": SEED,\n \"img_size\": IMG_SIZE,\n \"n_slices\": N_SLICES,\n \"max_series\": MAX_SERIES,\n \"cpu_workers\": CPU_WORKERS,\n \"dataloader_workers\": NUM_WORKERS,\n \"batch_size\": BATCH_SIZE,\n \"epochs\": EPOCHS,\n \"learning_rate\": LR,\n \"cache_version\": CACHE_VERSION,\n \"best_validation_macro_auc\": float(best_auc),\n \"competition_root\": str(ROOT),\n \"train_selected_series\": int(len(train_selected)),\n \"test_selected_series\": int(len(test_selected)),\n \"train_selected_dicom_files\": int(\n train_selected[\"n_slices\"].sum()\n ),\n \"test_selected_dicom_files\": int(\n test_selected[\"n_slices\"].sum()\n ),\n \"submission\": str(SUBMISSION)\n}\n\nmanifest_path = WORK / \"experiment_manifest.json\"\n\nmanifest_path.write_text(\n json.dumps(\n manifest,\n indent=2\n )\n)\n\nprint(\n json.dumps(\n manifest,\n indent=2\n )\n)","metadata":{},"outputs":[],"execution_count":null},{"id":"fa719277-a665-4f38-94b1-cc974489cbd6","cell_type":"markdown","source":"# 25. What V2.1 changes for the competition\n\n### Old V2\n\n```text\nall DICOM filesystem entries\n ↓\nfull index\n ↓\nselect series\n ↓\ndecode\n```\n\n### V2.1\n\n```text\ntrain_series.csv\n ↓\nrank series\n ↓\nselect ~5/study\n ↓\ndirectly enter only those folders\n ↓\ndecode\n ↓\ncache\n```\n\nThe biggest gain is that **the initial filesystem scan is no longer proportional to all 819k DICOM files**.\n\n---\n\n## Next accuracy phase\n\nOnce V2.1 is running reliably, do not immediately increase resolution.\n\nThe competition-winning research path should be:\n\n### V3\n5-fold study-level CV\n\n### V4\nstronger MRI encoders\n\n### V5\nslice attention instead of simple slice averaging\n\n### V6\ntarget-specific specialist heads\n\n### V7\nmultilingual report Transformer / weak supervision\n\n### V8\npseudo-labeling of unlabeled studies, using only competition-permitted information\n\n### V9\nOOF ensemble + calibration\n\n### V10\nfinal full-data ensemble and submission\n\nThe speed work in V2.1 is valuable because each of those experiments can reuse the same cached MRI representation.","metadata":{}},{"id":"04e2906e-9b1a-4960-aa77-5b89e75efc9a","cell_type":"markdown","source":"# Final note\n\n**Do not delete `/kaggle/working/rsna_knee_v21/cache` while experimenting.**\n\nThat directory is the main speed asset.\n\nIf you change only:\n- model architecture\n- learning rate\n- loss\n- augmentation\n- fold\n\nyou should be able to reuse the same DICOM cache.\n\nIf you change:\n- `IMG_SIZE`\n- `N_SLICES`\n- series selection\n- preprocessing\n\nthen create a new `CACHE_VERSION`.","metadata":{}}]}