![\"Head,](\"https://playbees.firstcry.com/blog/wp-content/uploads/2025/12/Head-Shoulders-Knees-and-Toes-Lyrics-Nursery-Rhymes-For-Kids.jpg\")
\n\n \n
Illustration supplied by the user via Bing Images / FirstCry PlayBees.
\n\n## A 0.935 knee-MRI checkpoint you can actually start from\n\nThis notebook is the inference checkpoint, not a leaderboard scrapbook. Give\nit the competition data and two T4s; it reconstructs each study from DICOM,\nruns three deliberately different model families, and writes the twelve-column\nsubmission in about the same time it takes to make coffee.\n\n### Quick start\n\n1. Join **RSNA Knee Abnormality Detection** so Kaggle can mount the test data.\n2. Leave the attached inputs unchanged, select **GPU T4 ×2**, keep internet off,\n and run **Save Version**.\n3. The visible test produces three rows. That is Kaggle's decoy; the scored\n rerun automatically expands to 1,322 hidden studies.\n\nThe exact H&S prediction version **V36** scored **0.935**\npublicly. This release changes its front page only; the four prediction cells\nare byte-identical to that DINOsaur V4 with the MaxSpan Raptor checkpoint.\n\n## The signal path\n\n| Leg | Representation | Job in the ensemble |\n|---|---|---|\n| DINOv2 + DINOv3 | six protocol-aware MRI slots | broad transformer consensus |\n| RadImageNet | reference and fat-sensitive layouts | medical-domain error diversity |\n| Raptor CoAtNet | 2.5D study windows | finding-specific spatial evidence |\n\nThe first two legs form a calibrated transformer/RadImageNet parent. Raptor is\nthen blended by target in rank space; the meniscus and fracture weights are\npreserved from the scored source. Ranking is important here: it lets models\nwith very different probability calibration vote on the same scale.\n\n## Provenance trail\n\nThis is a faithful package, not a claim that every ingredient began here.\n\n| Layer | Public source | What is retained |\n|---|---|---|\n| Scored graph | [Roman Tamrazov — DINOsaur V4](https://www.kaggle.com/code/romantamrazov/rsna-knee-dinosaur-v4) | four-cell 0.934 inference graph, Apache-2.0 |\n| CoAtNet fusion lineage | [Muhammad Aldo Fahrezy — CoAtNet Transformer Blend](https://www.kaggle.com/code/aldofhrzy/rsna-knee-coatnet-transformer-blend) | target-wise rank fusion |\n| Raptor checkpoint | [Dread Development — Raptor weights](https://www.kaggle.com/datasets/dreaddevelopment/raptor-knee-widedense) | study-level CoAtNet weights |\n| Reproduction bundle | [Tony Li — RSNA Knee reproduction assets](https://www.kaggle.com/datasets/tonylica/rsna-knee-bend-dinov3-0917-repro-assets) | consolidated DINO/RadImageNet assets |\n| Base encoder | [Meta DINOv2](https://github.com/facebookresearch/dinov2) | standard ViT-S/14 base |\n\nThe Kaggle **Inputs** panel is the definitive inventory and license surface.\nCompetition images, reports, labels, and row-wise predictions are not\nredistributed by this notebook.\n\n## Reproducibility contract\n\n- Scored source: `romantamrazov/rsna-knee-dinosaur-v4`, V1.\n- H&S inference parent: V36, four cells, unchanged after its\n scored run.\n- Parent contract: only the Raptor checkpoint/span/window cell differs from V35.\n- Final visible-output schema: `StudyInstanceUID` plus the twelve official\n targets, with finite cohort-rank scores.\n\nThis is a competition checkpoint, not a clinical device.\n","metadata":{"papermill":{"duration":0.004715,"end_time":"2026-08-25T08:12:58.968635+00:00","exception":false,"start_time":"2026-08-25T08:12:58.96392+00:00","status":"completed"},"tags":[]}},{"cell_type":"code","source":"\"\"\"Notebook cell source: capture the exact 336px DINO cache for public0033.\n\nThis file is embedded by ``build_public0033_meniscus10_notebook.py`` immediately\nbefore the unchanged upstream DINO prediction cell. The capture is deliberately\nvery narrow: it only observes the cache and mask allocation made by the upstream\n``build_cache`` call for a 6-slot / 12-slice / 336px test volume. It does not\nmodify the returned arrays or any parent prediction code.\n\"\"\"\n\nimport builtins as _p33_builtins\nimport inspect as _p33_inspect\nimport numpy as _p33_np\n\n\n_P33_CAPTURE_KEY = \"_public0033_cache_capture_v1\"\n\nif hasattr(_p33_builtins, _P33_CAPTURE_KEY):\n raise RuntimeError(\"public0033: cache capture state already exists\")\n\n_p33_original_zeros = _p33_np.zeros\n\n\ndef _p33_normalize_shape(_p33_shape):\n try:\n return tuple(int(_p33_value) for _p33_value in _p33_shape)\n except TypeError:\n return None\n\n\ndef _p33_caller_studies():\n \"\"\"Read only the local ``studies`` tuple from upstream ``build_cache``.\"\"\"\n\n _p33_frame = _p33_inspect.currentframe()\n try:\n _p33_wrapper = None if _p33_frame is None else _p33_frame.f_back\n _p33_caller = None if _p33_wrapper is None else _p33_wrapper.f_back\n if _p33_caller is None or _p33_caller.f_code.co_name != \"build_cache\":\n return None\n _p33_value = _p33_caller.f_locals.get(\"studies\")\n if not isinstance(_p33_value, (list, tuple)):\n return None\n _p33_studies = tuple(str(_p33_uid) for _p33_uid in _p33_value)\n if not _p33_studies or any(not _p33_uid for _p33_uid in _p33_studies):\n return None\n if len(set(_p33_studies)) != len(_p33_studies):\n raise RuntimeError(\"public0033: captured build_cache studies are not unique\")\n return _p33_studies\n finally:\n # ``inspect`` frames retain all parent locals, including large DICOM objects.\n del _p33_frame\n del _p33_wrapper\n\n\n_p33_state = {\n \"schema_version\": \"public0033_cache_capture_v1\",\n \"original_zeros\": _p33_original_zeros,\n \"numpy_module\": _p33_np,\n \"events\": [],\n \"cache_candidates\": [],\n \"mask_candidates\": [],\n}\n\n\ndef _p33_zeros(\n _p33_shape,\n dtype=float,\n order=\"C\",\n *,\n like=None,\n _p33_delegate=_p33_original_zeros,\n _p33_state_ref=_p33_state,\n):\n \"\"\"Delegate ``numpy.zeros`` while retaining only the prescribed allocations.\"\"\"\n\n if like is None:\n _p33_value = _p33_delegate(_p33_shape, dtype=dtype, order=order)\n else:\n _p33_value = _p33_delegate(\n _p33_shape, dtype=dtype, order=order, like=like\n )\n\n _p33_shape_tuple = _p33_normalize_shape(_p33_shape)\n _p33_dtype = _p33_np.dtype(dtype)\n _p33_studies = _p33_caller_studies()\n _p33_event = {\n \"shape\": list(_p33_shape_tuple) if _p33_shape_tuple is not None else None,\n \"dtype\": _p33_dtype.str,\n \"caller\": \"build_cache\" if _p33_studies is not None else None,\n \"study_count\": len(_p33_studies) if _p33_studies is not None else None,\n }\n _p33_state_ref[\"events\"].append(_p33_event)\n\n if _p33_studies is None:\n return _p33_value\n\n if (\n _p33_shape_tuple is not None\n and len(_p33_shape_tuple) == 5\n and _p33_shape_tuple[0] == len(_p33_studies)\n and _p33_shape_tuple[1:] == (6, 12, 336, 336)\n and _p33_dtype == _p33_np.dtype(_p33_np.uint8)\n ):\n _p33_state_ref[\"cache_candidates\"].append(\n {\"cache\": _p33_value, \"studies\": _p33_studies}\n )\n elif (\n _p33_shape_tuple is not None\n and len(_p33_shape_tuple) == 2\n and _p33_shape_tuple[0] == len(_p33_studies)\n and _p33_shape_tuple[1:] == (6,)\n and _p33_dtype == _p33_np.dtype(_p33_np.float32)\n ):\n _p33_state_ref[\"mask_candidates\"].append(\n {\"mask\": _p33_value, \"studies\": _p33_studies}\n )\n return _p33_value\n\n\n_p33_np.zeros = _p33_zeros\nsetattr(_p33_builtins, _P33_CAPTURE_KEY, _p33_state)\ndel _p33_state\ndel _p33_original_zeros\n","metadata":{},"outputs":[],"execution_count":null},{"cell_type":"code","source":"from __future__ import annotations\nimport os\nimport gc\nimport hashlib\nimport json\nimport re\nimport time\nimport traceback\nimport threading\nfrom concurrent.futures import ThreadPoolExecutor\nfrom pathlib import Path\nimport numpy as np\nimport pandas as pd\nimport pydicom\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n_ASSET_ROOTS = [\n Path('/kaggle/input/rsna-knee-bend-dinov3-0917-repro-assets'),\n Path('/kaggle/input/datasets/tonylica/rsna-knee-bend-dinov3-0917-repro-assets'),\n]\nASSET = next((path for path in _ASSET_ROOTS if (path / 'rsna-knee-weights' / 'manifest.json').is_file()),\n _ASSET_ROOTS[0])\n_COMPETITION_ROOTS = [\n Path('/kaggle/input/rsna-knee-abnormality-detection'),\n Path('/kaggle/input/competitions/rsna-knee-abnormality-detection'),\n]\nROOT = next((path for path in _COMPETITION_ROOTS if (path / 'train.csv').is_file()),\n _COMPETITION_ROOTS[0])\nDINO = Path('/kaggle/input/models/metaresearch/dinov2/pytorch/small/1')\nT0 = time.time()\nDEVS = [torch.device(f'cuda:{i}') for i in range(torch.cuda.device_count())]\nSEED = 2026\nTARGETS = ['ACL', 'MCL', 'Medial Meniscus', 'Lateral Meniscus', 'Medial OA', 'Lateral OA', 'PF OA', 'Effusion', 'Synovitis', \"Baker's\", 'Contusion', 'Fracture']\nCROP_MM = 130.0\nCACHE_IMG = 336\nGROUP = 3\nN_GROUP_MAX = 1\nCACHE_FRACTION = 0.45\nCACHE_BUDGET_MAX_GB = 24.0\nCACHE_BUDGET_GB = 12.0\nTEST_SHARE = 0.3\nHDR_THREADS = 16\nPIX_THREADS = 12\nORDER_THREADS = 32\nORDER_BUDGET_S = 5400\nAUG_ROT_DEG = 8.0\nAUG_SCALE = 0.08\nAUG_SHIFT = 0.05\nAUG_INTENSITY = 0.1\nLAT_MIN_OFFSET_MM = 20.0\nSLICE_BAND = (0.2, 0.8)\nRULES_NATIVE = {'order': 'normal', 'lat': 'centre', 'slot_fallback': False, 'decode_fill': 'nearest'}\nRULES_LEGACY = {'order': 'dominant_axis', 'lat': 'corner_x', 'slot_fallback': True, 'decode_fill': 'zero'}\nRULES = dict(RULES_NATIVE)\nLEGACY_LAT_OFFSET_MM = 5.0\nEVAL_BATCH = 8\nTIME_BUDGET = 8.0 * 3600\nSLOTS_RECOVERED = [('SAG_FLUID_FS', 'Sagittal', True, True), ('COR_FLUID_FS', 'Coronal', True, True), ('AX_FLUID_FS', 'Axial', True, True), ('SAG_FLUID_NOFS', 'Sagittal', True, False), ('COR_T1', 'Coronal', False, False), ('SAG_T1', 'Sagittal', False, False)]\nSLOTS_PUBLIC = [('SAG_FLUID', 'Sagittal', None, True), ('COR_FLUID', 'Coronal', None, True), ('AX_FLUID', 'Axial', None, True), ('SAG_STRUCT', 'Sagittal', None, False), ('COR_STRUCT', 'Coronal', None, False), ('AX_STRUCT', 'Axial', None, False)]\nSLOT_SCHEME = os.environ.get('SLOT_SCHEME', 'recovered')\nSLOTS = SLOTS_PUBLIC if SLOT_SCHEME == 'public' else SLOTS_RECOVERED\nN_SLOT = len(SLOTS)\nPOOL_PARTS = {'cls_mean': 2, 'cls_mean_focal': 3}\nSLOT_PRIOR_TABLE = {'ACL': (0, 3, 5), 'MCL': (1, 4), 'Medial Meniscus': (0, 1, 3, 4), 'Lateral Meniscus': (0, 1, 3, 4), 'Medial OA': (1, 4, 5), 'Lateral OA': (1, 4, 5), 'PF OA': (0, 2, 5), 'Effusion': (0, 2), 'Synovitis': (0, 2), \"Baker's\": (0,), 'Contusion': (0, 1, 2), 'Fracture': (0, 1, 2, 4, 5)}\nSLOT_PRIOR_STRENGTH = 0.55\nFATSAT_OPTS = {'FS', 'FATSAT', 'FAT_SAT', 'FSAT'}\n_SEP = re.compile('[_\\\\-.]')\n_FATSAT_RX = re.compile('\\\\bfs\\\\b|fatsat|fat sat|\\\\bstir\\\\b|\\\\bspair\\\\b|\\\\bspir\\\\b|\\\\bwe\\\\b|water excit|\\\\btirm\\\\b|\\\\bsting\\\\b|\\\\bfatsup\\\\b')\n_T1_RX = re.compile('\\\\bt1\\\\b|\\\\bt1w\\\\b')\n_T2_RX = re.compile('\\\\bt2\\\\b|\\\\bt2w\\\\b')\n_PD_RX = re.compile('\\\\bpd\\\\b|\\\\bpdw\\\\b|proton|\\\\bdp\\\\b|dens')\n\ndef log(msg):\n print(f'[{time.time() - T0:7.1f}s] {msg}', flush=True)\nIMG = CACHE_IMG\n\ndef available_gb():\n try:\n with open('/proc/meminfo') as fh:\n info = {k.strip(): v for k, v in (l.split(':', 1) for l in fh if ':' in l)}\n return int(info['MemAvailable'].split()[0]) / 1024 ** 2\n except Exception:\n return CACHE_BUDGET_GB / CACHE_FRACTION\n\ndef plan_cache(n_study, n_test=0):\n avail = available_gb()\n budget = min(avail * CACHE_FRACTION, CACHE_BUDGET_MAX_GB)\n n_total = n_study + max(n_test, int(TEST_SHARE * n_study))\n per_slice = n_total * N_SLOT * IMG * IMG\n afford = int(budget * 1024 ** 3 // max(per_slice, 1))\n groups = max(1, min(N_GROUP_MAX, afford // GROUP))\n log(f'memory: {avail:.1f} GB available, {budget:.1f} GB to the cache; sizing for {n_study} train + {n_total - n_study} test studies -> {groups} group(s) of {GROUP} = {groups * GROUP} slices per slot' + (f' (wanted {N_GROUP_MAX})' if groups < N_GROUP_MAX else ''))\n return groups\nN_GROUP = plan_cache(len(pd.read_csv(ROOT / 'train.csv')), len(pd.read_csv(ROOT / 'test.csv')))\nCACHE_SLICES = GROUP * N_GROUP\nHDR_TAGS = ['SeriesDescription', 'SequenceName', 'ScanOptions', 'ScanningSequence', 'RepetitionTime', 'EchoTime', 'Laterality', 'PixelSpacing', 'Rows', 'Columns', 'RescaleSlope', 'RescaleIntercept', 'ImagePositionPatient', 'ImageOrientationPatient']\n\ndef _hdr_vec(s, n):\n if not isinstance(s, str):\n return None\n try:\n v = [float(x) for x in s.split('|')]\n except ValueError:\n return None\n return np.array(v) if len(v) >= n else None\n\ndef side_from_geometry(h):\n cx = {}\n for r in h.itertuples(index=False):\n ipp = _hdr_vec(getattr(r, 'ImagePositionPatient', None), 3)\n iop = _hdr_vec(getattr(r, 'ImageOrientationPatient', None), 6)\n ps = _hdr_vec(getattr(r, 'PixelSpacing', None), 2)\n rows, cols = (getattr(r, 'Rows', None), getattr(r, 'Columns', None))\n if ipp is None or iop is None or ps is None or (not rows) or (not cols):\n continue\n try:\n c = ipp[:3] + iop[:3] * ps[1] * float(cols) / 2 + iop[3:6] * ps[0] * float(rows) / 2\n except (TypeError, ValueError):\n continue\n cx.setdefault(r.StudyInstanceUID, []).append(float(c[0]))\n out = {}\n for st, xs in cx.items():\n m = float(np.median(xs))\n out[st] = None if abs(m) < LAT_MIN_OFFSET_MM else 'R' if m < 0 else 'L'\n return out\n\ndef side_from_corner_x(h):\n out = {}\n for st, g in h.groupby('StudyInstanceUID'):\n xs = []\n for r in g.itertuples(index=False):\n ipp = _hdr_vec(getattr(r, 'ImagePositionPatient', None), 3)\n if ipp is not None and np.isfinite(ipp).all():\n xs.append(float(ipp[0]))\n if not xs:\n out[st] = None\n continue\n x = float(np.median(xs))\n out[st] = None if abs(x) < LEGACY_LAT_OFFSET_MM else 'R' if x < 0 else 'L'\n return out\n\ndef lat_of(h, tag=''):\n geo = side_from_corner_x(h) if RULES['lat'] == 'corner_x' else side_from_geometry(h)\n d, n_tag, n_geo, n_none, n_disagree = ({}, 0, 0, 0, 0)\n for st, g in h.groupby('StudyInstanceUID'):\n v = [str(x).strip().upper() for x in g['Laterality'].dropna()]\n if RULES['lat'] == 'corner_x' and 'ImageLaterality' in g.columns:\n v += [str(x).strip().upper() for x in g['ImageLaterality'].dropna()]\n v = [x[0] for x in v if x and x[0] in ('L', 'R')]\n side = v[0] if v else None\n if side is not None:\n n_tag += 1\n if geo.get(st) is not None and geo[st] != side:\n n_disagree += 1\n else:\n side = geo.get(st)\n n_geo += side is not None\n n_none += side is None\n d[st] = side\n log(f'{tag}laterality: {n_tag} from the tag, {n_geo} from geometry, {n_none} unresolved; tag and geometry disagree on {n_disagree} ({n_disagree / max(n_tag, 1):.1%} of the tagged)')\n return d\n\ndef probe(item):\n split, study, series, path = item\n row = {'split': split, 'StudyInstanceUID': study, 'SeriesInstanceUID': series, 'dir': path}\n try:\n files = sorted((e.name for e in os.scandir(path) if e.name.endswith('.dcm')))\n row['files'] = files\n row['n_slices'] = len(files)\n if not files:\n return row\n ds = pydicom.dcmread(os.path.join(path, files[len(files) // 2]), stop_before_pixels=True, force=True)\n for t in HDR_TAGS:\n v = getattr(ds, t, None)\n if v is None:\n row[t] = None\n elif isinstance(v, (list, tuple)) or type(v).__name__ == 'MultiValue':\n row[t] = '|'.join((str(x) for x in v))\n else:\n row[t] = str(v)\n except Exception as exc:\n row['err'] = str(exc)[:120]\n return row\n\ndef walk(split):\n base = ROOT / split\n items = []\n if not base.is_dir():\n return pd.DataFrame(columns=['split', 'StudyInstanceUID', 'SeriesInstanceUID', 'dir', 'files', 'n_slices'] + HDR_TAGS)\n for study in os.scandir(base):\n if study.is_dir():\n for series in os.scandir(study.path):\n if series.is_dir():\n items.append((split, study.name, series.name, series.path))\n with ThreadPoolExecutor(max_workers=HDR_THREADS) as pool:\n rows = list(pool.map(probe, items))\n return pd.DataFrame(rows)\n\ndef annotate(df):\n desc = df['SeriesDescription'].fillna('') + ' ' + df['SequenceName'].fillna('')\n desc = desc.str.lower().str.replace(_SEP, ' ', regex=True)\n opts = df['ScanOptions'].fillna('').str.upper().str.split('|')\n opts_fs = opts.apply(lambda ts: any((t.strip() in FATSAT_OPTS for t in ts)))\n df['fatsat'] = desc.str.contains(_FATSAT_RX) | opts_fs\n tr = pd.to_numeric(df['RepetitionTime'], errors='coerce')\n te = pd.to_numeric(df['EchoTime'], errors='coerce')\n gre = df['ScanningSequence'].fillna('').str.upper().str.contains('GR')\n t1, t2, pdw = (desc.str.contains(_T1_RX), desc.str.contains(_T2_RX), desc.str.contains(_PD_RX))\n df['weight'] = np.where(t1 & ~t2 & ~pdw, 'T1', np.where(t2 & ~pdw, 'T2', np.where(pdw, 'PD', np.where(gre, 'GRE', np.where(tr < 800, 'T1', np.where(te > 60, 'T2', np.where(tr >= 800, 'PD', 'UNK')))))))\n df['fluid'] = np.isin(df['weight'], ['PD', 'T2'])\n df['px'] = pd.to_numeric(df['PixelSpacing'].fillna('').str.split('|').str[0].replace('', np.nan), errors='coerce')\n return df\n\ndef pick_slots(series_df, plane_map):\n series_df = series_df.copy()\n series_df['plane'] = series_df['SeriesInstanceUID'].map(plane_map)\n out = {}\n for study, g in series_df.groupby('StudyInstanceUID'):\n chosen = {}\n for name, plane, fluid, fs in SLOTS:\n sel = (g['plane'] == plane) & (g['fatsat'] == fs)\n if fluid is not None:\n sel &= g['fluid'] == fluid\n cand = g[sel]\n if len(cand) == 0 and RULES['slot_fallback'] and (fluid is False):\n cand = g[(g['plane'] == plane) & ~g['fatsat']]\n if len(cand):\n chosen[name] = cand.sort_values('n_slices', ascending=False).iloc[0]\n out[study] = chosen\n return out\nORDER_TAGS = [(32, 50), (32, 55), (32, 19)]\nDECODE_FAILED = []\n\ndef _natural_key(name):\n return tuple((int(x) if x.isdigit() else x.lower() for x in re.split('(\\\\d+)', str(name))))\n\ndef _order_dominant_axis(rec):\n files, d = (rec['files'], rec['dir'])\n rows = []\n for pos, f in enumerate(files):\n ipp = inst = None\n try:\n ds = pydicom.dcmread(os.path.join(d, f), force=True, stop_before_pixels=True, specific_tags=['ImagePositionPatient', 'InstanceNumber'])\n raw = getattr(ds, 'ImagePositionPatient', None)\n if raw is not None and len(raw) >= 3:\n c = np.asarray(raw[:3], dtype=np.float64)\n if np.isfinite(c).all():\n ipp = c\n n = getattr(ds, 'InstanceNumber', None)\n if n is not None:\n inst = float(n)\n except Exception:\n pass\n rows.append((f, ipp, inst, pos))\n placed = [r for r in rows if r[1] is not None]\n need = max(2, int(0.8 * len(rows)))\n if len(placed) >= need:\n xyz = np.stack([r[1] for r in placed])\n axis = int(np.argmax(np.ptp(xyz, axis=0)))\n spare = float(np.nanmedian(xyz[:, axis]))\n rows.sort(key=lambda r: (float(r[1][axis]) if r[1] is not None else spare, r[2] if r[2] is not None else float('inf'), r[3]))\n elif sum((r[2] is not None for r in rows)) >= need:\n rows.sort(key=lambda r: (r[2] if r[2] is not None else float('inf'), r[3]))\n else:\n rows.sort(key=lambda r: _natural_key(r[0]))\n return ([r[0] for r in rows], True)\n\ndef order_slices(rec):\n if RULES['order'] == 'dominant_axis':\n return _order_dominant_axis(rec)\n files, d = (rec['files'], rec['dir'])\n keyed = []\n for f in files:\n k = None\n try:\n ds = pydicom.dcmread(os.path.join(d, f), force=True, stop_before_pixels=True, specific_tags=ORDER_TAGS)\n iop = np.asarray(ds.ImageOrientationPatient, dtype=float)\n ipp = np.asarray(ds.ImagePositionPatient, dtype=float)\n k = float(np.dot(ipp, np.cross(iop[:3], iop[3:])))\n except Exception:\n try:\n k = float(ds.InstanceNumber)\n except Exception:\n k = None\n keyed.append((k, f))\n if any((k is None for k, _ in keyed)):\n return (files, False)\n return ([f for _, f in sorted(keyed, key=lambda t: t[0])], True)\n\ndef read_slot(rec, n_slice=None, out_size=None):\n n_slice = GROUP if n_slice is None else n_slice\n out_size = IMG if out_size is None else out_size\n files, d, px = (rec.get('ordered') or rec['files'], rec['dir'], rec['px'])\n n = len(files)\n if n == 0:\n return None\n lo, hi = (int(SLICE_BAND[0] * (n - 1)), int(SLICE_BAND[1] * (n - 1)))\n idx = np.unique(np.linspace(lo, hi, n_slice).astype(int)) if hi > lo else np.array([n // 2])\n while len(idx) < n_slice:\n idx = np.append(idx, idx[-1])\n planes = []\n for i in idx[:n_slice]:\n try:\n ds = pydicom.dcmread(os.path.join(d, files[int(i)]), force=True)\n a = ds.pixel_array.astype(np.float32)\n sl = float(getattr(ds, 'RescaleSlope', 1) or 1)\n ic = float(getattr(ds, 'RescaleIntercept', 0) or 0)\n a = a * sl + ic\n except Exception:\n a = None\n planes.append(a)\n got = [k for k, p in enumerate(planes) if p is not None]\n if RULES['decode_fill'] == 'zero':\n if not got:\n DECODE_FAILED.append(rec.get('SeriesInstanceUID', d))\n planes = [np.zeros((out_size, out_size), np.float32) if p is None else p for p in planes]\n got = list(range(len(planes)))\n if not got:\n DECODE_FAILED.append(rec.get('SeriesInstanceUID', d))\n return None\n if len(got) < len(planes):\n DECODE_FAILED.append(rec.get('SeriesInstanceUID', d))\n for k, p in enumerate(planes):\n if p is None:\n planes[k] = planes[min(got, key=lambda j: abs(j - k))]\n shp = planes[0].shape\n planes = [p if p.shape == shp else np.zeros(shp, np.float32) for p in planes]\n vol = np.stack(planes)\n if px and np.isfinite(px) and (px > 0):\n want = int(round(CROP_MM / px))\n h, w = shp\n if 16 < want < min(h, w):\n cy, cx = (h // 2, w // 2)\n half = want // 2\n vol = vol[:, max(0, cy - half):cy + half, max(0, cx - half):cx + half]\n lo_v, hi_v = np.percentile(vol, [1, 99])\n vol = np.clip((vol - lo_v) / max(hi_v - lo_v, 1e-06), 0, 1)\n t = torch.from_numpy(np.ascontiguousarray(vol)).unsqueeze(0)\n t = F.interpolate(t, size=(out_size, out_size), mode='bilinear', align_corners=False)\n return (t.squeeze(0) * 255).round().clamp(0, 255).to(torch.uint8)\n\ndef normalise_laterality(img, plane, lat):\n if lat != 'R':\n return img\n if plane in ('Coronal', 'Axial'):\n return torch.flip(img, dims=[-1])\n return torch.flip(img, dims=[0])\nORDER_CACHE = os.environ.get('RSNA_ORDER_CACHE') or None\n\ndef build_cache(slot_map, plane_map, lat_map, tag):\n studies = sorted(slot_map)\n sidx = {s: i for i, s in enumerate(studies)}\n cache = np.zeros((len(studies), N_SLOT, CACHE_SLICES, IMG, IMG), np.uint8)\n mask = np.zeros((len(studies), N_SLOT), np.float32)\n log(f'{tag}: cache {cache.shape} = {cache.nbytes / 1024 ** 3:.1f} GB')\n jobs = [(st, k, plane, slot_map[st][name]) for st in studies for k, (name, plane, _, _) in enumerate(SLOTS) if name in slot_map[st]]\n n_job = len(jobs)\n t_ord = time.time()\n n_slice_total = sum((len(j[3]['files']) for j in jobs))\n log(f'{tag}: ordering {len(jobs)} slot-series ({n_slice_total} slice headers)')\n ok = done = 0\n CHUNK_O = 1024\n seen = {}\n if ORDER_CACHE and Path(ORDER_CACHE).is_file():\n try:\n import json as _json\n seen = _json.loads(Path(ORDER_CACHE).read_text())\n except (OSError, ValueError):\n seen = {}\n hit = 0\n for _, _, _, rec in jobs:\n e = seen.get(rec['SeriesInstanceUID'])\n if e and len(e['files']) == len(rec['files']):\n rec['ordered'] = e['files']\n ok += int(e['good'])\n hit += 1\n jobs = [j for j in jobs if 'ordered' not in j[3]]\n log(f'{tag}: {hit} slot-series ordered from {ORDER_CACHE}, {len(jobs)} to read')\n with ThreadPoolExecutor(max_workers=ORDER_THREADS) as pool:\n for c0 in range(0, len(jobs), CHUNK_O):\n block = jobs[c0:c0 + CHUNK_O]\n for (_, _, _, rec), (files, good) in zip(block, pool.map(lambda j: order_slices(j[3]), block)):\n rec['ordered'] = files\n ok += int(good)\n done += 1\n if ORDER_CACHE:\n seen[rec['SeriesInstanceUID']] = {'files': files, 'good': bool(good)}\n budget = min(ORDER_BUDGET_S, max(60.0, (TIME_BUDGET - (time.time() - T0)) * 0.35))\n if time.time() - t_ord > budget:\n log(f'{tag}: ordering budget spent at {done}/{len(jobs)}; the rest keep file order')\n break\n if ORDER_CACHE and done:\n import json as _json\n _t = Path(ORDER_CACHE).with_suffix('.tmp')\n _t.write_text(_json.dumps(seen))\n _t.replace(Path(ORDER_CACHE))\n log(f'{tag}: ordered {ok}/{n_job} by geometry ({n_job - ok} kept arbitrary) in {time.time() - t_ord:.0f}s')\n jobs = [(st, k, plane, slot_map[st][name]) for st in studies for k, (name, plane, _, _) in enumerate(SLOTS) if name in slot_map[st]]\n log(f'{tag}: decoding {len(jobs)} slot-series')\n n_failed_before = len(DECODE_FAILED)\n CHUNK = 512\n done = 0\n with ThreadPoolExecutor(max_workers=PIX_THREADS) as pool:\n for c0 in range(0, len(jobs), CHUNK):\n block = jobs[c0:c0 + CHUNK]\n for (st, k, plane, _), img in zip(block, pool.map(lambda j: read_slot(j[3], CACHE_SLICES, IMG), block)):\n done += 1\n if img is None:\n continue\n cache[sidx[st], k] = normalise_laterality(img, plane, lat_map.get(st)).numpy()\n mask[sidx[st], k] = 1.0\n if done % 4096 < CHUNK:\n log(f' {tag} {done}/{len(jobs)}')\n if time.time() - T0 > TIME_BUDGET:\n log(f' {tag}: time budget reached during decode')\n break\n n_failed = len(DECODE_FAILED) - n_failed_before\n log(f'{tag}: {int(mask.sum())}/{len(jobs)} slots filled' + (f'; {n_failed} series had a slice that would not decode' if n_failed else ''))\n gc.collect()\n return (studies, cache, mask)\n\nclass SlotHead(nn.Module):\n\n def __init__(self, dim, n_slot, n_out, hidden=256, p=0.2, prior=False):\n super().__init__()\n self.proj = nn.Sequential(nn.LayerNorm(dim), nn.Linear(dim, hidden), nn.GELU())\n self.slot_emb = nn.Parameter(torch.randn(n_slot, hidden) * 0.02)\n self.query = nn.Parameter(torch.randn(n_out, hidden) * 0.02)\n self.drop = nn.Dropout(p)\n self.out = nn.Linear(hidden, n_out)\n self.hidden = hidden\n p_ = torch.zeros(n_out, n_slot)\n if prior and n_slot == len(SLOTS) and (n_out == len(TARGETS)):\n for t, slots in SLOT_PRIOR_TABLE.items():\n if t in TARGETS:\n p_[TARGETS.index(t), list(slots)] = SLOT_PRIOR_STRENGTH\n self.prior = prior\n if prior:\n self.register_buffer('slot_prior', p_)\n\n def forward(self, x, mask):\n h = self.proj(x) + self.slot_emb\n att = torch.einsum('bsh,oh->bos', h, self.query) / self.hidden ** 0.5\n if self.prior:\n att = att + self.slot_prior.unsqueeze(0)\n att = att.masked_fill(mask.unsqueeze(1) < 0.5, -10000.0).softmax(-1)\n ctx = self.drop(torch.einsum('bos,bsh->boh', att, h))\n return (ctx * self.out.weight.unsqueeze(0)).sum(-1) + self.out.bias\n\nclass Model(nn.Module):\n\n def __init__(self, backbone, dim, pool='cls_mean', prior=False):\n super().__init__()\n self.backbone = backbone\n self.pool = pool\n self.head = SlotHead(dim * POOL_PARTS[pool], N_SLOT, len(TARGETS), prior=prior)\n self.register_buffer('mean', torch.tensor([0.485, 0.456, 0.406]).view(1, 3, 1, 1))\n self.register_buffer('std', torch.tensor([0.229, 0.224, 0.225]).view(1, 3, 1, 1))\n\n def forward(self, imgs, mask, img_size=None):\n B, S = imgs.shape[:2]\n x = imgs.reshape(B * S, *imgs.shape[2:]).float().div_(255.0)\n if img_size is not None and img_size != x.shape[-1]:\n x = F.interpolate(x, size=(img_size, img_size), mode='bilinear', align_corners=False)\n x = (x - self.mean) / self.std\n out = self.backbone(pixel_values=x).last_hidden_state\n patch = out[:, 1:]\n parts = [out[:, 0], patch.mean(1)]\n if self.pool == 'cls_mean_focal':\n k = max(1, patch.shape[1] // 8)\n parts.append(patch.topk(k, dim=1).values.mean(1))\n feat = torch.cat(parts, dim=1).reshape(B, S, -1)\n return self.head(feat, mask)\n\ndef build_model(unfreeze_last, source=None, variant='small', pool='cls_mean', prior=False):\n from transformers import AutoModel\n p = source if source is not None else find_dinov2(variant)\n if p is None:\n raise FileNotFoundError('DINOv2 weights not attached')\n bb = AutoModel.from_pretrained(str(p))\n n_layer = len(bb.encoder.layer)\n for prm in bb.parameters():\n prm.requires_grad = False\n for blk in bb.encoder.layer[max(0, n_layer - unfreeze_last):]:\n for prm in blk.parameters():\n prm.requires_grad = True\n for prm in bb.layernorm.parameters():\n prm.requires_grad = True\n dim = bb.config.hidden_size\n trainable = sum((p.numel() for p in bb.parameters() if p.requires_grad))\n log(f'backbone: {n_layer} blocks, last {unfreeze_last} trainable ({trainable / 1000000.0:.1f}M params), feature dim {dim * POOL_PARTS[pool]}')\n return Model(bb, dim, pool=pool, prior=prior)\nFINGERPRINT_TOL = 0.002\n\ndef fingerprint(model, dev, img_size, n_slot=None, group=None, seed=None):\n n_slot = N_SLOT if n_slot is None else n_slot\n group = GROUP if group is None else group\n seed = SEED if seed is None else seed\n g = torch.Generator().manual_seed(seed)\n imgs = torch.randint(0, 256, (2, n_slot, group, img_size, img_size), generator=g, dtype=torch.uint8).to(dev)\n mask = torch.ones(2, n_slot, device=dev)\n mask[1, -1] = 0.0\n was_training = model.training\n model.eval()\n with torch.no_grad():\n out = model(imgs, mask, img_size).float().cpu().numpy()\n if was_training:\n model.train()\n return out\n\ndef check_fingerprint(model, dev, img_size, expected, tol=FINGERPRINT_TOL, tag=''):\n got = fingerprint(model, dev, img_size)\n exp = np.asarray(expected, np.float32)\n if got.shape != exp.shape:\n raise WeightsError(f'{tag}fingerprint shape {got.shape} != stored {exp.shape}: the architecture is not the one these weights were fitted to')\n d = float(np.abs(got - exp).max())\n if d > tol:\n raise WeightsError(f'{tag}fingerprint differs by {d:.4g} (tolerance {tol:g}). The weights load but do not compute what they computed when fitted - preprocessing, resolution or architecture has moved between the two runs.')\n log(f'{tag}fingerprint matches within {d:.2g}')\n return d\n\nclass WeightsError(RuntimeError):\n pass\nTTA_OVERLAP = True\nTTA_POOL = 'prob'\nPUBLIC_FRONTIER_TARGET_POOL = {'Fracture': 'max', 'Contusion': 'max', 'Medial Meniscus': 'max', 'Lateral Meniscus': 'max', 'ACL': 'top2', 'MCL': 'top2', \"Baker's\": 'max'}\nTTA_TARGET_POOL = {**PUBLIC_FRONTIER_TARGET_POOL, 'Synovitis': 'original_mean'}\nLEGACY_FOLD_SOFTPOOL_BETA = {'ACL': 6.0, 'MCL': 6.0, 'Medial Meniscus': 8.0, 'Lateral Meniscus': 8.0, \"Baker's\": 8.0, 'Contusion': 8.0, 'Fracture': 10.0}\nLEGACY_FOLD_SOFTPOOL_ALPHA = {'ACL': 0.2, 'MCL': 0.2, 'Medial Meniscus': 0.25, 'Lateral Meniscus': 0.25, \"Baker's\": 0.2, 'Contusion': 0.2, 'Fracture': 0.15}\n\ndef window_starts(n_slice, group, overlap=None):\n overlap = TTA_OVERLAP if overlap is None else overlap\n if overlap and n_slice >= group:\n return list(range(n_slice - group + 1))\n return [g * group for g in range(max(n_slice // group, 1))]\n\ndef apply_target_window_pool(values, probs, logits, original_probs, mapping, target_idx):\n for target, mode in mapping.items():\n j = target_idx[target]\n if mode == 'max':\n values[:, j] = probs[:, :, j].max(0).values\n elif mode == 'mean':\n values[:, j] = probs[:, :, j].mean(0)\n elif mode == 'logit_mean':\n values[:, j] = torch.sigmoid(logits[:, :, j].mean(0))\n elif mode == 'original_mean':\n values[:, j] = original_probs[:, :, j].mean(0)\n elif mode in ('top2', 'top3'):\n k = min(int(mode[3:]), probs.shape[0])\n values[:, j] = probs[:, :, j].topk(k, dim=0).values.mean(0)\n else:\n raise ValueError(f'unknown TTA pooling mode for {target}: {mode}')\n return values\n\ndef legacy_fold_soft_window_pool(original_probs, target_idx):\n values = original_probs.mean(0).clone()\n for target, beta in LEGACY_FOLD_SOFTPOOL_BETA.items():\n j = target_idx[target]\n x = original_probs[:, :, j]\n weight = torch.softmax(float(beta) * x, dim=0)\n values[:, j] = (weight * x).sum(0)\n return values\n\n@torch.no_grad()\ndef predict_member(model, cache, mask, idx, dev, img_size, group=None, pool=None, starts=None, jitter=False, jitter_seed=SEED, return_public_frontier=False):\n group = GROUP if group is None else group\n pool = TTA_POOL if pool is None else pool\n starts = window_starts(cache.shape[2], group) if starts is None else list(starts)\n if not starts:\n raise ValueError('predict_member was given no windows to average over')\n target_idx = {t: j for j, t in enumerate(TARGETS)}\n unknown = (set(TTA_TARGET_POOL) | set(PUBLIC_FRONTIER_TARGET_POOL)) - set(target_idx)\n if unknown:\n raise ValueError(f'unknown target(s) in TTA_TARGET_POOL: {unknown}')\n jitter_gen = torch.Generator(device=dev)\n jitter_gen.manual_seed(int(jitter_seed) % (2 ** 63 - 1))\n model.eval()\n out, public_frontier_out, public_soft_out = ([], [], [])\n for b in range(0, len(idx), EVAL_BATCH):\n sel = idx[b:b + EVAL_BATCH]\n m = torch.from_numpy(mask[sel]).to(dev)\n win_probs, win_logits, win_original_probs = ([], [], [])\n for st in starts:\n rows = torch.from_numpy(np.ascontiguousarray(cache[sel, :, st:st + group])).to(dev)\n views = [rows] + ([augment(rows, generator=jitter_gen)] if jitter else [])\n view_probs, view_logits = ([], [])\n for view in views:\n with torch.autocast('cuda', enabled=dev.type == 'cuda'):\n z = model(view, m, img_size).float()\n view_logits.append(z)\n view_probs.append(torch.sigmoid(z))\n win_logits.append(torch.stack(view_logits).mean(0))\n win_probs.append(torch.stack(view_probs).mean(0))\n win_original_probs.append(view_probs[0])\n probs = torch.stack(win_probs)\n logits = torch.stack(win_logits)\n original_probs = torch.stack(win_original_probs)\n v = torch.sigmoid(logits.mean(0)) if pool == 'logit' else probs.mean(0)\n v = apply_target_window_pool(v, probs, logits, original_probs, TTA_TARGET_POOL, target_idx)\n out.append(v.cpu().numpy())\n if return_public_frontier:\n public_v = apply_target_window_pool(original_probs.mean(0), original_probs, logits, original_probs, PUBLIC_FRONTIER_TARGET_POOL, target_idx)\n public_frontier_out.append(public_v.cpu().numpy())\n public_soft = legacy_fold_soft_window_pool(original_probs, target_idx)\n public_soft_out.append(public_soft.cpu().numpy())\n primary = np.concatenate(out) if out else np.zeros((0, len(TARGETS)), np.float32)\n if not return_public_frontier:\n return primary\n public_frontier = np.concatenate(public_frontier_out) if public_frontier_out else np.zeros((0, len(TARGETS)), np.float32)\n public_soft = np.concatenate(public_soft_out) if public_soft_out else np.zeros((0, len(TARGETS)), np.float32)\n return (primary, public_frontier, public_soft)\nBUILD_LOCK = threading.Lock()\nSTATE_LOCK = threading.Lock()\n\ndef _run_member(path, m, dev, Cte, Mte, idx, starts, jitter):\n t0 = time.time()\n with BUILD_LOCK:\n if 'state' in m:\n state, fp = (m['state'], None)\n else:\n ck = torch.load(Path(path) / m['file'], map_location='cpu', weights_only=False)\n state, fp = (ck['model'], ck.get('fingerprint'))\n model = build_model(int(m['config']['unfreeze_last']), variant=m['config']['variant'], pool=m['config'].get('pool', 'cls_mean'), prior=bool(m['config'].get('prior', False))).to(dev)\n model.load_state_dict(state)\n if fp is not None:\n check_fingerprint(model, dev, IMG, fp, tag=f\"{m['id']}: \")\n else:\n log(f\" {m['id']}: no stored fingerprint (legacy bundle) -- accepted at reduced weight\")\n t_ready = time.time()\n jitter_seed = SEED + int(hashlib.sha256(str(m['id']).encode()).hexdigest()[:8], 16)\n public_member = 'state' not in m\n predicted = predict_member(model, Cte, Mte, idx, dev, IMG, starts=starts, jitter=jitter, jitter_seed=jitter_seed, return_public_frontier=public_member)\n if public_member:\n p, public_p, public_soft = predicted\n else:\n p, public_p, public_soft = (predicted, None, None)\n t_done = time.time()\n del model, state\n gc.collect()\n if dev.type == 'cuda':\n with torch.cuda.device(dev):\n torch.cuda.empty_cache()\n passes = len(starts) * (2 if jitter else 1)\n return (p, public_p, public_soft, (t_ready - t0, (t_done - t_ready) / max(passes, 1)))\n\ndef _combine(per_member):\n all_ids = sorted({s for m in per_member for s in m['ids']})\n pos = {s: i for i, s in enumerate(all_ids)}\n acc = np.zeros((len(all_ids), len(TARGETS)), np.float64)\n tot = np.zeros(len(TARGETS), np.float64)\n for m in per_member:\n target_weight = m.get('target_weight')\n w = np.asarray(target_weight if target_weight is not None else [float(m.get('weight', 1.0))] * len(TARGETS), dtype=np.float64)\n if w.shape != (len(TARGETS),) or np.any(w < 0):\n raise ValueError(f\"invalid target weights for {m.get('id')}: {w}\")\n r = pd.DataFrame(m['pred']).rank(pct=True).to_numpy()\n acc[[pos[s] for s in m['ids']]] += r * w[None, :]\n tot += w\n if np.any(tot <= 0):\n raise ValueError(f'at least one target has no ensemble vote: {tot}')\n return (all_ids, acc / tot[None, :])\n\ndef combine_public_members_by_fold(per_member, pred_key='pred'):\n all_ids = sorted({study for member in per_member for study in member['ids']})\n position = {study: i for i, study in enumerate(all_ids)}\n groups = {}\n for i, member in enumerate(per_member):\n fold = member.get('fold')\n key = f'fold_{fold}' if fold is not None else f'member_{i}'\n groups.setdefault(key, []).append(member)\n fold_ranks, diagnostics = ([], [])\n for key, members_in_fold in sorted(groups.items()):\n matrices = []\n for member in members_in_fold:\n values = np.full((len(all_ids), len(TARGETS)), np.nan, np.float64)\n values[[position[study] for study in member['ids']]] = np.asarray(member[pred_key], np.float64)\n if np.isnan(values).any():\n raise WeightsError(f\"{member.get('id')}: incomplete {pred_key} coverage\")\n matrices.append(values)\n raw_fold_mean = np.mean(matrices, axis=0)\n fold_ranks.append(pd.DataFrame(raw_fold_mean).rank(method='average', pct=True).to_numpy(np.float64))\n diagnostics.append({'ensemble_group': key, 'members': len(members_in_fold)})\n if len(fold_ranks) != 5:\n raise WeightsError(f'legacy branch requires five folds, found {len(fold_ranks)}')\n return (all_ids, np.mean(fold_ranks, axis=0), pd.DataFrame(diagnostics))\n\ndef blend_legacy_frontier_and_soft(frontier_rank, soft_rank):\n output = np.asarray(frontier_rank, np.float64).copy()\n for j, target in enumerate(TARGETS):\n alpha = float(LEGACY_FOLD_SOFTPOOL_ALPHA.get(target, 0.0))\n if alpha:\n output[:, j] = (1.0 - alpha) * frontier_rank[:, j] + alpha * soft_rank[:, j]\n return output\n\ndef infer_from_package(path, dev=None):\n man = json.loads((Path(path) / 'manifest.json').read_text())\n members = man['members']\n log(f'weights package: {len(members)} member(s) from {path}; {len(DEVS)} device(s)')\n test_df = pd.read_csv(ROOT / 'test.csv')\n test_series = pd.read_csv(ROOT / 'test_series.csv')\n plane_map = dict(zip(test_series['SeriesInstanceUID'], test_series['Anatomical_Plane']))\n hte = annotate(walk('test_series'))\n log(f'test header pass: {len(hte)} series')\n groups = {}\n for m in members:\n groups.setdefault(m['pixel_group'], []).append(m)\n groups.update(legacy_group_members())\n per_member, public_frontier_members = ([], [])\n est = {'fixed': None, 'win': None}\n\n def bank(m, ids, pred, starts, jitter, public_pred=None, public_soft=None):\n if float(np.std(pred)) < 1e-09:\n log(f\" {m['id']}: degenerate predictions; not banked\")\n return\n with STATE_LOCK:\n per_member.append({'id': m['id'], 'fold': m.get('fold'), 'ids': ids, 'pred': pred, 'weight': m.get('weight', 1.0), 'target_weight': m.get('target_weight'), 'holdout': m.get('holdout')})\n if public_pred is not None and len(starts) == len(starts_full):\n if float(np.std(public_pred)) < 1e-09:\n raise WeightsError(f\"{m['id']}: degenerate public-frontier prediction\")\n public_frontier_members.append({'id': m['id'], 'fold': m.get('fold'), 'ids': ids, 'pred': public_pred, 'soft_pred': public_soft})\n elif public_pred is not None:\n log(f\" {m['id']}: public-frontier vote omitted because only {len(starts)} / {len(starts_full)} windows completed\")\n all_ids, acc = _combine(per_member)\n write_submission(acc, all_ids, test_df, 'submission.csv')\n log(f\" banked {m['id']} fold {m.get('fold', '?')} ({len(starts)} window(s){(', jitter' if jitter else '')}); submission.csv = weighted rank mean of {len(per_member)} member(s)\")\n for gi, (key, gm) in enumerate(groups.items(), 1):\n cfg = json.loads(key)\n adopt_config_globals(cfg)\n log(f\"decode group {gi}/{len(groups)}: {cfg['img']}px x {cfg['slices']} slices, crop {cfg['crop_mm']} mm -> {len(gm)} member(s)\")\n st_te, Cte, Mte = build_cache(pick_slots(hte, plane_map), plane_map, lat_of(hte, 'test '), f'test g{gi}')\n idx = np.arange(len(st_te))\n starts_full = window_starts(Cte.shape[2], GROUP)\n pending = sorted(gm, key=lambda m: -(m.get('holdout') or 0))\n left_after = sum((len(g) for j, (_, g) in enumerate(groups.items(), 1) if j > gi))\n\n def pop_next():\n with STATE_LOCK:\n if not pending:\n return (None, None, False)\n left = TIME_BUDGET - (time.time() - T0)\n remaining = len(pending) + left_after\n slots_left = -(-remaining // len(DEVS))\n starts, jit = (starts_full, False)\n if est['fixed'] is not None and est['win'] is not None:\n afford = max(left * 0.9, 0.0)\n room = afford / max(slots_left, 1)\n if est['fixed'] + est['win'] > room:\n log(f' {left / 60:.0f} min left: surrendering {len(pending)} member(s); not one more fits')\n pending.clear()\n return (None, None, False)\n jit = est['fixed'] + 2 * len(starts_full) * est['win'] <= room * 0.6\n per_win = est['win'] * (2 if jit else 1)\n n_win = int((room - est['fixed']) / per_win) if per_win > 0 else len(starts_full)\n n_win = max(1, min(len(starts_full), n_win))\n if n_win < len(starts_full):\n mid = (len(starts_full) - n_win) // 2\n starts = starts_full[mid:mid + n_win]\n return (pending.pop(0), starts, jit)\n\n def worker(dev):\n others = [d for d in DEVS if d is not dev]\n while True:\n m, starts, jit = pop_next()\n if m is None:\n return\n for attempt, d in enumerate([dev] + others[:1]):\n try:\n p, public_p, public_soft, (fs, ws) = _run_member(path, m, d, Cte, Mte, idx, starts, jit)\n with STATE_LOCK:\n est['fixed'], est['win'] = (fs, ws)\n bank(m, st_te, p, starts, jit, public_p, public_soft)\n break\n except Exception as exc:\n log(f\" MEMBER {m['id']} failed on {d} ({type(exc).__name__}: {exc}); \" + ('retrying on peer device' if attempt == 0 and others else 'dropped -- costs one vote, not the run'))\n if d.type == 'cuda':\n with torch.cuda.device(d):\n torch.cuda.empty_cache()\n threads = [threading.Thread(target=worker, args=(d,)) for d in DEVS]\n for t in threads:\n t.start()\n for t in threads:\n t.join()\n del Cte, Mte\n gc.collect()\n if not per_member:\n raise WeightsError('no member produced predictions; submission stays at 0.5')\n all_ids, acc = _combine(per_member)\n sub = write_submission(acc, all_ids, test_df, 'submission.csv')\n log(f'final submission.csv = weighted rank mean of {len(per_member)} member(s); {sub.shape}; nulls {int(sub[TARGETS].isna().sum().sum())}')\n if len(public_frontier_members) == len(members):\n frontier_ids, frontier_acc = _combine(public_frontier_members)\n frontier_sub = write_submission(frontier_acc, frontier_ids, test_df, 'submission_public_0899.csv')\n log(f'submission_public_0899.csv = exact no-jitter public-frontier rank mean of {len(public_frontier_members)} member(s); {frontier_sub.shape}; nulls {int(frontier_sub[TARGETS].isna().sum().sum())}')\n fold_ids, fold_frontier, fold_diagnostics = combine_public_members_by_fold(public_frontier_members, 'pred')\n soft_ids, fold_soft, _ = combine_public_members_by_fold(public_frontier_members, 'soft_pred')\n if fold_ids != soft_ids:\n raise WeightsError('legacy hard/soft study order mismatch')\n legacy_prediction = blend_legacy_frontier_and_soft(fold_frontier, fold_soft)\n legacy_sub = write_submission(legacy_prediction, fold_ids, test_df, 'submission_legacy_fold_blend.csv')\n fold_diagnostics.to_csv('legacy_fold_diagnostics.csv', index=False)\n log(f'legacy DINO aggregation written from five folds; {legacy_sub.shape}')\n else:\n log(f'public-frontier fallback not emitted: {len(public_frontier_members)} / {len(members)} required public members completed')\n return sub\n\ndef adopt_config_globals(cfg):\n global IMG, CACHE_IMG, GROUP, CACHE_SLICES, N_GROUP, CROP_MM, SLICE_BAND, RULES\n CACHE_IMG = IMG = int(cfg['img'])\n GROUP = int(cfg['group'])\n CACHE_SLICES = int(cfg['slices'])\n N_GROUP = max(CACHE_SLICES // GROUP, 1)\n CROP_MM = float(cfg['crop_mm'])\n SLICE_BAND = tuple((float(x) for x in cfg['band']))\n rules = cfg.get('rules') or RULES_NATIVE\n unknown = {k: v for k, v in rules.items() if k not in RULES_NATIVE or v not in (RULES_NATIVE[k], RULES_LEGACY[k])}\n if unknown:\n raise WeightsError(f'the members record pixel rules this pipeline cannot reproduce: {unknown}')\n RULES = {**RULES_NATIVE, **rules}\n if [s[0] for s in SLOTS] != list(cfg['slots']):\n raise WeightsError(f\"the members were fitted on slots {cfg['slots']} and this pipeline defines {[s[0] for s in SLOTS]}; a weight would be read against the wrong slot\")\n\ndef augment(imgs, generator=None):\n lead = imgs.shape[:-3]\n x = imgs.reshape(-1, *imgs.shape[-3:]).float()\n n, dev = (x.shape[0], x.device)\n rot = (torch.rand(n, device=dev, generator=generator) - 0.5) * 2 * (AUG_ROT_DEG * np.pi / 180)\n sc = 1.0 + torch.rand(n, device=dev, generator=generator) * AUG_SCALE\n tx = (torch.rand(n, device=dev, generator=generator) - 0.5) * 2 * AUG_SHIFT\n ty = (torch.rand(n, device=dev, generator=generator) - 0.5) * 2 * AUG_SHIFT\n cos, sin = (torch.cos(rot) / sc, torch.sin(rot) / sc)\n theta = torch.zeros(n, 2, 3, device=dev, dtype=torch.float32)\n theta[:, 0, 0], theta[:, 0, 1], theta[:, 0, 2] = (cos, -sin, tx)\n theta[:, 1, 0], theta[:, 1, 1], theta[:, 1, 2] = (sin, cos, ty)\n grid = F.affine_grid(theta, x.shape, align_corners=False)\n x = F.grid_sample(x, grid, mode='bilinear', padding_mode='border', align_corners=False)\n scale = 1.0 + (torch.rand(n, 1, 1, 1, device=dev, generator=generator) - 0.5) * 2 * AUG_INTENSITY\n x = (x * scale).clamp(0, 255)\n return x.reshape(*lead, *x.shape[-3:]).to(imgs.dtype)\n\ndef write_submission(pred, studies, test_df, path):\n sub = pd.DataFrame(pd.DataFrame(pred).rank(pct=True).values, columns=TARGETS)\n sub.insert(0, 'StudyInstanceUID', studies)\n sub = test_df[['StudyInstanceUID']].merge(sub, on='StudyInstanceUID', how='left')\n sub[TARGETS] = sub[TARGETS].fillna(0.5)\n sub.to_csv(path, index=False)\n return sub\n\ndef find_dinov2(variant='small'):\n if not (DINO / 'config.json').is_file():\n raise FileNotFoundError(DINO)\n return DINO\n\ndef legacy_group_members():\n return {}\n\ndef run_dinov2():\n path = ASSET / 'rsna-knee-weights'\n infer_from_package(path, DEVS[0])\n public = Path('/kaggle/working/submission_public_0899.csv')\n if not public.is_file():\n raise RuntimeError('public DINOv2 frontier was not produced')\n public.replace('/kaggle/working/submission.csv')\n for name in ('submission_legacy_fold_blend.csv', 'legacy_fold_diagnostics.csv'):\n candidate = Path('/kaggle/working') / name\n if candidate.is_file():\n candidate.unlink()\nrun_dinov2()\n","metadata":{"execution":{"iopub.execute_input":"2026-08-25T08:12:58.978806Z","iopub.status.busy":"2026-08-25T08:12:58.978597Z","iopub.status.idle":"2026-08-25T08:14:07.339642Z","shell.execute_reply":"2026-08-25T08:14:07.339004Z"},"papermill":{"duration":68.368841,"end_time":"2026-08-25T08:14:07.341151+00:00","exception":false,"start_time":"2026-08-25T08:12:58.97231+00:00","status":"completed"},"tags":[]},"outputs":[],"execution_count":null},{"cell_type":"code","source":"\"\"\"Notebook cell source: invoke the bundled fullfit0033 cached inference runtime.\n\nThe preceding capture shim owns the only large array references. This cell\nselects exactly one cache/mask pair, restores ``numpy.zeros`` before loading\nany new code, then calls the hash-bound runtime from the private bundle.\nFailures are intentionally propagated: there is no prediction fallback.\n\"\"\"\n\nimport builtins as _p33_builtins\nimport gc as _p33_gc\nimport hashlib as _p33_hashlib\nimport importlib.util as _p33_importlib_util\nimport json as _p33_json\nimport os as _p33_os\nfrom pathlib import Path as _P33Path\n\n\n_P33_CAPTURE_KEY = \"_public0033_cache_capture_v1\"\n_P33_BUNDLE_SCHEMA = \"public0033_meniscus10_bundle_v1\"\n_P33_RUNTIME_ENTRYPOINT = \"run_cached_inference(cache, mask, studies, output_paths)\"\n_p33_work_dir = _P33Path(_p33_os.environ.get(\"PUBLIC0033_WORK_DIR\", \"/kaggle/working\"))\n_p33_output_paths = {\n \"bag_raw_csv\": str(_p33_work_dir / \"public0033_bag_raw.csv\"),\n \"receipt_json\": str(_p33_work_dir / \"public0033_cached_inference_receipt.json\"),\n \"work_dir\": str(_p33_work_dir),\n}\n\n\ndef _p33_sha256_file(_p33_path):\n _p33_digest = _p33_hashlib.sha256()\n with _p33_path.open(\"rb\") as _p33_handle:\n for _p33_block in iter(lambda: _p33_handle.read(8 << 20), b\"\"):\n _p33_digest.update(_p33_block)\n return _p33_digest.hexdigest()\n\n\ndef _p33_restore_zeros(_p33_state):\n _p33_numpy = _p33_state.get(\"numpy_module\")\n _p33_original = _p33_state.get(\"original_zeros\")\n if _p33_numpy is None or _p33_original is None:\n raise RuntimeError(\"public0033: cache capture lacks original numpy.zeros\")\n _p33_numpy.zeros = _p33_original\n\n\ndef _p33_capture_pair(_p33_state):\n _p33_caches = _p33_state.get(\"cache_candidates\")\n _p33_masks = _p33_state.get(\"mask_candidates\")\n if not isinstance(_p33_caches, list) or not isinstance(_p33_masks, list):\n raise RuntimeError(\"public0033: cache capture candidate registry is invalid\")\n _p33_pairs = []\n for _p33_cache_record in _p33_caches:\n for _p33_mask_record in _p33_masks:\n _p33_studies = _p33_cache_record.get(\"studies\")\n if _p33_studies != _p33_mask_record.get(\"studies\"):\n continue\n _p33_cache = _p33_cache_record.get(\"cache\")\n _p33_mask = _p33_mask_record.get(\"mask\")\n if _p33_cache is None or _p33_mask is None:\n continue\n _p33_expected_n = len(_p33_studies)\n if (\n tuple(_p33_cache.shape) == (_p33_expected_n, 6, 12, 336, 336)\n and str(_p33_cache.dtype) == \"uint8\"\n and tuple(_p33_mask.shape) == (_p33_expected_n, 6)\n and str(_p33_mask.dtype) == \"float32\"\n ):\n _p33_pairs.append((_p33_cache, _p33_mask, _p33_studies))\n if len(_p33_pairs) != 1:\n raise RuntimeError(\n \"public0033: expected exactly one captured 6x12x336 cache/mask pair, \"\n f\"got {len(_p33_pairs)}\"\n )\n return _p33_pairs[0]\n\n\ndef _p33_find_bundle():\n _p33_input = _P33Path(_p33_os.environ.get(\"PUBLIC0033_INPUT_ROOT\", \"/kaggle/input\"))\n if not _p33_input.is_dir():\n raise RuntimeError(\"public0033: /kaggle/input is unavailable\")\n _p33_matches = []\n # Kaggle currently mounts datasets below\n # ``/kaggle/input/datasets/