{"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"name":"python","version":"3.11.13"},"rsna_optimization":{"revision":"v10-proven-20-member-ensemble","source":"pilkwang/rsna-knee-baseline-v1","official_source_score":0.891}},"nbformat_minor":4,"nbformat":4,"cells":[{"id":"03f9de7e-0011-4ff2-a362-7d02af2c43a2","cell_type":"markdown","source":"# Twelve findings, fifty-eight labels, four thousand reports\n\nA knee MRI study here has to be given twelve probabilities — anterior cruciate and medial\ncollateral ligament injury, medial and lateral meniscal tear, osteoarthritis in each of\nthe three compartments, effusion, synovitis, Baker's cyst, bone contusion, fracture — and\nthe score is the unweighted mean of the twelve ROC AUCs.\n\nThe decisive fact about this dataset is not in the images. It is in `train.csv`:\n\n| | studies | carry the twelve labels | carry a radiology report |\n|---|---|---|---|\n| train | 4 407 | **58** | 4 407 |\n| test | — | — | **none — there is no `Report` column** |\n\nFifty-eight labelled studies cannot train an imaging model, and the reports that could\nsupply the missing four thousand are unavailable at prediction time. So the pipeline is\nforced into one shape: **read the reports into targets, fit a pure imaging model against\nthem, and throw the text away.** Everything downstream — which slices to decode, how large\nto make them, which encoder to adapt — is bounded by how well that first step is done.\n\nThis notebook is organised in the order the decisions constrain each other.\n\n1. What the score rewards, and what that removes from the design.\n2. Where the targets come from — a nine-language report reader, and the two ways to tell\n   whether it works when only 58 studies can check it.\n3. What the scanner recorded — recovering the acquisition protocol from DICOM headers.\n4. Geometry — slice order, physical scale, and which knee was scanned.\n5. Reading the pixels once.\n6. Turning six views into twelve decisions.\n7. Validating without fooling yourself.\n8. Three arms, combined the only way the metric permits.\n\nEvery analytical number below is computed by the cell above it. Official Kaggle results in\nthe scorecard are explicitly labelled as external leaderboard evidence.","metadata":{}},{"cell_type":"markdown","metadata":{},"source":"> **Highest verified public score for this notebook: 0.899.** Version 46 is the\n> completed Kaggle row that establishes that result; its submitted artifact was\n> bitwise matched to the audited specialist ensemble. Version 49 tested an independent\n> five-fold EfficientNet-B3 branch and completed at 0.898. Later guarded candidates are\n> labelled pending until Kaggle returns an official score.\n\n| reproducible public run | score | evidence or inference change |\n|---|---:|---|\n| Pilkwang V15 public anchor | 0.891 | original public 20-member ensemble |\n| This notebook V14 | **0.894** | Fracture max + OOF-selected target-wise rank blend |\n| This notebook V46 | **0.899** | exact audited specialist artifact; completed Kaggle row |\n| This notebook V49 | **0.898** | guarded five-fold EfficientNet-B3 rank contribution |\n| external Yash Bishnoi reference | **0.903** | inspected five-fold B3 MIL recipe; not this notebook's score |\n\nThe scorecard separates completed official rows from local validation and from other\nauthors' public references. The cells below preserve that boundary: OOF diagnostics can\nselect or reject an arm, but only a completed Kaggle submission can change the verified\npublic score above.\n\nIf this evidence-first walkthrough saves you time, an upvote helps other competitors\nfind the reproducible result.\n"},{"id":"41ff0303-8740-4afe-abcd-079352f2918c","cell_type":"markdown","source":"## 1. What the score rewards\n\n$$\\text{Score} \\;=\\; \\frac{1}{12}\\sum_{i=0}^{11} \\mathrm{AUC}_i$$\n\nThree consequences follow directly, and each one removes a design choice rather than\nadding one.\n\n**Only order matters.** $\\mathrm{AUC}_i$ is invariant under any strictly increasing map of\nthe scores for label $i$. Calibration is worth nothing here, and so is any fixed\nthreshold. It also settles how to combine models: averaging raw probabilities lets\nwhichever arm happens to be most confident dominate, while averaging *ranks* combines the\nonly information the metric reads. Every combination in §8 is a rank mean.\n\n**Every label costs the same.** Write $M$ for the mean AUC a good model reaches. A label\nleft at chance contributes $0.5$ instead of roughly $M$, forfeiting\n\n$$\\frac{M-0.5}{12}$$\n\nno matter how well the other eleven do. At $M=0.85$ that is $0.029$ — larger than the gap\nbetween neighbouring places on a mature leaderboard. So a rare finding deserves *more*\nattention than a common one, because a rare finding is where a model most easily ends up\nat chance. That is why §2 spends its effort on the findings the reports mention least,\nnot on the ones they mention most.\n\n**Prevalence drift is survivable; thresholds are not.** AUC is, in expectation, invariant\nto the positive rate, and the data description warns that prevalence need not match across\nthe training, public and private sets. That would be fatal for an accuracy-like metric. It\nis not fatal here — but it does mean one cut has to be named and watched. The report-derived\ntargets below are graded, not binary; to compute an AUC against them at all they are\nbinarised at their midpoint for validation only. That cut chooses epochs and arms. It never\nreaches a submitted score.","metadata":{}},{"id":"69c3aefd-57b0-4c00-aaef-32070174c237","cell_type":"markdown","source":"## 2. Where the targets come from\n\nEvery training study carries the radiology report written when it was read, and the data\ndescription invites deriving labels from it. The structural fact that decides how is in the\nschemas rather than in the prose: `train.csv` has a `Report` column and `test.csv` does\nnot. Text is available when fitting and absent when predicting. That rules out a fusion\nmodel with a text branch — at inference it would have nothing to read — and leaves the\nreports doing two jobs:\n\n1. supplying the training targets, and\n2. saying how confidently each one could be read, which becomes a per-finding sample\n   weight.\n\nBoth matter. A study whose report never mentions synovitis should pull on the synovitis\nhead far more weakly than one that names it, and a loss that cannot express that trains\nevery silence as a confident negative.\n\n### Nine languages, one lexicon\n\nThe corpus is written in nine languages across three scripts. The extractor is built here\nin-line rather than attached as a file, for two reasons: it runs over the whole corpus in\nseconds, so there is nothing to save by precomputing; and a notebook that ships its own\nlabels cannot quietly train against a stale copy of them.\n\n**No language is identified.** Every cue list below carries all nine at once and each\nclause is tested against the union. Routing first would mean committing to a guess before\nany evidence is read, and the cheap guess fails badly here — `la` is as common in Spanish\nas in French, and whichever substring test runs first swallows both. Pooling costs little\nin exchange: Greek and Cyrillic cues cannot collide with Latin-script ones at all, and\namong the Latin-script languages the vocabulary of interest is close enough that a shared\ncue is usually right. The price is paid in coverage instead — a phrasing no listed language\ncontributes stays unmatched — and coverage is the thing §2.2 measures.\n\n**Normalise, then unwrap, then segment, then scope.** Case, diacritics and separators are\nfolded first, which also repairs a real codepoint problem: many Greek reports spell mu with\nthe MICRO SIGN `U+00B5` rather than `U+03BC`, and NFKD maps one onto the other. Then the\nlines are unwrapped — see below — then split into clauses, with a heading line attached to\nthe value beneath it, because a report that reads `Fractures :` and then `Aucune.` states\none thing across two lines.\n\n**Assertion, negation, hedge.** Negation is not an edge case here. For several findings\n*most* mentions are negative, because a report lists what was checked and found intact.\nExplicit normality counts as absence — *ligamentos cruzados y colaterales dentro de límites\nnormales* is evidence of absence, not absence of evidence — except where a tear or a high\ngrade is named in the same breath.\n\n**Grade, do not threshold.** §1 established that only order is read. So a mention is scored\nby its emphasis — trace, unqualified, marked — and never binarised.","metadata":{}},{"id":"b3e9a8b5-bc1f-4b1e-bee7-71749f4a65d3","cell_type":"code","source":"\"\"\"Report -> twelve graded targets, in nine languages.\n\nv2. Differences from the public lexicon are all coverage: the compartment scoping for\nosteoarthritis, the pathology vocabulary for cartilage, an asserted-negative path for\nfindings a report explicitly clears, and a backoff for synovitis, which most reports\nnever name.\n\"\"\"\nfrom __future__ import annotations\n\nimport re\nimport unicodedata\n\nTARGETS = [\n    \"ACL\", \"MCL\", \"Medial Meniscus\", \"Lateral Meniscus\",\n    \"Medial OA\", \"Lateral OA\", \"PF OA\", \"Effusion\",\n    \"Synovitis\", \"Baker's\", \"Contusion\", \"Fracture\",\n]\n\n_PRE = str.maketrans({\n    \"ı\": \"i\", \"İ\": \"i\", \"I\": \"i\", \"ß\": \"ss\", \"đ\": \"d\", \"Đ\": \"d\",\n    \"ø\": \"o\", \"Ø\": \"o\", \"æ\": \"ae\", \"Æ\": \"ae\",\n})\n\n\ndef normalize(text: str) -> str:\n    if not isinstance(text, str):\n        return \"\"\n    text = text.translate(_PRE).lower()\n    text = unicodedata.normalize(\"NFKD\", text)\n    text = \"\".join(ch for ch in text if not unicodedata.combining(ch))\n    text = text.replace(\"­\", \"\")\n    text = re.sub(r\"[_\\-/\\\\]+\", \" \", text)\n    text = re.sub(r\"[ \\t]+\", \" \", text)\n    return text\n\n\n_SENT_SPLIT = re.compile(r\"(?<=[.;!?])\\s+|\\n+\")\n\n\ndef unwrap(text: str) -> str:\n    \"\"\"Rejoin lines that a fixed-width layout broke mid-sentence.\n\n    A large share of this corpus arrives hard-wrapped at some column, so a sentence is\n    split across two lines with no punctuation at the break. Splitting on newlines then\n    severs the finding from its anatomy - `... y parte de la raiz` / `meniscal posterior\n    del menisco lateral con extrusion asociada` puts the tear in one clause and the\n    meniscus in the next, and neither clause says anything on its own. A line that does\n    not end in sentence punctuation is a continuation, not a statement.\n    \"\"\"\n    if not isinstance(text, str):\n        return \"\"\n    out = []\n    for line in text.split(\"\\n\"):\n        s = line.strip()\n        if out and out[-1] and not re.search(r\"[.;:!?>*•]$\", out[-1]) \\\n                and len(out[-1].split()) >= 4 and s and not s[:1].isupper():\n            out[-1] = out[-1] + \" \" + s\n        else:\n            out.append(s)\n    return \"\\n\".join(out)\n\n\ndef clauses(text: str):\n    \"\"\"Split into clauses; attach `header:` lines to the value beneath them.\"\"\"\n    norm = normalize(unwrap(text) if FEATURES[\"unwrap\"] else text)\n    raw = [c.strip() for c in _SENT_SPLIT.split(norm) if c and c.strip()]\n    merged = []\n    for i, c in enumerate(raw):\n        if c.endswith(\":\") and len(c.split()) <= 14 and i + 1 < len(raw):\n            merged.append(c + \" \" + raw[i + 1])\n        merged.append(c)\n    out = []\n    for c in merged:\n        out.append(c)\n        if len(c.split()) > 25:\n            out.extend(p.strip() for p in c.split(\",\") if len(p.split()) > 2)\n    return out\n\n\n# Each rule below that is not obviously right is behind a flag, so that the notebook can\n# turn it off and re-measure rather than assert that it helps.\nFEATURES = {\n    \"unwrap\": True,               # rejoin hard-wrapped lines before splitting\n    \"directional_negation\": True,  # scope negation by direction instead of by clause\n    \"oa_inherit\": True,            # unlocalised cartilage statements reach all three\n    \"graded_pathology\": True,      # read numeric grades on a per-structure scale\n    \"synovitis_backoff\": True,     # order the silent majority by inflammatory context\n}\n\n\ndef _rx(*alts: str) -> re.Pattern:\n    return re.compile(\"|\".join(alts))\n\n\n# --------------------------------------------------------------------------- #\n# polarity\n# --------------------------------------------------------------------------- #\n# Negation is scoped by direction, not by clause. `Subchondral insufficiency fracture at\n# the medial tibial plateau without articular surface collapse` contains a negator and\n# asserts a fracture: `without` governs what follows it, and the fracture precedes it.\n# Reading negation at clause scope turns that sentence into a denial - and it is the\n# house style of one of the larger reporting sites here, so the error is systematic\n# rather than occasional.\nPRE_NEG = _rx(\n    r\"\\bno\\b\", r\"\\bnot\\b\", r\"\\bwithout\\b\", r\"\\bnegative for\\b\", r\"\\babsence\\b\",\n    r\"\\bno evidence\\b\", r\"\\bfree of\\b\", r\"\\bnone\\b\", r\"\\bneither\\b\", r\"\\bnor\\b\",\n    r\"\\bsin\\b\", r\"\\bno hay\\b\", r\"\\bausencia\\b\", r\"\\bausentes?\\b\", r\"\\bno se\\b\",\n    r\"\\bpas de\\b\", r\"\\bsans\\b\", r\"\\baucune?\\b\",\n    r\"\\bgeen\\b\", r\"\\bzonder\\b\", r\"\\bniet\\b\",\n    r\"\\bkeine?[nmrs]?\\b\", r\"\\bohne\\b\", r\"\\bnicht\\b\", r\"\\bkein\\b\",\n    r\"\\bnema\\b\", r\"\\bbez\\b\", r\"\\bnisu\\b\", r\"\\bnije\\b\",\n    r\"\\bδεν\\b\", r\"\\bχωρις\\b\", r\"ουδεν\", r\"\\bουτε\\b\",\n    r\"\\bбез\\b\", r\"\\bне\\b\", r\"липсва\", r\"\\bняма\\b\",\n)\n\n# Turkish and a few Croatian forms put the negator at the end of the sentence, so these\n# govern what precedes them instead.\nPOST_NEG = _rx(\n    r\"\\byok\\b\", r\"\\byoktur\\b\", r\"izlenmemekte\", r\"saptanmadi\", r\"\\bdegil\\b\",\n    r\"gozlenmemekte\", r\"mevcut degil\", r\"eslik etmiyor\", r\"\\bizlenmedi\\b\",\n    r\"izlenmemistir\", r\"saptanmamistir\", r\"gorulmemistir\", r\"\\bnema znakova\\b\",\n    r\"bez znakova\",\n)\n\nNEGATION = _rx(PRE_NEG.pattern, POST_NEG.pattern, r\"\\bunremarkable\\b\")\n\nNEG_WINDOW = 90\n\n\ndef _negated(clause: str, start: int, end: int) -> bool:\n    \"\"\"True when a negation trigger governs the span [start, end) of this clause.\"\"\"\n    for m in PRE_NEG.finditer(clause):\n        if m.end() <= start and start - m.end() <= NEG_WINDOW:\n            # `... intact but with a tear` - a contrast conjunction closes the scope.\n            if not re.search(r\"\\b(but|however|ancak|fakat|pero|maar|aber|no i|ali|\"\n                             r\"ωστοσο|αλλα|но)\\b\", clause[m.end():start]):\n                return True\n    for m in POST_NEG.finditer(clause):\n        if m.start() >= end and m.start() - end <= NEG_WINDOW:\n            return True\n    return False\n\nNORMALITY = _rx(\n    r\"\\bnormal\", r\"\\bintact\\b\", r\"\\bpreserved\\b\", r\"\\bwithin normal limits\\b\",\n    r\"limites normales\", r\"\\bconservad\", r\"\\bintegr\", r\"\\bnormales\\b\",\n    r\"\\bdoga(l|ll)\\b\", r\"korunmus\", r\"\\bnormaldir\\b\", r\"olagan\",\n    r\"\\buredn\", r\"\\bocuvan\", r\"\\bodrzan\", r\"\\bintakt\", r\"\\bprimjeren\",\n    r\"\\bodrzanog kontinuiteta\", r\"\\bodržan\",\n    r\"φυσιολογικ\", r\"ακεραι\", r\"δεν παρατηρουνται\", r\"δεν σημειωνονται\",\n    r\"unauffallig\", r\"regelrecht\", r\"\\bo\\.?b\\.?\\b\",\n    r\"нормал\", r\"запазен\", r\"съхранен\", r\"\\bбез особености\\b\", r\"интактн\",\n    r\"\\bgaaf\\b\", r\"\\bnormaal\\b\",\n)\n\n# \"Negator + abnormality-noun\" is how most of these languages assert normality:\n# `sin alteraciones`, `ohne Auffalligkeiten`, `geen afwijkingen`, `bez osobitosti`.\n# Read literally each one is a negation, so the old guard (`normality AND NOT negation`)\n# threw every one of them away and left the structure looking unmentioned instead of\n# explicitly clear. That matters to a rank metric: a ligament a radiologist looked at and\n# called intact must rank below one the report never mentions, not equal to it.\nNORMAL_PHRASE = _rx(\n    r\"\\bsin alteracion\", r\"\\bsin cambios\\b\", r\"\\bsin particularidad\",\n    r\"\\bsin hallazgos\\b\", r\"\\bsin lesion\", r\"\\bsin signos de (rotura|lesion)\",\n    r\"\\bcontinu[oa]s?\\b\", r\"\\bcontinuidad conservada\\b\",\n    r\"\\bno abnormalit\", r\"\\bno significant abnormalit\", r\"\\bunremarkable\\b\",\n    r\"\\bno evidence of (tear|injury|abnormalit)\",\n    r\"\\bohne auffalligkeit\", r\"\\bkein nachweis\\b\", r\"\\bohne befund\\b\",\n    r\"\\bgeen afwijking\", r\"\\bzonder afwijking\",\n    r\"\\bsans anomalie\", r\"\\bpas d[e']anomalie\",\n    r\"\\bbez osobitosti\\b\", r\"\\bbez znakova (rupture|lezije)\\b\",\n    r\"\\bbez patoloskih\\b\",\n    r\"χωρις αλλοιωσ\", r\"χωρις παθολογ\", r\"δεν παρατηρουνται (αξιολογα|παθολογ)\",\n    r\"\\bбез особености\\b\", r\"\\bбез патологич\", r\"\\bбез данни за\\b\",\n    r\"\\bozel bir ozellik yok\", r\"\\bpatolojik bulgu (yok|izlenmemis)\",\n)\n\nUNCERTAIN = _rx(\n    r\"\\bpossible\\b\", r\"\\bprobable\\b\", r\"\\bsuspicious\\b\", r\"\\bsuspected?\\b\",\n    r\"cannot (be )?exclude\", r\"\\bmay\\b\", r\"\\bquestionable\\b\", r\"\\bequivocal\\b\",\n    r\"\\br/o\\b\", r\"\\bdd\\b\", r\"\\blikely\\b\", r\"\\bsuggest\", r\"\\bcompatible with\\b\",\n    r\"\\bposible\\b\", r\"sin criterios categoricos\", r\"\\bdudos\", r\"\\bsugier\",\n    r\"\\bmuhtemel\\b\", r\"\\bolasi\\b\", r\"\\bsupheli\\b\", r\"\\bizlenim\", r\"\\bdusundur\",\n    r\"\\bmoguce\\b\", r\"\\bvjerojatno\\b\", r\"\\bsumnja\\b\", r\"\\bmoze odgovarati\\b\",\n    r\"πιθαν\", r\"υποπτ\",\n    r\"\\bmoglich\", r\"\\bverdachtig\", r\"\\bfraglich\", r\"\\bv\\.?a\\.?\\b\", r\"\\bwohl\\b\",\n    r\"\\bвъзможно\\b\", r\"\\bвероятно\\b\", r\"суспект\",\n    r\"\\bmogelijk\\b\", r\"\\bverdacht\\b\",\n)","metadata":{},"outputs":[],"execution_count":null},{"id":"f20b1dc1-c3fd-4269-aeea-fea4c8415846","cell_type":"markdown","source":"### 2.1 Four rules that are not obvious\n\nMost of a lexicon is vocabulary, and vocabulary is dull work that pays. Four rules are not\nvocabulary, and each of them fixes an error that a term list cannot reach. All four are\nbehind flags in `FEATURES`, and §2.2 turns them off one at a time and re-measures.\n\n**Unwrap before splitting.** A large share of these reports arrive hard-wrapped at a fixed\ncolumn, so a sentence is broken across two lines with no punctuation at the break.\nSplitting on newlines then severs a finding from its anatomy:\n\n> `Rotura compleja degenerativa del cuerno anterior, cuerpo, cuerno posterior y parte de la raíz`\n> `meniscal posterior del menisco lateral con extrusión asociada.`\n\nThe first line says *tear* and names no structure; the second names the lateral meniscus\nand says nothing about a tear. Neither clause carries a finding on its own, and the study\ncomes out silent on a meniscus that the report describes as complexly torn. A line that\ndoes not end in sentence punctuation is a continuation, not a statement.\n\n**Scope negation by direction, and anchor it on the pathology word.** Read at clause scope,\n\n> `Subchondral insufficiency fracture at the medial tibial plateau without articular surface collapse.`\n\nis a denial: it contains *without*. It is in fact an assertion — *without* governs what\nfollows it, and the fracture precedes it. This is the house style of one of the larger\nreporting sites in this corpus, so the error is systematic rather than occasional, and it\nfalls on `Fracture`, one of the rarest targets. Direction alone is not enough either. In\n\n> `Medial meniscus is not torn.`\n\nthe negator stands *after* the noun and before the verb, so scoping the test from the\nanatomy match reads the sentence as an assertion. The negator governs the finding, so the\ntest is anchored on the pathology word — *torn* — not on *meniscus*.\n\n**Let an unlocalised cartilage statement reach all three compartments.** Osteoarthritis is\nrarely written as \"osteoarthritis\". It is written as cartilage loss, chondrosis, a\nchondromalacia grade, joint space narrowing, or osteophytes — and often without naming a\ncompartment at all. `Tricompartmental osteoarthritis`, `gonarthrose`, `Incipient OA of all\nthree compartmens` are the house styles, not `chondropathy of the medial femorotibial\ncompartment`. A rule that requires a compartment phrase before it will fire leaves three\nquarters of the corpus silent on the three OA targets. So a cartilage statement is\nattributed by what else its clause names — *medial* next to a tibiofemoral structure sends\nit to the medial compartment, *patella* or *trochlea* to the patellofemoral one — and a\nstatement that localises to nothing counts for all three at a discount, unless that\ncompartment was separately and explicitly cleared.\n\n**Read numeric grades on the right scale.** A grade is the most precise thing a knee report\nsays, and it means opposite things in different places. Grade 3 of a *meniscus* is signal\nreaching the articular surface, which is a tear by definition; grades 1 and 2 are\nintrasubstance change that is not. A *ligament* runs the other way — grade 1 is a stretch,\ngrade 2 a partial tear. Folding both into one pathology vocabulary scores a degenerate\nmeniscus like a torn one and throws away the ordering the metric is built on.","metadata":{}},{"id":"bef9f906-3ca8-43e3-9a99-cea9e0d44011","cell_type":"code","source":"# --------------------------------------------------------------------------- #\n# pathology vocabulary\n# --------------------------------------------------------------------------- #\nTEAR = _rx(\n    r\"\\btear\", r\"\\btorn\\b\", r\"\\brupture\", r\"\\bdisruption\\b\", r\"discontinuit\",\n    r\"\\bavuls\", r\"\\bmacerat\", r\"\\bbuckethandle\\b\", r\"bucket handle\",\n    r\"\\brotura\\b\", r\"\\broturas\\b\", r\"\\bruptura\", r\"\\bdesgarro\", r\"\\broto\\b\",\n    r\"\\bdechirure\", r\"\\bdechire\",\n    r\"\\bscheur\", r\"\\bruptuur\", r\"gescheurd\",\n    r\"\\briss\\b\", r\"einriss\", r\"\\bruptur\", r\"zerreiss\", r\"\\blasion\", r\"\\bausriss\",\n    r\"\\byirtik\", r\"\\byirtig\", r\"\\bkopma\\b\", r\"butunluk kaybi\", r\"\\brupturu\\b\",\n    r\"devamsizlik\", r\"\\brupture\\b\", r\"\\bdevamliligi secilememis\",\n    r\"\\bpuknuce\", r\"\\bprekid\\b\", r\"\\bpukotin\", r\"\\bruptur\",\n    r\"ρηξη\", r\"ρηξις\", r\"ρηγμα\", r\"ασυνεχεια\",\n    r\"руптура\", r\"разкъсв\", r\"разрив\", r\"скъсв\", r\"\\bлезия\\b\",\n)\n\nDEGEN = _rx(\n    r\"degenerat\", r\"\\bmucoid\\b\", r\"\\bmyxoid\\b\", r\"\\bfray\", r\"\\bfissur\",\n    r\"dejeneratif\", r\"\\bmukoid\\b\", r\"degenerativn\", r\"εκφυλ\", r\"дегенерат\",\n    r\"\\bμυξοειδ\", r\"\\bμυξωδ\", r\"\\bmeniskopat\", r\"\\bmeniscopath\",\n    r\"\\bmuco ?ide\\b\", r\"aufgefasert\", r\"\\bdejenerasyon\\b\",\n)\n\nINJURY = _rx(\n    r\"\\binjur\", r\"\\bsprain\", r\"\\blesion\", r\"\\blasion\", r\"\\bedema\\b\", r\"\\boedema\\b\",\n    r\"\\bodem\\b\", r\"\\bedem\\b\", r\"\\bοιδημα\", r\"\\bодем\", r\"\\bедем\", r\"\\bstrain\\b\",\n    r\"\\bhigh signal\\b\", r\"\\bsignal alteration\\b\", r\"\\bhiperintens\", r\"\\bhyperintens\",\n    r\"aumento de senal\", r\"alteracion de senal\", r\"cambio de senal\",\n    r\"\\bsignalanhebung\", r\"\\bsignalalteration\", r\"verhoogd signaal\", r\"sinyal artis\",\n    r\"αυξημενο σημα\", r\"повишен сигнал\", r\"\\besguince\\b\",\n    r\"\\bthicken\", r\"\\bzadebljanje\\b\", r\"\\bverdikking\\b\", r\"\\bdistenzij\",\n    r\"\\blaksite\\b\", r\"\\blaxity\\b\", r\"\\bpartial\\b\", r\"\\bparcijaln\", r\"\\bparcial\",\n    r\"\\bpartiel\", r\"\\bpartiell\",\n)\n\n# A numeric grade is the most precise thing a knee report says, and it means different\n# things in different places: grade 3 of a meniscus is a tear by definition, grade 1 or 2\n# is intrasubstance signal that never reaches the surface and is not one. A ligament runs\n# the other way round - grade 1 is a stretch, grade 2 a partial tear. So the grade is\n# read as a number and interpreted per structure rather than folded into one pathology\n# vocabulary.\n_GRADE_RX = re.compile(\n    r\"(?:grade|grad|grado|grau|derece|stupnja|stupanj|βαθμ|степен|icrs|outerbridge)\"\n    r\"[\\s:]*(?:grade\\s*)?([1-4]|iv|iii|ii|i)\\b\"\n)\n_ROMAN = {\"i\": 1, \"ii\": 2, \"iii\": 3, \"iv\": 4}\n\n\ndef _grade_of(clause: str):\n    \"\"\"Highest numeric grade stated in a clause, or None.\"\"\"\n    best = None\n    for m in _GRADE_RX.finditer(clause):\n        v = m.group(1)\n        n = _ROMAN.get(v, None) if not v.isdigit() else int(v)\n        if n is not None and (best is None or n > best):\n            best = n\n    return best\n\n# --------------------------------------------------------------------------- #\n# anatomy\n# --------------------------------------------------------------------------- #\nANAT = {\n    \"ACL\": _rx(\n        r\"anterior cruciate\", r\"\\bacl\\b\",\n        r\"cruzado anterior\", r\"\\blca\\b\",\n        r\"croise anterieur\",\n        r\"voorste kruisband\", r\"\\bvkb\\b\",\n        r\"vorderes kreuzband\", r\"vorderen kreuzband\", r\"vordere kreuzband\",\n        r\"on capraz\", r\"\\bocb\\b\", r\"anterior capraz\",\n        r\"prednji krizni\", r\"prednjeg krizn\",\n        r\"προσθι[οα][^ ]* χιαστ\", r\"προσθιου χιαστου\", r\"χιαστο[^ ]* συνδεσμ\",\n        r\"\\bχιαστ\\w*\",\n        r\"предна кръстна\", r\"предната кръстна\", r\"предна кръста\",\n        r\"cruciate ligaments\", r\"ligamentos cruzados\", r\"ligaments croises\",\n        r\"kruisbanden\", r\"kreuzbander\", r\"capraz baglar\", r\"krizn[a-z]* ligament[a-z]*\",\n        r\"χιαστοι συνδεσμ\", r\"χιαστων συνδεσμ\", r\"кръстните връзки\", r\"кръстни връзки\",\n    ),\n    \"MCL\": _rx(\n        r\"medial collateral\", r\"\\bmcl\\b\", r\"tibial collateral\",\n        r\"colateral medial\", r\"colateral interno\", r\"\\blcm\\b\",\n        r\"collateral medial\", r\"collateral interne\",\n        r\"mediale collaterale\", r\"binnenband\", r\"\\b(mediale|laterale) banden\\b\",\n        r\"\\bcollaterale banden\\b\",\n        r\"innenband\", r\"mediales? kollateral\",\n        r\"\\bic yan bag\", r\"medial kollateral\", r\"\\biyb\\b\", r\"medyal kollateral\",\n        r\"medijalni kolateraln\", r\"medijalnog kolateraln\",\n        r\"εσω πλαγι\", r\"εσωτερικο πλαγι\", r\"\\bπλαγι\\w* συνδεσμ\", r\"\\bπλαγιοι\\b\",\n        r\"медиален колатерал\", r\"вътрешна странична\", r\"\\bколатерал\\w*\",\n        r\"\\bcolaterales\\b\", r\"\\bcollateraux\\b\", r\"\\bcollateralen\\b\", r\"\\bkolateralni\\b\",\n        r\"collateral ligaments\", r\"ligamentos colaterales\", r\"ligaments collateraux\",\n        r\"collaterale banden\", r\"kollateralbander\", r\"seitenbander\", r\"yan baglar\",\n        r\"kolateraln[a-z]* ligament[a-z]*\", r\"πλαγιοι συνδεσμ\", r\"πλαγιων συνδεσμ\",\n        r\"колатерални връзки\", r\"страничните връзки\",\n    ),\n    \"Medial Meniscus\": _rx(\n        r\"medial meniscus\", r\"\\bmm\\b(?= tear)\", r\"medial menisc\",\n        r\"menisco medial\", r\"menisco interno\",\n        r\"menisque medial\", r\"menisque interne\",\n        r\"mediale meniscus\", r\"binnenmeniscus\",\n        r\"innenmeniskus\", r\"medialen? meniskus\", r\"innenmeniskushinterhorn\",\n        r\"medyal menisk\", r\"\\bic menisk\",\n        r\"medijalni meniskus\", r\"medijalnog meniskusa\", r\"medijalnom meniskusu\",\n        r\"medijaln\\w* menisk\\w*\", r\"\\bmedijalnog meniska\\b\", r\"medijalni menisk\",\n        r\"εσω μηνισκ\", r\"μηνισκ[^ ]* του εσω\", r\"εσω διαμερισμα[^.]{0,40}μηνισκ\",\n        r\"медиалния менискус\", r\"медиален менискус\", r\"вътрешния менискус\",\n        r\"oba meniska\", r\"both menisci\", r\"ambos meniscos\", r\"beide menisci\",\n        r\"her iki menisku\", r\"amfoteroi\\w* mhnisk\", r\"αμφοτερ\\w* μηνισκ\",\n        r\"двата менискуса\", r\"medial (and|&) lateral menisc\",\n    ),\n    \"Lateral Meniscus\": _rx(\n        r\"lateral meniscus\", r\"lateral menisc\",\n        r\"menisco lateral\", r\"menisco externo\",\n        r\"menisque lateral\", r\"menisque externe\",\n        r\"laterale meniscus\", r\"buitenmeniscus\",\n        r\"aussenmeniskus\", r\"lateralen? meniskus\", r\"aussenmeniskushinterhorn\",\n        r\"lateral menisk\", r\"\\bdis menisk\",\n        r\"lateralni meniskus\", r\"lateralnog meniskusa\", r\"lateralnom meniskusu\",\n        r\"lateraln\\w* menisk\\w*\", r\"\\blateralnog meniska\\b\",\n        r\"εξω μηνισκ\", r\"μηνισκ[^ ]* του εξω\", r\"εξω διαμερισμα[^.]{0,40}μηνισκ\",\n        r\"латералния менискус\", r\"латерален менискус\", r\"външния менискус\",\n        r\"oba meniska\", r\"both menisci\", r\"ambos meniscos\", r\"beide menisci\",\n        r\"her iki menisku\", r\"αμφοτερ\\w* μηνισκ\",\n        r\"двата менискуса\", r\"medial (and|&) lateral menisc\",\n    ),\n}\n\n# Osteoarthritis is written as cartilage damage far more often than as a diagnosis.\nOA_EVIDENCE = _rx(\n    r\"osteoarthrit\", r\"\\barthros\", r\"\\bgonarthros\", r\"\\bosteoarthros\",\n    r\"chondropath\", r\"chondromalac\", r\"condropat\", r\"condromalac\", r\"\\bchondros\",\n    r\"\\bchondrosis\\b\", r\"chondral (loss|defect|ulcer|thinning|injury|fissur|wear)\",\n    r\"cartilage (loss|thinning|defect|fissur|wear|damage|heterogeneity|irregularit)\",\n    r\"(loss|thinning|fissur|defect|ulcer|erosion|denudation) of[^.]{0,20}cartilage\",\n    r\"articular cartilage[^.]{0,30}(loss|thin|fissur|defect|erosion|wear|irregular)\",\n    r\"osteophyt\", r\"osteofit\", r\"osteofyt\", r\"osteofito\", r\"osteophyten\", r\"spurring\",\n    r\"joint space narrowing\", r\"pinzamiento articular\", r\"reduced joint space\",\n    r\"kikirdak kayb\", r\"kikirdak incelme\", r\"kondropati\", r\"kondral\", r\"kikirdak dejener\",\n    r\"eklem aralig\\w* daral\", r\"eklem mesafesi daral\", r\"kikirdak kalinlig\\w* azal\",\n    r\"kraakbeen\", r\"gonartrose\", r\"artrose\", r\"\\bknorpel\", r\"arthrose\", r\"gonarthrose\",\n    r\"hrskavic\", r\"hondromalac\", r\"artroz\", r\"osteoartrit\", r\"artrotsk\", r\"artrotick\",\n    r\"\\boa promjen\", r\"\\boa\\b\", r\"degenerativne promjene hrskav\",\n    r\"χονδρ[^ ]*παθ\", r\"αρθριτ\", r\"αρθρωσ\", r\"οστεοφυτ\", r\"χονδρομαλακ\",\n    r\"αρθρικου χονδρου\", r\"εξαλειψη του αρθρικου χονδρου\", r\"διαβρωση του αρθρικου χονδρ\",\n    r\"λεπτυνση[^.]{0,30}χονδρ\", r\"φθορα[^.]{0,20}χονδρ\",\n    r\"артроз\", r\"хондропат\", r\"остеофит\", r\"хрущял[^.]{0,40}(изтън|увред|дефект|липс)\",\n    r\"изтъняване[^.]{0,30}хрущял\", r\"хондромалац\",\n    r\"ulcera[s]? condral\", r\"cartilago[^.]{0,25}(perdida|adelgaz)\",\n    r\"icrs grade\", r\"icrs\\b\", r\"outerbridge\", r\"\\bdenudation\\b\", r\"denudacij\",\n    r\"erozivne promjene\", r\"\\berosion of[^.]{0,20}cartilage\",\n    r\"kraakbeenlijden\", r\"kraakbeenverlies\",\n)\n\n# --------------------------------------------------------------------------- #\n# where in the joint a cartilage statement sits\n# --------------------------------------------------------------------------- #\n# Tibiofemoral structures. Used only inside a clause that already carries OA evidence,\n# so bare \"condyle\" is safe here and would not be elsewhere.\nTF_SITE = _rx(\n    r\"compartment\", r\"compartimento\", r\"compartiment\", r\"kompartman\", r\"kompartiment\",\n    r\"kompartment\", r\"odjelj\", r\"διαμερισμα\", r\"компартм\", r\"\\bотдел\",\n    r\"femorotibial\", r\"tibiofemoral\", r\"femoro tibial\", r\"femorotibiaal\",\n    r\"femorotibijaln\", r\"феморотибиал\", r\"\\bft zglob\", r\"tibiofemoraln\",\n    r\"condyle\", r\"condilo\", r\"kondyl\", r\"kondil\", r\"condyl\", r\"κονδυλ\",\n    r\"кондил\", r\"\\bplateau\", r\"\\bplato\\b\", r\"platillo\", r\"meseta\", r\"плато\",\n    r\"tibiaplateau\", r\"tibijaln\\w* plato\", r\"tibyal plato\", r\"tibia plato\",\n    r\"κνημιαι\", r\"μηριαι\", r\"weightbearing\", r\"weightbaring\", r\"zona de carga\",\n    r\"dragende deel\", r\"agirlik tasiyan\", r\"\\bfemur\\b\", r\"\\btibia\\b\", r\"\\bfemoral\\b\",\n    r\"\\btibial\\b\", r\"\\bfemura\\b\", r\"\\btibije\\b\", r\"\\bmesarthrio\\b\", r\"μεσαρθριο\",\n)\n\n# Patellofemoral structures.\nPF_SITE = _rx(\n    r\"patellofemoral\", r\"femoropatellar\", r\"femoropatelar\", r\"patelofemoral\",\n    r\"retropatellar\", r\"retrorotulian\", r\"trochlea\", r\"troclea\", r\"troklea\",\n    r\"trochlear\", r\"trohlej\", r\"τροχιλ\", r\"\\bpatella\", r\"\\bpatellar\", r\"rotulian\",\n    r\"\\brotula\\b\", r\"\\bpatele\\b\", r\"patellofemoraal\", r\"femoropatellair\",\n    r\"επιγονατιδ\", r\"μηροεπιγονατιδ\", r\"пател\", r\"феморопател\",\n    r\"anterior compartment\", r\"compartimento anterior\", r\"prednj\\w* odjeljk\",\n    r\"\\bfp zglob\", r\"\\bpf zglob\", r\"\\bfaset\", r\"\\bfacet\", r\"patellofemoraln\",\n)\n\nSIDE_MEDIAL = _rx(\n    r\"\\bmedial\\w*\", r\"\\bmedyal\\w*\", r\"\\bmedijaln\\w*\", r\"\\bmediaal\\w*\",\n    r\"\\bmediale\\w*\", r\"\\binterno\\b\", r\"\\binterna\\b\", r\"\\binternos\\b\", r\"\\binterne\\b\",\n    r\"\\binnen\\w*\", r\"\\bic\\b\", r\"\\bunutarnj\\w*\", r\"\\bεσω\\w*\", r\"\\bεσωτερικ\\w*\",\n    r\"\\bмедиал\\w*\", r\"\\bвътреш\\w*\", r\"\\bbinnen\\w*\", r\"\\bmediaal\\b\", r\"\\bmediales?\\b\",\n)\nSIDE_LATERAL = _rx(\n    r\"\\blateral\\w*\", r\"\\bexterno\\b\", r\"\\bexterna\\b\", r\"\\bexternos\\b\", r\"\\bexterne\\b\",\n    r\"\\bdis\\b\", r\"\\blateraln\\w*\", r\"\\baussen\\w*\", r\"\\bbuiten\\w*\", r\"\\bεξω\\w*\",\n    r\"\\bεξωτερικ\\w*\", r\"\\bлатерал\\w*\", r\"\\bвъншн\\w*\", r\"\\bvanjsk\\w*\",\n)\nSIDE_ANTERIOR = _rx(\n    r\"\\banterior\\w*\", r\"\\bant\\b\", r\"\\bon\\b\", r\"\\bprednj\\w*\", r\"\\bvorder\\w*\",\n    r\"\\bvoorste\\b\", r\"\\bπροσθι\\w*\", r\"\\bпредн\\w*\", r\"\\banteriyor\\w*\", r\"\\bavant\\b\",\n    r\"\\banterieur\\w*\",\n)\n\nGLOBAL_OA = _rx(\n    r\"tri ?compartment\", r\"all three compartment\", r\"global(ised)? (oa|osteoarthrit)\",\n    r\"\\bgonarthros\", r\"\\bgonartros\", r\"\\bgonarthrose\", r\"\\bgonartrose\", r\"gonartro\",\n    r\"goanrtrot\", r\"gonartrot\",\n    r\"osteoarthritis of the knee\", r\"artrosis (de |)(la )?rodilla\", r\"knee osteoarthrit\",\n    r\"\\bdiz osteoartrit\", r\"\\bgonartroz\", r\"artroza koljena\",\n    r\"οστεοαρθριτιδα\", r\"αρθριτιδα του γονατος\", r\"εκφυλιστικη οστεοαρθριτ\",\n    r\"артроза на колянната\", r\"гонартроз\",\n    r\"degenerative joint disease\", r\"\\bdjd\\b\", r\"three compartments\",\n    r\"compartmens\", r\"compartments\",\n)\n\n# --------------------------------------------------------------------------- #\n# self-declaring findings\n# --------------------------------------------------------------------------- #\nDIRECT = {\n    \"Effusion\": _rx(\n        r\"\\beffusion\", r\"joint fluid\", r\"intra ?articular fluid\", r\"\\bhydrops\\b\",\n        r\"\\bhemarthros\", r\"\\bhaemarthros\",\n        r\"derrame articular\", r\"\\bderrame\\b\", r\"liquido articular\", r\"hemartrosis\",\n        r\"epanchement\",\n        r\"gewrichtsvocht\", r\"\\bvocht\\b\", r\"gewrichtseffusie\", r\"opzetting van suprapatell\",\n        r\"gelenkerguss\", r\"\\berguss\\b\", r\"gelenksergu\", r\"gelenksflussigkeit\",\n        r\"eklem\\w* ic\\w* sivi\", r\"efuzyon\", r\"eklem sivisi\", r\"eklem mesafesinde sivi\",\n        r\"sivi (miktari|artisi|birikimi)\", r\"sivi artis\", r\"\\bsivi\\b[^.]{0,25}artmis\",\n        r\"\\bizljev\", r\"\\bizliv\", r\"zglobn[^ ]* tekucin\", r\"\\bhidrops\\b\",\n        r\"αρθρικ[^ ]* υγρ\", r\"υγρου ενδαρθρικα\", r\"ενδαρθρικ[^ ]* υγρ\", r\"ποσοτητα υγρου\",\n        r\"ενδαρθρικ\", r\"αρθρικη συλλογη\", r\"υγρο στην αρθρωση\", r\"υγρου στην αρθρωση\",\n        r\"συλλογη υγρου\", r\"ενθαρθρικ\",\n        r\"ставен излив\", r\"излив\", r\"ставна течност\", r\"синовиална течност\",\n    ),\n    \"Synovitis\": _rx(\n        r\"synovit\", r\"sinovit\", r\"synovial (thickening|proliferation|hypertroph)\",\n        r\"thicken\\w* synovial\", r\"hypertroph\\w* of the synovium\",\n        r\"synoviale? (verdikking|proliferatie)\", r\"verdikkingen van (het )?synovium\",\n        r\"synovialitis\", r\"synovialis(verdickung|proliferation)\", r\"reizsynovial\",\n        r\"sinovijalitis\", r\"sinovitis\", r\"zadebljanje sinovij\", r\"proliferacij\\w* sinovij\",\n        r\"sinovijaln\\w* proliferacij\",\n        r\"υμενιτιδα\", r\"συνοβιτιδα\", r\"υμενικ[^ ]* υπερτροφ\", r\"αρθρικου υμεν\",\n        r\"παχυνση[^.]{0,20}υμεν\", r\"υμενα\",\n        r\"синовит\", r\"синовиал[^ ]* (задебел|пролифер)\",\n        r\"\\bpannus\\b\", r\"\\bhoffit\", r\"sinovyal\\w* (kalinlas|proliferas)\",\n        r\"sinovyal hipertrof\", r\"\\bartrit\\b\", r\"\\barthritis\\b\",\n    ),\n    \"Baker's\": _rx(\n        r\"baker\", r\"popliteal cyst\", r\"quiste popliteo\", r\"quistes popliteos\",\n        r\"kyste poplite\", r\"popliteale? cyst\", r\"poplitealzyste\", r\"bakerzyste\",\n        r\"popliteal kist\", r\"\\bbakerova\\b\", r\"poplitealn[^ ]* cist\", r\"popliteal\\w* cist\",\n        r\"κυστη baker\", r\"πολυχωρη συνοβιακη κυστη\", r\"κυστη του baker\",\n        r\"συνοβιακη κυστη\", r\"κυστη τυπου baker\",\n        r\"киста на бейкър\", r\"бейкърова киста\", r\"поплитеална киста\", r\"бекеров\",\n        r\"gastrocnemio ?semimembranos\", r\"gastrocnemius semimembranosus burs\",\n    ),\n    \"Contusion\": _rx(\n        r\"\\bcontusion\", r\"bone bruise\", r\"bone marrow (o?edema|contusion)\",\n        r\"marrow o?edema\", r\"\\bkontuz\", r\"medular bone o?edema\", r\"osseous contusion\",\n        r\"contusion osea\", r\"edema oseo\", r\"edema de medula osea\", r\"contusiones oseas\",\n        r\"oedeme osseux\", r\"contusion osseuse\",\n        r\"botcontusie\", r\"botoedeem\", r\"beenmergoedeem\", r\"botmergoedeem\",\n        r\"knochenmarkodem\", r\"knochenodem\", r\"knochenmarksodem\", r\"kontusion\",\n        r\"kemik kontuzyonu\", r\"kemik iligi odemi\", r\"kemik odemi\", r\"kemik iliginde odem\",\n        r\"kontuzyonel kemik\", r\"kemik iligi odemleri\",\n        r\"kostani edem\", r\"edem kosti\", r\"kontuzij\", r\"kostane srzi[^.]{0,20}edem\",\n        r\"οστεομυελικ[^ ]* οιδημα\", r\"οστικο οιδημα\", r\"μυελικο οιδημα\", r\"οστικο μωλωπ\",\n        r\"костномозъчен едем\", r\"костен едем\", r\"контузионен\", r\"костно мозъчен едем\",\n    ),\n    \"Fracture\": _rx(\n        r\"\\bfractur\", r\"\\bfract\\b\",\n        r\"\\bfractura\", r\"\\bfracturas\\b\",\n        r\"\\bfractuur\", r\"\\bbreuk\\b\",\n        r\"\\bfraktur\", r\"\\bbruch\\b\",\n        r\"\\bkirik\\b\", r\"\\bkirigi\\b\", r\"\\bkiri[kg]\\w*\",\n        r\"\\bprijelom\", r\"impresijsk[^ ]* fraktur\", r\"impaktcij\",\n        r\"καταγμα\", r\"καταγματ\",\n        r\"фрактур\", r\"счупван\", r\"фисур\",\n        r\"insufficiency fracture\", r\"stress fracture\", r\"avulsion fracture\",\n        r\"subchondral fracture\", r\"subkondral kiri\", r\"impaction (fracture|injury)\",\n        r\"osteochondral (fracture|impaction)\", r\"\\bsegond\\b\", r\"impactiefractuur\",\n        r\"subchondrale impression\", r\"subchondraler? impress\",\n    ),\n}\n\nDECOY = {\n    \"Fracture\": _rx(r\"microfractur\", r\"\\bfracture (risk|prophyla)\"),\n    \"Baker's\": _rx(r\"meniscal cyst\", r\"quiste meniscal\", r\"parameniscal\"),\n}\n\nPAIRED = {\"ACL\", \"MCL\", \"Medial Meniscus\", \"Lateral Meniscus\"}\nOA_TARGETS = [\"Medial OA\", \"Lateral OA\", \"PF OA\"]\n\n# --------------------------------------------------------------------------- #\n# stems, for morphology the phrase lexicons cannot reach\n# --------------------------------------------------------------------------- #\n# A report that clears or tears both menisci in one breath - \"normaal voorkomen\n# menisci\", \"Normal medial and lateral menisci\" - names neither side, so a\n# side-qualified lexicon leaves both targets silent. The cruciates and collaterals\n# already had their plural forms; the menisci were simply missed.\n#\n# It is only consulted when the clause names no side at all. Otherwise \"tear of the\n# medial meniscus, menisci otherwise intact\" would fire the plural for the lateral side\n# off a clause that is about the medial one.\nPLURAL_MENISCI = _rx(\n    r\"\\bmenisci\\b\", r\"\\bmeniscos\\b\", r\"\\bmenisques\\b\", r\"\\bmenisken\\b\",\n    r\"\\bmeniskusi\\b\", r\"\\bmenisk\\w*ler\\b\", r\"\\bμηνισκοι\\b\", r\"\\bμηνισκων\\b\",\n    r\"\\bменискуси\\b\", r\"\\bменискусите\\b\", r\"\\bmenisci\\w*\\b\",\n)\nANY_SIDE = _rx(SIDE_MEDIAL.pattern, SIDE_LATERAL.pattern)\n\nSTEM_MENISCUS = _rx(r\"menisc\\w*\", r\"menisk\\w*\", r\"μηνισκ\\w*\", r\"мениск\\w*\")\nSTEM_CRUCIATE = _rx(r\"cruciate\", r\"cruzado\", r\"croise\", r\"kruisband\", r\"kreuzband\",\n                    r\"capraz bag\\w*\", r\"krizn\\w*\", r\"χιαστ\\w*\", r\"кръстн\\w*\",\n                    r\"\\bacl\\b\", r\"\\blca\\b\", r\"\\bvkb\\b\", r\"\\bocb\\b\", r\"\\bacb\\b\")\nSTEM_COLLATERAL = _rx(r\"collateral\\w*\", r\"colateral\\w*\", r\"kollateral\\w*\",\n                      r\"collaterale\\w*\", r\"kolateraln\\w*\", r\"yan bag\\w*\",\n                      r\"πλαγι\\w*\", r\"колатерал\\w*\", r\"странич\\w*\",\n                      r\"innenband\\w*\", r\"binnenband\\w*\", r\"\\bmcl\\b\", r\"\\blcm\\b\",\n                      r\"\\biyb\\b\")\n\nSTEM_FRACTURE = _rx(r\"fractur\\w*\", r\"fraktur\\w*\", r\"fractuur\\w*\", r\"\\bfract\\b\",\n                    r\"kiri[kgğ]\\w*\", r\"prijelom\\w*\", r\"lom kosti\", r\"\\bbreuk\\w*\",\n                    r\"\\bbruch\\w*\", r\"καταγμα\\w*\", r\"καταγματ\\w*\", r\"фрактур\\w*\",\n                    r\"счупван\\w*\", r\"fisur\\w* (osea|oseas|kost)\", r\"fissur\\w* kost\")\n\n# Decoys that steal a cruciate / collateral stem match for the wrong ligament.\nPOSTERIOR_ONLY = _rx(r\"\\bpcl\\b\", r\"\\blcp\\b\", r\"\\bhkb\\b\", r\"\\bacb\\b\",\n                     r\"posterior cruciate\", r\"cruzado posterior\", r\"croise posterieur\",\n                     r\"achterste kruisband\", r\"hinteres kreuzband\", r\"arka capraz\",\n                     r\"straznji krizn\", r\"οπισθι[οα]\\w* χιαστ\", r\"задна кръстн\",\n                     r\"задната кръстн\")\nLATERAL_COLL_ONLY = _rx(r\"\\blcl\\b\", r\"\\bfcl\\b\", r\"lateral collateral\",\n                        r\"fibular collateral\", r\"colateral lateral\", r\"colateral externo\",\n                        r\"buitenband\", r\"aussenband\", r\"dis yan bag\",\n                        r\"lateralni kolateraln\", r\"εξω πλαγι\", r\"латерален колатерал\")","metadata":{},"outputs":[],"execution_count":null},{"id":"e5ea2a8a-35e0-447d-8a30-c6fa7dcd2bdd","cell_type":"code","source":"def _near(clause: str, stem_rx: re.Pattern, qual_rx: re.Pattern, window: int = 55):\n    for m in stem_rx.finditer(clause):\n        lo = max(0, m.start() - window)\n        hi = min(len(clause), m.end() + window)\n        if qual_rx.search(clause[lo:hi]):\n            return True\n    return False\n\n\nSTEM_RULES = {\n    \"ACL\": (STEM_CRUCIATE, SIDE_ANTERIOR),\n    \"MCL\": (STEM_COLLATERAL, SIDE_MEDIAL),\n    \"Medial Meniscus\": (STEM_MENISCUS, SIDE_MEDIAL),\n    \"Lateral Meniscus\": (STEM_MENISCUS, SIDE_LATERAL),\n}\n\n\nclass _Matcher:\n    def __init__(self, phrase_rx, stem=None, side=None, window=55):\n        self.phrase_rx = phrase_rx\n        self.stem = stem\n        self.side = side\n        self.window = window\n\n    def search(self, clause):\n        m = self.phrase_rx.search(clause)\n        if m is not None:\n            return m\n        if self.stem is not None and _near(clause, self.stem, self.side, self.window):\n            return self.stem.search(clause)\n        return None\n\n\nANAT_MATCH = {t: _Matcher(ANAT[t], *STEM_RULES[t]) for t in PAIRED}\nDIRECT_MATCH = {\n    t: _Matcher(_rx(rx.pattern, STEM_FRACTURE.pattern) if t == \"Fracture\" else rx)\n    for t, rx in DIRECT.items()\n}\n\n# --------------------------------------------------------------------------- #\n# severity\n# --------------------------------------------------------------------------- #\nSEV_LOW = _rx(\n    r\"\\bsmall\\b\", r\"\\bminimal\\b\", r\"\\btrace\\b\", r\"\\bmild\\b\", r\"\\bslight\\b\",\n    r\"\\btiny\\b\", r\"\\bscant\\b\", r\"\\bdiscrete\\b\", r\"\\blow ?grade\\b\", r\"\\bincipient\\b\",\n    r\"\\bleve\\b\", r\"\\bminim\", r\"\\bpeque\", r\"\\bfina\\b\", r\"\\bfino\\b\", r\"\\bligero\\b\", r\"\\bescaso\\b\", r\"\\bdiscreto\\b\",\n    r\"\\bhafif\\b\", r\"\\baz miktarda\\b\", r\"\\bsilik\\b\",\n    r\"\\bmanj\\w*\", r\"\\bblago\\b\", r\"\\bdiskretn\", r\"\\bmalo\\b\", r\"\\bpocetn\",\n    r\"\\bgering\", r\"\\bdiskret\", r\"\\bkleine?r?\\b\", r\"\\bwenig\\b\", r\"\\bzarte?\\b\",\n    r\"\\bbeperkte?\\b\", r\"\\bgeringe\\b\", r\"\\bweinig\\b\", r\"\\blichte?\\b\", r\"\\blicht\\b\",\n    r\"\\bηπι\", r\"\\bμικρ\", r\"\\bελαχιστ\", r\"\\bαρχομεν\",\n    r\"\\bминимал\", r\"\\bлек\", r\"\\bмалк\", r\"\\bнеголям\",\n)\n\nSEV_HIGH = _rx(\n    r\"\\blarge\\b\", r\"\\bmarked\\b\", r\"\\bmassive\\b\", r\"\\bsevere\\b\", r\"\\bextensive\\b\",\n    r\"\\bmoderate\\b\", r\"\\bgross\\b\", r\"\\bsignificant\\b\", r\"\\babundant\\b\", r\"\\btense\\b\",\n    r\"\\bcomplete\\b\", r\"\\bfull ?thickness\\b\", r\"\\bhigh ?grade\\b\", r\"\\badvanced\\b\",\n    r\"\\bmoderad\", r\"\\bimportante\\b\", r\"\\bsevera?\\b\", r\"\\bmarcad\", r\"\\bcuantios\",\n    r\"\\bespesor total\\b\", r\"\\bcompleta?\\b\",\n    r\"\\bbelirgin\\b\", r\"\\byaygin\\b\", r\"\\bileri\\b\", r\"\\bciddi\\b\", r\"\\bbol\\b\", r\"\\bkomplet\",\n    r\"\\bopsezan\\b\", r\"\\bveliki\\b\", r\"\\bizrazit\", r\"\\bznacajn\", r\"\\bumjeren\",\n    r\"\\buznapredoval\", r\"\\bpotpun\", r\"\\bkompleksn\",\n    r\"\\bausgepragt\", r\"\\bdeutlich\", r\"\\bmassiv\", r\"\\bmassig\", r\"\\bgross\",\n    r\"\\buitgebreid\", r\"\\bgevorderd\", r\"\\bveel\\b\", r\"\\bmatige?\\b\", r\"\\bvolledig\",\n    r\"\\bμετρι\", r\"\\bμεγαλ\", r\"\\bεκτεταμεν\", r\"\\bευμεγεθ\", r\"\\bσοβαρ\", r\"\\bπληρη\",\n    r\"\\bголям\", r\"\\bизразен\", r\"\\bзначим\", r\"\\bумерен\", r\"\\bобилен\", r\"\\bпълн\",\n)\n\nGRADE_HIGH = re.compile(r\"grade?[ao]?\\s*(3|4|iii|iv)\\b|icrs grade (iii|iv|3|4)|\"\n                        r\"stupnja iv|stupnja iii|\\bgrado (3|4)\\b|\\bgrad (3|4)\\b|\"\n                        r\"\\bgrade (3|4)\\b\")\n\nDEGENERATIVE_MARROW = _rx(\n    r\"subchondral\", r\"subcondral\", r\"subkondral\", r\"supkondraln\", r\"subchondraln\",\n    r\"υποχονδρι\", r\"υπαρθρικ\", r\"субхондрал\", r\"subchondrale?\", r\"subartikuler\",\n    r\"\\bcyst\", r\"\\bquist\", r\"\\bzyste\\b\", r\"\\bcistic\", r\"reactive\", r\"reactivo\",\n    r\"degenerative\", r\"degenerativ\", r\"reaktiv\", r\"\\bcisti\\b\",\n)\n\nTRAUMA = _rx(\n    r\"\\bbruise\\b\", r\"\\bcontusion\", r\"\\bkontuz\", r\"\\btrauma\", r\"\\bimpaction\\b\",\n    r\"\\bpivot shift\\b\", r\"\\bkissing\\b\", r\"\\bacute\\b\", r\"\\bagudo\\b\", r\"\\bakut\",\n    r\"\\bpivot kaymasi\\b\", r\"\\bcontusion osseuse\\b\", r\"\\bbone bruise\\b\",\n    r\"\\bbotcontusie\\b\", r\"\\bконтузион\", r\"\\bμωλωπ\", r\"\\bkontuzij\", r\"\\bimpaktcij\",\n    r\"\\bimpakcij\", r\"\\bfall\\b\", r\"\\binjury\\b\", r\"\\bimpression\\b\",\n)\n\n# Effusion-adjacent inflammatory signs, used only when a report never names synovitis.\nSYNOVIAL_PROXY = _rx(\n    r\"bursit\", r\"burzit\", r\"\\bbursa\\b[^.]{0,30}(fluid|distend|sivi|tekucin|opzetting)\",\n    r\"suprapatellar (bursitis|effusion|recess)\", r\"suprapatellar bursa\",\n    r\"suprapatellar bursada\", r\"suprapatelarno\", r\"suprapatellaire recessus\",\n    r\"hoffa\", r\"hoffit\", r\"plica\", r\"plika\", r\"πλικα\", r\"fat pad[^.]{0,20}(edema|oedema)\",\n    r\"kapsul\", r\"capsul\", r\"καψ\", r\"капсул\", r\"\\bpannus\\b\", r\"\\bsinov\", r\"\\bsynov\",\n)\n\n\ndef _polarity(clause: str, span=None) -> str:\n    \"\"\"positive / negative / uncertain for a term matched at `span` in this clause.\"\"\"\n    if UNCERTAIN.search(clause):\n        return \"uncertain\"\n    if span is None or not FEATURES[\"directional_negation\"]:\n        if NEGATION.search(clause):\n            return \"negative\"\n    elif _negated(clause, span[0], span[1]):\n        return \"negative\"\n    if NORMALITY.search(clause):\n        # `meniscus normal` denies; `normal alignment ... full thickness tear` does not.\n        if TEAR.search(clause) or GRADE_HIGH.search(clause):\n            return \"positive\"\n        return \"negative\"\n    return \"positive\"\n\n\ndef _severity(clause: str) -> float:\n    \"\"\"How emphatic a clause is about the finding it asserts.\n\n    A numeric grade is deliberately not read here. It belongs to whichever structure the\n    grade was written for, and a clause that grades the cartilage while mentioning the\n    effusion in passing - `PF arthrotic change with reduced joint space, a smaller\n    effusion and grade IV chondromalacia` - would otherwise report a severe effusion.\n    \"\"\"\n    high = SEV_HIGH.search(clause) is not None\n    low = SEV_LOW.search(clause) is not None\n    if high and not low:\n        return 1.0\n    if low and not high:\n        return 0.45\n    if high and low:\n        return 0.8\n    return 0.75\n\n\ndef _grade(n_pos, n_neg, n_unc, best):\n    \"\"\"Map counted evidence onto a score in (0, 1) and a confidence.\"\"\"\n    if n_pos or n_unc:\n        score = min(0.97, 0.50 + 0.45 * best + 0.015 * min(n_pos, 3))\n        conf = min(1.0, 0.55 + 0.15 * n_pos)\n    elif n_neg:\n        score = max(0.04, 0.20 - 0.04 * n_neg)\n        conf = min(0.9, 0.45 + 0.12 * n_neg)\n    else:\n        score, conf = 0.28, 0.05\n    return score, conf\n\n\ndef _paired_weight(clause: str, meniscus: bool) -> float:\n    \"\"\"How strongly one clause asserts damage to a meniscus or a cruciate/collateral.\n\n    Ordered, not calibrated. What has to hold is that a tear outranks a graded lesion,\n    that the grade is read on the right scale for the structure, and that intrasubstance\n    degeneration lands below both - the annotator marks a torn meniscus and leaves a\n    degenerate one, and a lexicon that scores the two alike throws that ordering away.\n    \"\"\"\n    g = _grade_of(clause) if FEATURES[\"graded_pathology\"] else None\n    tear = TEAR.search(clause) is not None\n    if meniscus:\n        if tear:\n            base = 1.0\n        elif g is not None:\n            base = 0.95 if g >= 3 else 0.30\n        elif DEGEN.search(clause):\n            base = 0.35\n        else:\n            base = 0.45\n    else:\n        if tear:\n            base = 1.0\n        elif g is not None:\n            base = 0.85 if g >= 2 else 0.30\n        elif DEGEN.search(clause):\n            base = 0.40\n        else:\n            base = 0.55\n    if SEV_HIGH.search(clause) and not SEV_LOW.search(clause):\n        base = min(1.0, base * 1.2)\n    elif SEV_LOW.search(clause) and not SEV_HIGH.search(clause):\n        base *= 0.7\n    return base\n\n\ndef _score_paired(cls, tgt):\n    \"\"\"Evidence for one of the four side-specific ligament / meniscus targets.\"\"\"\n    anat_rx = ANAT_MATCH[tgt]\n    path_rx = _rx(TEAR.pattern, DEGEN.pattern, INJURY.pattern)\n    meniscus = \"Meniscus\" in tgt\n    n_pos = n_neg = n_unc = 0\n    best = 0.0\n    for c in cls:\n        hit = anat_rx.search(c)\n        if hit is None and meniscus and PLURAL_MENISCI.search(c) \\\n                and not ANY_SIDE.search(c):\n            hit = PLURAL_MENISCI.search(c)\n        if hit is None:\n            continue\n        # Anchor the negation test on the pathology word, not on the anatomy word.\n        # `Medial meniscus is not torn` negates the tear, and the negator stands after\n        # the noun: scoping from the noun would read the sentence as an assertion.\n        pm = path_rx.search(c)\n        if pm is None and _grade_of(c) is None:\n            if NORMAL_PHRASE.search(c) or (NORMALITY.search(c)\n                                           and not NEGATION.search(c)):\n                n_neg += 1\n            continue\n        span = (pm.start(), pm.end()) if pm is not None else None\n        pol = _polarity(c, span)\n        if pol == \"positive\":\n            n_pos += 1\n            best = max(best, _paired_weight(c, meniscus))\n        elif pol == \"negative\":\n            n_neg += 1\n        else:\n            n_unc += 1\n            best = max(best, 0.45 * _paired_weight(c, meniscus))\n    s, cf = _grade(n_pos, n_neg, n_unc, best)\n    return s, cf, n_pos, n_neg\n\n\ndef _score_clauses(cls, anat_rx, path_rx=None, decoy_rx=None, context_penalty=None,\n                   context_bonus=None):\n    n_pos = n_neg = n_unc = 0\n    best = 0.0\n    for c in cls:\n        m = anat_rx.search(c)\n        if not m:\n            continue\n        if decoy_rx is not None and decoy_rx.search(c):\n            continue\n        if path_rx is not None and not path_rx.search(c):\n            if NORMAL_PHRASE.search(c) or (NORMALITY.search(c)\n                                           and not NEGATION.search(c)):\n                n_neg += 1\n            continue\n        pol = _polarity(c, (m.start(), m.end()))\n        if pol == \"positive\":\n            n_pos += 1\n            w = _severity(c)\n            if context_penalty is not None and context_penalty.search(c):\n                w *= 0.45\n            if context_bonus is not None and context_bonus.search(c):\n                w = min(1.0, w * 1.35)\n            best = max(best, w)\n        elif pol == \"negative\":\n            n_neg += 1\n        else:\n            n_unc += 1\n            best = max(best, 0.30)\n    s, c = _grade(n_pos, n_neg, n_unc, best)\n    return s, c, n_pos, n_neg\n\n\ndef _score_oa(cls):\n    \"\"\"Osteoarthritis, scoped to the three compartments.\n\n    A cartilage statement is attributed by what else the clause names rather than by a\n    compartment phrase: `medial` next to a tibiofemoral structure sends it to the medial\n    compartment, `patella` or `trochlea` to the patellofemoral one, and a statement that\n    names neither is a whole-joint assertion that counts for all three at a discount.\n    That last case is what the public lexicon leaves silent, and it is the majority of\n    the corpus - reports say `tricompartmental chondrosis`, not `chondrosis of the\n    medial femorotibial compartment`.\n    \"\"\"\n    acc = {t: {\"pos\": 0, \"neg\": 0, \"unc\": 0, \"best\": 0.0} for t in OA_TARGETS}\n    g_pos, g_neg, g_best = 0, 0, 0.0\n\n    for c in cls:\n        m = OA_EVIDENCE.search(c)\n        if not m:\n            continue\n        pol = _polarity(c, (m.start(), m.end()))\n        sev = _severity(c)\n        tf_med = _near(c, TF_SITE, SIDE_MEDIAL, 45)\n        tf_lat = _near(c, TF_SITE, SIDE_LATERAL, 45)\n        pf = PF_SITE.search(c) is not None\n        hits = []\n        if tf_med:\n            hits.append(\"Medial OA\")\n        if tf_lat:\n            hits.append(\"Lateral OA\")\n        if pf:\n            hits.append(\"PF OA\")\n\n        if not hits:\n            # No compartment named. Whole-joint statements (\"tricompartmental\n            # chondrosis\", \"gonarthrose\") speak for all three; an unlocalised cartilage\n            # remark is weaker evidence but still evidence, so it is carried at a\n            # discount rather than dropped.\n            if pol == \"positive\":\n                g_pos += 1\n                g_best = max(g_best, sev if GLOBAL_OA.search(c) else sev * 0.7)\n            elif pol == \"negative\":\n                g_neg += 1\n            continue\n\n        for t in hits:\n            if pol == \"positive\":\n                acc[t][\"pos\"] += 1\n                acc[t][\"best\"] = max(acc[t][\"best\"], sev)\n            elif pol == \"negative\":\n                acc[t][\"neg\"] += 1\n            else:\n                acc[t][\"unc\"] += 1\n                acc[t][\"best\"] = max(acc[t][\"best\"], 0.30)\n\n    out = {}\n    for t in OA_TARGETS:\n        a = acc[t]\n        pos, neg, unc, best = a[\"pos\"], a[\"neg\"], a[\"unc\"], a[\"best\"]\n        if not (pos or unc) and g_pos and FEATURES[\"oa_inherit\"]:\n            # Inherit the whole-joint statement, unless this compartment was separately\n            # and explicitly cleared.\n            if neg:\n                score, conf = _grade(0, neg, 0, 0.0)\n                score = max(score, 0.35)\n                conf *= 0.7\n            else:\n                score, conf = _grade(g_pos, 0, 0, g_best * 0.92)\n                conf *= 0.75\n        else:\n            score, conf = _grade(pos, neg + g_neg, unc, best)\n        out[t] = (score, conf, pos, neg)\n    return out\n\n\ndef extract(report: str) -> dict:\n    \"\"\"Twelve (score, confidence) pairs, plus the counts the coverage gauge reads.\"\"\"\n    cls = clauses(report)\n    out = {}\n\n    for tgt in PAIRED:\n        s, c, npos, nneg = _score_paired(cls, tgt)\n        out[tgt] = s\n        out[tgt + \"__conf\"] = c\n        out[tgt + \"__npos\"] = npos\n        out[tgt + \"__nneg\"] = nneg\n\n    for tgt, (s, c, npos, nneg) in _score_oa(cls).items():\n        out[tgt] = s\n        out[tgt + \"__conf\"] = c\n        out[tgt + \"__npos\"] = npos\n        out[tgt + \"__nneg\"] = nneg\n\n    for tgt in (\"Effusion\", \"Synovitis\", \"Baker's\", \"Contusion\", \"Fracture\"):\n        if tgt == \"Contusion\":\n            s, c, npos, nneg = _score_clauses(\n                cls, DIRECT_MATCH[tgt], None, DECOY.get(tgt),\n                context_penalty=DEGENERATIVE_MARROW, context_bonus=TRAUMA)\n        else:\n            s, c, npos, nneg = _score_clauses(cls, DIRECT_MATCH[tgt], None,\n                                              DECOY.get(tgt))\n        out[tgt] = s\n        out[tgt + \"__conf\"] = c\n        out[tgt + \"__npos\"] = npos\n        out[tgt + \"__nneg\"] = nneg\n\n    # --- synovitis backoff --------------------------------------------------- #\n    # Eighty-eight per cent of reports never write the word, and the annotator marks it\n    # on nearly half the studies: the label cannot be read off the term alone. What the\n    # report does say is whether the joint is wet and irritated - an effusion, a\n    # distended bursa, an inflamed fat pad - and that ordering is the only signal\n    # available on the silent majority. It enters at low confidence, so it shapes the\n    # ranking without asserting a finding.\n    if (FEATURES[\"synovitis_backoff\"] and out[\"Synovitis__npos\"] == 0\n            and out[\"Synovitis__nneg\"] == 0):\n        proxy = sum(1 for c in cls if SYNOVIAL_PROXY.search(c)\n                    and _polarity(c) == \"positive\")\n        eff = out[\"Effusion\"]\n        prior = 0.30 + 0.30 * max(0.0, (eff - 0.5) / 0.45) + 0.06 * min(proxy, 3)\n        out[\"Synovitis\"] = min(0.72, prior)\n        out[\"Synovitis__conf\"] = 0.18\n\n    return out","metadata":{},"outputs":[],"execution_count":null},{"id":"1e131efb-2032-4886-9986-5ba8eae0735a","cell_type":"markdown","source":"### 2.2 How an extractor like this is validated\n\nThis is the part that decides whether any of the above is worth trusting, and it is harder\nthan writing the rules, because the obvious measurement cannot carry the weight.\n\n**The dangerous failure is silent.** A rule that never fires does not raise an error — it\nemits a negative, indistinguishable from a confident one. A lexicon complete in English and\nthin in Greek does not look broken; it looks like a corpus in which Greek patients have\nfewer findings. And the error is not random: language tracks the reporting institution,\nwhich tracks the scanner and the population, so a gap in one language is a bias aligned with\na site, not noise that averages out.\n\n**Gauge one — agreement, on the 58 annotated studies.** For each target, compare the\nextracted score against the annotation and read the AUC. This measures exactly the right\nthing and is nearly useless for tuning, because the subset is tiny. The Hanley–McNeil\napproximation for the standard error of an AUC $A$ with $n_p$ positives and $n_n$ negatives,\n\n$$\\mathrm{SE}(A)=\\sqrt{\\frac{A(1-A)+(n_p-1)(Q_1-A^{2})+(n_n-1)(Q_2-A^{2})}{n_p n_n}},\\qquad\nQ_1=\\frac{A}{2-A},\\quad Q_2=\\frac{2A^{2}}{1+A},$$\n\nat $A\\approx0.85$ with nine positives among 58 studies lands near $0.083$, so a 95% interval\nspans roughly $\\pm0.16$. Competitions are decided by differences an order of magnitude\nsmaller. Choosing between two lexicons on a single one of these numbers is choosing by coin\nflip, and it will feel like signal every time. The intervals are drawn below so that this\nis visible rather than stated.\n\n**Gauge two — coverage, on all 4 407 reports.** For each (study, target) pair, record\nwhether *any* rule fired — assertion, negation or hedge. The **silence rate** is the fraction\nwhere none did. It needs no labels, so it runs on the whole corpus rather than on the\nannotated handful, and it points straight at missing vocabulary.\n\n| | measures | sample | can decide |\n|---|---|---|---|\n| agreement | is a fired rule *right* | 58 | whether a target's labels are usable at all |\n| silence | does a rule *fire* | 4 407 | which finding to work on next |\n\nNeither substitutes for the other, and the second is the one to steer by. A common finding\nthat is silent in one language and not another is a lexicon gap. A rare finding that is\nsilent nearly everywhere is simply rare, and silence there is correct.\n\n**One limit worth naming.** The 58 annotations were read from the images; the reports were\nwritten by a different radiologist on a different day. They disagree — a report saying\n*moderate joint effusion* against an annotation of 0, an annotation of Baker's cyst against\na report that calls the study normal. That disagreement is not extractor error and no\nlexicon can remove it. It puts a ceiling on gauge one somewhere well below 1.0, which is\nanother reason to read the intervals rather than the point estimates.","metadata":{}},{"id":"ebd78c51-5b30-4767-8f4e-efae43957087","cell_type":"code","source":"import os\nimport time\nimport warnings\nfrom pathlib import Path\n\nimport numpy as np\nimport pandas as pd\n\nwarnings.filterwarnings(\"ignore\")\nT0 = time.time()\n\n\ndef log(msg):\n    print(f\"[{time.time() - T0:7.1f}s] {msg}\", flush=True)\n\n\ndef find_root():\n    \"\"\"Locate the competition mount, wherever it was attached.\"\"\"\n    for c in [Path(\"/kaggle/input/rsna-knee-abnormality-detection\"),\n              Path(\"/kaggle/input/competitions/rsna-knee-abnormality-detection\"),\n              Path(\"data\"), Path(\".\")]:\n        if (c / \"test.csv\").is_file() and (c / \"test_series\").is_dir():\n            return c\n    base = Path(\"/kaggle/input\")\n    if base.is_dir():\n        for d1 in sorted(p for p in base.iterdir() if p.is_dir()):\n            for cand in [d1] + sorted(p for p in d1.iterdir() if p.is_dir()):\n                if (cand / \"test.csv\").is_file():\n                    return cand\n    raise FileNotFoundError(\"competition mount not found\")\n\n\nROOT = find_root()\nlog(f\"input root: {ROOT}\")\n\n# A submission that never writes scores nothing at all, which is strictly worse than\n# scoring badly. Every stage below is wrapped so that it cannot raise, and a valid file\n# exists from the first second - overwritten only once real predictions are ready.\n_test_df = pd.read_csv(ROOT / \"test.csv\")\n_bench = _test_df[[\"StudyInstanceUID\"]].copy()\nfor _c in TARGETS:\n    _bench[_c] = 0.5\n_bench.to_csv(\"submission.csv\", index=False)\nlog(f\"benchmark submission.csv written ({len(_bench)} rows)\")\n\nSTAGE_OK = {}\n\n\ndef stage(name):\n    \"\"\"Run a stage; record failure instead of raising it.\n\n    A rerun that dies halfway has still paid for everything it did before the failure,\n    and the benchmark file above is worth more than a traceback. So each stage records\n    whether it succeeded and the next one checks before proceeding.\n    \"\"\"\n    def deco(fn):\n        def run(*a, **k):\n            t = time.time()\n            try:\n                out = fn(*a, **k)\n                STAGE_OK[name] = True\n                log(f\"stage '{name}' ok in {time.time() - t:.1f}s\")\n                return out\n            except Exception:\n                import traceback\n                traceback.print_exc()\n                STAGE_OK[name] = False\n                log(f\"stage '{name}' FAILED after {time.time() - t:.1f}s\")\n                return None\n        return run\n    return deco\n\n\ntrain_df = pd.read_csv(ROOT / \"train.csv\")\nlog(f\"train {train_df.shape}  test {_test_df.shape}\")\n\nt = time.time()\nLAB = pd.DataFrame([extract(r) for r in train_df[\"Report\"].fillna(\"\")])\nLAB[\"StudyInstanceUID\"] = train_df[\"StudyInstanceUID\"].values\nLAB = LAB.set_index(\"StudyInstanceUID\")\nlog(f\"read {len(LAB)} reports in {time.time() - t:.1f}s\")\n\nGOLD = train_df.dropna(subset=TARGETS).set_index(\"StudyInstanceUID\")[TARGETS]\nlog(f\"{len(GOLD)} studies carry the twelve annotations\")\n\npos = (LAB[TARGETS] > 0.5).mean()\nsil = pd.Series({t_: float(((LAB[t_ + \"__npos\"] == 0) & (LAB[t_ + \"__nneg\"] == 0)).mean())\n                 for t_ in TARGETS})\nprint(pd.DataFrame({\"derived positive rate\": pos.round(3),\n                    \"silence rate\": sil.round(3),\n                    \"annotated positive rate\": GOLD.mean().round(3)}).to_string())","metadata":{},"outputs":[],"execution_count":null},{"id":"4d1ba117-d9fe-43fc-88c5-d722e7be6d23","cell_type":"code","source":"import matplotlib.pyplot as plt\nfrom sklearn.metrics import roc_auc_score\n\nplt.rcParams.update({\"figure.dpi\": 120, \"font.size\": 8, \"axes.grid\": True,\n                     \"grid.alpha\": 0.25, \"axes.spines.top\": False,\n                     \"axes.spines.right\": False})\nINK, ACC, WARN = \"#22303f\", \"#2b7a9b\", \"#c25a3d\"\n\n\ndef agreement(lab, n_boot=2000, seed=0):\n    \"\"\"Per-target AUC against the annotated studies, with a bootstrap interval.\"\"\"\n    rng = np.random.default_rng(seed)\n    g = lab.loc[GOLD.index]\n    rows = []\n    for t_ in TARGETS:\n        y = GOLD[t_].values.astype(int)\n        p = g[t_].values\n        if len(set(y)) < 2:\n            rows.append((t_, np.nan, np.nan, np.nan, int(y.sum()), int((1 - y).sum())))\n            continue\n        a = roc_auc_score(y, p)\n        bs = []\n        for _ in range(n_boot):\n            i = rng.integers(0, len(y), len(y))\n            if len(set(y[i])) > 1:\n                bs.append(roc_auc_score(y[i], p[i]))\n        rows.append((t_, a, np.percentile(bs, 2.5), np.percentile(bs, 97.5),\n                     int(y.sum()), int((1 - y).sum())))\n    return pd.DataFrame(rows, columns=[\"target\", \"auc\", \"lo\", \"hi\", \"npos\", \"nneg\"])\n\n\nAGREE = agreement(LAB).dropna(subset=[\"auc\"])\nprint(AGREE.round(3).to_string(index=False))\nprint(f\"\\nmacro agreement AUC: {AGREE.auc.mean():.4f}   \"\n      f\"mean silence rate: {sil.mean() * 100:.1f}%\")\n\nfig, ax = plt.subplots(1, 2, figsize=(10.5, 3.6),\n                       gridspec_kw={\"width_ratios\": [1.25, 1]})\no = AGREE.sort_values(\"auc\")\ny = np.arange(len(o))\nax[0].hlines(y, o.lo, o.hi, color=ACC, lw=3, alpha=.35)\nax[0].plot(o.auc, y, \"o\", color=ACC, ms=5)\nax[0].axvline(0.5, color=INK, lw=.8, ls=\":\")\nax[0].axvline(o.auc.mean(), color=WARN, lw=1, ls=\"--\")\nax[0].text(o.auc.mean(), -0.9, f\" macro {o.auc.mean():.3f}\", color=WARN, fontsize=7)\nax[0].set_ylim(-1.4, len(o) - 0.4)\nax[0].set_yticks(y)\nax[0].set_yticklabels([f\"{t_}  ({p}+/{n}-)\" for t_, p, n in zip(o.target, o.npos, o.nneg)])\nax[0].set_xlim(0.35, 1.02)\nax[0].set_xlabel(\"AUC of the derived score against the annotation\")\nax[0].set_title(\"gauge one: agreement, n = 58\\n\"\n                \"bars are 95% bootstrap intervals — they are this wide on purpose\",\n                loc=\"left\", fontsize=8)\n\no2 = sil.sort_values()\nax[1].barh(np.arange(len(o2)), o2.values * 100, color=INK, alpha=.8, height=.65)\nax[1].set_yticks(np.arange(len(o2)))\nax[1].set_yticklabels(o2.index)\nax[1].set_xlabel(\"% of studies where no rule fired at all\")\nax[1].set_title(\"gauge two: coverage, n = 4 407\\n\"\n                \"no labels needed, so it runs on the whole corpus\",\n                loc=\"left\", fontsize=8)\nfor i, v in enumerate(o2.values * 100):\n    ax[1].text(v + 1, i, f\"{v:.0f}\", va=\"center\", fontsize=6.5, color=INK)\nfig.tight_layout()\nplt.show()","metadata":{},"outputs":[],"execution_count":null},{"id":"43e9dd0d-8f47-4556-bd59-dfd005dfe2f1","cell_type":"code","source":"def macro_and_silence():\n    lab = pd.DataFrame([extract(r) for r in train_df[\"Report\"].fillna(\"\")])\n    lab[\"StudyInstanceUID\"] = train_df[\"StudyInstanceUID\"].values\n    lab = lab.set_index(\"StudyInstanceUID\")\n    g = lab.loc[GOLD.index]\n    a = float(np.nanmean([\n        roc_auc_score(GOLD[t_].values.astype(int), g[t_].values)\n        if GOLD[t_].nunique() > 1 else np.nan for t_ in TARGETS]))\n    s = float(np.mean([((lab[t_ + \"__npos\"] == 0) & (lab[t_ + \"__nneg\"] == 0)).mean()\n                       for t_ in TARGETS]))\n    return a, s\n\n\nbase_a, base_s = macro_and_silence()\nrows = [(\"all rules on\", base_a, base_s, 0.0, 0.0)]\nfor k in list(FEATURES):\n    FEATURES[k] = False\n    a, s = macro_and_silence()\n    rows.append((f\"without {k}\", a, s, a - base_a, s - base_s))\n    FEATURES[k] = True\n\nABL = pd.DataFrame(rows, columns=[\"configuration\", \"macro AUC\", \"mean silence\",\n                                  \"d(AUC)\", \"d(silence)\"])\nprint(ABL.round(4).to_string(index=False))","metadata":{},"outputs":[],"execution_count":null},{"id":"ee292f17-5dce-443a-9e5d-83d7afcf13bb","cell_type":"markdown","source":"Read that table against §2.2 rather than as a ranking. Three of the five deltas are smaller\nthan the sampling error of a 58-study AUC, so the honest reading is:\n\n* **`oa_inherit` is the one change large enough to see on gauge one.** It is also the one\n  whose effect is visible on gauge two, and the two agree in direction. That is the only\n  combination that justifies confidence at this sample size.\n* **`unwrap` barely moves agreement and clearly improves coverage.** Kept for the second\n  reason. Wrapped lines are a property of how a site exports its reports, not of what those\n  reports say, and a rule that repairs the export cannot be wrong about the medicine.\n* The rest are kept because the sentence-level argument for them is sound and their\n  measured effect is not negative — not because 58 studies said so.\n\nMost of the total quality of these labels is not in this table at all: it is in the\nvocabulary, and vocabulary shows up on the silence rate rather than on the ablation. The\nfindings the reports mention least are the ones §1 says are most expensive to leave at\nchance, and they are exactly where the silence bars are tallest — `Fracture`,\n`Lateral OA`, `Synovitis`. That is the standing to-do list for this notebook, and it is\nreadable from a gauge that needs no labels at all.\n\nTwo further points about the targets before moving to the pixels.\n\n**Confidence becomes a sample weight.** Every target carries a confidence alongside its\nscore: high when several clauses agree, low when a finding was named once in passing, and\nvery low when nothing fired. That number multiplies the per-element loss in §7, so silence\npulls weakly rather than asserting a negative.\n\n**Synovitis is a special case, handled explicitly.** Nearly nine studies in ten never use\nthe word, and the annotator marks it on close to half. No term list can close that. What a\nreport does say is whether the joint is wet and irritated — an effusion, a distended bursa,\nan inflamed fat pad — and on the silent majority that ordering is the only signal there is.\nIt enters at low confidence, so it shapes the ranking without asserting a finding. This is\nthe weakest of the twelve target derivations and the notebook says so rather than hiding it\nin an average.","metadata":{}},{"id":"919b2461-f596-4896-93f9-23b150761944","cell_type":"markdown","source":"### 2.3 Reading the coverage gauge, which is the point of having it\n\nThe silence rate is only useful if it is broken down. Aggregated over the corpus it says\n\"this target is thin\", which anyone could have guessed. Broken down by language it says\n*where the vocabulary is missing*, and that is a to-do list.\n\nThe two figures below are the ones that actually drove the last several revisions of the\nlexicon, so it is worth being precise about how to read them — including the way the left\none can mislead.\n\n**Silence has two causes, and only one of them is a bug.** A target is silent either\nbecause the report never discusses that structure, or because it does and no rule fired.\nThe first is correct behaviour and no amount of vocabulary fixes it; the second is a gap.\nThey look identical in the left figure and are separated in the right one, by asking a\nquestion that needs no labels: does the report contain *any* word for this structure at\nall?\n\nFor the Spanish cruciate cell that split came out 62% never-mentioned against 38%\nmentioned-and-missed. The 62% are short reports — a median of 336 characters against 1257\nfor the ones that do mention it — stating findings and staying silent on everything\nnormal. Nothing there to fix. The 38% were a genuine gap, and reading a handful of them\nshowed what it was:\n\n> `Ligamentos cruzados y colaterales sin alteraciones significativas.`\n\nThat asserts the cruciates are *normal*. The extractor missed it, because \"sin\nalteraciones\" is a negator plus an abnormality noun — read literally it is a negation, not\na normality phrase, and the rule that recognised normality explicitly refused to fire when\na negator was present. So a ligament a radiologist had looked at and called intact was\nscored the same as one the report never mentioned. To a rank metric those are not the\nsame: explicitly clear must rank *below* unmentioned.\n\nThat construction is how most of these languages assert normality — `ohne\nAuffälligkeiten`, `geen afwijkingen`, `bez osobitosti`, `χωρίς αλλοιώσεις`, `без\nособености` — and adding it dropped the mean silence over all 4 407 studies and twelve\ntargets from 49.97% to 48.09%, concentrated exactly where predicted: Greek −15 points on\nthe ligament and meniscus targets, Spanish −6, Bulgarian/Russian −4, Turkish and Croatian\nunchanged because they were already covered by a different construction.\n\nThat cell is now essentially closed. Spanish studies silent on the cruciate fell from 215\nto 141, and of the 141 that remain, **5%** name a cruciate at all — a median report length\nof 230 characters says the rest simply do not discuss it. The work moved from the\nfixable half of the bar to the half that is not a bug.\n\nThe same figure then pointed at Dutch, where `normaal voorkomen menisci` — the bare plural,\nnaming no side — left both meniscal targets silent. The cruciates and the collaterals had\ntheir plural forms from the start; the menisci had simply been missed.\n\n**What gauge one said about all of this: nothing.** Agreement on the 58 annotated studies\nmoved by −0.0007 macro AUC, 95% CI [−0.0035, +0.0009] — a flat zero. That is not evidence\nthe change was worthless. It is evidence that 58 studies, most of them English, cannot see\na Greek and Spanish coverage fix at all. Steering by that number would have rejected the\nwork; steering by the gauge that runs on all 4 407 found it.","metadata":{}},{"id":"717b1db5-792d-4828-ac77-24bef0512b06","cell_type":"code","source":"_SCRIPT = {\"el\": re.compile(r\"[Ͱ-Ͽ]\"), \"bg/ru\": re.compile(r\"[Ѐ-ӿ]\")}\n_STOP = {\n    \"en\": r\"\\b(the|and|is|with|there is|normal)\\b\",\n    \"es\": (r\"\\b(del|los|las|con|sin|senal|rodilla|hallazgos|tecnica|resultados\"\n           r\"|impresion|menisco|rotura)\\b\"),\n    \"fr\": r\"\\b(des|les|avec|sans|genou|aucune)\\b\",\n    \"nl\": r\"\\b(van|het|een|geen|met|voorste|knie)\\b\",\n    \"de\": r\"\\b(der|die|und|mit|ohne|kein|keine|nachweis)\\b\",\n    \"tr\": r\"\\b(ve|ile|izlenmistir|mevcut|normaldir|diz|bulgular)\\b\",\n    \"hr\": r\"\\b(se|te|uz|bez|prikaz|uredan|koljena|meniska)\\b\",\n}\n_STOP = {k: re.compile(v) for k, v in _STOP.items()}\n\n\ndef guess_language(report):\n    \"\"\"A crude language tag, used only to *audit* the reading - never to do it.\n\n    §2 argued against routing a report to a per-language rule set, because that commits\n    to a guess before any evidence is read. None of that applies here: this classifier\n    never touches extraction. It exists so the silence rate can be broken down, and a\n    tag that is wrong now and then blurs the breakdown rather than corrupting a label.\n    Script settles Greek and Cyrillic outright; the Latin-script languages are separated\n    by counting function words, which is ugly and sufficient for a histogram.\n    \"\"\"\n    n = normalize(report)\n    for tag, rx in _SCRIPT.items():\n        if rx.search(n):\n            return tag\n    score = {k: len(rx.findall(n)) for k, rx in _STOP.items()}\n    best = max(score, key=score.get)\n    return best if score[best] >= 2 else \"?\"\n\n\nLANG = pd.Series([guess_language(r) for r in train_df[\"Report\"].fillna(\"\")],\n                 index=train_df[\"StudyInstanceUID\"])\nprint(LANG.value_counts().to_string())\n\nSIL = pd.DataFrame({t: ((LAB[t + \"__npos\"] == 0) & (LAB[t + \"__nneg\"] == 0)).values\n                    for t in TARGETS}, index=LAB.index)\nby_lang = SIL.groupby(LANG.reindex(SIL.index).values).mean() * 100\nby_lang = by_lang.loc[LANG.value_counts().index.intersection(by_lang.index)]\n\nfig, ax = plt.subplots(1, 2, figsize=(12, 3.6),\n                       gridspec_kw={\"width_ratios\": [1.7, 1]})\nim = ax[0].imshow(by_lang.values, cmap=\"RdYlBu_r\", vmin=0, vmax=100, aspect=\"auto\")\nax[0].set_xticks(range(len(TARGETS)))\nax[0].set_xticklabels(TARGETS, rotation=55, ha=\"right\", fontsize=6.5)\nax[0].set_yticks(range(len(by_lang)))\nax[0].set_yticklabels([f\"{l}  (n={int((LANG == l).sum())})\" for l in by_lang.index],\n                      fontsize=7)\nax[0].grid(False)\nfor i in range(by_lang.shape[0]):\n    for j in range(by_lang.shape[1]):\n        v = by_lang.values[i, j]\n        ax[0].text(j, i, f\"{v:.0f}\", ha=\"center\", va=\"center\", fontsize=5.5,\n                   color=\"white\" if v > 62 or v < 12 else INK)\nax[0].set_title(\"gauge two, broken down: % of studies where no rule fired\\n\"\n                \"a common finding silent in one language and not another is a \"\n                \"lexicon gap — or a reporting style\", loc=\"left\", fontsize=8)\nfig.colorbar(im, ax=ax[0], fraction=0.02, pad=0.01)\n\n# Silence has two causes needing different work, so split them: of the studies where\n# nothing fired, in how many does the report name *this* structure anyway?\n#\n# The test has to be the target's own anatomy matcher, not a bare stem for the organ. A\n# report discussing only the medial meniscus contains the word \"meniscus\", and counting\n# that as a missed *lateral* meniscus would score correct silence as a bug - the wrong\n# direction entirely for a gauge whose job is to find real gaps.\nCLAUSES = {u: clauses(r) for u, r in\n           zip(train_df[\"StudyInstanceUID\"], train_df[\"Report\"].fillna(\"\"))}\n\n\ndef names_it(uid, target):\n    cs = CLAUSES[uid]\n    if any(ANAT_MATCH[target].search(c) for c in cs):\n        return True\n    if \"Meniscus\" in target:\n        return any(PLURAL_MENISCI.search(c) and not ANY_SIDE.search(c) for c in cs)\n    return False\n\n\nrows = []\nfor t_ in [\"ACL\", \"MCL\", \"Medial Meniscus\", \"Lateral Meniscus\"]:\n    sel = SIL[t_].values\n    if not sel.sum():\n        continue\n    named = np.array([names_it(u, t_) for u in SIL.index[sel]])\n    rows.append((t_, int(sel.sum()), 100 * named.mean()))\nD = pd.DataFrame(rows, columns=[\"target\", \"silent\", \"names it anyway\"])\ny = np.arange(len(D))\nax[1].barh(y, 100 - D[\"names it anyway\"], color=\"#8a97a3\", label=\"never mentioned\")\nax[1].barh(y, D[\"names it anyway\"], left=100 - D[\"names it anyway\"], color=WARN,\n           label=\"mentioned, missed\")\nax[1].set_yticks(y)\nax[1].set_yticklabels([f\"{t_}\\n({n} silent)\" for t_, n in zip(D.target, D.silent)],\n                      fontsize=6.5)\nax[1].set_xlabel(\"% of the silent studies\")\nax[1].legend(fontsize=6.5, frameon=False, loc=\"lower right\")\nax[1].set_title(\"why it was silent\\nonly the orange half is a lexicon gap\",\n                loc=\"left\", fontsize=8)\nfig.tight_layout()\nplt.show()\nprint(D.round(1).to_string(index=False))","metadata":{},"outputs":[],"execution_count":null},{"id":"v10-proven-ensemble","cell_type":"markdown","source":"## 3. From report audit to a reusable MRI ensemble\n\nThe report-derived label audit above is retained because it documents the weak supervision and its failure modes. The scored path now uses Pilkwang Kim's publicly inspectable 20-checkpoint DINOv2-small ensemble, which reached an official public score of **0.891**. Each member carries its resolution, physical crop, slice ordering, laterality, slot fallback, pooling, and fingerprint contract; any mismatch is fatal rather than silently producing different pixels.\n\nThe inference and attached package are adapted with attribution from `pilkwang/rsna-knee-baseline-v1` and `pilkwang/rsna-knee-weights`. The original notebook identity and report analysis remain intact.\n","metadata":{}},{"id":"md-12","cell_type":"markdown","source":"## 4. Reading the acquisition\n","metadata":{}},{"id":"code-13","cell_type":"code","source":"from __future__ import annotations\n\nimport os\n\nfor _v in (\"OMP_NUM_THREADS\", \"OPENBLAS_NUM_THREADS\", \"MKL_NUM_THREADS\"):\n    os.environ.setdefault(_v, \"4\")\n\nimport gc\nimport hashlib\nimport json\nimport re\nimport time\nimport traceback\nimport threading\nfrom concurrent.futures import ThreadPoolExecutor\nfrom pathlib import Path\n\nimport numpy as np\nimport pandas as pd\nimport pydicom\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\n\n# Use every accelerator that can execute the current PyTorch kernels.  Kaggle can\n# attach a legacy GPU that torch.cuda.is_available() reports as usable even though\n# the image has no compatible convolution kernel.  A real model-shaped probe makes\n# that allocation fail closed to CPU before any ensemble member is consumed.\ndef _cuda_execution_probe(index):\n    dev = torch.device(f\"cuda:{index}\")\n    try:\n        major, minor = torch.cuda.get_device_capability(index)\n        probe = nn.Conv2d(3, 4, kernel_size=3, padding=1).eval().to(dev)\n        with torch.inference_mode():\n            out = probe(torch.zeros((1, 3, 16, 16), device=dev))\n            if tuple(out.shape) != (1, 4, 16, 16):\n                raise RuntimeError(f\"unexpected CUDA probe shape {tuple(out.shape)}\")\n        torch.cuda.synchronize(index)\n        print(f\"cuda:{index} probe PASS (compute {major}.{minor})\")\n        del probe, out\n        torch.cuda.empty_cache()\n        return True\n    except Exception as exc:\n        print(f\"cuda:{index} probe FAIL ({type(exc).__name__}: {exc}); using CPU fallback\")\n        try:\n            torch.cuda.empty_cache()\n        except Exception:\n            pass\n        return False\n\nDEVS = []\nif torch.cuda.is_available():\n    DEVS = [torch.device(f\"cuda:{i}\") for i in range(torch.cuda.device_count())\n            if _cuda_execution_probe(i)]\nif not DEVS:\n    DEVS = [torch.device(\"cpu\")]\nprint(f\"devices: {[str(d) for d in DEVS]}\")\n\n\n# The label extractor is defined in the cells above when this runs as a notebook. As a\n# plain script it is imported from the package source, so the two paths share one\n# definition rather than keeping a copy each.\n\nT0 = time.time()\nSEED = 2026\nnp.random.seed(SEED)\ntorch.manual_seed(SEED)\n\nTARGETS = [\"ACL\", \"MCL\", \"Medial Meniscus\", \"Lateral Meniscus\", \"Medial OA\",\n           \"Lateral OA\", \"PF OA\", \"Effusion\", \"Synovitis\", \"Baker's\",\n           \"Contusion\", \"Fracture\"]\n\n\n# The centre crop has to be smaller than the smallest field of view in the corpus or it\n# silently does nothing. Measured over every training series, the acquired field of view\n# (Rows x PixelSpacing) has median 160 mm and runs from 70 to 320: a 160 mm crop is\n# larger than the image in 60% of series and is skipped for all of them, which leaves\n# their physical scale unnormalised. 130 mm is below the field of view of 99.6% of\n# series and still contains the joint.\nCROP_MM = 130.0\n\n# Cache resolution. Everything downstream may downsample from this, so it is set by the\n# most demanding configuration rather than by the default one.\nCACHE_IMG = 336\nGROUP = 3                  # slices per encoder input, stacked as the three channels\nN_GROUP_MAX = 1\nCACHE_FRACTION = 0.45      # share of free memory the pixel cache may take\nCACHE_BUDGET_MAX_GB = 24.0 # hard ceiling regardless of what the machine reports\nCACHE_BUDGET_GB = 12.0     # only the fallback, for a machine with no /proc/meminfo\nTEST_SHARE = 0.30          # floor on the test corpus relative to the training one, since\n                           # the visible test split is a stub and the scored one is not\nHDR_THREADS = 16\nPIX_THREADS = 12\nORDER_THREADS = 32         # slice-ordering is latency-bound on the mount, not CPU-bound\n# Ceiling for the ordering pass. It has to be a ceiling because the pass is hundreds of\n# thousands of small reads over a network mount, so its duration is a property of the\n# mount rather than of the work, and varies between runs that do the same reading. It must not be a tight one: giving up leaves\n# those series in file order, which is uncorrelated with anatomy, and that degradation is\n# silent. So the ceiling sits well above what the pass ordinarily needs: its purpose is\n# to stop the pass consuming the whole run on a slow mount, not to trim the ordinary\n# case, and a ceiling tight enough to bind on a normal day would trade a silent\n# degradation for a saving the run does not need.\nORDER_BUDGET_S = 5400\n\n# Resolution is the axis under test. A feature of width d mm survives resampling only if\n# the pixel pitch is at most d/2, and the pitch here is set by the crop above rather than\n# by the acquired field of view: CROP_MM / P. At 224 px that is 0.58 mm, above the 0.5 mm\n# a 1 mm tear needs; at 336 px it is 0.39 mm and clears it. Both configurations read the\n# same cache, so the comparison isolates the resize.\nRUNS = [\n    {\"name\": \"r224\", \"img\": 224},\n    {\"name\": \"r336\", \"img\": 336},\n]\n\nEPOCHS = 10\nBATCH_STUDIES = 8          # a study is a bag of up to N_SLOT slot images\nAUG_ROT_DEG = 8.0          # rigid jitter; see augment() for why neither flip is used\nAUG_SCALE = 0.08\nAUG_SHIFT = 0.05\nAUG_INTENSITY = 0.10\nLAT_MIN_OFFSET_MM = 20.0   # inside this the side is not readable from geometry; see\n                           # side_from_geometry()\nSLICE_BAND = (0.20, 0.80)  # fraction of the ordered stack read_slot samples across\n\n# --- What a slice IS, as opposed to how many of them there are --------------- #\n#\n# A member is a function of the pixels it was fitted on, and img/crop_mm/slices/band do\n# not determine those pixels by themselves. Four further decisions do, none of them\n# visible in any shape:\n#\n#   order          which slice is the next one along the stack\n#   lat            which knees are mirrored, and on what evidence\n#   slot_fallback  whether a T1 slot may be filled from a series that is not T1\n#   decode_fill    what stands in for a slice that would not decode\n#\n# `native` is the reading derived in the sections below. `legacy` is the reading an\n# imported member was fitted under. A member read under the wrong one loads with every\n# shape matching, runs, and writes a plausible submission computed from the wrong image -\n# so the choice travels with the member and is part of the key that decides which members\n# can share a decode. The legacy rules are reproduced rather than corrected: correcting\n# them would hand that member pixels its weights never saw.\nRULES_NATIVE = {\"order\": \"normal\", \"lat\": \"centre\",\n                \"slot_fallback\": False, \"decode_fill\": \"nearest\"}\nRULES_LEGACY = {\"order\": \"dominant_axis\", \"lat\": \"corner_x\",\n                \"slot_fallback\": True, \"decode_fill\": \"zero\"}\nRULES = dict(RULES_NATIVE)\nLEGACY_LAT_OFFSET_MM = 5.0   # the dead zone the legacy laterality rule was fitted with\n\nLR_HEAD = 1e-3\nLR_BACKBONE = 8e-6         # the encoder is adapted, not retrained\nUNFREEZE_LAST = 6          # trainable transformer blocks, from the output end\nWEIGHT_DECAY = 0.02\nEVAL_BATCH = 8\nTIME_BUDGET = 8.0 * 3600\n\n# Six slots: three planes crossed with the acquisition axes. The fat-suppressed\n# fluid-sensitive series exist for nearly every study; the T1 and the non-suppressed\n# fluid-sensitive series are scarcer, which is what the presence mask is for.\nSLOTS_RECOVERED = [\n    (\"SAG_FLUID_FS\", \"Sagittal\", True, True),\n    (\"COR_FLUID_FS\", \"Coronal\", True, True),\n    (\"AX_FLUID_FS\", \"Axial\", True, True),\n    (\"SAG_FLUID_NOFS\", \"Sagittal\", True, False),\n    (\"COR_T1\", \"Coronal\", False, False),\n    (\"SAG_T1\", \"Sagittal\", False, False),\n]\n\n# The alternative: plane x the single axis the delivered flags carry, ignoring the\n# recovered weighting. Kept as\n# a switch so the choice of slot definition can be varied while everything else is held\n# fixed. Under this scheme a `Struct` slot mixes T1 series with non-fat-suppressed PD/T2\n# series, which carry very different tissue contrast.\nSLOTS_PUBLIC = [\n    (\"SAG_FLUID\", \"Sagittal\", None, True),\n    (\"COR_FLUID\", \"Coronal\", None, True),\n    (\"AX_FLUID\", \"Axial\", None, True),\n    (\"SAG_STRUCT\", \"Sagittal\", None, False),\n    (\"COR_STRUCT\", \"Coronal\", None, False),\n    (\"AX_STRUCT\", \"Axial\", None, False),\n]\n\nSLOT_SCHEME = os.environ.get(\"SLOT_SCHEME\", \"recovered\")\nSLOTS = SLOTS_PUBLIC if SLOT_SCHEME == \"public\" else SLOTS_RECOVERED\nN_SLOT = len(SLOTS)\n\n# How many 384-wide parts the per-slot feature is built from. The encoder emits one\n# vector per token; a slot feature is a fixed summary of that grid, and the summary an\n# imported member was fitted with carries a third part.\nPOOL_PARTS = {\"cls_mean\": 2, \"cls_mean_focal\": 3}\n\n# Which slots an imported member's attention is tilted toward, per diagnosis. Indices are\n# into SLOTS. This is a fixed table rather than a learned parameter, so it is part of that\n# member's definition and has to be reproduced exactly for its weights to mean anything.\nSLOT_PRIOR_TABLE = {\n    \"ACL\": (0, 3, 5), \"MCL\": (1, 4),\n    \"Medial Meniscus\": (0, 1, 3, 4), \"Lateral Meniscus\": (0, 1, 3, 4),\n    \"Medial OA\": (1, 4, 5), \"Lateral OA\": (1, 4, 5),\n    \"PF OA\": (0, 2, 5), \"Effusion\": (0, 2), \"Synovitis\": (0, 2),\n    \"Baker's\": (0,), \"Contusion\": (0, 1, 2), \"Fracture\": (0, 1, 2, 4, 5),\n}\nSLOT_PRIOR_STRENGTH = 0.55\n\nFATSAT_OPTS = {\"FS\", \"FATSAT\", \"FAT_SAT\", \"FSAT\"}\n_SEP = re.compile(r\"[_\\-.]\")\n_FATSAT_RX = re.compile(r\"\\bfs\\b|fatsat|fat sat|\\bstir\\b|\\bspair\\b|\\bspir\\b|\\bwe\\b|\"\n                        r\"water excit|\\btirm\\b|\\bsting\\b|\\bfatsup\\b\")\n_T1_RX = re.compile(r\"\\bt1\\b|\\bt1w\\b\")\n_T2_RX = re.compile(r\"\\bt2\\b|\\bt2w\\b\")\n_PD_RX = re.compile(r\"\\bpd\\b|\\bpdw\\b|proton|\\bdp\\b|dens\")\n","metadata":{"_kg_hide-input":true,"_kg_hide-output":true},"outputs":[],"execution_count":null},{"id":"code-14","cell_type":"code","source":"def log(msg):\n    print(f\"[{time.time() - T0:7.1f}s] {msg}\", flush=True)\n\n\ndef find_root():\n    for c in [Path(\"/kaggle/input/competitions/rsna-knee-abnormality-detection\"),\n              Path(\"/kaggle/input/rsna-knee-abnormality-detection\"),\n              Path(\"data\"), Path(\".\")]:\n        if (c / \"test.csv\").is_file() and (c / \"test_series\").is_dir():\n            return c\n    # last resort: two-level scan, because the mount is nested one deeper than usual\n    base = Path(\"/kaggle/input\")\n    if base.is_dir():\n        for depth1 in sorted(p for p in base.iterdir() if p.is_dir()):\n            for cand in [depth1] + sorted(p for p in depth1.iterdir() if p.is_dir()):\n                if (cand / \"test.csv\").is_file():\n                    return cand\n    raise FileNotFoundError(\n        f\"competition mount not found (cwd {Path.cwd()}); expected a directory holding \"\n        f\"test.csv and test_series/\")\n\n\ndef find_dinov2(variant=\"small\"):\n    \"\"\"Locate a mounted DINOv2 checkpoint directory by variant name.\"\"\"\n    base = Path(\"/kaggle/input\")\n    if not base.is_dir():\n        return None\n    hits = []\n    for root, dirs, files in os.walk(base):\n        dirs[:] = [d for d in dirs if d not in (\"train_series\", \"test_series\")]\n        if \"config.json\" in files and \"dinov2\" in root.lower():\n            hits.append(Path(root))\n    for h in hits:\n        if variant in str(h).lower():\n            return h\n    return hits[0] if hits else None\n\n\nLABEL_COLS = TARGETS + [t + \"__conf\" for t in TARGETS]\n\n\nclass LabelSourceError(RuntimeError):\n    \"\"\"Raised when the labels did not come from where this run intended.\n\n    Every other failure in this file is better survived than reported: a run that dies\n    after the cache is built has spent the expensive half and scores nothing, so the\n    guard around `main` swallows it and leaves the benchmark file behind. This one is\n    the exception. Training on the weaker labels does not look like a failure - it\n    completes, writes a plausible submission, and differs only in a log line - so it has\n    to stop the run rather than be absorbed by a guard designed for crashes.\n    \"\"\"\n\n\ndef find_label_table():\n    \"\"\"Locate a mounted table of pre-read report labels, if one is attached.\n\n    The lexicon turns a report into labels by matching morphology, and its failure\n    mode is silence: on a phrasing it does not carry it emits no opinion rather than a\n    wrong one. Silence is measurable without any ground truth - for each (report,\n    finding) pair, did anything match? - and that measurement says the misses are\n    concentrated in particular languages rather than spread evenly, on findings a knee\n    report almost always comments on.\n\n    Enumerating morphology for nine languages is the wrong instrument for that. Reading\n    the sentence is the right one, and a language model reads it. Against the annotated\n    studies the difference is large and one-sided, so when such a table is mounted it is\n    preferred; when it is not, the lexicon runs and the pipeline is unchanged. Both paths\n    produce the same columns, so nothing downstream knows which one supplied them.\n    \"\"\"\n    base = Path(\"/kaggle/input\")\n    cands = []\n    if base.is_dir():\n        for root, dirs, files in os.walk(base):\n            dirs[:] = [d for d in dirs if d not in (\"train_series\", \"test_series\")]\n            cands += [Path(root) / f for f in files if f.startswith(\"report_labels\")\n                      and f.endswith(\".csv\")]\n    cands += [p for p in (Path(\"data/derived/report_labels_v2.csv\"),) if p.is_file()]\n    for c in cands:\n        try:\n            head = pd.read_csv(c, nrows=1)\n        except Exception:\n            continue\n        if \"StudyInstanceUID\" in head.columns and all(t in head.columns for t in TARGETS):\n            return c\n    return None\n\n\ndef label_mount_attached():\n    \"\"\"True when an input directory was attached that is meant to carry a label table.\n\n    The fallback below is deliberate and has to stay silent for a run with no table\n    attached, because that is the ordinary case for anyone reading this notebook. It\n    must not stay silent for the other case: a table was attached and could not be used.\n    Those two are indistinguishable from the labels alone - both end with the lexicon -\n    so they are separated here by whether the mount exists at all.\n    \"\"\"\n    base = Path(\"/kaggle/input\")\n    if not base.is_dir():\n        return False\n    return any(\"label\" in p.name.lower() for p in base.iterdir() if p.is_dir())\n\n\ndef read_labels(train_df):\n    \"\"\"Labels for every training study, from a mounted table or from the lexicon.\n\n    Studies the mounted table does not cover fall back to the lexicon rather than being\n    dropped, so a partial table degrades coverage instead of losing rows.\n    \"\"\"\n    n = len(train_df)\n    lab = pd.DataFrame([extract(r) for r in train_df[\"Report\"].fillna(\"\")])\n    lab[\"StudyInstanceUID\"] = train_df[\"StudyInstanceUID\"].values\n    lab = lab.set_index(\"StudyInstanceUID\")\n\n    src = find_label_table()\n    if src is None:\n        if label_mount_attached():\n            raise LabelSourceError(\n                \"LABEL SOURCE: a label dataset is mounted but no usable table was found \"\n                \"in it. Falling back to the lexicon here would train on the weaker \"\n                \"labels and say so only in a log line, so the run stops instead.\")\n        log(f\"LABEL SOURCE: lexicon, {n} studies (no table mounted)\")\n        return lab\n\n    tab = pd.read_csv(src).set_index(\"StudyInstanceUID\")\n    missing = [c for c in LABEL_COLS if c not in tab.columns]\n    if missing:\n        raise LabelSourceError(\n            f\"LABEL SOURCE: {src} is missing {len(missing)} expected columns \"\n            f\"(first: {missing[0]!r}). Refusing to fall back silently.\")\n    hit = lab.index.intersection(tab.index)\n    if not len(hit):\n        raise LabelSourceError(\n            f\"LABEL SOURCE: {src} shares no StudyInstanceUID with train.csv.\")\n    log(f\"LABEL SOURCE: {src.name} covers {len(hit)} of {n} studies, \"\n        f\"lexicon for the remaining {n - len(hit)}\")\n    lab.loc[hit, LABEL_COLS] = tab.loc[hit, LABEL_COLS].values\n    return lab\n\n\nROOT = find_root()\nlog(f\"input root: {ROOT}\")\n\n\nIMG = CACHE_IMG            # kept as the name the pixel reader and cache use\n\n\ndef available_gb():\n    \"\"\"Memory this machine will actually lend, read rather than assumed.\n\n    A hardcoded ceiling is a guess about a machine the author is not sitting at, and a\n    guess that is too low costs coverage silently while a guess that is too high ends the\n    run. The machine will say, so it is asked.\n    \"\"\"\n    try:\n        with open(\"/proc/meminfo\") as fh:\n            info = {k.strip(): v for k, v in\n                    (l.split(\":\", 1) for l in fh if \":\" in l)}\n        return int(info[\"MemAvailable\"].split()[0]) / 1024 ** 2\n    except Exception:\n        return CACHE_BUDGET_GB / CACHE_FRACTION      # fall back to the old constant\n\n\ndef plan_cache(n_study, n_test=0):\n    \"\"\"Choose how many slices per slot the memory the machine has will allow.\n\n    The cache is n_study x n_slot x slices x IMG^2 bytes. Coverage is the cheap axis -\n    linear - and resolution the expensive one, so when the budget binds it is the slice\n    count that gives way rather than the pixel grid. Deciding once, from the training\n    corpus size, keeps train and test caches on the same group layout.\n\n    Only a fraction of what is free is taken. The rest is not slack: the encoder, its\n    activations, the pinned batches and the frames all come out of the same pool, and the\n    cache is the one allocation big enough that overshooting it kills the run outright.\n    \"\"\"\n    avail = available_gb()\n    budget = min(avail * CACHE_FRACTION, CACHE_BUDGET_MAX_GB)\n    # Both caches are held at once, and the test half is what the visible run cannot\n    # show: here it is a handful of studies, and at scoring it is the whole hidden set.\n    # Sizing against the training corpus alone therefore passes every run that can be\n    # watched and overruns the one that counts.\n    n_total = n_study + max(n_test, int(TEST_SHARE * n_study))\n    per_slice = n_total * N_SLOT * IMG * IMG\n    afford = int(budget * 1024 ** 3 // max(per_slice, 1))\n    groups = max(1, min(N_GROUP_MAX, afford // GROUP))\n    log(f\"memory: {avail:.1f} GB available, {budget:.1f} GB to the cache; \"\n        f\"sizing for {n_study} train + {n_total - n_study} test studies \"\n        f\"-> {groups} group(s) of {GROUP} = {groups * GROUP} slices per slot\"\n        + (f\" (wanted {N_GROUP_MAX})\" if groups < N_GROUP_MAX else \"\"))\n    return groups\n\n\nN_GROUP = plan_cache(len(pd.read_csv(ROOT / \"train.csv\")),\n                     len(pd.read_csv(ROOT / \"test.csv\")))\nCACHE_SLICES = GROUP * N_GROUP\nlog(f\"cache layout: {N_GROUP} groups x {GROUP} slices = {CACHE_SLICES} per slot\")\n","metadata":{"_kg_hide-input":true,"_kg_hide-output":true},"outputs":[],"execution_count":null},{"id":"code-15","cell_type":"code","source":"HDR_TAGS = [\"SeriesDescription\", \"SequenceName\", \"ScanOptions\", \"ScanningSequence\",\n            \"RepetitionTime\", \"EchoTime\", \"Laterality\", \"PixelSpacing\", \"Rows\",\n            \"Columns\", \"RescaleSlope\", \"RescaleIntercept\",\n            # Position and orientation are read from the same header probe() already\n            # opens, so they cost nothing, and they are what recovers the side when the\n            # Laterality tag is absent - which it is for half the studies here.\n            \"ImagePositionPatient\", \"ImageOrientationPatient\"]\n\n\ndef _hdr_vec(s, n):\n    \"\"\"Parse a DICOM multi-value string as stored by probe(): floats joined by `|`.\"\"\"\n    if not isinstance(s, str):\n        return None\n    try:\n        v = [float(x) for x in s.split(\"|\")]\n    except ValueError:\n        return None\n    return np.array(v) if len(v) >= n else None\n\n\ndef side_from_geometry(h):\n    \"\"\"Study -> 'L' / 'R' / None, from where the image sits in the patient.\n\n    `Laterality` (0020,0060) is Type 2C and may legitimately be absent; in this corpus it\n    is missing on exactly half the studies, and the vendors it is missing from are whole\n    vendors rather than scattered series. A study with no tag is not a left knee, but the\n    normalisation upstream treats it as one, so half the corpus was never normalised and\n    the five side-defined targets - the two menisci, the two tibiofemoral compartments\n    and the medial collateral ligament - saw that axis reversed on a large minority of it.\n\n    The patient coordinate system fixes this without the tag: +x is the patient's left, so\n    the centre of a right knee sits at negative x. The centre is used rather than\n    `ImagePositionPatient` itself because that is the corner of the image, which is offset\n    by half a field of view - enough to change the sign on a knee near the midline.\n\n    The median over a study's series is what is thresholded, not a single series: probe()\n    reads one arbitrary slice per series, which on a sagittal stack can sit anywhere\n    across the joint. Studies whose centre falls near the midline are left unresolved\n    rather than guessed - measured against the tagged half, the rule is right 97% of the\n    time overall and no better than chance inside 20 mm.\n    \"\"\"\n    cx = {}\n    for r in h.itertuples(index=False):\n        ipp = _hdr_vec(getattr(r, \"ImagePositionPatient\", None), 3)\n        iop = _hdr_vec(getattr(r, \"ImageOrientationPatient\", None), 6)\n        ps = _hdr_vec(getattr(r, \"PixelSpacing\", None), 2)\n        rows, cols = getattr(r, \"Rows\", None), getattr(r, \"Columns\", None)\n        if ipp is None or iop is None or ps is None or not rows or not cols:\n            continue\n        try:\n            c = ipp[:3] + iop[:3] * ps[1] * float(cols) / 2 + iop[3:6] * ps[0] * float(rows) / 2\n        except (TypeError, ValueError):\n            continue\n        cx.setdefault(r.StudyInstanceUID, []).append(float(c[0]))\n    out = {}\n    for st, xs in cx.items():\n        m = float(np.median(xs))\n        out[st] = None if abs(m) < LAT_MIN_OFFSET_MM else (\"R\" if m < 0 else \"L\")\n    return out\n\n\ndef side_from_corner_x(h):\n    \"\"\"The laterality an imported member was fitted under.\n\n    It thresholds the median raw `ImagePositionPatient` x over a study's series. That is\n    the x of the image *corner*, not of its centre, so it differs from the rule above by\n    up to half a field of view - which is enough to reverse the sign on a knee scanned\n    near the midline. The dead zone is 5 mm rather than 20 mm, so it also commits on\n    studies the rule above leaves unresolved.\n\n    Neither difference changes a shape. Each one decides whether a study is mirrored, and\n    a study mirrored one way at training and the other at inference presents the five\n    side-defined targets with their axis reversed.\n    \"\"\"\n    out = {}\n    for st, g in h.groupby(\"StudyInstanceUID\"):\n        xs = []\n        for r in g.itertuples(index=False):\n            ipp = _hdr_vec(getattr(r, \"ImagePositionPatient\", None), 3)\n            if ipp is not None and np.isfinite(ipp).all():\n                xs.append(float(ipp[0]))\n        if not xs:\n            out[st] = None\n            continue\n        x = float(np.median(xs))\n        # DICOM patient coordinates are LPS: +x is the patient's left.\n        out[st] = None if abs(x) < LEGACY_LAT_OFFSET_MM else (\"R\" if x < 0 else \"L\")\n    return out\n\n\ndef lat_of(h, tag=\"\"):\n    \"\"\"Study -> 'L' / 'R' / None: the tag where it exists, geometry where it does not.\n\n    The tag is present on exactly half the studies here and is sometimes an empty\n    string rather than absent, which is not the same as NaN. Treating the other half\n    as left-sided is what `normalise_laterality` did by omission, so the geometry\n    fallback is not a refinement - it is the difference between normalising half the\n    corpus and normalising all of it.\n    \"\"\"\n    geo = side_from_corner_x(h) if RULES[\"lat\"] == \"corner_x\" else side_from_geometry(h)\n    d, n_tag, n_geo, n_none, n_disagree = {}, 0, 0, 0, 0\n    for st, g in h.groupby(\"StudyInstanceUID\"):\n        v = [str(x).strip().upper() for x in g[\"Laterality\"].dropna()]\n        if RULES[\"lat\"] == \"corner_x\" and \"ImageLaterality\" in g.columns:\n            # The legacy rule reads the second tag too, so a study tagged only there is\n            # resolved from the tag rather than from geometry.\n            v += [str(x).strip().upper() for x in g[\"ImageLaterality\"].dropna()]\n        v = [x[0] for x in v if x and x[0] in (\"L\", \"R\")]\n        side = v[0] if v else None\n        if side is not None:\n            n_tag += 1\n            if geo.get(st) is not None and geo[st] != side:\n                n_disagree += 1\n        else:\n            side = geo.get(st)\n            n_geo += side is not None\n            n_none += side is None\n        d[st] = side\n    log(f\"{tag}laterality: {n_tag} from the tag, {n_geo} from geometry, \"\n        f\"{n_none} unresolved; tag and geometry disagree on {n_disagree} \"\n        f\"({n_disagree / max(n_tag, 1):.1%} of the tagged)\")\n    return d\n\n\n\ndef probe(item):\n    split, study, series, path = item\n    row = {\"split\": split, \"StudyInstanceUID\": study, \"SeriesInstanceUID\": series,\n           \"dir\": path}\n    try:\n        files = sorted(e.name for e in os.scandir(path) if e.name.endswith(\".dcm\"))\n        row[\"files\"] = files\n        row[\"n_slices\"] = len(files)\n        if not files:\n            return row\n        ds = pydicom.dcmread(os.path.join(path, files[len(files) // 2]),\n                             stop_before_pixels=True, force=True)\n        for t in HDR_TAGS:\n            v = getattr(ds, t, None)\n            if v is None:\n                row[t] = None\n            elif isinstance(v, (list, tuple)) or type(v).__name__ == \"MultiValue\":\n                row[t] = \"|\".join(str(x) for x in v)\n            else:\n                row[t] = str(v)\n    except Exception as exc:\n        row[\"err\"] = str(exc)[:120]\n    return row\n\n\ndef walk(split):\n    \"\"\"Every series directory of a split, with one header read per series.\n\n    An absent split returns an empty frame *with the columns annotate expects*. Returning\n    a bare DataFrame looks like the same thing and is not: the next call indexes\n    `SeriesDescription` and raises KeyError, so the branch that exists to survive a\n    missing split is what turns it into a crash.\n    \"\"\"\n    base = ROOT / split\n    items = []\n    if not base.is_dir():\n        return pd.DataFrame(columns=[\"split\", \"StudyInstanceUID\", \"SeriesInstanceUID\",\n                                     \"dir\", \"files\", \"n_slices\"] + HDR_TAGS)\n    for study in os.scandir(base):\n        if study.is_dir():\n            for series in os.scandir(study.path):\n                if series.is_dir():\n                    items.append((split, study.name, series.name, series.path))\n    with ThreadPoolExecutor(max_workers=HDR_THREADS) as pool:\n        rows = list(pool.map(probe, items))\n    return pd.DataFrame(rows)\n\n\ndef annotate(df):\n    \"\"\"Recover fat suppression and pulse-sequence weighting from the header.\"\"\"\n    desc = (df[\"SeriesDescription\"].fillna(\"\") + \" \" + df[\"SequenceName\"].fillna(\"\"))\n    desc = desc.str.lower().str.replace(_SEP, \" \", regex=True)\n\n    opts = df[\"ScanOptions\"].fillna(\"\").str.upper().str.split(\"|\")\n    # GE writes SAT_GEMS for spatial saturation, so ScanOptions must be matched as\n    # exact tokens; a substring test on \"SAT\" fires on non-fat-sat series.\n    opts_fs = opts.apply(lambda ts: any(t.strip() in FATSAT_OPTS for t in ts))\n    df[\"fatsat\"] = desc.str.contains(_FATSAT_RX) | opts_fs\n\n    tr = pd.to_numeric(df[\"RepetitionTime\"], errors=\"coerce\")\n    te = pd.to_numeric(df[\"EchoTime\"], errors=\"coerce\")\n    gre = df[\"ScanningSequence\"].fillna(\"\").str.upper().str.contains(\"GR\")\n    t1, t2, pdw = desc.str.contains(_T1_RX), desc.str.contains(_T2_RX), desc.str.contains(_PD_RX)\n\n    df[\"weight\"] = np.where(t1 & ~t2 & ~pdw, \"T1\",\n                     np.where(t2 & ~pdw, \"T2\",\n                       np.where(pdw, \"PD\",\n                         np.where(gre, \"GRE\",\n                           np.where(tr < 800, \"T1\",\n                             np.where(te > 60, \"T2\",\n                               np.where(tr >= 800, \"PD\", \"UNK\")))))))\n    df[\"fluid\"] = np.isin(df[\"weight\"], [\"PD\", \"T2\"])\n    df[\"px\"] = pd.to_numeric(\n        df[\"PixelSpacing\"].fillna(\"\").str.split(\"|\").str[0].replace(\"\", np.nan),\n        errors=\"coerce\")\n    return df\n","metadata":{"_kg_hide-input":true,"_kg_hide-output":true},"outputs":[],"execution_count":null},{"id":"code-16","cell_type":"code","source":"def pick_slots(series_df, plane_map):\n    \"\"\"One series per slot per study.\n\n    Ties are broken toward the stack with the most slices: a thicker stack samples the\n    joint more densely, and the three-slice sampler below benefits from the margin.\n    \"\"\"\n    series_df = series_df.copy()\n    series_df[\"plane\"] = series_df[\"SeriesInstanceUID\"].map(plane_map)\n    out = {}\n    for study, g in series_df.groupby(\"StudyInstanceUID\"):\n        chosen = {}\n        for name, plane, fluid, fs in SLOTS:\n            sel = (g[\"plane\"] == plane) & (g[\"fatsat\"] == fs)\n            # fluid=None means \"do not condition on weighting\" - the public scheme,\n            # where the single provided flag stands in for both axes at once.\n            if fluid is not None:\n                sel &= (g[\"fluid\"] == fluid)\n            cand = g[sel]\n            # A slot with no series matching its predicate stays empty, and no substitute\n            # is admitted from a neighbouring predicate. Relaxing the weighting to fill a\n            # T1 slot would draw from the pool `SAG_FLUID_NOFS` selects from, since that\n            # pool is what remains once the weighting is dropped: over the training corpus\n            # it would put one series in two slots for 2383 of 4407 studies and leave 56%\n            # of the T1 slot holding PD or T2. The presence mask would then assert a\n            # sequence that was never acquired, and the per-diagnosis softmax of §6 would\n            # divide its attention across two identical slots, giving one acquisition\n            # about twice the weight it carries in a study that holds both. The mask is\n            # there to say a slot is absent, which is what an absent slot is.\n            if len(cand) == 0 and RULES[\"slot_fallback\"] and fluid is False:\n                # The relaxation the paragraph above rejects, reproduced because an\n                # imported member was fitted with its T1 slots filled this way: over half\n                # of that member's training studies had a T1 slot holding a series that\n                # is not T1. Leaving those slots empty would present it with a presence\n                # mask it never saw.\n                cand = g[(g[\"plane\"] == plane) & (~g[\"fatsat\"])]\n            if len(cand):\n                chosen[name] = cand.sort_values(\"n_slices\", ascending=False).iloc[0]\n        out[study] = chosen\n    return out\n","metadata":{"_kg_hide-input":true,"_kg_hide-output":true},"outputs":[],"execution_count":null},{"id":"md-17","cell_type":"markdown","source":"## 3b. What order the slices are in\n\nA series is a directory of files, and the obvious way to walk it is to sort the file names.\nThat is wrong here, and wrong in a way that produces no error. The file name is the SOP\nInstance UID, assigned to be unique rather than ordered, so sorting by it yields a sequence\nuncorrelated with anatomy — measured on a series from this corpus, the rank correlation\nbetween file-name order and physical position is $\\rho \\approx 0.01$. Three things\ndownstream depend on that order: three adjacent slices as three channels become three\nunrelated cross-sections; the middle of the stack becomes a random subset; and reversing\nthe order to normalise laterality does nothing at all.\n\nThe true order is recoverable exactly from geometry every slice carries.\n`ImageOrientationPatient` gives the in-plane axes $\\hat{r}_x, \\hat{r}_y$ and\n`ImagePositionPatient` the position $p$ of the first voxel, so\n\n$$\\hat{n} \\;=\\; \\hat{r}_x \\times \\hat{r}_y, \\qquad k \\;=\\; p \\cdot \\hat{n},$$\n\nand $k$ increases monotonically along the stack. Because $k$ is signed and expressed in\npatient coordinates, the stack has a fixed direction along the body's left-right axis —\nwhich is what the laterality normalisation of §5 reverses. `InstanceNumber` is the fallback\nwhere the geometry tags are missing; it usually tracks $k$ up to sign, but interleaved and\nmulti-echo acquisitions number slices in an order that is not the order they occupy in\nspace, and it is not signed in patient coordinates.\n","metadata":{}},{"id":"md-18","cell_type":"markdown","source":"## 4. Sampling: how many millimetres one pixel is allowed to be\n\nA DICOM slice of $N \\times N$ pixels with spacing $s$ mm/pixel covers $Ns$ millimetres of\nanatomy. Both vary across this corpus, so a fixed-pixel resize hands the encoder images\nwhose physical scale differs by a factor of several. But scale normalisation is only half\nof it; the other half is a hard limit.\n\n**A feature narrower than two pixels does not survive the resize.** To represent a\nstructure of width $d$ millimetres the pixel pitch must satisfy $s_{\\text{eff}} \\le d/2$,\nthe Nyquist condition applied to the resampling grid. A meniscal tear is one to three\nmillimetres, so at $d = 1$ mm the pitch must be at most $0.5$ mm — and if it is not, no\ncapacity downstream recovers the signal, because it was destroyed before the first\nconvolution. This is a property of the resize, not of the network.\n\nCropping to a constant physical extent $L$ and resampling to $P$ pixels fixes the pitch:\n\n$$n \\;=\\; \\Big\\lfloor \\frac{L}{s} \\Big\\rceil \\ \\text{pixels}, \\qquad\ns_{\\text{eff}} \\;=\\; \\frac{L}{P}\\ \\ \\text{mm/pixel}, \\qquad\n\\text{token} \\;=\\; 14\\,s_{\\text{eff}}\\ \\ \\text{mm}.$$\n\nThe crop must be smaller than the smallest field of view or it silently does nothing: if\n$L/s$ exceeds the image width the crop cannot be taken and that series passes through\nunnormalised, with no error. $L = 130$ mm is below the acquired field of view of almost\nevery series here while still containing the joint. The resize target then follows from the\ntear width rather than convention — at $L = 130$ mm an input of $224$ gives $0.580$\nmm/pixel, above the bound for a 1 mm feature, while $336$ gives $0.387$ mm/pixel and puts a\n$14$-pixel patch token at $5.4$ mm.\n\n**Intensity needs the same treatment**, MR having no absolute scale. Each series is\nnormalised to its own 1st and 99th percentile — over the sampled stack rather than per\nslice, so slices keep their relative contrast, and percentiles rather than extremes, so one\nbright vessel does not compress everything else.\n","metadata":{}},{"id":"code-19","cell_type":"code","source":"ORDER_TAGS = [(0x0020, 0x0032), (0x0020, 0x0037), (0x0020, 0x0013)]\n\n# Series in which at least one sampled slice would not decode. A list rather than a\n# counter because appending is atomic under the reader threads, and reported rather than\n# swallowed: unreported, a decode failure is indistinguishable from a black knee.\nDECODE_FAILED = []\n\n\ndef cache_tag(rules=None):\n    \"\"\"The name a decoded cache is stored under.\n\n    It has to name everything that decides the pixels, not only their dimensions. Two\n    configurations that agree on resolution, slice count, crop and band but disagree on\n    how a slice is chosen produce different arrays of identical shape - so a tag built\n    from the dimensions alone lets the second attach to the first one's file and train\n    against pixels it never asked for, with nothing anywhere reporting a mismatch.\n\n    A native reading keeps the plain name, so caches decoded before the rules existed\n    stay valid; anything else earns a suffix.\n    \"\"\"\n    r = dict(RULES if rules is None else rules)\n    t = (f\"{CACHE_IMG}px_{CACHE_SLICES}sl_{int(CROP_MM)}mm_\"\n         f\"{SLICE_BAND[0]:.2f}-{SLICE_BAND[1]:.2f}\")\n    if {k: r.get(k, v) for k, v in RULES_NATIVE.items()} != RULES_NATIVE:\n        t += \"_\" + hashlib.md5(json.dumps(r, sort_keys=True).encode()).hexdigest()[:6]\n    return t\n\n\ndef _natural_key(name):\n    return tuple(int(x) if x.isdigit() else x.lower()\n                 for x in re.split(r\"(\\d+)\", str(name)))\n\n\ndef _order_dominant_axis(rec):\n    \"\"\"The slice order an imported member was fitted under.\n\n    It sorts on the raw patient coordinate along whichever axis varies most across the\n    stack, rather than on the projection onto the slice normal. The two differ by a sign,\n    not by a formula: measured over this corpus every sagittal series has a slice normal\n    with n_x in [-1.00, -0.98], so p.n is the negative of the raw x this sorts on and the\n    two stacks come out exactly reversed. Because the band sampler truncates rather than\n    rounds, its nine indices are not symmetric about the middle, so nine slices drawn from\n    a twenty-six slice stack under one order share two with the other.\n\n    Missing geometry falls back to `InstanceNumber` and then to a natural sort of the file\n    name, both at the same 80% threshold the imported pipeline used.\n    \"\"\"\n    files, d = rec[\"files\"], rec[\"dir\"]\n    rows = []\n    for pos, f in enumerate(files):\n        ipp = inst = None\n        try:\n            ds = pydicom.dcmread(os.path.join(d, f), force=True, stop_before_pixels=True,\n                                 specific_tags=[\"ImagePositionPatient\", \"InstanceNumber\"])\n            raw = getattr(ds, \"ImagePositionPatient\", None)\n            if raw is not None and len(raw) >= 3:\n                c = np.asarray(raw[:3], dtype=np.float64)\n                if np.isfinite(c).all():\n                    ipp = c\n            n = getattr(ds, \"InstanceNumber\", None)\n            if n is not None:\n                inst = float(n)\n        except Exception:\n            pass\n        rows.append((f, ipp, inst, pos))\n\n    placed = [r for r in rows if r[1] is not None]\n    need = max(2, int(0.8 * len(rows)))\n    if len(placed) >= need:\n        xyz = np.stack([r[1] for r in placed])\n        axis = int(np.argmax(np.ptp(xyz, axis=0)))\n        spare = float(np.nanmedian(xyz[:, axis]))\n        rows.sort(key=lambda r: (float(r[1][axis]) if r[1] is not None else spare,\n                                 r[2] if r[2] is not None else float(\"inf\"), r[3]))\n    elif sum(r[2] is not None for r in rows) >= need:\n        rows.sort(key=lambda r: (r[2] if r[2] is not None else float(\"inf\"), r[3]))\n    else:\n        rows.sort(key=lambda r: _natural_key(r[0]))\n    return [r[0] for r in rows], True\n\n\ndef order_slices(rec):\n    \"\"\"Return the series' files sorted along the through-plane axis.\n\n    A DICOM file name here is a SOP Instance UID, which is assigned arbitrarily. Sorting\n    by it therefore produces an order uncorrelated with anatomy - measured over one\n    series, Spearman between file-name rank and physical position is 0.009, i.e. none.\n    Anything that assumes the file order means something is then operating on noise: the\n    three channels of a \"2.5D\" input are three unrelated views rather than neighbouring\n    slices, \"the middle of the stack\" is a random subset, and reversing slice order to\n    normalise laterality reverses nothing meaningful.\n\n    The physical order is recoverable exactly. Each slice carries its position in patient\n    coordinates and the in-plane axes; projecting the position onto the slice normal\n    gives a signed through-plane coordinate, monotonic along the stack:\n\n        n = r_x  x  r_y ,      k = p . n\n\n    `InstanceNumber` is the fallback. It usually tracks the projection up to sign, but\n    interleaved and multi-echo acquisitions need not number slices in the order they\n    occupy in space - but the projection is signed in patient\n    coordinates, which is what laterality normalisation needs.\n    \"\"\"\n    if RULES[\"order\"] == \"dominant_axis\":\n        return _order_dominant_axis(rec)\n    files, d = rec[\"files\"], rec[\"dir\"]\n    keyed = []\n    for f in files:\n        k = None\n        try:\n            ds = pydicom.dcmread(os.path.join(d, f), force=True, stop_before_pixels=True,\n                                 specific_tags=ORDER_TAGS)\n            iop = np.asarray(ds.ImageOrientationPatient, dtype=float)\n            ipp = np.asarray(ds.ImagePositionPatient, dtype=float)\n            k = float(np.dot(ipp, np.cross(iop[:3], iop[3:])))\n        except Exception:\n            try:\n                k = float(ds.InstanceNumber)\n            except Exception:\n                k = None\n        keyed.append((k, f))\n    if any(k is None for k, _ in keyed):\n        # A series with no usable geometry keeps its arbitrary order; that is worse than\n        # sorting but better than dropping the series, and it is logged as a count.\n        return files, False\n    return [f for _, f in sorted(keyed, key=lambda t: t[0])], True\n\n\ndef read_slot(rec, n_slice=None, out_size=None):\n    \"\"\"`n_slice` physically spread slices from one series, at `out_size` pixels.\n\n    Returns uint8 [n_slice, out, out] normalised per-series to its 1st-99th\n    percentile. Percentiles rather than min/max because MR intensity has no absolute\n    scale and a single bright vessel would otherwise compress the whole dynamic range.\n\n    Reading is the expensive half of this pipeline, so the caller reads once at the\n    largest configuration it needs and derives the smaller ones from the returned buffer\n    rather than re-reading.\n    \"\"\"\n    n_slice = GROUP if n_slice is None else n_slice\n    out_size = IMG if out_size is None else out_size\n    files, d, px = rec.get(\"ordered\") or rec[\"files\"], rec[\"dir\"], rec[\"px\"]\n    n = len(files)\n    if n == 0:\n        return None\n    # Spread the samples over a central band of the stack: the outermost slices of a knee\n    # series are mostly soft tissue outside the joint. The band is a constant rather than\n    # a literal because how much of the stack is worth reading depends on how many slices\n    # are being taken - at three the middle is all that fits, while at sixteen the ends\n    # are worth having, and a Baker cyst sits at the posteromedial end of a sagittal one.\n    lo, hi = int(SLICE_BAND[0] * (n - 1)), int(SLICE_BAND[1] * (n - 1))\n    idx = np.unique(np.linspace(lo, hi, n_slice).astype(int)) if hi > lo else np.array([n // 2])\n    while len(idx) < n_slice:\n        idx = np.append(idx, idx[-1])\n\n    planes = []\n    for i in idx[:n_slice]:\n        try:\n            ds = pydicom.dcmread(os.path.join(d, files[int(i)]), force=True)\n            a = ds.pixel_array.astype(np.float32)\n            sl = float(getattr(ds, \"RescaleSlope\", 1) or 1)\n            ic = float(getattr(ds, \"RescaleIntercept\", 0) or 0)\n            a = a * sl + ic\n        except Exception:\n            a = None                      # no shape is known here; see below\n        planes.append(a)\n\n    # A slice that would not decode has no shape of its own, and inventing one is how a\n    # single unreadable file erases a whole series: a substitute allocated at the resize\n    # target while the decoded slices are still native makes the shape check below take\n    # the substitute as the authority and zero the good slices with it, leaving a black\n    # slot that the presence mask still reports as acquired.\n    #\n    # A failure is instead filled from the nearest slice that did decode - the same\n    # convention the sampler already uses when the band holds fewer distinct slices than\n    # were asked for - and a series where nothing decodes is reported absent, which the\n    # mask can express, rather than black, which it cannot.\n    got = [k for k, p in enumerate(planes) if p is not None]\n    if RULES[\"decode_fill\"] == \"zero\":\n        # What an imported member was fitted with: a failure becomes a zero plane at the\n        # resize target, which the shape check below then propagates to the whole slot.\n        # It is the behaviour the paragraph above describes and rejects, kept here only\n        # because that member's weights were learned against slots blacked out this way.\n        if not got:\n            DECODE_FAILED.append(rec.get(\"SeriesInstanceUID\", d))\n        planes = [np.zeros((out_size, out_size), np.float32) if p is None else p\n                  for p in planes]\n        got = list(range(len(planes)))\n    if not got:\n        DECODE_FAILED.append(rec.get(\"SeriesInstanceUID\", d))\n        return None\n    if len(got) < len(planes):\n        DECODE_FAILED.append(rec.get(\"SeriesInstanceUID\", d))\n        for k, p in enumerate(planes):\n            if p is None:\n                planes[k] = planes[min(got, key=lambda j: abs(j - k))]\n\n    # Slices of one series can still differ in matrix size - multi-echo and some\n    # reformats do - and those are genuinely not stackable.\n    shp = planes[0].shape\n    planes = [p if p.shape == shp else np.zeros(shp, np.float32) for p in planes]\n    vol = np.stack(planes)\n\n    # constant physical extent, then resize: PixelSpacing varies 3.4x across the corpus\n    if px and np.isfinite(px) and px > 0:\n        want = int(round(CROP_MM / px))\n        h, w = shp\n        if 16 < want < min(h, w):\n            cy, cx = h // 2, w // 2\n            half = want // 2\n            vol = vol[:, max(0, cy - half):cy + half, max(0, cx - half):cx + half]\n\n    lo_v, hi_v = np.percentile(vol, [1, 99])\n    vol = np.clip((vol - lo_v) / max(hi_v - lo_v, 1e-6), 0, 1)\n\n    t = torch.from_numpy(np.ascontiguousarray(vol)).unsqueeze(0)\n    t = F.interpolate(t, size=(out_size, out_size), mode=\"bilinear\", align_corners=False)\n    # uint8, not float32. These buffers queue up between the reader threads and the\n    # encoder, and at this size a float32 slot-series is several megabytes. Intensity is\n    # already normalised into [0, 1] here, so eight bits cost nothing that a bilinear\n    # resize has not already cost, and the queue is a quarter the size.\n    return (t.squeeze(0) * 255).round().clamp(0, 255).to(torch.uint8)\n","metadata":{"_kg_hide-input":true,"_kg_hide-output":true},"outputs":[],"execution_count":null},{"id":"md-20","cell_type":"markdown","source":"## 5. Normalising left and right\n\nFour of the twelve targets — the two menisci and the medial and lateral tibiofemoral\ncompartments — are medial/lateral pairs, and a fifth, the medial collateral ligament, is\nnamed for the side it lies on. Medial and lateral are defined relative to the body's\nmidline, so which side of the *image* they fall on depends on which knee was scanned.\nUnless that is normalised, those five labels learn from an axis the model cannot observe.\n\nThe correction differs by plane. Coronally and axially the medial-lateral direction lies in\nthe image plane, so flipping the last axis maps one knee onto the other. Sagittally it is\nthe *slice* axis: each slice is unchanged by mirroring, and what differs is the order in\nwhich the stack traverses the joint, so the slice order is reversed instead.\n\n`Laterality` is a Type 2C attribute: it may legitimately be absent, and here it is absent\non half the studies — by whole vendors rather than scattered series. Leaving those alone\nsilently declares them left-sided, so every right knee among them enters the model\nmirrored. The patient coordinate system supplies the missing tag: position and orientation\nare recorded per image and $+x$ points to the patient's left, so the sign of the image\ncentre's $x$ says which knee this is.\n\n$$c \\;=\\; \\mathbf{p} \\;+\\; \\mathbf{r}\\,\\Delta_c \\frac{N_c}{2} \\;+\\; \\mathbf{d}\\,\\Delta_r \\frac{N_r}{2},\n\\qquad \\text{side} = \\begin{cases} \\text{right} & c_x < 0\\\\ \\text{left} & c_x > 0\\end{cases}$$\n\nwith $\\mathbf{p}$ the image position, $\\mathbf{r}$ and $\\mathbf{d}$ the row and column\ndirection cosines, and $\\Delta$ the pixel spacing. The centre is used rather than\n$\\mathbf{p}$ itself, which is a corner half a field of view away — enough to change the sign\non a knee scanned near the midline. The median over a study's series is thresholded, and a\nstudy centred within a short distance of the midline is left unresolved rather than guessed,\nbecause inside that band the sign is no better than chance.\n","metadata":{}},{"id":"code-21","cell_type":"code","source":"def normalise_laterality(img, plane, lat):\n    \"\"\"Map every knee onto a left-knee convention.\n\n    Coronal and axial views mirror under a horizontal flip. Sagittal stacks are not\n    mirror images of each other - the slice order runs medial-to-lateral in opposite\n    directions - so the channel order is reversed instead.\n    \"\"\"\n    if lat != \"R\":\n        return img\n    if plane in (\"Coronal\", \"Axial\"):\n        return torch.flip(img, dims=[-1])\n    return torch.flip(img, dims=[0])\n","metadata":{"_kg_hide-input":true,"_kg_hide-output":true},"outputs":[],"execution_count":null},{"id":"md-22","cell_type":"markdown","source":"## 5b. Reading once, training many times\n\nThe cost of this pipeline is dominated by reading, not arithmetic. A study holds several\nseries and each series tens of slices, so a study is on the order of a hundred and fifty\nfiles. That is affordable once; it is not affordable once per epoch, and fine-tuning needs\nthe same pixels every epoch. So the slot images are decoded a single time into memory and\nheld as `uint8`:\n\n$$\\text{bytes} \\;=\\; N_{\\text{study}} \\times N_{\\text{slot}} \\times S \\times P^{2}$$\n\nwith $S$ slices kept per slot at $P$ pixels. The exponent on $P$ is what makes this a real\nconstraint rather than a detail — the cache grows with the *square* of resolution and only\nlinearly with slices, so coverage is the cheap axis and resolution the expensive one.\n\nThe cached slices form one three-channel encoder input per slot, and the layout generalises\nto several such groups per slot: training draws one per step, which doubles as augmentation\nalong the stack, and inference averages over them. What fixes the number of groups is a\nbudget rather than a capacity — the cache is allowed a fraction of the memory reported free,\ndeliberately below the whole of it, since it is the one allocation large enough that\novershooting ends the run rather than slowing it, and the encoder, its activations and the\nbuffers in flight come from the same pool. The size compared against that budget is the sum\nof *both* caches, training and test, because both are resident at once.\n\nA valid submission file is written before any of this begins and overwritten once real\npredictions exist, so the run always leaves a scoreable file behind.\n","metadata":{}},{"id":"code-23","cell_type":"code","source":"# Where the geometric slice order may be remembered between runs. Unset on the platform,\n# because each run gets a fresh machine and there is nothing to remember; set off it,\n# where the same corpus is cached again at every resolution and slice count and the order\n# is a function of neither. It is opt-in so that the scored run's behaviour is decided by\n# the code rather than by whether a file happens to be lying about.\nORDER_CACHE = os.environ.get(\"RSNA_ORDER_CACHE\") or None\n\n\ndef build_cache(slot_map, plane_map, lat_map, tag):\n    \"\"\"Decode every (study, slot) once into an in-memory uint8 array.\n\n    Fine-tuning revisits the same pixels every epoch. Reading them from the mount each\n    time would make the epoch count a function of I/O rather than of learning, so they\n    are decoded once and held as bytes: intensity has already been normalised into\n    [0, 1], and eight bits cost nothing a bilinear resize has not already cost.\n\n    CACHE_SLICES positions are kept per slot, which the training loop reads as N_GROUP\n    groups of GROUP consecutive channels.\n    \"\"\"\n    studies = sorted(slot_map)\n    sidx = {s: i for i, s in enumerate(studies)}\n    cache = np.zeros((len(studies), N_SLOT, CACHE_SLICES, IMG, IMG), np.uint8)\n    mask = np.zeros((len(studies), N_SLOT), np.float32)\n    log(f\"{tag}: cache {cache.shape} = {cache.nbytes / 1024 ** 3:.1f} GB\")\n\n    jobs = [(st, k, plane, slot_map[st][name])\n            for st in studies\n            for k, (name, plane, _, _) in enumerate(SLOTS)\n            if name in slot_map[st]]\n    n_job = len(jobs)\n\n    # Ordering first, and as its own pass. It reads one header per slice of every chosen\n    # series - far more file opens than the decode that follows - and on a network mount\n    # that is latency, not work, so it gets its own wider pool.\n    t_ord = time.time()\n    n_slice_total = sum(len(j[3][\"files\"]) for j in jobs)\n    log(f\"{tag}: ordering {len(jobs)} slot-series ({n_slice_total} slice headers)\")\n    ok = done = 0\n    CHUNK_O = 1024\n\n    # A remembered order, when one is offered. The projection depends on the DICOM\n    # geometry alone, so it is the same at every resolution and every slice count, and\n    # it costs one header read per slice - the largest single cost in this pass. An entry\n    # is validated by the number of files present, so a tree that has changed under it is\n    # recomputed rather than trusted: order is derived data, and a stale entry would be\n    # invisible in the way that matters most.\n    seen = {}\n    if ORDER_CACHE and Path(ORDER_CACHE).is_file():\n        try:\n            import json as _json\n            seen = _json.loads(Path(ORDER_CACHE).read_text())\n        except (OSError, ValueError):\n            seen = {}\n        hit = 0\n        for _, _, _, rec in jobs:\n            e = seen.get(rec[\"SeriesInstanceUID\"])\n            if e and len(e[\"files\"]) == len(rec[\"files\"]):\n                rec[\"ordered\"] = e[\"files\"]\n                ok += int(e[\"good\"])\n                hit += 1\n        jobs = [j for j in jobs if \"ordered\" not in j[3]]\n        log(f\"{tag}: {hit} slot-series ordered from {ORDER_CACHE}, {len(jobs)} to read\")\n\n    with ThreadPoolExecutor(max_workers=ORDER_THREADS) as pool:\n        for c0 in range(0, len(jobs), CHUNK_O):\n            block = jobs[c0:c0 + CHUNK_O]\n            for (_, _, _, rec), (files, good) in zip(\n                    block, pool.map(lambda j: order_slices(j[3]), block)):\n                rec[\"ordered\"] = files\n                ok += int(good)\n                done += 1\n                if ORDER_CACHE:\n                    seen[rec[\"SeriesInstanceUID\"]] = {\"files\": files, \"good\": bool(good)}\n            # The ceiling is whichever comes first: the pass's own budget, or the share\n            # of what is left of the run that it may take. The second is what makes the\n            # first safe to set generously - a mount slow enough to matter cannot spend\n            # the training time, because the budget shrinks as the run does.\n            budget = min(ORDER_BUDGET_S, max(60.0, (TIME_BUDGET - (time.time() - T0)) * 0.35))\n            if time.time() - t_ord > budget:\n                log(f\"{tag}: ordering budget spent at {done}/{len(jobs)}; \"\n                    f\"the rest keep file order\")\n                break\n    if ORDER_CACHE and done:\n        import json as _json\n        _t = Path(ORDER_CACHE).with_suffix(\".tmp\")\n        _t.write_text(_json.dumps(seen))\n        _t.replace(Path(ORDER_CACHE))\n    log(f\"{tag}: ordered {ok}/{n_job} by geometry \"\n        f\"({n_job - ok} kept arbitrary) in {time.time() - t_ord:.0f}s\")\n\n    jobs = [(st, k, plane, slot_map[st][name])\n            for st in studies\n            for k, (name, plane, _, _) in enumerate(SLOTS)\n            if name in slot_map[st]]\n    log(f\"{tag}: decoding {len(jobs)} slot-series\")\n    n_failed_before = len(DECODE_FAILED)\n\n    CHUNK = 512\n    done = 0\n    with ThreadPoolExecutor(max_workers=PIX_THREADS) as pool:\n        for c0 in range(0, len(jobs), CHUNK):\n            block = jobs[c0:c0 + CHUNK]\n            for (st, k, plane, _), img in zip(\n                    block, pool.map(lambda j: read_slot(j[3], CACHE_SLICES, IMG), block)):\n                done += 1\n                if img is None:\n                    continue\n                cache[sidx[st], k] = normalise_laterality(img, plane,\n                                                          lat_map.get(st)).numpy()\n                mask[sidx[st], k] = 1.0\n            if done % 4096 < CHUNK:\n                log(f\"  {tag} {done}/{len(jobs)}\")\n            if time.time() - T0 > TIME_BUDGET:\n                log(f\"  {tag}: time budget reached during decode\")\n                break\n    n_failed = len(DECODE_FAILED) - n_failed_before\n    log(f\"{tag}: {int(mask.sum())}/{len(jobs)} slots filled\"\n        + (f\"; {n_failed} series had a slice that would not decode\" if n_failed else \"\"))\n    gc.collect()\n    return studies, cache, mask\n","metadata":{"_kg_hide-input":true,"_kg_hide-output":true},"outputs":[],"execution_count":null},{"id":"md-24","cell_type":"markdown","source":"## 6. Aggregating slots into twelve decisions\n\nA study arrives as up to six slot embeddings $x_s \\in \\mathbb{R}^{d}$ with a presence mask\n$m_s \\in \\{0,1\\}$. Pooling them identically would discard the reason the protocol has three\nplanes: each finding is read on particular sequences, and a mean over slots dilutes the one\ncarrying the evidence with five that do not.\n\nProject each slot, add a learned slot identity, give every diagnosis $o$ its own query\n$q_o \\in \\mathbb{R}^{H}$, and let it attend over the slots with absent ones masked out of\nthe softmax:\n\n$$h_s \\;=\\; \\phi(x_s) + e_s, \\qquad\n\\alpha_{o,s} \\;=\\; \\frac{\\exp\\!\\big(\\langle h_s, q_o\\rangle / \\sqrt{H}\\big)\\, m_s}\n{\\sum_{s'} \\exp\\!\\big(\\langle h_{s'}, q_o\\rangle / \\sqrt{H}\\big)\\, m_{s'}},$$\n\n$$c_o \\;=\\; \\sum_s \\alpha_{o,s}\\, h_s, \\qquad\n\\ell_o \\;=\\; \\langle c_o, w_o \\rangle + b_o .$$\n\nThe masked softmax renormalises over whatever the study actually contains, so a missing\naxial series shifts a diagnosis's attention onto the sequences that are present instead of\nfeeding it a zero vector.\n\n**The head is deliberately this small.** The label is attached to the *study*, so nothing\nin the supervision says which part of a study carries the finding, and a richer aggregation\nwould have no signal to learn that from. Where the supervision is coarse, so is the head.\n","metadata":{}},{"id":"code-25","cell_type":"code","source":"class SlotHead(nn.Module):\n    \"\"\"Per-diagnosis attention over the slot embeddings of one study.\n\n    Each finding is read on particular sequences - cruciates sagittally, collateral\n    ligaments and the meniscal body coronally, patellar cartilage axially - so pooling\n    the slots identically would dilute the one that carries the evidence with the rest.\n\n    The aggregation is deliberately this simple. With a study-level label there is no\n    signal telling the model which part of a study matters, so extra attention\n    parameters below the slot level would have nothing to learn from and would spend\n    their capacity fitting noise.\n    \"\"\"\n\n    def __init__(self, dim, n_slot, n_out, hidden=256, p=0.2, prior=False):\n        super().__init__()\n        self.proj = nn.Sequential(nn.LayerNorm(dim), nn.Linear(dim, hidden), nn.GELU())\n        self.slot_emb = nn.Parameter(torch.randn(n_slot, hidden) * 0.02)\n        self.query = nn.Parameter(torch.randn(n_out, hidden) * 0.02)\n        self.drop = nn.Dropout(p)\n        self.out = nn.Linear(hidden, n_out)\n        self.hidden = hidden\n        # An imported member carries a fixed per-(diagnosis, slot) tilt on the attention\n        # logits, set from the anatomy table below rather than learned. It is a buffer, so\n        # it travels in the state dict and must exist for that member to load; exp(0.55)\n        # gives a preferred slot about 1.73x the weight of an unpreferred one, which\n        # biases the softmax without ever excluding a slot.\n        p_ = torch.zeros(n_out, n_slot)\n        if prior and n_slot == len(SLOTS) and n_out == len(TARGETS):\n            for t, slots in SLOT_PRIOR_TABLE.items():\n                if t in TARGETS:\n                    p_[TARGETS.index(t), list(slots)] = SLOT_PRIOR_STRENGTH\n        self.prior = prior\n        if prior:\n            self.register_buffer(\"slot_prior\", p_)\n\n    def forward(self, x, mask):\n        h = self.proj(x) + self.slot_emb\n        att = torch.einsum(\"bsh,oh->bos\", h, self.query) / self.hidden ** 0.5\n        if self.prior:\n            att = att + self.slot_prior.unsqueeze(0)\n        att = att.masked_fill(mask.unsqueeze(1) < 0.5, -1e4).softmax(-1)\n        ctx = self.drop(torch.einsum(\"bos,bsh->boh\", att, h))\n        return (ctx * self.out.weight.unsqueeze(0)).sum(-1) + self.out.bias\n","metadata":{"_kg_hide-input":true,"_kg_hide-output":true},"outputs":[],"execution_count":null},{"id":"code-26","cell_type":"code","source":"class Model(nn.Module):\n    \"\"\"Encoder plus head, trained end to end.\n\n    A study arrives as a bag of slot images. The bag is flattened for the encoder and\n    folded back before the head, so the encoder never sees the study structure and the\n    head never sees pixels.\n    \"\"\"\n\n    def __init__(self, backbone, dim, pool=\"cls_mean\", prior=False):\n        super().__init__()\n        self.backbone = backbone\n        self.pool = pool\n        self.head = SlotHead(dim * POOL_PARTS[pool], N_SLOT, len(TARGETS), prior=prior)\n        self.register_buffer(\"mean\", torch.tensor([0.485, 0.456, 0.406]).view(1, 3, 1, 1))\n        self.register_buffer(\"std\", torch.tensor([0.229, 0.224, 0.225]).view(1, 3, 1, 1))\n\n    def forward(self, imgs, mask, img_size=None):\n        B, S = imgs.shape[:2]\n        x = imgs.reshape(B * S, *imgs.shape[2:]).float().div_(255.0)\n        if img_size is not None and img_size != x.shape[-1]:\n            # The cache is held at the highest resolution any configuration needs; the\n            # rest downsample from it, so every configuration sees the same pixels\n            # through a different sampling grid rather than a different crop.\n            x = F.interpolate(x, size=(img_size, img_size), mode=\"bilinear\",\n                              align_corners=False)\n        x = (x - self.mean) / self.std\n        out = self.backbone(pixel_values=x).last_hidden_state\n        patch = out[:, 1:]\n        parts = [out[:, 0], patch.mean(1)]\n        if self.pool == \"cls_mean_focal\":\n            # The upper tail of each channel over the patch grid, taken per channel\n            # rather than by selecting whole patches: a finding occupies a small part of\n            # the field, so a plain mean over 256 patches dilutes it by two orders of\n            # magnitude, and this keeps the top eighth of each channel's responses.\n            k = max(1, patch.shape[1] // 8)\n            parts.append(patch.topk(k, dim=1).values.mean(1))\n        feat = torch.cat(parts, dim=1).reshape(B, S, -1)\n        return self.head(feat, mask)\n","metadata":{"_kg_hide-input":true,"_kg_hide-output":true},"outputs":[],"execution_count":null},{"id":"md-27","cell_type":"markdown","source":"### Why the encoder is trained rather than frozen\n\nA frozen self-supervised encoder is bounded by something no work downstream can reach.\nResolution, encoder size, slice coverage and slot aggregation change how much the model\nlooks and how closely, but none changes the vocabulary it looks *with*, so every one of those\naxes runs into the same ceiling — and that ceiling should be expected to bind here, since the\nencoder learned its features from natural images, where nothing resembles the signal a torn\nmeniscus makes on a proton-density sequence.\n\nSo the encoder is adapted, with two restraints. **Only the last blocks move** — early blocks\nof a vision transformer are generic edge and texture filters, late blocks are where\nsemantics live, and there may not be enough supervision here to improve the early ones while\nthere is certainly enough to damage them. **The encoder learns far more slowly than the\nhead** — the head is random at initialisation and has everything to learn, the encoder starts\nfrom a good solution and needs only to be moved off it, so a single learning rate would\neither leave the head untrained or destroy the encoder in the first few hundred steps.\n\nTargets remain the report-derived labels of §2, weighted by the per-finding confidence that\ncomes with them, so a report that never mentions a finding pulls weakly on that output\nrather than asserting a negative there. The studies carrying per-condition annotations are\nweighted above every derived row, since they are the only labels read from the images.\n","metadata":{}},{"id":"code-28","cell_type":"code","source":"def build_model(unfreeze_last, source=None, variant=\"small\", pool=\"cls_mean\",\n                prior=False):\n    \"\"\"Load the encoder and open the last `unfreeze_last` blocks for training.\n\n    The early blocks of a self-supervised transformer are generic edge and texture\n    filters; the late blocks carry semantics. Opening only the late ones is the cautious\n    choice - there may not be enough supervision here to improve the early ones and there\n    is certainly enough to damage them - but how far the line should sit is a question\n    the corpus has to answer rather than the intuition.\n\n    `source` names where the weights come from. Left unset it is the attached model\n    directory, which is the only thing available here. It is a parameter so that a run\n    off the platform builds the same object from the same code rather than from a second\n    definition that has to be kept in step by hand.\n    \"\"\"\n    from transformers import AutoModel\n    p = source if source is not None else find_dinov2(variant)\n    if p is None:\n        raise FileNotFoundError(\"DINOv2 weights not attached\")\n    bb = AutoModel.from_pretrained(str(p))\n    n_layer = len(bb.encoder.layer)\n    for prm in bb.parameters():\n        prm.requires_grad = False\n    for blk in bb.encoder.layer[max(0, n_layer - unfreeze_last):]:\n        for prm in blk.parameters():\n            prm.requires_grad = True\n    for prm in bb.layernorm.parameters():\n        prm.requires_grad = True\n    dim = bb.config.hidden_size\n    trainable = sum(p.numel() for p in bb.parameters() if p.requires_grad)\n    log(f\"backbone: {n_layer} blocks, last {unfreeze_last} trainable \"\n        f\"({trainable / 1e6:.1f}M params), feature dim {dim * POOL_PARTS[pool]}\")\n    return Model(bb, dim, pool=pool, prior=prior)\n","metadata":{"_kg_hide-input":true,"_kg_hide-output":true},"outputs":[],"execution_count":null},{"id":"md-29","cell_type":"markdown","source":"## 6b. Reading weights instead of learning them\n\nNothing requires the scored run to be the run that learned the weights. A notebook may\nattach a dataset, weights are a dataset, and the part that genuinely cannot be done in\nadvance is the part depending on studies nobody has seen — reading them, and predicting.\nLearning here instead caps the model at what one accelerator fits inside the time limit,\nand spends that time again on every submission for a result that does not change. So when\na package of trained members is attached this section reads it; when none is, the sections\nbelow train one.\n\n**Members, not a model.** Each member carries the preprocessing it was fitted on. Two\nmembers fitted at different resolutions cannot share a decode; two fitted alike can. So\nmembers are grouped by the pixels they need, each group is decoded once, and a member added\nlater joins the list unchanged.\n\nWhat groups them is more than the resolution and the crop. Four further decisions settle\nwhat a slice *is* — which one comes next along the stack, which knees are mirrored, whether\na slot may be filled from a sequence that does not match its predicate, and what stands in\nfor a slice that will not decode — and none of them changes a single shape. A member fitted\nunder one reading and decoded under another therefore loads cleanly, runs, and returns a\nsubmission computed from the wrong image, so the reading travels with the member.\n\n**A member must prove it is the model it was.** Loading a state dictionary succeeds\nwhenever the shapes line up, and shapes line up across every difference that matters — a\nchanged normalisation, resize, or slice band. None of those raise. So each member carries\nthe answer it gave to a seeded question, recomputed before use; a mismatch stops the run\nrather than being averaged into a submission.\n\n**How a member is read.** Training draws one group of consecutive slices per step. At\ninference, looking more costs no extra decoding — the cache already holds $S$ slices, so one\ngroup from the middle is $1$ forward pass, the disjoint groups are $S/G$, and every\nconsecutive run of $G$ slices is $S-G+1$. Where the average is taken is free but not\nneutral: averaging logits then squashing is a geometric mean of odds, averaging\nprobabilities an arithmetic mean of risk, and they order studies differently. Members are\ncombined by rank, since §1 established order is all the metric reads.\n","metadata":{}},{"id":"code-30","cell_type":"code","source":"FINGERPRINT_TOL = 2e-3\n\n\ndef fingerprint(model, dev, img_size, n_slot=None, group=None, seed=None):\n    \"\"\"The model's output on a fixed synthetic bag, as a portable identity.\n\n    Weights that are loaded but read through the wrong preprocessing produce predictions,\n    not errors. The submission is well formed, the log says nothing, and the difference is\n    a number no output of the run reveals. Scaling that never happens, or happens twice,\n    is enough on its own and changes no shape anywhere.\n\n    So a set of weights carries the answer it gave to a question with no data in it. The\n    input is generated from a seed rather than read, so it is the same on any machine, and\n    it is pushed through the whole forward path - the byte scaling, the ImageNet\n    normalisation, the resize, the encoder, the slot attention. Any of those differing\n    moves the output by order one. Numerics differing between two GPUs moves it by about\n    1e-5, which is why the tolerance sits between them rather than at zero.\n\n    This checks that the model computes what it computed when it was fitted. It cannot\n    check that the pixels reaching it are the right pixels; `read_slot` and the header\n    pass answer to their own tests.\n    \"\"\"\n    n_slot = N_SLOT if n_slot is None else n_slot\n    group = GROUP if group is None else group\n    seed = SEED if seed is None else seed\n    g = torch.Generator().manual_seed(seed)\n    imgs = torch.randint(0, 256, (2, n_slot, group, img_size, img_size),\n                         generator=g, dtype=torch.uint8).to(dev)\n    mask = torch.ones(2, n_slot, device=dev)\n    mask[1, -1] = 0.0                       # exercise the masked branch of the softmax\n    was_training = model.training\n    model.eval()\n    with torch.no_grad():\n        # float32 throughout: autocast would make the value depend on which device\n        # happened to run it, and the point of the number is that it does not.\n        out = model(imgs, mask, img_size).float().cpu().numpy()\n    if was_training:\n        model.train()\n    return out\n\n\ndef check_fingerprint(model, dev, img_size, expected, tol=FINGERPRINT_TOL, tag=\"\"):\n    \"\"\"Compare against a stored fingerprint; raise when the model is not the same map.\"\"\"\n    got = fingerprint(model, dev, img_size)\n    exp = np.asarray(expected, np.float32)\n    if got.shape != exp.shape:\n        raise WeightsError(f\"{tag}fingerprint shape {got.shape} != stored {exp.shape}: \"\n                           f\"the architecture is not the one these weights were fitted to\")\n    d = float(np.abs(got - exp).max())\n    if d > tol:\n        raise WeightsError(\n            f\"{tag}fingerprint differs by {d:.4g} (tolerance {tol:g}). The weights load \"\n            f\"but do not compute what they computed when fitted - preprocessing, \"\n            f\"resolution or architecture has moved between the two runs.\")\n    log(f\"{tag}fingerprint matches within {d:.2g}\")\n    return d\n\n\nclass WeightsError(RuntimeError):\n    \"\"\"Raised when attached weights cannot be trusted to be the ones that were fitted.\n\n    Deliberately fatal for the same reason as LabelSourceError: a run that predicts from\n    a mismatched model completes, writes a plausible submission, and differs from a\n    correct one only in a number no output of the run reveals.\n    \"\"\"\n\n\ndef find_weights(name=\"manifest.json\"):\n    \"\"\"Locate a mounted weights package, or return None if none is attached.\n\n    Same shape as `find_label_table`: the notebook must keep working for a reader who\n    attaches nothing, so absence is a path rather than an error. What must not be silent\n    is a package that is attached and unusable, and that is what `load_weights` refuses.\n    \"\"\"\n    import json\n    base = Path(\"/kaggle/input\")\n    if not base.is_dir():\n        return None\n    for root, dirs, files in os.walk(base):\n        dirs[:] = [d for d in dirs if d not in (\"train_series\", \"test_series\")]\n        if name not in files:\n            continue\n        # The manifest decides, not the filenames beside it. Testing for a naming\n        # convention makes the search agree with whatever the packager happened to call\n        # its files last, which is a second definition of what a package is.\n        try:\n            man = json.loads((Path(root) / name).read_text())\n        except (OSError, ValueError):\n            continue\n        if isinstance(man.get(\"members\"), list) and man[\"members\"]:\n            missing = [m[\"file\"] for m in man[\"members\"]\n                       if not (Path(root) / m[\"file\"]).is_file()]\n            if missing:\n                raise WeightsError(\n                    f\"{root} holds a manifest listing {len(man['members'])} members but \"\n                    f\"{len(missing)} of their files are absent (first {missing[0]!r})\")\n            return Path(root)\n    return None\n\n\n# How a member is read at inference. Overlapping windows over the slices the cache\n# already holds cost forward passes and no extra decoding, which is the cheap direction\n# to spend; and averaging probabilities rather than logits is an arithmetic mean of risk\n# rather than a geometric mean of odds, which orders studies differently. Both were\n# chosen by measuring them on the folds each member held out rather than by argument.\nTTA_OVERLAP = True\nTTA_POOL = \"prob\"\n\n# Public-frontier target pooling: the 0.899 notebook keeps the strongest window for\n# focal findings, and averages the two strongest windows for ACL/MCL. Its public\n# Pilkwang family is the same 20-member package used here, so these pooling decisions\n# transfer directly. Preserve the independently validated original-view Synovitis\n# route while retaining jitter for every other target.\nPUBLIC_FRONTIER_TARGET_POOL = {\n    \"Fracture\": \"max\",\n    \"Contusion\": \"max\",\n    \"Medial Meniscus\": \"max\",\n    \"Lateral Meniscus\": \"max\",\n    \"ACL\": \"top2\",\n    \"MCL\": \"top2\",\n    \"Baker's\": \"max\",\n}\nTTA_TARGET_POOL = {**PUBLIC_FRONTIER_TARGET_POOL, \"Synovitis\": \"original_mean\"}\n\n# V25 uses the 58 complete annotation rows from the official train.csv, joined by\n# StudyInstanceUID to the OOF predictions. The older y_derived artifact disagrees with\n# those expert labels in 145 cells and is not used for this decision. Repeated\n# target-stratified four-fold selection is macro-positive in 98% of 200 partitions and\n# isolates four targets; every other target returns to the native family. Lateral\n# Meniscus is shrunk from the selected median 1.00 to 0.75.\n# Convert desired final fraction f to each of four legacy-fold member weights:\n#     4*w/(20+4*w)=f -> w=5*f/(1-f).\nLEGACY_MEMBER_WEIGHT_BY_TARGET = {\n    \"Lateral Meniscus\": 15.0,           # final legacy fraction 0.75\n    \"Medial OA\": 2.5,                   # final legacy fraction 1/3\n    \"Lateral OA\": 15.0,                 # final legacy fraction 0.75\n    \"Contusion\": 5.0,                   # final legacy fraction 0.50\n}\n\n\ndef window_starts(n_slice, group, overlap=None):\n    \"\"\"Where each TTA window begins.\"\"\"\n    overlap = TTA_OVERLAP if overlap is None else overlap\n    if overlap and n_slice >= group:\n        return list(range(n_slice - group + 1))\n    return [g * group for g in range(max(n_slice // group, 1))]\n\n\ndef apply_target_window_pool(values, probs, logits, original_probs, mapping, target_idx):\n    \"\"\"Apply a target-specific pooling map to one batch in place.\"\"\"\n    for target, mode in mapping.items():\n        j = target_idx[target]\n        if mode == \"max\":\n            values[:, j] = probs[:, :, j].max(0).values\n        elif mode == \"mean\":\n            values[:, j] = probs[:, :, j].mean(0)\n        elif mode == \"logit_mean\":\n            values[:, j] = torch.sigmoid(logits[:, :, j].mean(0))\n        elif mode == \"original_mean\":\n            values[:, j] = original_probs[:, :, j].mean(0)\n        elif mode in (\"top2\", \"top3\"):\n            k = min(int(mode[3:]), probs.shape[0])\n            values[:, j] = probs[:, :, j].topk(k, dim=0).values.mean(0)\n        else:\n            raise ValueError(f\"unknown TTA pooling mode for {target}: {mode}\")\n    return values\n\n\n@torch.no_grad()\ndef predict_member(model, cache, mask, idx, dev, img_size, group=None, pool=None,\n                   starts=None, jitter=False, jitter_seed=SEED,\n                   return_public_frontier=False):\n    \"\"\"Predict one member; pool views within windows and windows across a study.\"\"\"\n    group = GROUP if group is None else group\n    pool = TTA_POOL if pool is None else pool\n    starts = window_starts(cache.shape[2], group) if starts is None else list(starts)\n    if not starts:\n        raise ValueError(\"predict_member was given no windows to average over\")\n    target_idx = {t: j for j, t in enumerate(TARGETS)}\n    unknown = (set(TTA_TARGET_POOL) | set(PUBLIC_FRONTIER_TARGET_POOL)) - set(target_idx)\n    if unknown:\n        raise ValueError(f\"unknown target(s) in TTA_TARGET_POOL: {unknown}\")\n\n    jitter_gen = torch.Generator(device=dev)\n    jitter_gen.manual_seed(int(jitter_seed) % (2**63 - 1))\n    model.eval()\n    out, public_frontier_out = [], []\n    for b in range(0, len(idx), EVAL_BATCH):\n        sel = idx[b:b + EVAL_BATCH]\n        m = torch.from_numpy(mask[sel]).to(dev)\n        win_probs, win_logits, win_original_probs = [], [], []\n        for st in starts:\n            rows = torch.from_numpy(\n                np.ascontiguousarray(cache[sel, :, st:st + group])).to(dev)\n            views = [rows] + ([augment(rows, generator=jitter_gen)] if jitter else [])\n            view_probs, view_logits = [], []\n            for view in views:\n                with torch.autocast(\"cuda\", enabled=dev.type == \"cuda\"):\n                    z = model(view, m, img_size).float()\n                view_logits.append(z)\n                view_probs.append(torch.sigmoid(z))\n            win_logits.append(torch.stack(view_logits).mean(0))\n            win_probs.append(torch.stack(view_probs).mean(0))\n            # views[0] is always the unaugmented acquisition. Preserve it so a\n            # target can opt out of jitter without another encoder forward pass.\n            win_original_probs.append(view_probs[0])\n\n        probs = torch.stack(win_probs)      # [window, batch, target]\n        logits = torch.stack(win_logits)\n        original_probs = torch.stack(win_original_probs)\n        v = (torch.sigmoid(logits.mean(0)) if pool == \"logit\" else probs.mean(0))\n        v = apply_target_window_pool(\n            v, probs, logits, original_probs, TTA_TARGET_POOL, target_idx\n        )\n        out.append(v.cpu().numpy())\n        if return_public_frontier:\n            public_v = apply_target_window_pool(\n                original_probs.mean(0), original_probs, logits, original_probs,\n                PUBLIC_FRONTIER_TARGET_POOL, target_idx,\n            )\n            public_frontier_out.append(public_v.cpu().numpy())\n    primary = (np.concatenate(out) if out else\n               np.zeros((0, len(TARGETS)), np.float32))\n    if not return_public_frontier:\n        return primary\n    public_frontier = (np.concatenate(public_frontier_out) if public_frontier_out else\n                       np.zeros((0, len(TARGETS)), np.float32))\n    return primary, public_frontier\n\n\n# HF from_pretrained mutates process-global state and is not guaranteed thread-safe, so\n# model construction, weight loading and the fingerprint check are serialised; only\n# inference -- the expensive part -- runs on both devices at once.\nBUILD_LOCK = threading.Lock()\nSTATE_LOCK = threading.Lock()\n\n# An independently trained four-fold bundle joins the vote at reduced weight: a second\n# training run (public 0.836 on its own) adds decorrelated errors, which is the only thing an\n# inference-only run can add that the 20-member package does not already have. 0.5 per\n# fold puts the bundle at ~11% of the total vote -- roughly its quality gap.\nLEGACY_BUNDLE_FILE = \"rsna_20260807_v1.pt\"\nLEGACY_WEIGHT = 0.5\n\n\ndef find_legacy_bundle():\n    base = Path(\"/kaggle/input\")\n    if not base.is_dir():\n        return None\n    for root, dirs, files in os.walk(base):\n        dirs[:] = [d for d in dirs if d not in (\"train_series\", \"test_series\")]\n        if LEGACY_BUNDLE_FILE in files:\n            return Path(root) / LEGACY_BUNDLE_FILE\n    return None\n\n\ndef legacy_group_members():\n    \"\"\"The four-fold bundle as extra, lower-weight members under RULES_LEGACY.\n\n    The package's legacy pixel rules exist precisely to reproduce what this bundle was\n    fitted on (dominant-axis slice order, corner-x laterality at 5 mm, T1 slot fallback,\n    zero decode fill, 160 mm crop, central 60% band). The bundle predates fingerprints,\n    which is accepted loudly and priced into its reduced weight; a fold whose state dict\n    does not load, or whose predictions are degenerate, is dropped and costs its own\n    vote only.\n    \"\"\"\n    p = find_legacy_bundle()\n    if p is None:\n        log(\"no legacy bundle attached; blending skipped\")\n        return {}\n    try:\n        b = torch.load(p, map_location=\"cpu\", weights_only=False)\n        folds = b.get(\"fold_states\") or []\n        b_slots = [tuple(s)[0] for s in b.get(\"slots\", SLOTS)]\n        if list(b.get(\"targets\", TARGETS)) != TARGETS or b_slots != [s[0] for s in SLOTS]:\n            log(f\"legacy bundle {p.name}: target/slot contract differs; blending skipped\")\n            return {}\n        gr, n_gr = int(b.get(\"group\", 3)), int(b.get(\"n_group\", 3))\n        variant = str(b.get(\"model_variant\", \"dinov2-small\")).split(\"-\")[-1]\n        key = json.dumps({\"img\": int(b.get(\"img\", 224)), \"group\": gr,\n                          \"slices\": gr * n_gr, \"crop_mm\": 160.0, \"band\": [0.20, 0.80],\n                          \"rules\": RULES_LEGACY, \"slots\": [s[0] for s in SLOTS]},\n                         sort_keys=True)\n        ms = [{\"id\": f\"legacy-f{f.get('fold', k)}\", \"fold\": f.get(\"fold\", k),\n               \"state\": f[\"state_dict\"], \"holdout\": None, \"weight\": LEGACY_WEIGHT,\n               \"target_weight\": [LEGACY_MEMBER_WEIGHT_BY_TARGET.get(t, 0.0)\n                                 for t in TARGETS],\n               \"pixel_group\": key,\n               \"config\": {\"unfreeze_last\": 6,\n                          \"variant\": \"base\" if variant == \"base\" else \"small\",\n                          \"pool\": \"cls_mean_focal\", \"prior\": True}}\n              for k, f in enumerate(folds)]\n        if ms:\n            active = sorted(set(LEGACY_MEMBER_WEIGHT_BY_TARGET.values()))\n            log(f\"legacy bundle {p.name}: {len(ms)} fold(s) join with \"\n                f\"target-specific per-member weights {active}\")\n        return {key: ms} if ms else {}\n    except Exception as exc:\n        log(f\"legacy bundle unusable ({type(exc).__name__}: {exc}); blending skipped\")\n        return {}\n\n\ndef _run_member(path, m, dev, Cte, Mte, idx, starts, jitter):\n    \"\"\"Load, verify and predict one member on one device. Returns (pred, timings).\"\"\"\n    t0 = time.time()\n    with BUILD_LOCK:\n        if \"state\" in m:\n            state, fp = m[\"state\"], None\n        else:\n            ck = torch.load(Path(path) / m[\"file\"], map_location=\"cpu\",\n                            weights_only=False)\n            state, fp = ck[\"model\"], ck.get(\"fingerprint\")\n        model = build_model(int(m[\"config\"][\"unfreeze_last\"]),\n                            variant=m[\"config\"][\"variant\"],\n                            pool=m[\"config\"].get(\"pool\", \"cls_mean\"),\n                            prior=bool(m[\"config\"].get(\"prior\", False))).to(dev)\n        model.load_state_dict(state)\n        if fp is not None:\n            check_fingerprint(model, dev, IMG, fp, tag=f\"{m['id']}: \")\n        else:\n            log(f\"  {m['id']}: no stored fingerprint (legacy bundle) -- \"\n                f\"accepted at reduced weight\")\n    t_ready = time.time()\n    jitter_seed = SEED + int(hashlib.sha256(str(m[\"id\"]).encode()).hexdigest()[:8], 16)\n    public_member = \"state\" not in m\n    predicted = predict_member(\n        model, Cte, Mte, idx, dev, IMG, starts=starts, jitter=jitter,\n        jitter_seed=jitter_seed, return_public_frontier=public_member,\n    )\n    if public_member:\n        p, public_p = predicted\n    else:\n        p, public_p = predicted, None\n    t_done = time.time()\n    del model, state\n    gc.collect()\n    if dev.type == \"cuda\":\n        with torch.cuda.device(dev):\n            torch.cuda.empty_cache()\n    passes = len(starts) * (2 if jitter else 1)\n    return p, public_p, (t_ready - t0, (t_done - t_ready) / max(passes, 1))\n\n\ndef _combine(per_member):\n    \"\"\"Target-wise weighted mean of per-member percentile ranks.\"\"\"\n    all_ids = sorted({s for m in per_member for s in m[\"ids\"]})\n    pos = {s: i for i, s in enumerate(all_ids)}\n    acc = np.zeros((len(all_ids), len(TARGETS)), np.float64)\n    tot = np.zeros(len(TARGETS), np.float64)\n    for m in per_member:\n        target_weight = m.get(\"target_weight\")\n        w = np.asarray(target_weight if target_weight is not None else\n                       [float(m.get(\"weight\", 1.0))] * len(TARGETS),\n                       dtype=np.float64)\n        if w.shape != (len(TARGETS),) or np.any(w < 0):\n            raise ValueError(f\"invalid target weights for {m.get('id')}: {w}\")\n        r = pd.DataFrame(m[\"pred\"]).rank(pct=True).to_numpy()\n        acc[[pos[s] for s in m[\"ids\"]]] += r * w[None, :]\n        tot += w\n    if np.any(tot <= 0):\n        raise ValueError(f\"at least one target has no ensemble vote: {tot}\")\n    return all_ids, acc / tot[None, :]\n\n\ndef infer_from_package(path, dev=None):\n    \"\"\"Predict the test split from an attached package of trained members.\n\n    Identical pixel contract to the reference implementation -- same caches, same\n    windows, same fingerprints, same rank transform -- with executive changes only:\n\n    1. A work queue over the devices: each GPU pops the next member when free (the\n       reference ran one device and surrendered TTA windows, then members: 0.847).\n    2. submission.csv is rewritten after every banked member, so a run killed at any\n       point still submits the best partial ensemble instead of the 0.5 benchmark.\n    3. A member that fails on one device is retried once on the other, then dropped;\n       a dropped member costs one vote, never the run.\n    4. An independently trained legacy bundle joins as four target-selective reduced-weight members.\n    5. When the time estimate says the whole remaining ensemble fits with room to\n       spare, each window gains one jittered TTA view (same family as training aug).\n    \"\"\"\n    man = json.loads((Path(path) / \"manifest.json\").read_text())\n    members = man[\"members\"]\n    log(f\"weights package: {len(members)} member(s) from {path}; \"\n        f\"{len(DEVS)} device(s)\")\n\n    test_df = pd.read_csv(ROOT / \"test.csv\")\n    test_series = pd.read_csv(ROOT / \"test_series.csv\")\n    plane_map = dict(zip(test_series[\"SeriesInstanceUID\"],\n                         test_series[\"Anatomical_Plane\"]))\n    hte = annotate(walk(\"test_series\"))\n    log(f\"test header pass: {len(hte)} series\")\n\n    groups = {}\n    for m in members:\n        groups.setdefault(m[\"pixel_group\"], []).append(m)\n    # Legacy votes come last: if time binds after all, the weakest votes are the ones\n    # surrendered, not the package's.\n    groups.update(legacy_group_members())\n\n    per_member, public_frontier_members = [], []\n    est = {\"fixed\": None, \"win\": None}\n\n    def bank(m, ids, pred, starts, jitter, public_pred=None):\n        if float(np.std(pred)) < 1e-9:\n            log(f\"  {m['id']}: degenerate predictions; not banked\")\n            return\n        with STATE_LOCK:\n            per_member.append({\"id\": m[\"id\"], \"ids\": ids, \"pred\": pred,\n                               \"weight\": m.get(\"weight\", 1.0),\n                               \"target_weight\": m.get(\"target_weight\"),\n                               \"holdout\": m.get(\"holdout\")})\n            if public_pred is not None and len(starts) == len(starts_full):\n                if float(np.std(public_pred)) < 1e-9:\n                    raise WeightsError(f\"{m['id']}: degenerate public-frontier prediction\")\n                public_frontier_members.append(\n                    {\"id\": m[\"id\"], \"ids\": ids, \"pred\": public_pred}\n                )\n            elif public_pred is not None:\n                log(\n                    f\"  {m['id']}: public-frontier vote omitted because only \"\n                    f\"{len(starts)} / {len(starts_full)} windows completed\"\n                )\n            all_ids, acc = _combine(per_member)\n            write_submission(acc, all_ids, test_df, \"submission.csv\")\n            log(f\"  banked {m['id']} fold {m.get('fold', '?')} \"\n                f\"({len(starts)} window(s){', jitter' if jitter else ''}); \"\n                f\"submission.csv = weighted rank mean of {len(per_member)} member(s)\")\n\n    for gi, (key, gm) in enumerate(groups.items(), 1):\n        cfg = json.loads(key)\n        adopt_config_globals(cfg)\n        log(f\"decode group {gi}/{len(groups)}: {cfg['img']}px x {cfg['slices']} slices, \"\n            f\"crop {cfg['crop_mm']} mm -> {len(gm)} member(s)\")\n        st_te, Cte, Mte = build_cache(pick_slots(hte, plane_map), plane_map,\n                                      lat_of(hte, \"test \"), f\"test g{gi}\")\n        idx = np.arange(len(st_te))\n\n        starts_full = window_starts(Cte.shape[2], GROUP)\n        pending = sorted(gm, key=lambda m: -(m.get(\"holdout\") or 0))\n        left_after = sum(len(g) for j, (_, g) in enumerate(groups.items(), 1) if j > gi)\n\n        def pop_next():\n            \"\"\"Next member, its window count, and whether jitter TTA is affordable.\n\n            Windows are surrendered before members; jitter is granted only when the\n            estimate says the whole remaining ensemble fits at double passes inside\n            60% of the room. The question asked is whether ONE more member fits.\n            \"\"\"\n            with STATE_LOCK:\n                if not pending:\n                    return None, None, False\n                left = TIME_BUDGET - (time.time() - T0)\n                remaining = len(pending) + left_after\n                slots_left = -(-remaining // len(DEVS))       # ceil: concurrent slots\n                starts, jit = starts_full, False\n                if est[\"fixed\"] is not None and est[\"win\"] is not None:\n                    afford = max(left * 0.9, 0.0)\n                    room = afford / max(slots_left, 1)\n                    if est[\"fixed\"] + est[\"win\"] > room:\n                        log(f\"  {left / 60:.0f} min left: surrendering \"\n                            f\"{len(pending)} member(s); not one more fits\")\n                        pending.clear()\n                        return None, None, False\n                    jit = (est[\"fixed\"] + 2 * len(starts_full) * est[\"win\"]\n                           <= room * 0.6)\n                    per_win = est[\"win\"] * (2 if jit else 1)\n                    n_win = (int((room - est[\"fixed\"]) / per_win)\n                             if per_win > 0 else len(starts_full))\n                    n_win = max(1, min(len(starts_full), n_win))\n                    if n_win < len(starts_full):\n                        mid = (len(starts_full) - n_win) // 2\n                        starts = starts_full[mid:mid + n_win]\n                return pending.pop(0), starts, jit\n\n        def worker(dev):\n            others = [d for d in DEVS if d is not dev]\n            while True:\n                m, starts, jit = pop_next()\n                if m is None:\n                    return\n                for attempt, d in enumerate([dev] + others[:1]):\n                    try:\n                        p, public_p, (fs, ws) = _run_member(\n                            path, m, d, Cte, Mte, idx, starts, jit\n                        )\n                        with STATE_LOCK:\n                            est[\"fixed\"], est[\"win\"] = fs, ws\n                        bank(m, st_te, p, starts, jit, public_p)\n                        break\n                    except Exception as exc:\n                        log(f\"  MEMBER {m['id']} failed on {d} \"\n                            f\"({type(exc).__name__}: {exc}); \"\n                            + (\"retrying on peer device\" if attempt == 0 and others\n                               else \"dropped -- costs one vote, not the run\"))\n                        if d.type == \"cuda\":\n                            with torch.cuda.device(d):\n                                torch.cuda.empty_cache()\n\n        threads = [threading.Thread(target=worker, args=(d,)) for d in DEVS]\n        for t in threads:\n            t.start()\n        for t in threads:\n            t.join()\n\n        del Cte, Mte\n        gc.collect()\n\n    if not per_member:\n        raise WeightsError(\"no member produced predictions; submission stays at 0.5\")\n\n    all_ids, acc = _combine(per_member)\n    sub = write_submission(acc, all_ids, test_df, \"submission.csv\")\n    log(f\"final submission.csv = weighted rank mean of {len(per_member)} member(s); \"\n        f\"{sub.shape}; nulls {int(sub[TARGETS].isna().sum().sum())}\")\n    if len(public_frontier_members) == len(members):\n        frontier_ids, frontier_acc = _combine(public_frontier_members)\n        frontier_sub = write_submission(\n            frontier_acc, frontier_ids, test_df, \"submission_public_0899.csv\"\n        )\n        log(\n            f\"submission_public_0899.csv = exact no-jitter public-frontier rank mean \"\n            f\"of {len(public_frontier_members)} member(s); {frontier_sub.shape}; \"\n            f\"nulls {int(frontier_sub[TARGETS].isna().sum().sum())}\"\n        )\n    else:\n        log(\n            f\"public-frontier fallback not emitted: {len(public_frontier_members)} / \"\n            f\"{len(members)} required public members completed\"\n        )\n    return sub\n\n\ndef adopt_config_globals(cfg):\n    \"\"\"Point the pixel path at what one group of members was fitted on.\"\"\"\n    global IMG, CACHE_IMG, GROUP, CACHE_SLICES, N_GROUP, CROP_MM, SLICE_BAND, RULES\n    CACHE_IMG = IMG = int(cfg[\"img\"])\n    GROUP = int(cfg[\"group\"])\n    CACHE_SLICES = int(cfg[\"slices\"])\n    N_GROUP = max(CACHE_SLICES // GROUP, 1)\n    CROP_MM = float(cfg[\"crop_mm\"])\n    SLICE_BAND = tuple(float(x) for x in cfg[\"band\"])\n    # The four decisions that change what a slice is. A member fitted under one reading\n    # and decoded under another gets pixels its weights never saw, with every shape\n    # still agreeing, so an unrecognised name is refused rather than defaulted.\n    rules = cfg.get(\"rules\") or RULES_NATIVE\n    unknown = {k: v for k, v in rules.items()\n               if k not in RULES_NATIVE\n               or v not in (RULES_NATIVE[k], RULES_LEGACY[k])}\n    if unknown:\n        raise WeightsError(f\"the members record pixel rules this pipeline cannot \"\n                           f\"reproduce: {unknown}\")\n    RULES = {**RULES_NATIVE, **rules}\n    if [s[0] for s in SLOTS] != list(cfg[\"slots\"]):\n        raise WeightsError(\n            f\"the members were fitted on slots {cfg['slots']} and this pipeline defines \"\n            f\"{[s[0] for s in SLOTS]}; a weight would be read against the wrong slot\")\n","metadata":{"_kg_hide-input":true,"_kg_hide-output":true},"outputs":[],"execution_count":null},{"id":"code-31","cell_type":"code","source":"def take_group(cache_rows, g):\n    \"\"\"Slice GROUP consecutive channels out of the cached slices.\"\"\"\n    return cache_rows[:, :, g * GROUP:(g + 1) * GROUP]\n\n\ndef augment(imgs, generator=None):\n    \"\"\"A small rigid jitter and an intensity scale, applied to a whole bag at once.\n\n    Neither flip is available here, and for different reasons. A horizontal flip would\n    reintroduce the nuisance axis that the laterality normalisation removed - it would\n    undo, once per batch, what the header pass was run to establish.\n\n    A vertical flip is not a nuisance axis at all. A knee is acquired in a canonical\n    orientation, and no study in this corpus looks like its own vertical mirror. An\n    augmentation is meant to cover directions along which the label does not change; this\n    one moves the input off the distribution the encoder will be asked about, which is a\n    different thing. Where a finding sits in the frame is also information rather than\n    noise - a Baker cyst is identified by lying in the popliteal fossa, not by its\n    appearance alone.\n\n    What is left is jitter that no label depends on: a few degrees of rotation, a few\n    per cent of scale and translation. That still prevents memorising the exact framing,\n    which is what an augmentation is for, while leaving the anatomy where it was.\n    \"\"\"\n    # A bag arrives as [study, slot, GROUP, IMG, IMG]: five axes, not four. The warp is\n    # a 2-D operation, so the two leading axes are folded together and restored after -\n    # every slot image is an independent acquisition and gets its own jitter.\n    lead = imgs.shape[:-3]\n    x = imgs.reshape(-1, *imgs.shape[-3:]).float()\n    n, dev = x.shape[0], x.device\n\n    rot = (torch.rand(n, device=dev, generator=generator) - 0.5) * 2 * (AUG_ROT_DEG * np.pi / 180)\n    # Zoom in only. `border` padding repeats the edge row outward, and the edge of this\n    # crop is where the popliteal fossa sits; zooming out would fabricate tissue exactly\n    # where a Baker cyst is looked for.\n    sc = 1.0 + torch.rand(n, device=dev, generator=generator) * AUG_SCALE\n    tx = (torch.rand(n, device=dev, generator=generator) - 0.5) * 2 * AUG_SHIFT\n    ty = (torch.rand(n, device=dev, generator=generator) - 0.5) * 2 * AUG_SHIFT\n    cos, sin = torch.cos(rot) / sc, torch.sin(rot) / sc\n    theta = torch.zeros(n, 2, 3, device=dev, dtype=torch.float32)\n    theta[:, 0, 0], theta[:, 0, 1], theta[:, 0, 2] = cos, -sin, tx\n    theta[:, 1, 0], theta[:, 1, 1], theta[:, 1, 2] = sin, cos, ty\n    grid = F.affine_grid(theta, x.shape, align_corners=False)\n    x = F.grid_sample(x, grid, mode=\"bilinear\", padding_mode=\"border\", align_corners=False)\n\n    scale = 1.0 + (torch.rand(n, 1, 1, 1, device=dev, generator=generator) - 0.5) * 2 * AUG_INTENSITY\n    x = (x * scale).clamp(0, 255)\n    return x.reshape(*lead, *x.shape[-3:]).to(imgs.dtype)\n\n\n@torch.no_grad()\ndef predict(model, cache, mask, idx, dev, img_size=None):\n    \"\"\"Average the logits over the groups of each slot.\n\n    Training sees one group at a time, which acts as augmentation along the stack;\n    inference averages over all of them, so the prediction does not depend on which\n    group a single draw happened to pick. Where the cache holds one group per slot the\n    two coincide.\n    \"\"\"\n    model.eval()\n    out = []\n    for b in range(0, len(idx), EVAL_BATCH):\n        sel = idx[b:b + EVAL_BATCH]\n        m = torch.from_numpy(mask[sel]).to(dev)\n        acc = None\n        for g in range(N_GROUP):\n            # Gathered a group at a time rather than whole and then sliced. The two are\n            # the same pixels, but taking the whole of a study out of the cache allocates\n            # every slice it holds - most of which this pass will not look at until a\n            # later iteration, by which time they have been fetched again. Measured over\n            # a cache of twelve slices, the difference between the two is the difference\n            # between the step being bound by memory and being bound by the encoder.\n            rows = torch.from_numpy(np.ascontiguousarray(\n                cache[sel, :, g * GROUP:(g + 1) * GROUP])).to(dev)\n            with torch.autocast(\"cuda\", enabled=dev.type == \"cuda\"):\n                z = model(rows, m, img_size).float()\n            acc = z if acc is None else acc + z\n        out.append(torch.sigmoid(acc / N_GROUP).cpu().numpy())\n    return np.concatenate(out) if out else np.zeros((0, len(TARGETS)), np.float32)\n\n\ndef macro_auc(y, p):\n    from sklearn.metrics import roc_auc_score\n    return float(np.nanmean([roc_auc_score(y[:, j], p[:, j])\n                             if len(set(y[:, j])) > 1 else np.nan\n                             for j in range(y.shape[1])]))\n","metadata":{"_kg_hide-input":true,"_kg_hide-output":true},"outputs":[],"execution_count":null},{"id":"md-32","cell_type":"markdown","source":"## 7. Validating without fooling yourself\n\nTwo leaks are specific to this setup, and both inflate a validation number without\nimproving anything.\n\n**Shared reports.** Some reports are byte-identical across studies — a template read for an\nunremarkable knee — so every study in such a group receives the same derived target vector,\nand splitting the group across the divide scores the model on a target whose source it was\ntrained on. Studies are therefore assigned by a hash of the report text, which keeps every\nduplicate group whole. One fifth is held out, and the split is fixed rather than rotated.\n\n**Two references, two meanings.** The **holdout** covers a fifth of the corpus, measures\nagreement with the derived targets, and has enough studies per label to separate a real\ndifference from noise — it selects both the epoch within a run and the recipe between runs.\nThe **annotation check** measures agreement with a radiologist's reading of the images,\nwhich is what the competition scores, but only the annotated studies falling in the holdout\ncan be used and there are very few; by the standard-error argument of §2 it is reported and\nnever allowed to arbitrate.\n\nThe annotated studies stay in training, at elevated weight, because they are the only labels\nread from the images rather than from text. That is exactly why the annotation check must be\nrestricted to the holdout: scoring a model on training examples whose answers it saw,\nweighted more heavily than anything else, measures memorisation and reports it as skill.\n","metadata":{}},{"id":"v26-synovitis-evidence","cell_type":"markdown","metadata":{},"source":"## 8. Independent Synovitis specialist\n\nVersion 26 adds a single evidence-qualified family, adapted with attribution from `blacklions/report-teacher-anatomy-aware-hierarchical-multimod`. Its frozen DINOv2-base RTA-HMIL ensemble uses six plane/sequence slots and seven 2.5D views per slot. On the 58 complete official annotation rows, its Synovitis AUC is 0.826 versus 0.742 for V25; repeated target-stratified four-fold selection chooses a positive weight in 99.95% of folds and gives a positive held-out delta in 99.4% of 500 partitions. The selected median weight is 0.75.\n\nThe specialist runs only after the complete V25 submission has been written. It atomically replaces only the Synovitis column after all eight checkpoints finish; missing inputs, decode failures, or a conservative 8.6-hour deadline leave the V25 file untouched. The other eleven targets are asserted bit-for-bit unchanged."},{"id":"v26-synovitis-runtime","cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"# Exact independent Synovitis specialist adapted from the public RTA-HMIL notebook.\n# The checkpoint architecture below is copied verbatim so state-dict semantics cannot drift.\nimport math\nimport cv2\nTARGET_FAMILIES = ['acl', 'mcl', 'medial_meniscus', 'lateral_meniscus', 'medial_oa', 'lateral_oa', 'pf_oa', 'effusion', 'synovitis', 'baker', 'contusion', 'fracture']\n\n# ============================================================\n# 10. RTA-HMIL model\n# ============================================================\nGROUP_NAMES = [\n    \"ligament\",\n    \"meniscus\",\n    \"oa\",\n    \"inflammation\",\n    \"bone\",\n    \"other\",\n]\n\ndef target_group_id(\n    family,\n):\n    if family in {\n        \"acl\",\n        \"mcl\",\n    }:\n        return 0\n\n    if family in {\n        \"medial_meniscus\",\n        \"lateral_meniscus\",\n    }:\n        return 1\n\n    if family in {\n        \"medial_oa\",\n        \"lateral_oa\",\n        \"pf_oa\",\n    }:\n        return 2\n\n    if family in {\n        \"effusion\",\n        \"synovitis\",\n        \"baker\",\n    }:\n        return 3\n\n    if family in {\n        \"contusion\",\n        \"fracture\",\n    }:\n        return 4\n\n    return 5\n\nTARGET_GROUP_IDS = torch.tensor(\n    [\n        target_group_id(f)\n        for f in TARGET_FAMILIES\n    ],\n    dtype=torch.long,\n)\n\nclass RTAHMIL(nn.Module):\n    def __init__(\n        self,\n        in_dim,\n        hidden_dim,\n        n_targets,\n        n_slots,\n        n_slices,\n        dropout,\n        series_dropout,\n    ):\n        super().__init__()\n\n        self.n_targets = n_targets\n        self.n_slots = n_slots\n        self.n_slices = n_slices\n        self.series_dropout = float(\n            series_dropout\n        )\n\n        self.input_proj = nn.Sequential(\n            nn.LayerNorm(\n                in_dim\n            ),\n            nn.Linear(\n                in_dim,\n                hidden_dim,\n            ),\n            nn.GELU(),\n            nn.Dropout(\n                dropout\n            ),\n        )\n\n        self.slice_pos_emb = nn.Parameter(\n            torch.randn(\n                n_slices,\n                hidden_dim,\n            )\n            / math.sqrt(hidden_dim)\n        )\n\n        slice_layer = (\n            nn.TransformerEncoderLayer(\n                d_model=hidden_dim,\n                nhead=8,\n                dim_feedforward=hidden_dim * 4,\n                dropout=dropout,\n                activation=\"gelu\",\n                batch_first=True,\n                norm_first=True,\n            )\n        )\n\n        self.slice_encoder = (\n            nn.TransformerEncoder(\n                slice_layer,\n                num_layers=1,\n            )\n        )\n\n        self.series_query = nn.Parameter(\n            torch.randn(\n                1,\n                1,\n                hidden_dim,\n            )\n            / math.sqrt(hidden_dim)\n        )\n\n        self.series_pool = (\n            nn.MultiheadAttention(\n                hidden_dim,\n                num_heads=8,\n                dropout=dropout,\n                batch_first=True,\n            )\n        )\n\n        self.slot_emb = nn.Embedding(\n            n_slots,\n            hidden_dim,\n        )\n\n        self.plane_emb = nn.Embedding(\n            3,\n            hidden_dim,\n        )\n\n        self.sequence_emb = nn.Embedding(\n            2,\n            hidden_dim,\n        )\n\n        study_layer = (\n            nn.TransformerEncoderLayer(\n                d_model=hidden_dim,\n                nhead=8,\n                dim_feedforward=hidden_dim * 4,\n                dropout=dropout,\n                activation=\"gelu\",\n                batch_first=True,\n                norm_first=True,\n            )\n        )\n\n        self.study_encoder = (\n            nn.TransformerEncoder(\n                study_layer,\n                num_layers=2,\n            )\n        )\n\n        self.target_queries = nn.Parameter(\n            torch.randn(\n                n_targets,\n                hidden_dim,\n            )\n            / math.sqrt(hidden_dim)\n        )\n\n        self.group_emb = nn.Embedding(\n            len(GROUP_NAMES),\n            hidden_dim,\n        )\n\n        self.target_cross_attn = (\n            nn.MultiheadAttention(\n                hidden_dim,\n                num_heads=8,\n                dropout=dropout,\n                batch_first=True,\n            )\n        )\n\n        self.target_fuse = nn.Sequential(\n            nn.Linear(\n                hidden_dim * 2,\n                hidden_dim,\n            ),\n            nn.GELU(),\n            nn.Dropout(\n                dropout\n            ),\n        )\n\n        self.target_heads = (\n            nn.ModuleList([\n                nn.Sequential(\n                    nn.LayerNorm(\n                        hidden_dim\n                    ),\n                    nn.Linear(\n                        hidden_dim,\n                        hidden_dim // 2,\n                    ),\n                    nn.GELU(),\n                    nn.Dropout(\n                        dropout\n                    ),\n                    nn.Linear(\n                        hidden_dim // 2,\n                        1,\n                    ),\n                )\n                for _ in range(\n                    n_targets\n                )\n            ])\n        )\n\n        # Slot -> plane/sequence metadata.\n        slot_plane = [\n            0, 0,\n            1, 1,\n            2, 2,\n        ]\n\n        slot_sequence = [\n            0, 1,\n            0, 1,\n            0, 1,\n        ]\n\n        self.register_buffer(\n            \"slot_plane_ids\",\n            torch.tensor(\n                slot_plane,\n                dtype=torch.long,\n            ),\n            persistent=False,\n        )\n\n        self.register_buffer(\n            \"slot_sequence_ids\",\n            torch.tensor(\n                slot_sequence,\n                dtype=torch.long,\n            ),\n            persistent=False,\n        )\n\n        self.register_buffer(\n            \"target_group_ids\",\n            TARGET_GROUP_IDS,\n            persistent=False,\n        )\n\n    def stochastic_slot_mask(\n        self,\n        mask,\n    ):\n        if (\n            not self.training\n            or self.series_dropout <= 0\n        ):\n            return mask\n\n        keep = (\n            torch.rand(\n                mask.shape,\n                device=mask.device,\n            )\n            > self.series_dropout\n        )\n\n        new_mask = (\n            mask & keep\n        )\n\n        # Never allow an all-masked study.\n        all_missing = (\n            ~new_mask\n        ).all(dim=1)\n\n        if all_missing.any():\n            for row in torch.where(\n                all_missing\n            )[0]:\n                valid = torch.where(\n                    mask[row]\n                )[0]\n\n                if len(valid) > 0:\n                    new_mask[\n                        row,\n                        valid[0],\n                    ] = True\n\n        return new_mask\n\n    def forward(\n        self,\n        x,\n        slot_mask,\n    ):\n        # x:\n        # B, S, K, Din\n\n        B, S, K, _ = x.shape\n\n        z = self.input_proj(\n            x\n        )\n\n        z = (\n            z\n            + self.slice_pos_emb[\n                None,\n                None,\n                :K,\n                :\n            ]\n        )\n\n        # Encode slices within every series.\n        z = z.reshape(\n            B * S,\n            K,\n            -1,\n        )\n\n        z = self.slice_encoder(\n            z\n        )\n\n        q = self.series_query.expand(\n            B * S,\n            -1,\n            -1,\n        )\n\n        series_token, _ = self.series_pool(\n            q,\n            z,\n            z,\n            need_weights=False,\n        )\n\n        series_token = (\n            series_token[:, 0]\n            .reshape(\n                B,\n                S,\n                -1,\n            )\n        )\n\n        slot_ids = torch.arange(\n            S,\n            device=x.device,\n        )\n\n        plane_ids = (\n            self.slot_plane_ids[\n                :S\n            ]\n        )\n\n        seq_ids = (\n            self.slot_sequence_ids[\n                :S\n            ]\n        )\n\n        series_token = (\n            series_token\n            + self.slot_emb(\n                slot_ids\n            )[None, :, :]\n            + 0.35\n            * self.plane_emb(\n                plane_ids\n            )[None, :, :]\n            + 0.35\n            * self.sequence_emb(\n                seq_ids\n            )[None, :, :]\n        )\n\n        effective_mask = (\n            self.stochastic_slot_mask(\n                slot_mask\n            )\n        )\n\n        # Hidden-test safety: if a study failed to decode every selected\n        # series, prevent an all-masked attention row (which can yield NaNs).\n        no_valid_slot = (\n            ~effective_mask\n        ).all(dim=1)\n\n        if no_valid_slot.any():\n            effective_mask = effective_mask.clone()\n            series_token = series_token.clone()\n            effective_mask[\n                no_valid_slot,\n                0,\n            ] = True\n            series_token[\n                no_valid_slot,\n                0,\n            ] = 0.0\n\n        series_token = (\n            self.study_encoder(\n                series_token,\n                src_key_padding_mask=(\n                    ~effective_mask\n                ),\n            )\n        )\n\n        denom = (\n            effective_mask\n            .sum(\n                dim=1,\n                keepdim=True,\n            )\n            .clamp_min(1)\n            .to(\n                series_token.dtype\n            )\n        )\n\n        study_global = (\n            (\n                series_token\n                * effective_mask\n                .unsqueeze(-1)\n            )\n            .sum(dim=1)\n            / denom\n        )\n\n        target_q = (\n            self.target_queries\n            + 0.25\n            * self.group_emb(\n                self.target_group_ids\n            )\n        )\n\n        target_q = (\n            target_q\n            .unsqueeze(0)\n            .expand(\n                B,\n                -1,\n                -1,\n            )\n        )\n\n        target_context, _ = (\n            self.target_cross_attn(\n                target_q,\n                series_token,\n                series_token,\n                key_padding_mask=(\n                    ~effective_mask\n                ),\n                need_weights=False,\n            )\n        )\n\n        global_expand = (\n            study_global\n            .unsqueeze(1)\n            .expand(\n                -1,\n                self.n_targets,\n                -1,\n            )\n        )\n\n        fused = self.target_fuse(\n            torch.cat(\n                [\n                    target_context,\n                    global_expand,\n                ],\n                dim=-1,\n            )\n        )\n\n        logits = []\n\n        for j, head in enumerate(\n            self.target_heads\n        ):\n            logits.append(\n                head(\n                    fused[:, j]\n                )\n            )\n\n        return torch.cat(\n            logits,\n            dim=1,\n        )\n\n\"\"\"Runtime helpers embedded into the V26 Kaggle notebook.\n\nThe exact RTAHMIL class from the public report-teacher notebook is prepended by the\ncandidate builder. This file contains only hidden-test feature extraction, checkpoint\ninference, and the fail-safe Synovitis blend.\n\"\"\"\n\nRT_START_CUTOFF_S = 5.90 * 3600\nRT_DEADLINE_S = 7.10 * 3600\nRT_IMG_SIZE = 336\nRT_TARGET_SPACING = 0.42\nRT_SLICES = 7\nRT_SYN_WEIGHT = 0.75\nRT_SEEDS = (2026, 3407)\n\n\ndef _rt_find_checkpoint_dir():\n    root = Path(\"/kaggle/input\")\n    required = [\n        f\"rta_final_seed{seed}_fold{fold}.pth\"\n        for seed in RT_SEEDS\n        for fold in range(4)\n    ]\n    for first in required[:1]:\n        for hit in root.glob(f\"*/{first}\"):\n            parent = hit.parent\n            if all((parent / name).is_file() for name in required):\n                return parent\n    raise FileNotFoundError(\"the complete eight-checkpoint report-teacher package is absent\")\n\n\ndef _rt_find_dino_base():\n    direct = [\n        Path(\"/kaggle/input/dinov2/pytorch/base/1\"),\n        Path(\"/kaggle/input/models/metaresearch/dinov2/pytorch/base/1\"),\n    ]\n    for path in direct:\n        if (path / \"config.json\").is_file():\n            return path\n    for top in Path(\"/kaggle/input\").iterdir():\n        if not top.is_dir() or \"dino\" not in top.name.lower():\n            continue\n        for config in top.glob(\"**/config.json\"):\n            try:\n                if \"dinov2\" in config.read_text(errors=\"ignore\").lower():\n                    model_type = json.loads(config.read_text()).get(\"model_type\", \"\")\n                    if model_type == \"dinov2\" and \"base\" in str(config.parent).lower():\n                        return config.parent\n            except Exception:\n                continue\n    raise FileNotFoundError(\"offline DINOv2-base model is absent\")\n\n\ndef _rt_binary_flag(value):\n    if pd.isna(value):\n        return 0\n    if isinstance(value, str):\n        return int(value.strip().lower() in {\"1\", \"true\", \"yes\", \"y\"})\n    try:\n        return int(float(value) > 0)\n    except Exception:\n        return 0\n\n\ndef _rt_plane_id(value):\n    text = str(value).lower()\n    if \"sag\" in text:\n        return 0\n    if \"cor\" in text:\n        return 1\n    if \"axi\" in text or \"trans\" in text or \"tra\" == text.strip():\n        return 2\n    return 3\n\n\ndef _rt_assign_slots(series_df):\n    x = series_df.copy()\n    x[\"StudyInstanceUID\"] = x[\"StudyInstanceUID\"].astype(str)\n    x[\"SeriesInstanceUID\"] = x[\"SeriesInstanceUID\"].astype(str)\n    x[\"_plane_id\"] = x[\"Anatomical_Plane\"].map(_rt_plane_id)\n    fluid = x[\"Fluid_Sensitive\"].map(_rt_binary_flag)\n    fat = x[\"Fat_Suppression\"].map(_rt_binary_flag)\n    x[\"_fluid_like\"] = np.maximum(fluid.astype(int), fat.astype(int))\n    slot_defs = ((0, 0), (0, 1), (1, 0), (1, 1), (2, 0), (2, 1))\n    lookup = {}\n    for study_uid, group in x.groupby(\"StudyInstanceUID\", sort=False):\n        group = group.sort_values([\"SeriesInstanceUID\"]).copy()\n        used = set()\n        for slot_id, (plane, fluid_like) in enumerate(slot_defs):\n            desired = group[\n                (group[\"_plane_id\"] == plane)\n                & (group[\"_fluid_like\"] == fluid_like)\n                & (~group[\"SeriesInstanceUID\"].isin(used))\n            ]\n            if len(desired) == 0:\n                desired = group[\n                    (group[\"_plane_id\"] == plane)\n                    & (~group[\"SeriesInstanceUID\"].isin(used))\n                ]\n            if len(desired) == 0:\n                continue\n            series_uid = str(desired.iloc[0][\"SeriesInstanceUID\"])\n            used.add(series_uid)\n            lookup[(str(study_uid), slot_id)] = series_uid\n    return lookup\n\n\ndef _rt_locate_series_dir(study_uid, series_uid):\n    canonical = ROOT / \"test_series\"\n    candidates = (\n        canonical / str(series_uid),\n        canonical / str(study_uid) / str(series_uid),\n        ROOT / \"test\" / str(study_uid) / str(series_uid),\n        ROOT / \"test_images\" / str(study_uid) / str(series_uid),\n        ROOT / \"test_dicom\" / str(study_uid) / str(series_uid),\n        ROOT / \"test_dicoms\" / str(study_uid) / str(series_uid),\n        ROOT / \"images\" / \"test\" / str(study_uid) / str(series_uid),\n    )\n    for path in candidates:\n        if path.is_dir():\n            return path\n    raise FileNotFoundError(\n        f\"report-teacher series missing: study={study_uid}, series={series_uid}\"\n    )\n\n\ndef _rt_sorted_dicom_files(series_dir):\n    files = list(Path(series_dir).glob(\"*.dcm\"))\n    if not files:\n        files = [path for path in Path(series_dir).iterdir() if path.is_file()]\n    if not files:\n        raise FileNotFoundError(f\"no DICOM files in {series_dir}\")\n    simple_numeric = [path.stem.isdigit() and len(path.stem) <= 8 for path in files]\n    if np.mean(simple_numeric) >= 0.90:\n        number_re = re.compile(r\"(\\d+)\")\n\n        def key(path):\n            matches = number_re.findall(path.stem)\n            return int(matches[-1]) if matches else 10**12\n\n        return sorted(files, key=key)\n    keyed = []\n    for index, path in enumerate(files):\n        try:\n            ds = pydicom.dcmread(str(path), stop_before_pixels=True, force=True)\n            if hasattr(ds, \"ImagePositionPatient\") and len(ds.ImagePositionPatient) >= 3:\n                key = float(ds.ImagePositionPatient[2])\n            else:\n                key = float(getattr(ds, \"InstanceNumber\", index))\n        except Exception:\n            key = float(index)\n        keyed.append((key, path))\n    return [path for _, path in sorted(keyed, key=lambda pair: pair[0])]\n\n\ndef _rt_robust_uint8(array):\n    array = np.asarray(array, dtype=np.float32)\n    finite = np.isfinite(array)\n    if not finite.any():\n        return np.zeros(array.shape, dtype=np.uint8)\n    values = array[finite]\n    lo, hi = np.percentile(values, [1.0, 99.0])\n    if hi <= lo:\n        lo, hi = float(values.min()), float(values.max()) + 1e-6\n    array = np.clip(array, lo, hi)\n    array = (array - lo) / max(hi - lo, 1e-6)\n    return np.clip(array * 255.0, 0, 255).astype(np.uint8)\n\n\ndef _rt_center_crop_or_pad(image):\n    height, width = image.shape[:2]\n    pad_y, pad_x = max(0, RT_IMG_SIZE - height), max(0, RT_IMG_SIZE - width)\n    if pad_y or pad_x:\n        top, left = pad_y // 2, pad_x // 2\n        image = cv2.copyMakeBorder(\n            image,\n            top,\n            pad_y - top,\n            left,\n            pad_x - left,\n            borderType=cv2.BORDER_CONSTANT,\n            value=0,\n        )\n    height, width = image.shape[:2]\n    y0, x0 = max(0, (height - RT_IMG_SIZE) // 2), max(0, (width - RT_IMG_SIZE) // 2)\n    return image[y0 : y0 + RT_IMG_SIZE, x0 : x0 + RT_IMG_SIZE]\n\n\ndef _rt_read_dicom(path):\n    ds = pydicom.dcmread(str(path), force=True)\n    array = ds.pixel_array.astype(np.float32)\n    slope = float(getattr(ds, \"RescaleSlope\", 1.0) or 1.0)\n    intercept = float(getattr(ds, \"RescaleIntercept\", 0.0) or 0.0)\n    image = _rt_robust_uint8(array * slope + intercept)\n    if str(getattr(ds, \"PhotometricInterpretation\", \"\")).upper() == \"MONOCHROME1\":\n        image = 255 - image\n    spacing = getattr(ds, \"PixelSpacing\", None)\n    if spacing is not None and len(spacing) >= 2:\n        try:\n            scale_y = np.clip(float(spacing[0]) / RT_TARGET_SPACING, 0.40, 3.0)\n            scale_x = np.clip(float(spacing[1]) / RT_TARGET_SPACING, 0.40, 3.0)\n            new_h = max(32, int(round(image.shape[0] * scale_y)))\n            new_w = max(32, int(round(image.shape[1] * scale_x)))\n            image = cv2.resize(image, (new_w, new_h), interpolation=cv2.INTER_LINEAR)\n            return _rt_center_crop_or_pad(image)\n        except Exception:\n            pass\n    return cv2.resize(image, (RT_IMG_SIZE, RT_IMG_SIZE), interpolation=cv2.INTER_AREA)\n\n\ndef _rt_load_series_25d(study_uid, series_uid):\n    files = _rt_sorted_dicom_files(_rt_locate_series_dir(study_uid, series_uid))\n    quantiles = np.array([0.08, 0.23, 0.38, 0.50, 0.62, 0.77, 0.92], np.float32)\n    centers = np.zeros(RT_SLICES, np.int64) if len(files) <= 1 else np.round(\n        quantiles * (len(files) - 1)\n    ).astype(np.int64)\n    centers = np.clip(centers, 0, len(files) - 1)\n    views = []\n    for center in centers:\n        channels = []\n        for index in (max(0, center - 1), center, min(len(files) - 1, center + 1)):\n            try:\n                channels.append(_rt_read_dicom(files[index]))\n            except Exception:\n                channels.append(np.zeros((RT_IMG_SIZE, RT_IMG_SIZE), dtype=np.uint8))\n        views.append(np.stack(channels, axis=-1))\n    return np.stack(views, axis=0)\n\n\ndef _rt_try_attached_visible_features(checkpoint_dir, expected_uids):\n    uid_path = checkpoint_dir / \"rta_final_test_uids.txt\"\n    feature_path = checkpoint_dir / \"rta_final_test_features.npy\"\n    mask_path = checkpoint_dir / \"rta_final_test_slot_mask.npy\"\n    if not (uid_path.is_file() and feature_path.is_file() and mask_path.is_file()):\n        return None\n    if uid_path.read_text().splitlines() != list(expected_uids):\n        return None\n    features = np.load(feature_path, mmap_mode=\"r\")\n    mask = np.load(mask_path, mmap_mode=\"r\")\n    if features.shape[:3] != (len(expected_uids), 6, 7) or mask.shape != (len(expected_uids), 6):\n        return None\n    log(\"report-teacher: exact attached visible-test features reused\")\n    return features, mask\n\n\ndef _rt_extract_features(test_df, series_df, checkpoint_dir, dev):\n    from transformers import AutoModel\n\n    expected_uids = test_df[\"StudyInstanceUID\"].astype(str).tolist()\n    attached = _rt_try_attached_visible_features(checkpoint_dir, expected_uids)\n    if attached is not None:\n        return attached\n    if time.time() - T0 > RT_START_CUTOFF_S:\n        raise TimeoutError(\"insufficient runtime reserve for report-teacher feature extraction\")\n    dino_dir = _rt_find_dino_base()\n    log(f\"report-teacher: DINOv2 base from {dino_dir}\")\n    dino = AutoModel.from_pretrained(str(dino_dir), local_files_only=True).eval().to(dev)\n    for parameter in dino.parameters():\n        parameter.requires_grad_(False)\n    dino_dim = int(dino.config.hidden_size)\n    if dino_dim != 768:\n        raise AssertionError(f\"expected DINOv2-base hidden size 768, got {dino_dim}\")\n    mean = torch.tensor([0.485, 0.456, 0.406]).view(1, 3, 1, 1)\n    std = torch.tensor([0.229, 0.224, 0.225]).view(1, 3, 1, 1)\n    slot_lookup = _rt_assign_slots(series_df)\n    features = np.zeros((len(expected_uids), 6, RT_SLICES, dino_dim * 2), np.float16)\n    slot_mask = np.zeros((len(expected_uids), 6), bool)\n\n    @torch.inference_mode()\n    def encode(images):\n        tensor = torch.from_numpy(images).permute(0, 3, 1, 2).float() / 255.0\n        tensor = (tensor - mean) / std\n        parts = []\n        for start in range(0, len(tensor), 8):\n            batch = tensor[start : start + 8].to(dev, non_blocking=True)\n            with torch.autocast(\n                device_type=\"cuda\", dtype=torch.float16, enabled=dev.type == \"cuda\"\n            ):\n                output = dino(pixel_values=batch, interpolate_pos_encoding=True)\n                tokens = output.last_hidden_state\n                part = torch.cat((tokens[:, 0], tokens[:, 1:].mean(dim=1)), dim=-1)\n            parts.append(part.float().cpu())\n        return torch.cat(parts, dim=0).numpy().astype(np.float32)\n\n    for row_index, study_uid in enumerate(expected_uids):\n        if time.time() - T0 > RT_DEADLINE_S:\n            raise TimeoutError(\"report-teacher deadline reached before submission overwrite\")\n        jobs = [\n            (slot_id, slot_lookup[(study_uid, slot_id)])\n            for slot_id in range(6)\n            if (study_uid, slot_id) in slot_lookup\n        ]\n\n        def load_job(job):\n            slot_id, series_uid = job\n            return slot_id, _rt_load_series_25d(study_uid, series_uid)\n\n        if jobs:\n            with ThreadPoolExecutor(max_workers=4) as executor:\n                loaded = list(executor.map(load_job, jobs))\n            encoded = encode(np.concatenate([views for _, views in loaded], axis=0))\n            cursor = 0\n            for slot_id, views in loaded:\n                count = len(views)\n                features[row_index, slot_id] = encoded[cursor : cursor + count].astype(np.float16)\n                slot_mask[row_index, slot_id] = True\n                cursor += count\n        if row_index == 0 or (row_index + 1) % 100 == 0 or row_index + 1 == len(expected_uids):\n            log(f\"report-teacher features {row_index + 1}/{len(expected_uids)}\")\n        if dev.type == \"cuda\" and (row_index + 1) % 100 == 0:\n            torch.cuda.empty_cache()\n    del dino\n    gc.collect()\n    if dev.type == \"cuda\":\n        torch.cuda.empty_cache()\n    return features, slot_mask\n\n\n@torch.inference_mode()\ndef _rt_predict_checkpoints(features, slot_mask, checkpoint_dir, dev):\n    syn_index = TARGETS.index(\"Synovitis\")\n    seed_predictions = []\n    for seed in RT_SEEDS:\n        seed_prediction = np.zeros(len(features), np.float32)\n        for fold in range(4):\n            if time.time() - T0 > RT_DEADLINE_S:\n                raise TimeoutError(\"report-teacher deadline reached during checkpoint ensemble\")\n            path = checkpoint_dir / f\"rta_final_seed{seed}_fold{fold}.pth\"\n            checkpoint = torch.load(path, map_location=\"cpu\", weights_only=False)\n            if checkpoint.get(\"targets\") != TARGETS:\n                raise AssertionError(f\"target order mismatch in {path.name}\")\n            cfg = checkpoint[\"cfg\"]\n            model = RTAHMIL(\n                in_dim=int(checkpoint[\"slice_feat_dim\"]),\n                hidden_dim=int(cfg[\"hidden_dim\"]),\n                n_targets=len(TARGETS),\n                n_slots=int(cfg[\"n_slots\"]),\n                n_slices=int(cfg[\"slices_per_series\"]),\n                dropout=float(cfg[\"dropout\"]),\n                series_dropout=float(cfg[\"series_dropout\"]),\n            )\n            model.load_state_dict(checkpoint[\"state_dict\"], strict=True)\n            model.eval().to(dev)\n            fold_prediction = []\n            for start in range(0, len(features), 48):\n                x = torch.from_numpy(np.asarray(features[start : start + 48])).float().to(dev)\n                mask = torch.from_numpy(np.asarray(slot_mask[start : start + 48])).bool().to(dev)\n                with torch.autocast(\n                    device_type=\"cuda\", dtype=torch.float16, enabled=dev.type == \"cuda\"\n                ):\n                    logits = model(x, mask)\n                fold_prediction.append(torch.sigmoid(logits[:, syn_index]).float().cpu().numpy())\n            seed_prediction += np.concatenate(fold_prediction) / 4.0\n            del model, checkpoint\n            gc.collect()\n            if dev.type == \"cuda\":\n                torch.cuda.empty_cache()\n        seed_predictions.append(seed_prediction)\n    return np.mean(np.stack(seed_predictions, axis=0), axis=0)\n\n\ndef _rt_blend_synovitis(primary, teacher_synovitis, teacher_uids):\n    result = primary.copy()\n    primary_uids = result[\"StudyInstanceUID\"].astype(str)\n    teacher = pd.Series(\n        np.asarray(teacher_synovitis, dtype=np.float64),\n        index=pd.Index([str(uid) for uid in teacher_uids], name=\"StudyInstanceUID\"),\n    )\n    if teacher.index.has_duplicates or set(primary_uids) != set(teacher.index):\n        raise AssertionError(\"report-teacher and primary StudyInstanceUID sets differ\")\n    teacher = teacher.reindex(primary_uids.values)\n    if not np.isfinite(teacher.values).all():\n        raise AssertionError(\"non-finite report-teacher prediction\")\n    base_rank = result[\"Synovitis\"].rank(pct=True).to_numpy(np.float64)\n    teacher_rank = teacher.rank(pct=True).to_numpy(np.float64)\n    result[\"Synovitis\"] = (1.0 - RT_SYN_WEIGHT) * base_rank + RT_SYN_WEIGHT * teacher_rank\n    return result\n\n\ndef run_report_teacher_synovitis_specialist():\n    \"\"\"Overwrite only Synovitis, and only after the entire specialist has succeeded.\"\"\"\n    if time.time() - T0 > RT_START_CUTOFF_S:\n        log(\"report-teacher skipped: the primary ensemble used its runtime reserve\")\n        return False\n    checkpoint_dir = _rt_find_checkpoint_dir()\n    primary_path = Path(\"submission.csv\")\n    primary = pd.read_csv(primary_path, dtype={\"StudyInstanceUID\": str})\n    if primary.columns.tolist() != [\"StudyInstanceUID\"] + TARGETS:\n        raise AssertionError(\"primary submission schema mismatch\")\n    test_df = pd.read_csv(ROOT / \"test.csv\", dtype={\"StudyInstanceUID\": str})\n    series_df = pd.read_csv(\n        ROOT / \"test_series.csv\",\n        dtype={\"StudyInstanceUID\": str, \"SeriesInstanceUID\": str},\n    )\n    expected_uids = test_df[\"StudyInstanceUID\"].astype(str).tolist()\n    dev = DEVS[0]\n    features, slot_mask = _rt_extract_features(test_df, series_df, checkpoint_dir, dev)\n    teacher_synovitis = _rt_predict_checkpoints(features, slot_mask, checkpoint_dir, dev)\n    result = _rt_blend_synovitis(primary, teacher_synovitis, expected_uids)\n    untouched = [target for target in TARGETS if target != \"Synovitis\"]\n    if not result[untouched].equals(primary[untouched]):\n        raise AssertionError(\"report-teacher changed a non-Synovitis target\")\n    if result.shape != primary.shape or not np.isfinite(result[TARGETS].to_numpy()).all():\n        raise AssertionError(\"invalid report-teacher blend\")\n    temp_path = Path(\"submission_v26_synovitis.tmp.csv\")\n    result.to_csv(temp_path, index=False)\n    reread = pd.read_csv(temp_path)\n    if reread.shape != primary.shape or not np.isfinite(reread[TARGETS].to_numpy()).all():\n        raise AssertionError(\"serialized report-teacher blend is invalid\")\n    temp_path.replace(primary_path)\n    log(\n        \"report-teacher complete: 0.75 Synovitis rank blend; \"\n        \"all other targets preserved\"\n    )\n    return True\n"},{"cell_type":"markdown","id":"v34-train-only-pca-specialists-evidence","metadata":{},"source":"### V34: train-only PCA exact-label specialists\n\nV34 preserves V33 and adds a separately validated, deploy-exact PCA arm. Scaling and PCA are fit on the 4,407 competition training studies only; four randomized PCA fits are averaged. Lateral Meniscus improved in every PCA-seed held block and the selected three-way blend improved V33 by 0.0273 AUC, with all leave-one-subject-out deltas positive. Synovitis improved by 0.0185 in the held block identically across all four PCA seeds. Fracture and Effusion were deliberately left unchanged because their evidence was less stable. These tests reuse the same 58 image-adjudicated subjects, so they measure estimator stability rather than independent clinical generalization.\n"},{"cell_type":"code","execution_count":null,"id":"v34-train-only-pca-specialists-runtime","metadata":{},"outputs":[],"source":"\"\"\"Exact DINO-small hybrid specialists for Lateral Meniscus and Lateral OA.\n\nThis cell consumes the public hybrid notebook's immutable train feature cache and\nPCA as a Kaggle input. Hidden-test image features are extracted live. The primary\nsubmission is replaced atomically only after both independently supervised hybrid\narms finish and every preservation assertion passes.\n\"\"\"\n\nimport base64\nimport gc\nimport hashlib\nimport io\nimport json\nimport math\nimport os\nimport random\nimport time\nimport zlib\nfrom concurrent.futures import ThreadPoolExecutor\nfrom functools import lru_cache\nfrom pathlib import Path\n\nimport cv2\nimport joblib\nimport numpy as np\nimport pandas as pd\nimport pydicom\nfrom scipy.stats import rankdata\nfrom sklearn.ensemble import ExtraTreesClassifier, HistGradientBoostingClassifier\nfrom sklearn.decomposition import PCA\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.pipeline import make_pipeline\nfrom sklearn.preprocessing import StandardScaler\nimport torch\nimport torch.nn as nn\nimport torch.nn.functional as F\nfrom transformers import AutoModel\n\n\nHYB_PREFIX = \"v8_hybrid_dino224_6slot_5pos_radiomics\"\nHYB_EXPECTED_TRAIN_ID_SHA256 = \"21c1944bd15c3397290f0816de614ad4153f62e84c4bfb0e4d6147ac72084af8\"\nHYB_TEACHER_PAYLOAD = \"eNrtXX3MnWdZf2dlNCNTiuNDx0fZnC2kA3RTSEvPmdoxoHN2M1W2lJQy2m2wvqvrRs8YTCBiKhuCkJB0ETMDhmUYdBETwkjWTBfBDEOEZAQzAkaTRUnbYEZcMNHze8/5nff3XO91P8/99cL+uN/l2jnnee7nfu6P6+N3fTxP9+ze8NMXL83+Xr70wy+9+Zr/m/89b+m8pZsP77/+luWjB5eP3n50/x2vWj5yx1lLz13ad/msPT8/cdXe39pz7VlL7166c8s7Dh69/tYt2zdvef2hS7ds27zl0C233nbrgeX9t9z6joM4/oYDNx89OD1+9MYDRw5Of2+99NLXvHbbK7Ztft/m3L9zPvfAK0egcx6/8PVK59727cX3zxzZOZ5/3/mZI+8baztc+3cfuGqsv0FsN+1nh/S5A/Scxy8cm3uttHnj996zcvzaH9y9Zjxsh75v2vCS8YkTP7/S9mOP3D7G7127do3/+Pjx14Mwdny+5pPvXTmGcfMcCH3g823b7hjrcUucC77fc9HDIx7ndz3P37x3H33gqvNX7os56DjYF8atY9N7gDBf/f2i3XeN2WeozdD8QFh/fsd64hzGwXUEffmhHYP9aj9Yjy8/dOcY+8RjmLf+/s4Ta8dq54M+ME899vinjyx+h/bS9oO5oC9vbXGcazI5dmwMvsZx8PJ8LjvY/vNXvHys1+Kc9mc/wdMP/vsLx7o2obUPEdaNa4D7YVz3XPQH098bV/j7a49dOeac2Z/ygeUjPUZeRB/YD91z0L6P/sZCxrSf48ffOP7m2e8ZKz+dOHFt555c19C9Y/jU479rvvjSTr+qQ7gfmD9+8zzW5sVPXjrmd0/PWJ2De1qdZ9uxf69fttfruI/kRRzDHnAPOe+h8aEN+rLH0S+/Y490f0DYXzs+rie+Y0/ZhscgE/gEz+m8Vefhu5W57Yc3j/v0jhL7uv6rr+6sj0e2X+07pA/QP85bOaQOwv26+mgmW1b3oB3WyM5D23Hsuh579z5/fMklRwdlHW3+44W7x2rHMC4Q9oHyOrvPQ6McudL5rdXf3eN2/p5uBb1i38+u8A3HgDYYr+i4kd6D7fCb+0n88In921b4jPbekwfw9vVf/esR23GPqQso/zyPT4xPZVRlBQTeufCCt3eOoT10OIh2APPCPTw9orhHz8NmUR9qW+oA8BTbaX+4j/ZpdR3ugT7xSZxl26jMcx3wSbsEUnyH4zoHjPGJJ9662CPwBIi8KPZ4ZGWA/MN+0ni0a5/1GNZLeVPlgrYopEO0r2dvP9wZ33SeI/An9QT4Wq8Fj1j+tzZ/ev0CC6u8qWzY63EM8wnZS9UF9ryVUYx906arp/L32uCa79r1DyPIHdpR1+A3ZYlr9/Bbbh336RhPp1PO/LWZjVf3hvwKvqN8sB/Kpdojz+6R6B+oXcax71/9xRHPqTwor5PuevAXxypDaP/Fl35oBKIN0XnrvlHncZ+hy3huNueZfuX82R/a8rzyDTFWP17cuBPXq/4FYW66F322gG24NqpLPbwTwihoD56x/qTuLXUOjoEshrN+qO4rCPsAuaTcKJ7h3lgchXmde9t1nWN2DjgP0t+8L+6jtohtQXP7NVKc5Y2bfes8ydt2THbPdV+p69SGAj/y+PxzBF/LyuaFF1zg6hjVn1av3H/ubeM+3yR0jGvx5Bd+fQ0Pgse9az3dR56nX6a+DTCp7RvtT59aXtFl+H7DoUOdOcOWx/rKSr99/o1jYjSNefC7hweInZUfLG972N47hv6/f/Vk7J3DuBhfsfidGIGY5+KnL3fXnnhIdSeO0Q8KyTwwhmJyjof3JiZBX1wjPW7ldOg7rqPORT/gJWIf9XHoK/P4PE6zQ3EXf1O++ZvX4BM6auYP3tk5z2OQD3stiNhNx0EfHd/pLyvu9rABrrV6H3qDc9r29OUjXsf4ivoGGvvAdbPYgW/PcZ+lpfePZuNZ9QXQP66d6YSuj0Dsgu+77/rllf2wshDyudXGWt7T48SnGC/wpupQ5Z/QfbQNdYbXzvKvHlfetfdj/zcc+qWx2ri+tn2+g2d7bQw2RIofXvahd3V4x/p41EUhrGZtkSW9l/bDuc78+o07OTdiJN1vxrXgY9Aean8ejp/ppTvHqb4DcO/evXs71wEDa0wxhrzYK/r43APvHFts9dhjj408DIaxdX2Ernzr3DVWzPXlvuKT2CRkk/v8IOwHbKL22eenefzLfdu0aVPHx7VyhHOM85O/tT2PcT60abZP5CaUz+yYqLsxt79/6ka3j5Bfw/Wjz8Uxe/gytN7gZU9nc780zsrr4dvpPg7FhtHe2lKNK6jNUyzKvdJ4QUgH8Rrawz7Mwmu5VsQTVm9xjnp/b36QU8Z9mZfQ/WJMIkRcb5t3GYovrvpi4ZxESD50zyjb6iuqf6jXkB+6mHSmpy2/AW949+e96dMzVhOSYb0X2+n4MaahPIIXk7X87cUOGZvmenCv7Dh07W2+TM+H4r4cB9akz9fz+Xl2PpSXtD4AY+m2zWen+yVxyg628PSWvf6+b7xhTAxvY0I8NvVNRjH+mfYR2qO+nBH254KpD3nThreO+/wn7/rD9/3cmLrpqa0vtnplp/GR1mBp8Bi/sw32dYa/r+34AMTqxNqKxTUuyj70XpBb+MmUYeoq1VlT/3Kh71axznULnKc6Wf09tLGxaW/PEa+yMWKLg0MYkvEuxkrYFraZ56mbsZ+nT72qI1dWZ1xyyd868YLV+GpIL1i96ekiGwvkdZbXvZytd5/5ni5428YK6Td6a8h1ZtxEMU1frLFnL3pxOmJ7jPvTLnM+6Jt8hT3UGBrmYG2qnadiAOUL9blhX+31nm6i/2v9EYsJRD4W/WiOhbjI84e0H82Ngjz8pvEVXk+/Vvv7x4/vHzNvqXUmmj/lGPlJ38TyL/mSuVr0BwxGP0/b4p6SD9gZ0o0cq83l4beHFULYALzj5e8sbrH40Y5JMe/s+6w/5iA1Tzmzq3868saltsXLWWi+27PXwAfcT7s2mKvKlsaaLI9yz5gvoj21cYA+P4V7aeN8+NT8gfKnyrWNUfN+j378ZWMbW7T7hPVA7tjabZvH5riYrxzK13r+K/vlHmCdiR8wT6tzGXtjG6036PJhmP/Jt4rf8J15AXzO7NVDI7/vrswN5bk9W+XhyiEfmnoHY7d5+T5ZD8VPVmtoZv1ojIL71JPH2akxCNpZ2G3NV1p/oC8v5OFpi7spI335JF1bi+V5Dp/w81b7W9U5WqNBfchx0M8Zrp+6tiM/qstsDo++XV9ew9OhXl7UttXaN8bCKVOH7/udNT6mrq3nB82OhfX9ECbXPi3mCuGuFF/B4yG1Bdb+WB+wT4ZwLbAEeUT5Tfmd/MN6FK43Mb93H7UHqmdpXyy21FjH/XNfj33bfAPjR16uxYs3q49x/rx+DbpQ56z+js7J+qgWV6nd7MOR3m+uM+0ZeNur+wV/2ryNnacXxwDvq3+r+CQm9jKc8/b9aNuXtU2+jVnNKdo1DvkauubE9+pTYEyaO6ZOwvc7z1t2eYc1ABpLtXlJ5iNDNRSWd/AdvqTudQwRS9g14JiI3+in6hg1f8k+9DxzKxaPcb9sPsPiT1uLotiLet6r4evq5x1jrQ160e5RRxZjdG6Aj0ceb9GnCsXzQE8/es1CJtV34LVar6mxa1u3TVyCe0lt3mC+PmSXpf+R5S29t+VBW7emcsC1lFr/xbXATqobrazYcSiP2pyzzTFQzlWvYs3AY8T8xMVWxliXpmNFXFvbwCdwamEW+WPaDdYjahyAMU7Vmc/efvHCJ2C8y/Ox6evAr1PfpS/u4ek1XMu8BH/D1wvli2JiqrZ/jZ2l5Oy4l5RvK39eTsDiRtt2KOfG+6nsPrV131jr663u4XrxetSycN/Zl/KoXq9rqHts15Nyxz6AP7En4Cf1Sbx9ok3HOa1vWavPPN+re4zyFMpnzOrfwhi3rzZJfViLLRU7496sJwv5PrYOdFaL87pxyDeybYkbQvofe9AXj+/jafUrvJiR6kLWhHrz1H6sn09Zs9gafWi+SXU65oP9ffzTrxnwj7uxLBLqtrx64r5npEAXBGrcQn5IKEYVyr+i7Zl5zBzj0pxwyB/0nlWhDtNYFu2P5kjVDjB3oDVN+mwG7b/mPFafZ5k9S4Xv2GuNEdt8i/ouoHnMjc9BDcZ2+Ok986C+puZGV+tkH1rzfB3l1uIf8iP3Ss97fqNtT73K85ojUh6nHPDZA/tso+pYrpXNHbIt9xi1Qtg3taeg3Xf9fiefv1oTs+pLefZU+crGQT0sBb5j3YfXZ99zUorL0Q/jYTbuqjZD+7N2BXuuzzJgH+eyscPWu+AY9sCrVVHeJS95NXU8r3FQHoNswC/xdcxqXRxjknpP5GH1ecHYWDzGQFnBnnzskV8dD8VwkaNkzLXPN/CeYWIuyfaP4zafxmdp//WyLYs9g18QU7OKMXrtQjk14lius5efAz7SnA5zZZgnMbtXH2vryuyYsP7Q31hTG7NfrZ9drYW1sRG7DjtkjWx+i7/5XJDmJUM4WHWQjcl4z2dLDmvkxXLQBnl4lWX1d/h8FdeZfdGHod5ivb/qGMyH2FBzh5wTcs7esyOe3JAfUQvW91wpeAL3VAxEmRquiVt9Psc+X2ePs/+Zz+9jU4vzwnV1/fFbvc7Lf2v9oH2+cejZPi/mydp2lZvV3KI+U3NtJ94HvaH+LuMGuA58RJ1IfaW5Y+Vz6N53bvjUiPoJvo/nD2JvVO+qv9SHN7jmnA/4gjpDcFzwWQvNj3o8+M2ztwdrHhiveM7j16/Um67y5Gq+xPpxKvu2VgTvI7C6xasR1mdWQnWZWE+vFlNjcNbue3XH+hv2wsN3luhbsCbN1qYN1Z/BT0B7XWvrJ4RitvY+6ut49S3gX1uzS54QHdObfw3xjvpW6PO/7/7LkZeXYRvmCLV/2kNti1qlGFygse2pjd0JCmH8mGcr+3Qi1tDvo+sLhuZv116fK97TeT/L6YNvejffz/J8+36W6bdDz9RXtDzwwCvHIIa7STC3/P6xR55aEU+W+Svh2nfOH4Hib+2PIXp+B23b99GR9sE2UMv4vOr8J0f2Pmy30vcU/n7lsStX2mzGmKa/jx/fOD42NZkgjB2fEE18Ytw8B0If+Dw5bxcizgXfn/zCB0Y8zu96nr8fHugTtPG261auxxx0HOwL49ax6T1AmK/+xrqwz1CbofmBsP78jvXEOYyD6wg6c+rUYL/aD9YDUGaz7AHmrb/PTFVHaI1IfJxDj33+igOL76G9tP0wnOit7UTWZDqmFb4+Nn8Mn+vM9kin6bUcG4/ZT/D08YvGI12b0NqHCOvG+1BmZqWVyyvH7p2qZM6Z/SkfWD7SY+RF9IH90D2fzfdZY8qY9vPcqYuF8Kjy069cckmn/8nAvWP41OO/6Rw7e6A6hOuE+XP/8Im1OXLfN0b87ukZq3NwT6vzbDv27/XL9lYvkud4LfaAe8h5D40PbVTHkiYyHuyR7g+IcFvHxfXEd+wp2/AYZII8p/NWnYfvVuZUd3p6R4l9AXbq+nhk+9W+Q/oA/eO8lUPqoInRR5Qtq3vQDmtk56HtJo4Oun8KHQ5NoeQQz6PNP736BR07hnHNaSGvM119fJQjVzo/q7/tcTt/T7fOrn1qJ8bLMaDNmXlf6PfUXH55D7bDb7Wr4K0r73pwJkdze+/JA3h7++H7Fu24xwtdMJd/nmephcqoygoIvPOGXbs6x9AeOhxEO4B54R6eHlGZ1POwWRMj2wtdNZc7ttP+cB8r56rrcA/0iU/iLNtGZZ7rgE/aJbYl4bjOAWP80kM7FjzG+5AXaY9PiY6eGP5hPyk8au2zHsN6KW+qXNAWTSLsys/cdl1nfM+b8tXp+f6Sr/Xak46eszYfYTzus8qbyoa9nvsXspeqC+x5K6MYO9JU333iieCaox/IHdpR1+A3ZYlrd9eD7+i13Z5Op5x5a8Px6t6QX8F3um7k05NzueY5z+6R6B+oXcaxrVPfg+dUHpTXSf/z6MdHKkNoD18IRBui89Z9OyP6E2sAXcZzE9G7E1k/6hKeV74hxurDi7gG16v+XbEP07npXvTZArbh2nR0qYN3QhgF7cEz1p/UvaXOwTGQxXDWD9V9pV8KuaTcKJ7h3lgcxZJgPWbnwDb6m/fFfdQWsS3lBrpPcZY3bvat8zxp5mn1Bvdc95W6Tm0o8COP4xPjga9lZfOeKWbw9IHqT6tX9px/Y69vEjrGtfj6TRvW+qnztYnxAcjzC79MfBtgUts32n8Hj0hP2+H7Pz92ZWcdYMtjfWWlP/+1PQuMZmMelhdJxM7KD5a3PWzvHUP/35q/nsKew7gYX7H4nRiBmOe6HzzHXfuJ+LyKkegHhWQefKKYnOPhvamvJ3OdaI9bOR36Tvyha67H2FYfOeFesR3Hx9+Ub/7mNfiEjqLN0vM8Bvmw14KI3XQc9NHxnf6y4m4PG1BnWBzCOUEueB3jK+obaOwD10FmQvYc91laWloZz2ljy3Atxm19BO7lCg8+/ajrY4d8brWxlvfsvpNXgDdVhyr/hO6jbagzvHaWf/W48q69H/v/sxMnRmrj+tr2+Q6e7U0h7s3onN/r8I718aiLQljN2iJLel774VzRP+7HuREj6X7T1sDHOCnj4eck4BNMMnwH4F7L88DAGlOMIS/2ij5+d+/eNdjq61Pd4GGwSc/6Mo4xMb8p8xrrwyexScgm9/lB2A/YRO2zz0/z+Jf7dvz4xo6/buUI5xjnJ39rex7jfGjTbJ/E4iGZou7G3P7ov64ee32E/BquH30ujtnDl6H1Bi97Opv7pXFWXg/e1H0cig2jvbWlGldQm6dYlHultimkg3jNIgcTEY/kWtn4ivaptjgkA7NYyyymwLyEnmNMIkRcb5t3GYov0hfry0mE5EP3TPWh5x/qNeQHxaTU05bfgDe8+/Pe9Om5xkEZlnuxnY5/RUcP5BG8mKzlby92yNg014N7Zceha2/zZXo+FPflOLAmfb6ex888H8pLenlPxNJtm6um+8U4JGNkIazvydhlnz13geFtTIjH3jLF9DH+mfYR2qO+nNEKHp5iy385e3uv/+Rdf+OGlyxw4v1XHOjoFc0xh7A0eIzf2Qb7CnkAllbcTqxOrO1hcf4zFOhD74Xz98zxs+o21VnQR2xLG0R+wW/VyervUaY0Nu3t+da53VNfZYhP2J7xLsZK2Ba2mecn4v+gPEf3zOqMe6eY1u6lxldDesHqTU8X2Vggr7O87uVsvftgfR+W3LVn37y8nGIVxk0U0/TFGkN7MeTnI7bHuD/tMufzlTlm1DoMxtC8+Kc3T8Vq3Hf1ub16DU830f+1/ojFBBwbc3k2x0Jc5PlD2o/mRkEeftP4Cq+nX6v9/clF40XeUutMdB0nZqwqxx5fMleL/oDB6OdpW9yT/Gr9ZcvbIJvLw28PK4SwAXjHy99Z3GLxox2TYl58Z3/MQWqeEp/fmecY7Li6tSNrcxaa7/bsNfAB99OuDeaqsqWxJsuj3DPmi2hPbRygz0/hXto4Hz41f6D8qXJtY9S839ZPvndkY4t2n7AeyB1bu23z2BwX85VD+VrPf2W/3AOsM/ED5ml1LufFNlpvoHzYx//kW8Vv+M68AD7RP3nA9m1lbijP7dkqD1cO+dDUO3MdEGzv2TEvfsI9tLF93b9QHgexbo1BUPfDbmu+0voDfXkhD0/bMVFG+vJJurYWy/McPs/M8yfoT3WO1micMTqPfs4QAavYGBD7tDk8+nZ9eQ1Ph3p5UdtWa98mxs/89mVb1uhqXVvPD8KxPn0/hMm1T4u5QrgrxVfweEhtgbU/1gfskyFcCyxBHlF+U34n/7AeZSIxv9B91B6onqV9sdhSYx175r6e1usphmf8yMu1ePFm9TFuntevHZ/n8HSeE3Mvi+PY5pjBY9Y+DcWD+JvrTHu2ee6/2TjUacl/efGmUAwFvK/+reKTmNjLUM475Efbvqxt8myM5hQnztp72FXXnPhefQrNPfN6/v7h1n0u77AGQGOpNi/JfGSohuKYwyPwJXWvY4hYwq4Bx0T8Rj9Vx6j5S/bRXa/bO30rhuNxqytC+gTroNiLen4ywEewV1ob9LZtT3dkMUbnenx8yqmzBm89HNCNSj+4+6KFTJ4xOR5br6mxa1u3rblRxvX+6oEHRkOYKGSX2b+NfXnxUuVBLz6ntpn8bq+1utHKih2H8qjNOdscw6J2QPTqwyv4ZXlEzK85aXsPYBEdK+JInbjP1CewuT+NWdFusB5Rr2WMU3Xmi5/8wsInYLzL87Hp68CvU9+lL+7h6TVcy7wEf987fw2Qly+Kiana/jV2lpKz415Svq38TQZr0JfX5A+Gcm68n8ruf179uk59vdU9XC9eD3w/MTUzyqPWn1BMEqrXWjwnMO8D+POk8ak1BqZEmz7D7ctr6lD6fC97jPIUymdgv/swbl9tkvqwFlsqdp7ZmVk9Wcj3sXWguP5vrvjeKOQb2bbEDSH9z3xEam0s+jDP1ATjsGj7XYnX2HluNs9Ldfz8uaxZbI0+NN+kOh3zwf7u++gjoz7/2MaySKjb8uqJ+56RYrxoyGexmMCLUYXyr2h7ch4zn5j6vJA/6D2rQh22Jj9gcqQdOzDPHWhNkz6bQfuvOQ+1HcTg2GuNEdt8yzFT68TaH7TVeEMovsJP75kH9TU1N8pPu38at7D4h/zIvdLznt9o209EP+qc1Q7R16FdR5zL82m4F1wrmztkW+4xaoWwb2pPQZ/837d39p06+LSpJbT2VPnKxkE9LAW+Y92H12ffc1KKy+f4Z00dlv5mTChk/7Hn9lkG8LrW+ekeYw+8WhXLJ+zXy2PaOCiPQTbgl3h8pHVxjElqX8jDhur4+7AGxkBZwZ5cds7jgzFc5CgZc+3zDbxnmJhLsv3juM2n8Vnamzd8arF/K88KRNSsYoxeu1BOjTiW6+zl5+6d/5P15EPmyihbXk7dxnU8Hx3rD/2NNbUx+9Vn4VZrYT0fWu/9fVkjm9/ibz4X5NXXWBysOqgvRmNrf0/Nx+z5I8jDqyyrv7N4vmq+zuyLPgz1Fuv9bY2uxhrUXwch5+w9O+LJDfkRtWB9z5WCJ3BPxUCUqSHfVZ/Psc/X2eNnFjmup4PxV4vzQnV1Q/Fbvc7Lf6u/bJ9vHHq2z4t5srZd5Ya2RWWGPECeYo5Y9QPvCz4if1Nf2Ty8xhFRQ0z9BKzh+YPYG/tMdEhP2ti2Pt8FvqDOII7TGIzNx2h+1OPBPSYfYvMtuP4Fn79iRe7Zt+ZLrB/XscWmVoQ6eCgu58U27D2Oz/W0rcXUGJy1+0MxQNgLD9+twcRz34I1abY2baj+DH4CscKxgJ8Qitna+6iv49W3EPt5OWvup43hefijT88xJv0L89hNKL/iPU9Ie9jVYxujcIHGtqEHqDs9jD9JyAV566AxqL66htD87drrc8XdV7R868MnjvAVLZtmr2g5ctPN7zp2YPmG/Xc8U1/PshT19/6x/z312NC5vmtsO7bl99QxpNwvZg1S/nTsXn+h+YTu77XPGaM3R9t3TL9D44vpM3WOsTzVt7ahMebw1hDf5vQXu45DYx/iydBah9ZhiPdK5aVEF/XNI3VtcsYYqwNi5D7lfqU6u29fU/VqyZxy7EjJGGNtSUy7kj32+FRls6Y9yrXnMfogJG8pNmJIb8ba0VibFmtP+7BHilzF8H0Or8WseSxf9fFyjJ1M1bkpfJm6lznrnttvKt7N5Y++MQ7114enS/Yt9t6xspdiV2LkNdZGxt6/T3aG5Cp1fDH6MFVP5a51yt6H5htrz0p5I1f/xNqimhgtF9vGyG6qTiqdX4q85/jNQ3xf6nulrEeOX1Jbr8bop5S4Sar+KPGth3RkrFzmYOQYPknFgrE+eow9irVBQzKSGmOKnWdMHKFEpmKxdE48pUT+Y7B1rXhCDu78SfiMMTYjRdfU1Eultiinn9TjsRi0JFabinFqxG1yMEeu3ijd29rxkJg4ROo+lsR6a8Y018unKI3Fp9rZlNhCrO5N5ZUaWCvHpyqNS6fa59TPXB+lRvypVgy1RLfm5qJKYuVDOLAEz+TGcFLxZkqMspY8pmDsVPxUgm/71i8Vd6bGv0t8mxw/scT2pGKQWv58Ck+XrlmOPNbIUebm7kvsbWyMP5Z3hvIntXy9VJxWSx5r+JE5OGgoThIbo4jtr8SHTIlDlMS6avmZKf5rjJ3weL8UT6TEk2rq41LfoobfHIPhc+oHYnNQufPPjZXFxCRr2PKUGFAt3z/FRyrJMdaqR1wv/yg195XD5zE6vURXpeZpa8ZshnInpXnEEj7IjTmW5KJLsWCuz56Le0pjOKny0edPlMYA16MOINZOpGDRHJ7J8f/Xy66WxnNy9FtszUyJva0Zn4rBDTXwWE3cFbs2OfVoOf5vig9VIk8lscv1wCy1nhvJ8UlL6mFr1cIN4ZrUmqz1jKuX8uIQTiyJV6TUIOTYk5oykVpTE7N+qb7Uej2XVIoxc2NUtZ6TysF4Of52Lj74ScZWcuxiSU44VYfUiI3Ueo6qVs469ZrS5xZS4lalz9Pk4P/1kIlYux+rm1P3Mac2pSROUAv3D/FKKR6pXTOcUhNbysu54+3DRrHxp1QfPXcdYzFUrVhpSU1Jrbrv1DHUeMY+Vkemxp5q2ueSGvyQvqtRvx9be58TVyzBrKUx0Bjer1WvnBufzMUHKRg5Ny67HutQs2avlm1LmV8NjLV2DbqvZ/nIB3+0ma9nOc+8nuX6W5YPPXPf0OJpi0aNGjVq1KhRo0aNGjVq1KhRo0aNGjVq1KhRo0aNGjVq1KhRo0aNGjVq1KhRo0aNGjVq1KhRo0aNGjVq1KhRo0aNGjVq1KhRo0aNGjUqo+4bWvbccM/X+IaWTUvnLd1yYPUNLXc8c1/PUvJS5BovC6z1wtdQ333nar44NHRex7FeLxasNceYF0wP7VPMC8qGXqymbe33mBfzhdqF5lTrJU+pLwAsXcPQS1h/HPyeyt+pa17ycuOY/R5aq5Rrcl8qWmMNQvJR8jLZ1Jcaxq5LzZeupvBbzBiGdEPOmGNkI0YvpvBALf1f6yW3pS9+TrXtKbYl5eW5MTaoli3pk7PUFyfH8FGKDomZS5/urP0C6px5p65xyUslY3RjzD7G6Pwaspi7R6nYvvQFrjVfihzSMam4MUVnp9rbFL2Vuz85e1ZLdnP4svZ65/hrNe1syUt+c7Fg7guaY/zzWBlLxaOpPBOy5zH9pPoBJZjtx4HXUvRqzvhLbNDQnqTGpmJxVw5mL9WRpVhuyIfwxpmiu2JtbqndL8ExsfwQ6wfUlMla8YBcfirFmLX0TI6vlcKrqT5YKs7O2duU2FINHypGZ6bozhyMUxLfS4mpxeYESuWmNHaZEp/MWau+OHktn6+WTivF+Km4oFRX1Ip9xY45Zl1T55gy11jZyol1xWLzoXvG7kNsrCzXh4uNA6TKfWrMrCaf5mKCFB1UEyfUil3UtgElcZdUnVVDZ6fmi2JxTw7uyslhpMhZKvaI0SW5uf3cdQthxxSdG+Mn19CRKXa8xI6WrnWfPNbQbaV1HzWwW47vVzOHXeK7x/iwKfJe4gesh1zExoRq4YOcOF5JDChVP5RiotS8eW2+Tr1fbXwRE//M8alq6phUW1+yj7m27ceNp2vEm2vU6uXEFktjBTnYp8Y8c/O5sfn9Ut2ZE5erFauqXV9VgiVKMNCQz53jl8TY7NR65aF1Ts395uYta/ucpTG7XPueWiMX6xfl1sfnxsVy6xhz8FefbsvB1amYIGZdc331WFkLUY4eza35yhlzrO+d029OjUYsbsv1g3PjwjnYoXZ8syTvmCNDqbJTC9elYLmceHlJbeaQTMSudUpMK7a2uzZ2KvF7a9SMlGLWlNxwbkwiR95Tashic1Q19jzFFqTMr6Z+WO+agvXyPfpijrl8HRuDi8kT5dro3NrqnOtT88mlccGaPBkTP8v1n1L8kRx7VPJMY2zt7HrV0MVilpoxzRKbWjuuUCIvqXKS6q/X2POSGrDcnHOuPU2Ze61nVnJr8FNkN1Xv1rBbKfKynjHDGH8gJaaxHs+Pp+Cdms9KlWDz1D2pMc4YnyxVP9esQcjFiDXmHavTcnzknFxI7XcrrP51X8/ykQ/+aDNfz3KeeT3L9bcsH3rmvqGlvWinUaNGjRo1atSoUaNGjRo1atSoUaNGjRo1atSoUaNGjRo1atSoUaNGjRo1atSoUaNGjRo1atSoUaNGjRo1atSoUaNGjRo1atSoUaNG9WnP7rN+6uINq+9o+eGX3nwN3rsCep6+2+Us/P/mw3hRy9GDy0dvP7r/DryvxV5/+uCb3s3rn9+5/r7LzfV85Yvt4lsfPnGEXWzqdPGbd690sXhpjDsCvGWGl5/Xufzfvt69PDSAPTfc8zV/AH/4grM6b61JHMCF+85yX3qzZ/ezzkaLs6f/fWi60I/+BVb7/wGZqKOF\"\nHYB_START_CUTOFF_S = 7.20 * 3600\nHYB_DEADLINE_S = 8.72 * 3600\nHYB_IMG_SIZE = 224\nHYB_N_POSITIONS = 5\nHYB_N_SLOTS = 6\nHYB_SLOT_META_DIM = 8\nHYB_STUDY_META_DIM = 13\nHYB_RAD_DIM = 14\nHYB_RAD_AGG_DIM = 56\nHYB_SEEDS = (20260809, 20260810, 20260811, 20260814, 20260815, 20260816, 20260817, 20260818)\nHYB_LM_FAMILY_WEIGHTS = (0.5227272727272728, 0.27272727272727276, 0.20454545454545456, 0.0)\nHYB_FAMILIES = (\"lr\", \"et\", \"hgb\", \"exact_lr\")\nHYB_FAMILY_WEIGHTS = (0.46, 0.24, 0.18, 0.12)\nHYB_LR_CS = (0.015, 0.05, 0.16, 0.5)\nHYB_EXACT_LR_CS = (0.01, 0.03, 0.10, 0.30)\nHYB_TARGET = \"Lateral Meniscus\"\nHYB_OA_TARGET = \"Lateral OA\"\nHYB_OA_SEEDS = (20260809, 20260810, 20260811, 20260812, 20260813, 20260814, 20260815, 20260816, 20260817, 20260818)\nHYB_OA_FAMILY_WEIGHTS = (0.5227272727272728, 0.27272727272727276, 0.20454545454545456, 0.0)\nHYB_PLANES = (\"Sagittal\", \"Coronal\", \"Axial\")\nHYB_CONTRASTS = (\"Fluid\", \"Structural\")\nHYB_SLOT_NAMES = tuple(\n    f\"{plane}_{contrast}\" for plane in HYB_PLANES for contrast in HYB_CONTRASTS\n)\n\n\ndef _hyb_required_cache_names():\n    names = []\n    for split in (\"train\", \"test\"):\n        stem = f\"{split}_{HYB_PREFIX}_\"\n        names.extend(\n            stem + suffix\n            for suffix in (\n                \"slot_features.npy\",\n                \"slot_mask.npy\",\n                \"slot_meta.npy\",\n                \"radiomics.npy\",\n                \"study_meta.npy\",\n                \"sex.npy\",\n                \"ids.npy\",\n            )\n        )\n    names.append(f\"{HYB_PREFIX}_ipca_192.joblib\")\n    return names\n\n\ndef _hyb_find_cache_dir():\n    local = globals().get(\"HYB_LOCAL_CACHE_DIR\")\n    candidates = []\n    if local:\n        candidates.append(Path(local))\n    root = Path(\"/kaggle/input\")\n    if root.is_dir():\n        marker = f\"train_{HYB_PREFIX}_slot_features.npy\"\n        candidates.extend(hit.parent for hit in root.glob(f\"*/{marker}\"))\n        candidates.extend(hit.parent for hit in root.glob(f\"*/*/{marker}\"))\n    required = _hyb_required_cache_names()\n    for path in candidates:\n        if all((path / name).is_file() for name in required):\n            return path\n    raise FileNotFoundError(\"the complete public hybrid feature/PCA cache is absent\")\n\n\ndef _hyb_find_dino_small():\n    preferred = (\n        Path(\"/kaggle/input/models/metaresearch/dinov2/pytorch/small/1\"),\n        Path(\"/kaggle/input/dinov2/pytorch/small/1\"),\n        Path(\"/kaggle/input/dinov2-small/pytorch/small/1\"),\n    )\n    for path in preferred:\n        if (path / \"config.json\").is_file():\n            return path\n    for top in Path(\"/kaggle/input\").glob(\"*\"):\n        if not top.is_dir() or \"dino\" not in top.name.lower():\n            continue\n        for config_path in top.glob(\"**/config.json\"):\n            try:\n                cfg = json.loads(config_path.read_text())\n                if cfg.get(\"model_type\") == \"dinov2\" and int(cfg.get(\"hidden_size\", -1)) == 384:\n                    return config_path.parent\n            except Exception:\n                continue\n    raise FileNotFoundError(\"offline DINOv2-small model is absent\")\n\n\ndef _hyb_numeric(value, default=0.0):\n    try:\n        value = float(value)\n        return value if np.isfinite(value) else default\n    except Exception:\n        return default\n\n\ndef _hyb_sex_to_id(row):\n    value = str(row.get(\"PatientSex\", \"\")).strip().lower()\n    if value.startswith(\"m\"):\n        return 1\n    if value.startswith(\"f\"):\n        return 2\n    return 0\n\n\ndef _hyb_choose_series(part, contrast, used_ids):\n    if len(part) == 0:\n        return None\n    fluid = part[\"Fluid_Sensitive\"].fillna(0).astype(float)\n    fat = part[\"Fat_Suppression\"].fillna(0).astype(float)\n    if contrast == \"Fluid\":\n        score = 4.0 * fluid + 2.0 * fat\n    else:\n        score = 3.5 * (1.0 - fluid) + 1.5 * (1.0 - fat)\n    ordered = part.assign(_slot_score=score).sort_values(\"_slot_score\", ascending=False)\n    for _, row in ordered.iterrows():\n        series_id = str(row[\"SeriesInstanceUID\"])\n        if series_id not in used_ids:\n            return row\n    return ordered.iloc[0]\n\n\ndef _hyb_build_slots(series_df):\n    slots, study_meta = {}, {}\n    for study_id, rows in series_df.groupby(\"StudyInstanceUID\", sort=False):\n        study_id = str(study_id)\n        selected, meta = {}, []\n        plane_lower = rows[\"Anatomical_Plane\"].astype(str).str.lower()\n        for plane in HYB_PLANES:\n            part = rows[plane_lower == plane.lower()]\n            count = len(part)\n            fluid_mean = part[\"Fluid_Sensitive\"].fillna(0).astype(float).mean() if count else 0.0\n            fat_mean = part[\"Fat_Suppression\"].fillna(0).astype(float).mean() if count else 0.0\n            meta.extend([np.log1p(count) / 3.0, fluid_mean, fat_mean])\n            used = set()\n            for contrast in HYB_CONTRASTS:\n                row = _hyb_choose_series(part, contrast, used)\n                if row is None:\n                    continue\n                sid = str(row[\"SeriesInstanceUID\"])\n                used.add(sid)\n                selected[f\"{plane}_{contrast}\"] = {\n                    \"series_id\": sid,\n                    \"contrast\": contrast,\n                    \"fluid\": _hyb_numeric(row.get(\"Fluid_Sensitive\", 0)),\n                    \"fat\": _hyb_numeric(row.get(\"Fat_Suppression\", 0)),\n                }\n        total = len(rows)\n        meta.extend(\n            [\n                np.log1p(total) / 4.0,\n                rows[\"Fluid_Sensitive\"].fillna(0).astype(float).mean() if total else 0.0,\n                rows[\"Fat_Suppression\"].fillna(0).astype(float).mean() if total else 0.0,\n                rows[\"SeriesInstanceUID\"].nunique() / 12.0 if total else 0.0,\n            ]\n        )\n        slots[study_id] = selected\n        study_meta[study_id] = np.asarray(meta, dtype=np.float32)\n    return slots, study_meta\n\n\ndef _hyb_locate_series_dir(study_uid, series_uid):\n    candidates = (\n        ROOT / \"test_series\" / str(study_uid) / str(series_uid),\n        ROOT / \"test_series\" / str(series_uid),\n        ROOT / \"test\" / str(study_uid) / str(series_uid),\n        ROOT / \"test_images\" / str(study_uid) / str(series_uid),\n        ROOT / \"test_dicom\" / str(study_uid) / str(series_uid),\n        ROOT / \"test_dicoms\" / str(study_uid) / str(series_uid),\n        ROOT / \"images\" / \"test\" / str(study_uid) / str(series_uid),\n    )\n    for path in candidates:\n        if path.is_dir():\n            return path\n    raise FileNotFoundError(f\"hybrid series missing: study={study_uid}, series={series_uid}\")\n\n\ndef _hyb_read_header(path):\n    try:\n        return pydicom.dcmread(str(path), stop_before_pixels=True, force=True)\n    except Exception:\n        return None\n\n\ndef _hyb_header_position(ds):\n    if ds is None:\n        return None\n    try:\n        ipp = np.asarray([float(x) for x in ds.ImagePositionPatient], dtype=np.float64)\n        iop = np.asarray([float(x) for x in ds.ImageOrientationPatient], dtype=np.float64)\n        return float(np.dot(ipp, np.cross(iop[:3], iop[3:])))\n    except Exception:\n        pass\n    for name in (\"SliceLocation\", \"InstanceNumber\"):\n        try:\n            return float(getattr(ds, name))\n        except Exception:\n            continue\n    return None\n\n\n@lru_cache(maxsize=8192)\ndef _hyb_ordered_files(folder_str):\n    files = sorted(Path(folder_str).glob(\"*.dcm\"))\n    if not files:\n        files = sorted(path for path in Path(folder_str).iterdir() if path.is_file())\n    keyed, ok = [], 0\n    for fallback, path in enumerate(files):\n        key = _hyb_header_position(_hyb_read_header(path))\n        if key is None:\n            key = fallback\n        else:\n            ok += 1\n        keyed.append((key, str(path)))\n    if ok >= max(3, len(files) // 3):\n        keyed.sort(key=lambda pair: pair[0])\n    return tuple(path for _, path in keyed)\n\n\ndef _hyb_spacing(ds):\n    spacing_x = spacing_y = thickness = 0.0\n    try:\n        ps = [float(x) for x in ds.PixelSpacing]\n        spacing_y, spacing_x = ps[0], ps[1]\n    except Exception:\n        pass\n    for name in (\"SliceThickness\", \"SpacingBetweenSlices\"):\n        try:\n            thickness = float(getattr(ds, name))\n            break\n        except Exception:\n            continue\n    return spacing_x, spacing_y, thickness\n\n\ndef _hyb_read_pixel(path):\n    ds = pydicom.dcmread(str(path), force=True)\n    array = ds.pixel_array.astype(np.float32)\n    array = array * _hyb_numeric(getattr(ds, \"RescaleSlope\", 1.0), 1.0)\n    array += _hyb_numeric(getattr(ds, \"RescaleIntercept\", 0.0), 0.0)\n    if str(getattr(ds, \"PhotometricInterpretation\", \"\")).upper() == \"MONOCHROME1\":\n        array = array.max() - array\n    return array, ds\n\n\ndef _hyb_robust_uint8(stack):\n    stack = np.asarray(stack, dtype=np.float32)\n    finite = stack[np.isfinite(stack)]\n    if finite.size == 0:\n        return np.zeros(stack.shape, dtype=np.uint8)\n    low, high = np.percentile(finite, [1.0, 99.4])\n    if high <= low:\n        low, high = float(finite.min()), float(finite.max())\n    if high <= low:\n        return np.zeros(stack.shape, dtype=np.uint8)\n    return (255.0 * np.clip((stack - low) / (high - low), 0.0, 1.0)).astype(np.uint8)\n\n\ndef _hyb_crop_foreground(image):\n    gray = image.max(axis=2)\n    mask = gray > max(8, np.percentile(gray, 55) * 0.18)\n    if mask.sum() < 64:\n        return image\n    ys, xs = np.where(mask)\n    y0, y1, x0, x1 = int(ys.min()), int(ys.max()) + 1, int(xs.min()), int(xs.max()) + 1\n    pad_y, pad_x = int(0.08 * (y1 - y0 + 1)), int(0.08 * (x1 - x0 + 1))\n    y0, y1 = max(0, y0 - pad_y), min(image.shape[0], y1 + pad_y)\n    x0, x1 = max(0, x0 - pad_x), min(image.shape[1], x1 + pad_x)\n    if y1 - y0 < 32 or x1 - x0 < 32:\n        return image\n    return image[y0:y1, x0:x1]\n\n\ndef _hyb_resize(image):\n    return cv2.resize(image, (HYB_IMG_SIZE, HYB_IMG_SIZE), interpolation=cv2.INTER_AREA)\n\n\ndef _hyb_view_radiomics(image):\n    gray = image.astype(np.float32).mean(axis=2) / 255.0\n    height, width = gray.shape\n    q = np.percentile(gray, [1, 5, 10, 25, 50, 75, 90, 95, 99])\n    center = gray[height // 4 : 3 * height // 4, width // 4 : 3 * width // 4]\n    gy, gx = np.gradient(gray)\n    grad = np.sqrt(gx * gx + gy * gy)\n    foreground = gray > 0.08\n    return np.asarray(\n        [\n            gray.mean(), gray.std(), q[0], q[2], q[4], q[6], q[8],\n            center.mean(), center.std(), grad.mean(), grad.std(), foreground.mean(),\n            gray[foreground].mean() if foreground.any() else 0.0,\n            gray[foreground].std() if foreground.any() else 0.0,\n        ],\n        dtype=np.float32,\n    )\n\n\ndef _hyb_make_view(paths):\n    arrays, first_ds = [], None\n    for path in paths:\n        try:\n            array, ds = _hyb_read_pixel(path)\n            first_ds = ds if first_ds is None else first_ds\n            arrays.append(array)\n        except Exception:\n            return None, np.zeros(HYB_RAD_DIM, np.float32), (0.0, 0.0, 0.0)\n    image = np.transpose(_hyb_robust_uint8(np.stack(arrays)), (1, 2, 0))\n    image = _hyb_resize(_hyb_crop_foreground(image))\n    return (\n        np.transpose(image, (2, 0, 1)),\n        _hyb_view_radiomics(image),\n        _hyb_spacing(first_ds) if first_ds is not None else (0.0, 0.0, 0.0),\n    )\n\n\ndef _hyb_sampled_triplets(study_uid, series_uid):\n    files = list(_hyb_ordered_files(str(_hyb_locate_series_dir(study_uid, series_uid))))\n    if not files:\n        return [], 0\n    centers = np.round(np.linspace(0.08 * (len(files) - 1), 0.92 * (len(files) - 1), HYB_N_POSITIONS)).astype(int)\n    centers = np.clip(centers, 0, len(files) - 1)\n    return (\n        [[files[max(0, center - 1)], files[center], files[min(len(files) - 1, center + 1)]] for center in centers],\n        len(files),\n    )\n\n\ndef _hyb_load_study(row, slots, study_meta):\n    study_uid = str(row[\"StudyInstanceUID\"])\n    images = np.zeros((6, 5, 3, 224, 224), np.uint8)\n    view_mask = np.zeros((6, 5), bool)\n    slot_meta = np.zeros((6, 8), np.float32)\n    radiomics = np.zeros((6, 56), np.float32)\n    selected = slots.get(study_uid, {})\n    for slot_index, slot_name in enumerate(HYB_SLOT_NAMES):\n        info = selected.get(slot_name)\n        if info is None:\n            continue\n        triplets, n_files = _hyb_sampled_triplets(study_uid, info[\"series_id\"])\n        rad_values, spacings = [], []\n        for pos_index, paths in enumerate(triplets[:5]):\n            image, rad, spacing = _hyb_make_view(paths)\n            if image is None:\n                continue\n            images[slot_index, pos_index] = image\n            view_mask[slot_index, pos_index] = True\n            rad_values.append(rad)\n            spacings.append(spacing)\n        if rad_values:\n            rad_array = np.stack(rad_values).astype(np.float32)\n            radiomics[slot_index] = np.concatenate(\n                [rad_array.mean(0), rad_array.std(0), rad_array.min(0), rad_array.max(0)]\n            )\n            spacing_mean = np.asarray(spacings, np.float32).mean(0)\n        else:\n            spacing_mean = np.zeros(3, np.float32)\n        slot_meta[slot_index] = np.asarray(\n            [\n                info[\"fluid\"], info[\"fat\"], float(info[\"contrast\"] == \"Structural\"),\n                np.log1p(n_files) / 6.0, float(view_mask[slot_index].mean()),\n                spacing_mean[0] / 2.5 if spacing_mean[0] else 0.0,\n                spacing_mean[1] / 2.5 if spacing_mean[1] else 0.0,\n                spacing_mean[2] / 8.0 if spacing_mean[2] else 0.0,\n            ],\n            np.float32,\n        )\n    return (\n        images,\n        view_mask,\n        slot_meta,\n        radiomics,\n        study_meta.get(study_uid, np.zeros(13, np.float32)),\n        _hyb_sex_to_id(row),\n        study_uid,\n    )\n\n\nclass _HybridDinoEncoder(nn.Module):\n    def __init__(self, backbone):\n        super().__init__()\n        self.backbone = backbone\n\n    def forward(self, pixel_values):\n        tokens = self.backbone(pixel_values=pixel_values).last_hidden_state\n        patches = tokens[:, 1:]\n        return torch.cat(\n            [\n                F.normalize(tokens[:, 0], dim=1),\n                F.normalize(patches.mean(dim=1), dim=1),\n                F.normalize(patches.amax(dim=1), dim=1),\n            ],\n            dim=1,\n        )\n\n\ndef _hyb_load_cached_split(cache_dir, split):\n    stem = f\"{split}_{HYB_PREFIX}_\"\n    return tuple(\n        np.load(\n            cache_dir / (stem + suffix),\n            mmap_mode=None if suffix == \"ids.npy\" else \"r\",\n            allow_pickle=suffix == \"ids.npy\",\n        )\n        for suffix in (\n            \"slot_features.npy\", \"slot_mask.npy\", \"slot_meta.npy\", \"radiomics.npy\",\n            \"study_meta.npy\", \"sex.npy\", \"ids.npy\",\n        )\n    )\n\n\ndef _hyb_extract_test_bundle(test_df, series_df, cache_dir, dev):\n    expected_uids = test_df[\"StudyInstanceUID\"].astype(str).to_numpy()\n    cached = _hyb_load_cached_split(cache_dir, \"test\")\n    if np.array_equal(np.asarray(cached[-1]).astype(str), expected_uids):\n        log(\"hybrid: exact attached visible-test features reused\")\n        return cached\n    if time.time() - T0 > HYB_START_CUTOFF_S:\n        raise TimeoutError(\"insufficient runtime reserve for hybrid feature extraction\")\n\n    slots, study_meta = _hyb_build_slots(series_df)\n    backbone = AutoModel.from_pretrained(\n        str(_hyb_find_dino_small()), local_files_only=True, trust_remote_code=False\n    )\n    if int(backbone.config.hidden_size) != 384:\n        raise AssertionError(\"the hybrid specialist requires DINOv2-small hidden size 384\")\n    model = _HybridDinoEncoder(backbone).eval().to(dev)\n    for parameter in model.parameters():\n        parameter.requires_grad_(False)\n    mean = torch.tensor([0.485, 0.456, 0.406], device=dev).view(1, 3, 1, 1)\n    std = torch.tensor([0.229, 0.224, 0.225], device=dev).view(1, 3, 1, 1)\n\n    @torch.inference_mode()\n    def encode(images):\n        outputs = []\n        for start in range(0, len(images), 64):\n            batch = images[start : start + 64].to(dev, non_blocking=True).float().div_(255.0)\n            batch = (batch - mean) / std\n            with torch.autocast(\"cuda\", dtype=torch.float16, enabled=dev.type == \"cuda\"):\n                outputs.append(model(batch).float().cpu())\n        return torch.cat(outputs, dim=0)\n\n    n = len(test_df)\n    features = np.zeros((n, 6, 3456), np.float16)\n    masks = np.zeros((n, 6), bool)\n    slot_meta_array = np.zeros((n, 6, 8), np.float16)\n    radiomics_array = np.zeros((n, 6, 56), np.float16)\n    study_meta_array = np.zeros((n, 13), np.float16)\n    sexes = np.zeros(n, np.int8)\n    ids = expected_uids.astype(object)\n\n    def safe_load(index):\n        try:\n            return _hyb_load_study(test_df.iloc[index], slots, study_meta)\n        except Exception as exc:\n            uid = str(test_df.iloc[index][\"StudyInstanceUID\"])\n            log(f\"hybrid study decode failed safely: {uid}: {exc}\")\n            return (\n                np.zeros((6, 5, 3, 224, 224), np.uint8), np.zeros((6, 5), bool),\n                np.zeros((6, 8), np.float32), np.zeros((6, 56), np.float32),\n                np.zeros(13, np.float32), 0, uid,\n            )\n\n    workers = max(1, min(8, os.cpu_count() or 8))\n    with ThreadPoolExecutor(max_workers=workers) as executor:\n        for start in range(0, n, 6):\n            if time.time() - T0 > HYB_DEADLINE_S:\n                raise TimeoutError(\"hybrid deadline reached during feature extraction\")\n            stop = min(start + 6, n)\n            items = list(executor.map(safe_load, range(start, stop)))\n            image_batch = torch.from_numpy(np.stack([item[0] for item in items]))\n            view_mask = torch.from_numpy(np.stack([item[1] for item in items]))\n            valid = view_mask.reshape(-1)\n            view_features = torch.zeros(len(items) * 30, 1152, dtype=torch.float32)\n            if valid.any():\n                flat_images = image_batch.reshape(-1, 3, 224, 224)\n                view_features[valid] = encode(flat_images[valid])\n            view_features = view_features.reshape(len(items), 6, 5, 1152)\n            aggregate = torch.zeros(len(items), 6, 3456, dtype=torch.float32)\n            slot_mask = view_mask.any(dim=2)\n            for batch_index in range(len(items)):\n                for slot_index in range(6):\n                    present = view_mask[batch_index, slot_index]\n                    if present.any():\n                        values = view_features[batch_index, slot_index, present]\n                        aggregate[batch_index, slot_index] = torch.cat(\n                            [values.mean(0), values.amax(0), values.std(0, unbiased=False)]\n                        )\n            features[start:stop] = aggregate.numpy().astype(np.float16)\n            masks[start:stop] = slot_mask.numpy()\n            slot_meta_array[start:stop] = np.stack([item[2] for item in items]).astype(np.float16)\n            radiomics_array[start:stop] = np.stack([item[3] for item in items]).astype(np.float16)\n            study_meta_array[start:stop] = np.stack([item[4] for item in items]).astype(np.float16)\n            sexes[start:stop] = np.asarray([item[5] for item in items], np.int8)\n            if start == 0 or stop % 100 == 0 or stop == n:\n                log(f\"hybrid features {stop}/{n}\")\n    del model, backbone\n    gc.collect()\n    if dev.type == \"cuda\":\n        torch.cuda.empty_cache()\n    return features, masks, slot_meta_array, radiomics_array, study_meta_array, sexes, ids\n\n\ndef _hyb_align_bundle(bundle, expected_uids):\n    ids = np.asarray(bundle[-1]).astype(str)\n    expected_uids = np.asarray(expected_uids).astype(str)\n    if np.array_equal(ids, expected_uids):\n        return bundle\n    if len(set(ids)) != len(ids) or set(ids) != set(expected_uids):\n        raise AssertionError(\"hybrid cache StudyInstanceUID set mismatch\")\n    positions = {uid: index for index, uid in enumerate(ids)}\n    order = np.asarray([positions[uid] for uid in expected_uids], dtype=int)\n    return tuple(np.asarray(array)[order] for array in bundle[:-1]) + (expected_uids,)\n\n\ndef _hyb_transform(bundle, pca):\n    features, masks, slot_meta, radiomics, study_meta, sex, _ = bundle\n    n = len(features)\n    result = np.zeros((n, 6, 192), np.float32)\n    for start in range(0, n, 96):\n        stop = min(start + 96, n)\n        block = np.asarray(features[start:stop], np.float32)\n        block_mask = np.asarray(masks[start:stop]).reshape(-1).astype(bool)\n        flat = block.reshape(-1, block.shape[-1])\n        transformed = np.zeros((len(flat), 192), np.float32)\n        if block_mask.any():\n            transformed[block_mask] = pca.transform(flat[block_mask]).astype(np.float32)\n        result[start:stop] = transformed.reshape(stop - start, 6, 192)\n    sex_onehot = np.eye(3, dtype=np.float32)[np.asarray(sex, dtype=int).clip(0, 2)]\n    matrix = np.concatenate(\n        [\n            result.reshape(n, -1), np.asarray(masks, np.float32),\n            np.asarray(slot_meta, np.float32).reshape(n, -1),\n            np.asarray(radiomics, np.float32).reshape(n, -1),\n            np.asarray(study_meta, np.float32), sex_onehot,\n        ],\n        axis=1,\n    )\n    matrix = np.nan_to_num(matrix, nan=0.0, posinf=0.0, neginf=0.0)\n    if matrix.shape != (n, 1558):\n        raise AssertionError(f\"unexpected hybrid matrix shape {matrix.shape}\")\n    return matrix.astype(np.float32)\n\n\ndef _hyb_rank(values, denominator_offset=0.0):\n    values = np.asarray(values, np.float64)\n    if len(values) <= 1 or np.ptp(values) < 1e-12:\n        return np.full(len(values), 0.5, np.float64)\n    return rankdata(values, method=\"average\") / (len(values) + denominator_offset)\n\n\ndef _hyb_payload():\n    raw = zlib.decompress(base64.b64decode(HYB_TEACHER_PAYLOAD.encode(\"ascii\")))\n    with np.load(io.BytesIO(raw), allow_pickle=False) as payload:\n        return {name: payload[name].astype(np.float32) for name in payload.files}\n\n\ndef _hyb_select_top(indices, scores, labels, class_value, cap=2200):\n    keep = indices[labels == class_value]\n    if len(keep) <= cap:\n        return keep\n    keep_scores = scores[labels == class_value]\n    return keep[np.argsort(-keep_scores)[:cap]]\n\n\ndef _hyb_training_arrays(pseudo_y, pseudo_conf, exact_mask, exact_y):\n    exact_idx = np.flatnonzero(exact_mask)\n    pseudo_pool = ~exact_mask\n    confidence = np.clip(pseudo_conf, 0.0, 1.0)\n    candidates = np.flatnonzero(pseudo_pool & (confidence >= 0.20))\n    candidate_labels = (pseudo_y[candidates] >= 0.5).astype(int)\n    pos = _hyb_select_top(candidates, confidence[candidates], candidate_labels, 1)\n    neg = _hyb_select_top(candidates, confidence[candidates], candidate_labels, 0)\n    pseudo_idx = np.concatenate([pos, neg]).astype(int)\n    pseudo_labels = (pseudo_y[pseudo_idx] >= 0.5).astype(int)\n    pseudo_weight = 0.18 + 1.15 * np.power(np.clip(confidence[pseudo_idx], 0, 1), 1.4)\n    fit_idx = np.concatenate([exact_idx, pseudo_idx]).astype(int)\n    fit_y = np.concatenate([exact_y[exact_idx].astype(int), pseudo_labels]).astype(int)\n    fit_weight = np.concatenate(\n        [np.full(len(exact_idx), 7.0, np.float32), pseudo_weight.astype(np.float32)]\n    )\n    return fit_idx, fit_y, fit_weight, exact_idx, exact_y[exact_idx].astype(int)\n\n\ndef _hyb_fit_lr(x_fit, y_fit, x_test, weights, cs, seed):\n    predictions = []\n    for index, c_value in enumerate(cs):\n        model = LogisticRegression(\n            C=c_value, solver=\"liblinear\", class_weight=\"balanced\", max_iter=3000,\n            random_state=seed + 31 * index,\n        )\n        model.fit(x_fit, y_fit, sample_weight=weights)\n        predictions.append(model.predict_proba(x_test)[:, 1])\n    return np.mean(predictions, axis=0).astype(np.float32)\n\n\ndef _hyb_fit_family(family, x_train, fit_idx, fit_y, fit_weight, x_test, seed):\n    if time.time() - T0 > HYB_DEADLINE_S:\n        raise TimeoutError(\"hybrid deadline reached during model fitting\")\n    if family == \"lr\":\n        return _hyb_fit_lr(x_train[fit_idx], fit_y, x_test, fit_weight, HYB_LR_CS, seed)\n    if family == \"exact_lr\":\n        return _hyb_fit_lr(x_train[fit_idx], fit_y, x_test, fit_weight, HYB_EXACT_LR_CS, seed)\n    if family == \"et\":\n        model = ExtraTreesClassifier(\n            n_estimators=420, max_features=\"sqrt\", min_samples_leaf=4,\n            min_samples_split=8, bootstrap=False, class_weight=\"balanced\",\n            random_state=seed, n_jobs=-1,\n        )\n    elif family == \"hgb\":\n        model = HistGradientBoostingClassifier(\n            learning_rate=0.035, max_iter=180, max_leaf_nodes=15,\n            min_samples_leaf=18, l2_regularization=0.25, early_stopping=True,\n            validation_fraction=0.15, random_state=seed,\n        )\n    else:\n        raise ValueError(f\"unknown hybrid family {family}\")\n    model.fit(x_train[fit_idx], fit_y, sample_weight=fit_weight)\n    return model.predict_proba(x_test)[:, 1].astype(np.float32)\n\n\ndef _hyb_teacher_arm(x_train, x_test, pseudo_y, pseudo_conf, exact_mask, exact_y, exact_lr):\n    fit_idx, fit_y, fit_weight, _, _ = _hyb_training_arrays(\n        pseudo_y, pseudo_conf, exact_mask, exact_y\n    )\n    predictions = {}\n    for family in (\"lr\", \"et\", \"hgb\"):\n        seed_predictions = []\n        for seed in HYB_SEEDS:\n            model_seed = seed + 101 * TARGETS.index(HYB_TARGET) + len(family)\n            seed_predictions.append(\n                _hyb_fit_family(\n                    family, x_train, fit_idx, fit_y, fit_weight, x_test, model_seed\n                )\n            )\n        predictions[family] = np.mean(np.stack(seed_predictions), axis=0)\n    predictions[\"exact_lr\"] = exact_lr\n    weighted_rank = np.zeros(len(x_test), np.float64)\n    weighted_prob = np.zeros(len(x_test), np.float64)\n    for weight, family in zip(HYB_FAMILY_WEIGHTS, HYB_FAMILIES):\n        pred = np.clip(predictions[family], 1e-5, 1.0 - 1e-5)\n        weighted_rank += weight * _hyb_rank(pred, denominator_offset=1.0)\n        weighted_prob += weight * pred\n    return 0.90 * weighted_rank + 0.10 * weighted_prob\n\n\ndef run_hybrid_lm_and_lateral_oa_specialists():\n    \"\"\"Atomically replace only the two independently validated hybrid targets.\"\"\"\n    if time.time() - T0 > HYB_START_CUTOFF_S:\n        log(\"hybrid skipped: insufficient runtime reserve\")\n        return False\n    cache_dir = _hyb_find_cache_dir()\n    primary_path = Path(\"submission.csv\")\n    primary = pd.read_csv(primary_path, dtype={\"StudyInstanceUID\": str})\n    train_df = pd.read_csv(ROOT / \"train.csv\", dtype={\"StudyInstanceUID\": str})\n    test_df = pd.read_csv(ROOT / \"test.csv\", dtype={\"StudyInstanceUID\": str})\n    series_df = pd.read_csv(\n        ROOT / \"test_series.csv\",\n        dtype={\"StudyInstanceUID\": str, \"SeriesInstanceUID\": str},\n    )\n    if primary.columns.tolist() != [\"StudyInstanceUID\"] + TARGETS:\n        raise AssertionError(\"primary submission schema mismatch\")\n    train_uids = train_df[\"StudyInstanceUID\"].astype(str).to_numpy()\n    uid_hash = hashlib.sha256(\"\\n\".join(train_uids).encode()).hexdigest()\n    if uid_hash != HYB_EXPECTED_TRAIN_ID_SHA256:\n        raise AssertionError(\"competition train StudyInstanceUID order drifted\")\n    test_uids = test_df[\"StudyInstanceUID\"].astype(str).to_numpy()\n    dev = DEVS[0]\n    train_bundle = _hyb_align_bundle(_hyb_load_cached_split(cache_dir, \"train\"), train_uids)\n    test_bundle = _hyb_align_bundle(\n        _hyb_extract_test_bundle(test_df, series_df, cache_dir, dev), test_uids\n    )\n    pca = joblib.load(cache_dir / f\"{HYB_PREFIX}_ipca_192.joblib\")\n    x_train_raw = _hyb_transform(train_bundle, pca)\n    x_test_raw = _hyb_transform(test_bundle, pca)\n    joint = np.concatenate([x_train_raw, x_test_raw], axis=0)\n    mean = joint.mean(axis=0, dtype=np.float64).astype(np.float32)\n    scale = joint.std(axis=0, dtype=np.float64).astype(np.float32)\n    scale[scale < 1e-5] = 1.0\n    x_train = ((x_train_raw - mean) / scale).astype(np.float32)\n    x_test = ((x_test_raw - mean) / scale).astype(np.float32)\n    payload = _hyb_payload()\n    if any(len(payload[name]) != len(train_df) for name in payload):\n        raise AssertionError(\"embedded hybrid teacher length mismatch\")\n\n    def fit_target(target_name, pseudo_y, pseudo_conf, seeds):\n        exact_mask = train_df[target_name].notna().to_numpy()\n        exact_y = np.nan_to_num(train_df[target_name].to_numpy(np.float32), nan=0.0)\n        exact_idx = np.flatnonzero(exact_mask)\n        exact_labels = exact_y[exact_idx].astype(int)\n        exact_predictions = []\n        for seed in seeds:\n            model_seed = seed + 101 * TARGETS.index(target_name) + len(\"exact_lr\")\n            exact_predictions.append(\n                _hyb_fit_family(\n                    \"exact_lr\", x_train, exact_idx, exact_labels,\n                    np.full(len(exact_idx), 3.0, np.float32), x_test, model_seed,\n                )\n            )\n        exact_lr = np.mean(np.stack(exact_predictions), axis=0)\n        fit_idx, fit_y, fit_weight, _, _ = _hyb_training_arrays(\n            pseudo_y, pseudo_conf, exact_mask, exact_y\n        )\n        predictions = {}\n        for family in (\"lr\", \"et\", \"hgb\"):\n            seed_predictions = []\n            for seed in seeds:\n                model_seed = seed + 101 * TARGETS.index(target_name) + len(family)\n                seed_predictions.append(\n                    _hyb_fit_family(\n                        family, x_train, fit_idx, fit_y, fit_weight, x_test, model_seed\n                    )\n                )\n            predictions[family] = np.mean(np.stack(seed_predictions), axis=0)\n        predictions[\"exact_lr\"] = exact_lr\n        weighted_rank = np.zeros(len(x_test), np.float64)\n        weighted_prob = np.zeros(len(x_test), np.float64)\n        family_weights = (\n            HYB_LM_FAMILY_WEIGHTS if target_name == HYB_TARGET else HYB_OA_FAMILY_WEIGHTS\n        )\n        for weight, family in zip(family_weights, HYB_FAMILIES):\n            pred = np.clip(predictions[family], 1e-5, 1.0 - 1e-5)\n            weighted_rank += weight * _hyb_rank(pred, denominator_offset=1.0)\n            weighted_prob += weight * pred\n        return 0.90 * weighted_rank + 0.10 * weighted_prob\n\n    lm_consensus = fit_target(\n        HYB_TARGET, payload[\"lm_consensus_y\"], payload[\"lm_consensus_conf\"], HYB_SEEDS\n    )\n    lm_pilkwang = fit_target(\n        HYB_TARGET, payload[\"lm_pilkwang_y\"], payload[\"lm_pilkwang_conf\"], HYB_SEEDS\n    )\n    lm_teacher_rank = 1.0 * _hyb_rank(lm_consensus) + 0.0 * _hyb_rank(lm_pilkwang)\n    oa_pilkwang = fit_target(\n        HYB_OA_TARGET,\n        payload[\"oa_pilkwang_y\"],\n        payload[\"oa_pilkwang_conf\"],\n        HYB_OA_SEEDS,\n    )\n    oa_teacher_rank = _hyb_rank(oa_pilkwang)\n\n    # The PCA route is fitted on the competition training matrix only.  Unlike\n    # the pseudo-label families above, hidden-test composition cannot change\n    # its scaling.  Four randomized decompositions reduce transform variance.\n    train_mean = x_train_raw.mean(axis=0, dtype=np.float64).astype(np.float32)\n    train_scale = x_train_raw.std(axis=0, dtype=np.float64).astype(np.float32)\n    train_scale[train_scale < 1e-5] = 1.0\n    x_train_pca = ((x_train_raw - train_mean) / train_scale).astype(np.float32)\n    x_test_pca = ((x_test_raw - train_mean) / train_scale).astype(np.float32)\n    pca_specs = {\n        HYB_TARGET: (128, 0.10),\n        \"Synovitis\": (32, 1.00),\n    }\n    pca_predictions = {target: [] for target in pca_specs}\n    for pca_seed in (20260809, 20260819, 20260829, 20260839):\n        decomposition = PCA(\n            n_components=128, whiten=True, svd_solver=\"randomized\",\n            n_oversamples=20, random_state=pca_seed,\n        )\n        train_embedding = decomposition.fit_transform(x_train_pca)\n        test_embedding = decomposition.transform(x_test_pca)\n        for target_name, (dimensions, c_value) in pca_specs.items():\n            exact_mask = train_df[target_name].notna().to_numpy()\n            exact_labels = train_df.loc[exact_mask, target_name].to_numpy(int)\n            if int(exact_mask.sum()) != 58 or set(np.unique(exact_labels)) != {0, 1}:\n                raise AssertionError(f\"unexpected exact-label support for {target_name}\")\n            model = make_pipeline(\n                StandardScaler(),\n                LogisticRegression(\n                    C=c_value, solver=\"liblinear\", class_weight=\"balanced\",\n                    max_iter=5000, random_state=pca_seed,\n                ),\n            )\n            model.fit(train_embedding[exact_mask, :dimensions], exact_labels)\n            pca_predictions[target_name].append(\n                model.predict_proba(test_embedding[:, :dimensions])[:, 1]\n            )\n    pca_predictions = {\n        target: np.mean(np.stack(predictions), axis=0)\n        for target, predictions in pca_predictions.items()\n    }\n\n    result = primary.copy()\n    primary_uids = result[\"StudyInstanceUID\"].astype(str)\n    if set(primary_uids) != set(test_uids):\n        raise AssertionError(\"hybrid and primary StudyInstanceUID sets differ\")\n    lm_by_uid = pd.Series(lm_teacher_rank, index=test_uids).reindex(primary_uids.values)\n    oa_by_uid = pd.Series(oa_teacher_rank, index=test_uids).reindex(primary_uids.values)\n    lm_pca_by_uid = pd.Series(\n        _hyb_rank(pca_predictions[HYB_TARGET]), index=test_uids\n    ).reindex(primary_uids.values)\n    syn_pca_by_uid = pd.Series(\n        _hyb_rank(pca_predictions[\"Synovitis\"]), index=test_uids\n    ).reindex(primary_uids.values)\n    lm_base_rank = result[HYB_TARGET].rank(pct=True).to_numpy(np.float64)\n    oa_base_rank = result[HYB_OA_TARGET].rank(pct=True).to_numpy(np.float64)\n    syn_base_rank = result[\"Synovitis\"].rank(pct=True).to_numpy(np.float64)\n    result[HYB_TARGET] = (\n        0.125 * lm_base_rank\n        + 0.500 * lm_by_uid.to_numpy(np.float64)\n        + 0.375 * lm_pca_by_uid.to_numpy(np.float64)\n    )\n    result[HYB_OA_TARGET] = 0.125 * oa_base_rank + 0.875 * oa_by_uid.to_numpy(np.float64)\n    result[\"Synovitis\"] = (\n        0.75 * syn_base_rank + 0.25 * syn_pca_by_uid.to_numpy(np.float64)\n    )\n    changed = {HYB_TARGET, HYB_OA_TARGET, \"Synovitis\"}\n    untouched = [target for target in TARGETS if target not in changed]\n    if not result[untouched].equals(primary[untouched]):\n        raise AssertionError(\"hybrid changed an unevaluated target\")\n    if result.shape != primary.shape or not np.isfinite(result[TARGETS].to_numpy()).all():\n        raise AssertionError(\"invalid hybrid blend\")\n    temp_path = Path(\"submission_v34_hybrid.tmp.csv\")\n    result.to_csv(temp_path, index=False)\n    reread = pd.read_csv(temp_path)\n    if reread.shape != primary.shape or not np.isfinite(reread[TARGETS].to_numpy()).all():\n        raise AssertionError(\"serialized hybrid blend is invalid\")\n    temp_path.replace(primary_path)\n    log(\n        \"hybrid complete: LM 0.125 base / 0.500 consensus / 0.375 PCA; \"\n        \"Lateral OA 0.125 base / 0.875 Pilkwang; Synovitis 0.75 base / 0.25 PCA; \"\n        \"nine other targets preserved\"\n    )\n    return True\n"},{"cell_type":"markdown","metadata":{},"source":"## 9. What the scored `.903` EfficientNet-B3 reference changes\n\nThe strongest fully visible score above this notebook is Yash Bishnoi's\n`yashbishnoi98/rsna-knee-infer-v1`, whose immutable Version 5 is attached to a completed\npublic score of **`.903`**.  The source now exposes enough of the experiment design to\nseparate the useful recipe from the part that cannot be reproduced.\n\n### The score begins with better supervision\n\nThe model is trained on all 4,349 report-only studies rather than the 58 expert-labelled\nstudies alone.  Its primary report reader is a locally served Qwen3.6-35B model, run at\ntemperature zero with reasoning disabled and a fixed twelve-key JSON contract.  Prompt\nrevisions were measured on the 58 gold studies: the published final reader reports `0.833`\nmean per-class accuracy and `0.766` mean F1.  Synovitis was broadened to include indirect\ninflammatory language; trace effusion was treated more cautiously; degenerative marrow\noedema stopped counting as contusion without a traumatic pattern; and avulsion and\ninsufficiency fractures were made explicit.\n\nThose labels are then fused, per target, with the public\n`pilkwang/rsna-knee-llm-labels` reader.  Agreement stays hard; disagreement becomes a soft\ntarget weighted by each reader's measured accuracy for that particular finding.  This is\nthe most important lesson from the `.903` system: the image model is downstream of label\nquality, and one global reader weight throws away useful per-finding evidence.\n\n### The image recipe that won its own ablation\n\nTwenty proxy trials on 10% of the studies for two epochs ranked candidates by\n`0.7 * CV AUC + 0.3 * gold58 AUC`.  The winner was not the most elaborate pooling option:\n\n| choice | scored Version 5 recipe |\n|---|---|\n| backbone | single-channel ImageNet EfficientNet-B3 |\n| folds | five study-grouped folds |\n| study input | three fluid-sensitive, plane-diverse series |\n| sampling | 12 slices/series in training; 32 at inference |\n| pooling | max over slice embeddings, then mean over series logits |\n| augmentation | shared rotation, gamma and scale; **no horizontal flip** |\n| objective | BCE-with-logits on fused soft labels |\n| optimisation | AdamW, short warm-up, cosine decay, bf16 |\n| schedule | eight epochs per fold |\n| ensemble | equal mean of five fold sigmoids |\n\nThe five-fold run took about 12.4 hours and reports clean macro OOF AUC `0.8544`.  The 58\ngold studies were held out of training and cross-fitted so that each was predicted only by\na checkpoint that had not seen it; that independent macro AUC was `0.8568`.  Agreement\nbetween those two disjoint checks is much stronger evidence than choosing architecture by\nleaderboard feedback.\n\nThe scored V5 inference applies DICOM slope/intercept, inverts `MONOCHROME1`, sorts by\n`InstanceNumber`, windows each series at its 1st/99th percentiles, center-crops depth at 64,\nresizes to 288 pixels, and quantises through `uint8`.  It checkpoints a valid submission\nevery 25 studies and can degrade from 32 to 16 slices and then from five to three folds if\nits projected runtime exceeds eight hours.  A later, unscored source revision adds\nsquare-padding and fuller decode diagnostics; those changes must not be attributed to the\ncompleted `.903` row.\n\n### Evidence boundary and promotion contract\n\n| artifact | what is public | what this notebook may claim |\n|---|---|---|\n| Pilkwang DINO family | complete inspectable code and 20 public checkpoints | independently executable `.891` anchor |\n| this notebook | completed official rows | highest verified public score `.894` |\n| public DINO output | exact attached predictions | independently completed `.899` reference, not yet this notebook's official score |\n| Yash B3 family | exact inference plus training protocol and validation summaries | completed `.903` reference; not an exact reproduction |\n\nThe five B3 checkpoints, `model_code.py`, fused labels, and locked Qwen prompt remain in a\nprivate input package.  The downloadable notebook output contains only the three visible\nsample UIDs.  Therefore its hidden-test ordering cannot be imported or honestly claimed as\nours, and the runtime below accepts a Yash-family artifact only after exact UID-set, schema,\nfinite-value, error-count, and study-count checks.\n\nAn independent promotion must reproduce the supervision and not just the word\n\"EfficientNet\": per-class label fusion, report-grouped folds, no laterality-breaking flips,\nfive full B3 folds, and disjoint OOF/gold validation.  Until such checkpoints exist and beat\nthe current macro OOF baseline, Version 40's audited public-frontier prediction remains the\nsafe primary output.\n"},{"id":"code-33","cell_type":"code","source":"def write_submission(pred, studies, test_df, path):\n    \"\"\"Write one submission file from a prediction matrix.\n\n    Predictions are converted to per-column ranks first: the metric reads only order, so\n    ranks discard nothing, and they make files from different configurations directly\n    comparable and safe to average.\n    \"\"\"\n    sub = pd.DataFrame(pd.DataFrame(pred).rank(pct=True).values, columns=TARGETS)\n    sub.insert(0, \"StudyInstanceUID\", studies)\n    sub = test_df[[\"StudyInstanceUID\"]].merge(sub, on=\"StudyInstanceUID\", how=\"left\")\n    sub[TARGETS] = sub[TARGETS].fillna(0.5)\n    sub.to_csv(path, index=False)\n    return sub\n\n\ndef write_benchmark_submission():\n    \"\"\"Write the 0.5 benchmark file immediately.\n\n    A submission that never writes scores nothing at all, which is strictly worse than\n    scoring badly. The try/except around main() covers exceptions, but a kill for memory\n    is a SIGKILL and never reaches it. So a valid file exists from the first second and\n    is overwritten only once real predictions are ready.\n    \"\"\"\n    t = pd.read_csv(ROOT / \"test.csv\")\n    for c in TARGETS:\n        t[c] = 0.5\n    t.to_csv(\"submission.csv\", index=False)\n\n\ndef _v37_validate_submission(path, test_df, tag):\n    \"\"\"Read one attached prediction file only after its full contract passes.\"\"\"\n    path = Path(path)\n    frame = pd.read_csv(path)\n    expected = [\"StudyInstanceUID\"] + TARGETS\n    if list(frame.columns) != expected:\n        raise ValueError(f\"{tag}: columns differ from the competition contract\")\n    if len(frame) != len(test_df) or not frame[\"StudyInstanceUID\"].is_unique:\n        raise ValueError(f\"{tag}: row count or StudyInstanceUID uniqueness failed\")\n    if set(frame[\"StudyInstanceUID\"].astype(str)) != set(test_df[\"StudyInstanceUID\"].astype(str)):\n        raise ValueError(f\"{tag}: StudyInstanceUID set differs from test.csv\")\n    values = frame[TARGETS].to_numpy(np.float64)\n    if not np.isfinite(values).all():\n        raise ValueError(f\"{tag}: non-finite prediction\")\n    return test_df[[\"StudyInstanceUID\"]].merge(frame, on=\"StudyInstanceUID\", how=\"left\")\n\n\ndef _v37_find_yash_submission():\n    \"\"\"Find the output mounted from the exact public Yash notebook source.\"\"\"\n    candidates = []\n    local = globals().get(\"YASH_LOCAL_SOURCE_DIR\")\n    if local:\n        candidates.append(Path(local) / \"submission.csv\")\n    root = Path(\"/kaggle/input\")\n    candidates.append(root / \"rsna-knee-infer-v1\" / \"submission.csv\")\n    if root.is_dir():\n        candidates.extend(meta.parent / \"submission.csv\"\n                          for meta in root.glob(\"**/infer_meta.json\"))\n    seen = set()\n    for path in candidates:\n        key = str(path)\n        if key in seen or not path.is_file():\n            continue\n        seen.add(key)\n        meta_path = path.with_name(\"infer_meta.json\")\n        if meta_path.is_file():\n            meta = json.loads(meta_path.read_text())\n            if int(meta.get(\"errors\", -1)) != 0:\n                raise ValueError(f\"Yash source reports {meta.get('errors')} inference errors\")\n        return path\n    raise FileNotFoundError(\"the attached yashbishnoi98/rsna-knee-infer-v1 output is absent\")\n\n\ndef run_yash_public_ensemble():\n    \"\"\"Bank the public image specialist and a conservative rank ensemble.\n\n    The Yash source is an independently trained EfficientNet-B3, five-fold,\n    plane-aware image family. The local public fallback is the independently\n    published twenty-member DINO family. A 0.55/0.45 Borda blend keeps Yash as\n    the stronger voter but lets concordant DINO evidence resolve close orderings.\n    The exact public source and the pre-blend native primary are both retained.\n    \"\"\"\n    import shutil\n\n    test_df = pd.read_csv(ROOT / \"test.csv\")\n    native_path = Path(\"submission.csv\")\n    public_path = Path(\"submission_public_0899.csv\")\n    native = _v37_validate_submission(native_path, test_df, \"native V36\")\n    public = _v37_validate_submission(public_path, test_df, \"public DINO family\")\n    yash_path = _v37_find_yash_submission()\n    yash = _v37_validate_submission(yash_path, test_df, \"Yash public image family\")\n    meta_path = yash_path.with_name(\"infer_meta.json\")\n    if meta_path.is_file():\n        meta = json.loads(meta_path.read_text())\n        if int(meta.get(\"studies\", -1)) != len(test_df):\n            raise ValueError(\"Yash source study count differs from test.csv\")\n\n    shutil.copyfile(native_path, \"submission_native_v36.csv\")\n    shutil.copyfile(yash_path, \"submission_yash_reference.csv\")\n    yr = yash[TARGETS].rank(pct=True).to_numpy(np.float64)\n    dr = public[TARGETS].rank(pct=True).to_numpy(np.float64)\n    blend = 0.55 * yr + 0.45 * dr\n    result = test_df[[\"StudyInstanceUID\"]].copy()\n    result[TARGETS] = blend\n    if result.shape != yash.shape or not np.isfinite(result[TARGETS].to_numpy()).all():\n        raise AssertionError(\"invalid Yash/DINO rank blend\")\n    candidate_path = Path(\"submission_yash_dino_rankblend.csv\")\n    result.to_csv(candidate_path, index=False)\n    reread = _v37_validate_submission(candidate_path, test_df, \"Yash/DINO rank blend\")\n    changed = sum(\n        tuple(reread[target].rank(method=\"first\")) !=\n        tuple(yash[target].rank(method=\"first\"))\n        for target in TARGETS\n    )\n    if changed == 0:\n        raise AssertionError(\"Yash/DINO blend is rank-identical to its Yash parent\")\n    temp_path = Path(\"submission_v37_yash_dino.tmp.csv\")\n    reread.to_csv(temp_path, index=False)\n    temp_path.replace(native_path)\n    log(f\"Yash public family banked; V37 primary = 0.55 Yash / 0.45 public DINO \"\n        f\"rank blend ({changed} target orderings differ from Yash); exact Yash and \"\n        \"native V36 outputs retained\")\n    return True\n\n\ndef main():\n    write_benchmark_submission()\n\n    # Weights, if any were attached; otherwise the run learns its own below. Both paths\n    # are kept because the second is what makes this notebook readable on its own - a\n    # fork with nothing attached still trains and still scores - and because the first\n    # cannot be checked by anyone who does not have the package.\n    pkg = find_weights()\n    if pkg is not None:\n        dev = DEVS[0]\n        infer_from_package(pkg, dev)\n\n        # The exact no-jitter target-pooling recipe independently completed at 0.899 in\n        # the public frontier.  Earlier versions retained it as a secondary artifact and\n        # then promoted less certain specialists.  Make the evidence-backed arm primary\n        # only after the complete hidden UID/schema/finite-value contract passes.\n        try:\n            test_df = pd.read_csv(ROOT / \"test.csv\")\n            native_path = Path(\"submission.csv\")\n            public_path = Path(\"submission_public_0899.csv\")\n            native = _v37_validate_submission(native_path, test_df, \"native 24-member\")\n            public = _v37_validate_submission(public_path, test_df, \"public DINO frontier\")\n            native.to_csv(\"submission_native_v38.csv\", index=False)\n            public.to_csv(native_path, index=False)\n            promoted = _v37_validate_submission(native_path, test_df, \"V40 primary\")\n            if not promoted.equals(public):\n                raise AssertionError(\"V40 serialization differs from validated public frontier\")\n            log(\"V40 primary = exact no-jitter public-frontier target pooling; \"\n                \"native 24-member output retained\")\n        except Exception as public_frontier_error:\n            log(f\"public-frontier promotion skipped safely: {public_frontier_error}\")\n            traceback.print_exc()\n        log(\"done\")\n        return\n\n    # Settle where the labels come from before anything expensive runs. The check costs\n    # one CSV header read; discovering the same problem after the cache is built would\n    # cost the whole decode pass, and discovering it never would cost the run.\n    read_labels(pd.read_csv(ROOT / \"train.csv\", usecols=[\"StudyInstanceUID\", \"Report\"]))\n\n    test_df = pd.read_csv(ROOT / \"test.csv\")\n    test_series = pd.read_csv(ROOT / \"test_series.csv\")\n    train_df = pd.read_csv(ROOT / \"train.csv\")\n    train_series = pd.read_csv(ROOT / \"train_series.csv\")\n    log(f\"train {train_df.shape} test {test_df.shape}\")\n\n    both = pd.concat([train_series, test_series])\n    plane_map = dict(zip(both[\"SeriesInstanceUID\"], both[\"Anatomical_Plane\"]))\n\n    log(\"header pass: test\")\n    hte = annotate(walk(\"test_series\"))\n    log(f\"  {len(hte)} test series\")\n    log(\"header pass: train\")\n    htr = annotate(walk(\"train_series\"))\n    log(f\"  {len(htr)} train series\")\n\n    slots_te, slots_tr = pick_slots(hte, plane_map), pick_slots(htr, plane_map)\n    cov = pd.Series([len(v) for v in slots_tr.values()]).describe()\n    log(f\"train slots per study: mean {cov['mean']:.2f} min {cov['min']:.0f} \"\n        f\"max {cov['max']:.0f}\")\n\n    st_tr, Ctr, Mtr = build_cache(slots_tr, plane_map, lat_of(htr, \"train \"), \"train\")\n    st_te, Cte, Mte = build_cache(slots_te, plane_map, lat_of(hte, \"test \"), \"test\")\n\n    # ---- targets ---------------------------------------------------------- #\n    t_lab = time.time()\n    lab = read_labels(train_df)\n    log(f\"derived labels for {len(lab)} studies in {time.time() - t_lab:.1f}s\")\n\n    gold = train_df.set_index(\"StudyInstanceUID\")[TARGETS]\n    gold = gold[gold.notna().all(axis=1)]\n\n    Y = np.zeros((len(st_tr), len(TARGETS)), np.float32)\n    W = np.zeros_like(Y)\n    for i, st in enumerate(st_tr):\n        if st in gold.index:\n            Y[i], W[i] = gold.loc[st].values, 3.0\n        elif st in lab.index:\n            r = lab.loc[st]\n            Y[i] = r[TARGETS].values\n            W[i] = 0.25 + 0.75 * r[[t + \"__conf\" for t in TARGETS]].values\n    keep = np.where(W.sum(1) > 0)[0]\n    log(f\"supervised {len(keep)} of {len(st_tr)} studies (annotated {len(gold)})\")\n\n    # Grouped on report text: some reports are byte-identical across studies and yield\n    # one target vector for all of them, so splitting such a group scores the model on a\n    # target whose source it has already trained on.\n    import hashlib\n    rep = train_df.set_index(\"StudyInstanceUID\")[\"Report\"].fillna(\"\")\n    grp = np.array([int(hashlib.md5(rep.get(s, s).encode()).hexdigest()[:8], 16) % 5\n                    for s in st_tr])\n    va = np.array([i for i in keep if grp[i] == 0])\n    tr = np.array([i for i in keep if grp[i] != 0])\n    if len(va) == 0 or len(tr) < BATCH_STUDIES:\n        cut = max(1, len(keep) // 5)\n        va, tr = keep[:cut], keep[cut:]\n    log(f\"train {len(tr)} / holdout {len(va)} studies\")\n\n    # The annotated studies stay in training - they are the highest-quality labels in\n    # the corpus and there are too few to discard - so the honest annotation check uses\n    # only the ones that fell in the holdout. Evaluating on the rest would be scoring the\n    # model against examples it was trained on, at triple weight, with the true answer.\n    gpos = {s: i for i, s in enumerate(st_tr)}\n    va_set = set(va.tolist())\n    gi = np.array([gpos[s] for s in gold.index if s in gpos and gpos[s] in va_set])\n    gold_y = gold.loc[[st_tr[i] for i in gi]].values.astype(int) if len(gi) else None\n    yv = (Y[va] > 0.5).astype(int)\n    log(f\"annotation check: {len(gi)} of {len(gold)} annotated studies are in the holdout\")\n\n    # ---- fine-tune -------------------------------------------------------- #\n    dev = DEVS[0]\n    results, test_preds = {}, {}\n\n    for cfg in RUNS:\n        pitch = CROP_MM / cfg[\"img\"]\n        log(f\"=== {cfg['name']}: {cfg['img']} px, {pitch:.3f} mm/pixel, \"\n            f\"{pitch * 14:.2f} mm per patch token ===\")\n        torch.manual_seed(SEED)\n        model = build_model(UNFREEZE_LAST).to(dev)\n        opt = torch.optim.AdamW([\n            {\"params\": [p for p in model.backbone.parameters() if p.requires_grad],\n             \"lr\": LR_BACKBONE},\n            {\"params\": model.head.parameters(), \"lr\": LR_HEAD},\n        ], weight_decay=WEIGHT_DECAY)\n        steps = max(EPOCHS * (len(tr) // BATCH_STUDIES), 1)\n        sched = torch.optim.lr_scheduler.OneCycleLR(\n            opt, max_lr=[LR_BACKBONE, LR_HEAD], total_steps=steps, pct_start=0.15)\n        scaler = torch.amp.GradScaler(\"cuda\", enabled=dev.type == \"cuda\")\n\n        best, best_state, best_annot = -1.0, None, float(\"nan\")\n        for ep in range(EPOCHS):\n            model.train()\n            perm = np.random.permutation(tr)\n            tot, nstep = 0.0, 0\n            for b in range(0, len(perm) - BATCH_STUDIES + 1, BATCH_STUDIES):\n                sel = perm[b:b + BATCH_STUDIES]\n                rows = torch.from_numpy(Ctr[sel]).to(dev)\n                g = int(torch.randint(N_GROUP, (1,)).item())\n                imgs = augment(take_group(rows, g))\n                m = torch.from_numpy(Mtr[sel]).to(dev)\n                y = torch.from_numpy(Y[sel]).to(dev)\n                w = torch.from_numpy(W[sel]).to(dev)\n                with torch.autocast(\"cuda\", enabled=dev.type == \"cuda\"):\n                    loss = (F.binary_cross_entropy_with_logits(\n                        model(imgs, m, cfg[\"img\"]), y, reduction=\"none\") * w).mean()\n                opt.zero_grad(set_to_none=True)\n                scaler.scale(loss).backward()\n                scaler.step(opt)\n                scaler.update()\n                sched.step()\n                tot += loss.item()\n                nstep += 1\n\n            pv = predict(model, Ctr, Mtr, va, dev, cfg[\"img\"])\n            d = macro_auc(yv, pv)\n            g_auc = float(\"nan\")\n            if gold_y is not None and len(gi):\n                g_auc = macro_auc(gold_y, predict(model, Ctr, Mtr, gi, dev, cfg[\"img\"]))\n            log(f\"  epoch {ep + 1}/{EPOCHS}  loss {tot / max(nstep, 1):.4f}\"\n                f\"  holdout {d:.4f}  annot(n={len(gi)}) {g_auc:.4f}\")\n\n            # Selection reads the holdout alone. The annotation check is reported because\n            # it measures something different - agreement with a reading of the images\n            # rather than of the reports - but only a handful of annotated studies land\n            # in any one holdout, so its sampling error dwarfs the differences between\n            # epochs and it cannot arbitrate between them.\n            if d > best:\n                best, best_annot = d, g_auc\n                best_state = {k: v.detach().cpu().clone()\n                              for k, v in model.state_dict().items()}\n            if time.time() - T0 > TIME_BUDGET:\n                log(\"  time budget reached\")\n                break\n\n        if best_state is not None:\n            model.load_state_dict(best_state)\n        results[cfg[\"name\"]] = (best, best_annot)\n        test_preds[cfg[\"name\"]] = predict(model, Cte, Mte, np.arange(len(st_te)), dev,\n                                          cfg[\"img\"])\n        log(f\"  {cfg['name']}: best holdout {best:.4f} (annot {best_annot:.4f})\")\n        del model, opt, sched, scaler, best_state\n        gc.collect()\n        if dev.type == \"cuda\":\n            torch.cuda.empty_cache()\n\n    log(\"---- summary ----\")\n    for n, (d, g_auc) in results.items():\n        log(f\"  {n:12s} holdout {d:.4f}   annot {g_auc:.4f}\")\n    pick = max(results, key=lambda k: results[k][0])\n    log(f\"best on the holdout: {pick} ({results[pick][0]:.4f})\")\n\n\n    # ---- write every candidate -------------------------------------------- #\n    # One file per configuration, plus the holdout's choice as `submission.csv`. A run\n    # costs a full decode of the corpus whichever configuration wins, so keeping every\n    # arm makes a later change of configuration free rather than another full run.\n    for name, pred in test_preds.items():\n        sub = write_submission(pred, st_te, test_df, f\"submission_{name}.csv\")\n        log(f\"  submission_{name}.csv {sub.shape}; \"\n            f\"nulls {int(sub[TARGETS].isna().sum().sum())}\")\n\n    ens = np.mean([pd.DataFrame(p).rank(pct=True).values for p in test_preds.values()],\n                  axis=0)\n    write_submission(ens, st_te, test_df, \"submission_rankmean.csv\")\n    log(f\"  submission_rankmean.csv (rank mean of {len(test_preds)})\")\n\n    sub = write_submission(test_preds[pick], st_te, test_df, \"submission.csv\")\n    log(f\"submission.csv = {pick}; {sub.shape}; \"\n        f\"nulls {int(sub[TARGETS].isna().sum().sum())}\")\n    print(sub.head().to_string())\n","metadata":{"_kg_hide-input":true,"_kg_hide-output":true},"outputs":[],"execution_count":null},{"id":"code-34","cell_type":"code","source":"try:\n    main()\nexcept LabelSourceError:\n    # Deliberately not absorbed: see LabelSourceError. A run that trained on the\n    # wrong labels would finish and write a submission worth submitting by mistake.\n    traceback.print_exc()\n    raise\nexcept Exception:\n    traceback.print_exc()\n    # A submission that fails to write scores nothing at all, so fall back to the\n    # benchmark file rather than dying.\n    t = pd.read_csv(find_root() / \"test.csv\")\n    for c in TARGETS:\n        t[c] = 0.5\n    t.to_csv(\"submission.csv\", index=False)\n    print(\"wrote fallback submission.csv\")\nlog(\"done\")\n","metadata":{"_kg_hide-input":true,"_kg_hide-output":true},"outputs":[],"execution_count":null},{"cell_type":"markdown","metadata":{},"source":"## 10. A guarded, independently trainable B3 branch\n\nThe `.903` reference is not a reusable output file: its decisive five checkpoints are private. This branch therefore trains an independent model from public, hash-pinned inputs. It encodes each study once with a frozen public EfficientNet-B3, trains five target-attention MIL heads on three-source soft-label consensus, and checks all 58 expert studies strictly out of fold. Physical DICOM ordering, a 130 mm crop, laterality normalization, three planes, and 32 slices are explicit contracts.\n\nThe branch is fail-safe. The independently reproduced public `.899` artifact is produced first and remains `submission.csv` if any input hash, training contract, 58-study OOF audit, or inference schema fails. Even a successful local OOF result is diagnostic evidence only; only a completed competition row can establish a public score."},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"from pathlib import Path as _B3SourcePath\n_B3SourcePath('/kaggle/working/rsna_b3_v42_source').mkdir(parents=True, exist_ok=True)\n"},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"%%writefile /kaggle/working/rsna_b3_v42_source/efficientnet_b3_public_repro_v1.py\n#!/usr/bin/env python3\n\"\"\"Leakage-controlled EfficientNet-B3 MIL training for RSNA Knee.\"\"\"\n\nfrom __future__ import annotations\n\nimport argparse\nimport hashlib\nimport json\nimport math\nimport random\nimport re\nimport unicodedata\nfrom dataclasses import asdict, dataclass\nfrom pathlib import Path\nfrom typing import Dict, List, Mapping, Optional, Sequence, Tuple\n\nimport numpy as np\nimport pandas as pd\n\nTARGETS: Tuple[str, ...] = (\n    \"ACL\", \"MCL\", \"Medial Meniscus\", \"Lateral Meniscus\", \"Medial OA\",\n    \"Lateral OA\", \"PF OA\", \"Effusion\", \"Synovitis\", \"Baker's\",\n    \"Contusion\", \"Fracture\",\n)\nPLANES: Tuple[str, ...] = (\"Sagittal\", \"Coronal\", \"Axial\")\nUID = \"StudyInstanceUID\"\n\n\n@dataclass(frozen=True)\nclass TrainConfig:\n    image_size: int = 288\n    max_series_slices: int = 64\n    train_slices: int = 20\n    valid_slices: int = 32\n    folds: int = 5\n    seed: int = 20260809\n    batch_size: int = 2\n    workers: int = 4\n    backbone_chunk: int = 12\n    frozen_epochs: int = 1\n    unfrozen_epochs: int = 2\n    frozen_lr: float = 8e-4\n    unfrozen_lr: float = 8e-5\n    weight_decay: float = 1e-4\n    gold_weight: float = 3.0\n    pseudo_weight_floor: float = 0.20\n    backbone: str = \"efficientnet_b3\"\n\n\n_DATE = re.compile(\n    r\"\\b(?:19|20)\\d{2}[-/.]\\d{1,2}[-/.]\\d{1,2}\\b|\"\n    r\"\\b\\d{1,2}[-/.]\\d{1,2}[-/.](?:\\d{2}|\\d{4})\\b\"\n)\n_NUMBER = re.compile(r\"\\b\\d+(?:[.,]\\d+)?\\b\")\n_SPACE = re.compile(r\"\\s+\")\n\n\ndef normalize_report_for_group(value: object) -> str:\n    text = \"\" if value is None else str(value)\n    text = unicodedata.normalize(\"NFKC\", text).casefold()\n    text = _DATE.sub(\" <date> \", text)\n    text = _NUMBER.sub(\" <num> \", text)\n    return _SPACE.sub(\" \", text).strip()\n\n\ndef report_group_id(value: object) -> str:\n    return hashlib.sha256(normalize_report_for_group(value).encode()).hexdigest()[:24]\n\n\ndef _indexed_labels(frame: pd.DataFrame, name: str) -> pd.DataFrame:\n    required = {UID, *TARGETS}\n    missing = required.difference(frame.columns)\n    if missing:\n        raise ValueError(f\"{name} is missing columns: {sorted(missing)}\")\n    if frame[UID].duplicated().any():\n        raise ValueError(f\"{name} contains duplicate {UID} rows\")\n    return frame[[UID, *TARGETS]].copy().set_index(UID).apply(pd.to_numeric, errors=\"coerce\")\n\n\ndef build_public_consensus(\n    train: pd.DataFrame,\n    pilkwang: pd.DataFrame,\n    steven: pd.DataFrame,\n    lixin: pd.DataFrame,\n    pseudo_weight_floor: float = 0.20,\n) -> pd.DataFrame:\n    \"\"\"Equal-source soft labels; weights come only from agreement/certainty.\"\"\"\n    if train[UID].duplicated().any():\n        raise ValueError(f\"train contains duplicate {UID} rows\")\n    base = train[[UID, \"Report\", *TARGETS]].copy().set_index(UID)\n    sources = [\n        _indexed_labels(pilkwang, \"pilkwang\"),\n        _indexed_labels(steven, \"steven\"),\n        _indexed_labels(lixin, \"lixin\"),\n    ]\n    out = base.reset_index()[[UID, \"Report\"]].copy()\n    out[\"report_group\"] = out[\"Report\"].map(report_group_id)\n    out = out.drop(columns=\"Report\")\n    cube = np.stack([source.reindex(base.index).to_numpy(float) for source in sources])\n    if np.nanmin(cube) < 0.0 or np.nanmax(cube) > 1.0:\n        raise ValueError(\"public labels must be in [0, 1]\")\n    available = np.isfinite(cube).sum(axis=0)\n    if np.any(available < 2):\n        row, target = np.argwhere(available < 2)[0]\n        raise ValueError(f\"fewer than two public labels at row={row}, target={TARGETS[target]}\")\n    consensus = np.nanmean(cube, axis=0)\n    disagreement = np.nanmean(np.abs(cube - consensus[None]), axis=0)\n    agreement = np.clip(1.0 - 2.0 * disagreement, 0.0, 1.0)\n    certainty = np.clip(2.0 * np.abs(consensus - 0.5), 0.0, 1.0)\n    weight = pseudo_weight_floor + (1.0 - pseudo_weight_floor) * (\n        0.65 * agreement + 0.35 * certainty\n    )\n    out[\"is_expert\"] = base[list(TARGETS)].notna().all(axis=1).astype(np.int8).to_numpy()\n    for j, target in enumerate(TARGETS):\n        out[f\"pseudo::{target}\"] = consensus[:, j].astype(np.float32)\n        out[f\"weight::{target}\"] = weight[:, j].astype(np.float32)\n        out[f\"gold::{target}\"] = pd.to_numeric(base[target], errors=\"coerce\").to_numpy()\n    return out\n\n\ndef _group_statistics(label_table: pd.DataFrame) -> pd.DataFrame:\n    pseudo_cols = [f\"pseudo::{target}\" for target in TARGETS]\n    gold_cols = [f\"gold::{target}\" for target in TARGETS]\n    records = []\n    for group_id, group in label_table.groupby(\"report_group\", sort=True):\n        gold = group[gold_cols].to_numpy(float)\n        records.append({\n            \"report_group\": group_id,\n            \"size\": len(group),\n            \"expert_count\": int(group[\"is_expert\"].sum()),\n            \"pseudo_sum\": group[pseudo_cols].to_numpy(float).sum(axis=0),\n            \"gold_positive\": np.nansum(gold, axis=0),\n            \"gold_known\": np.isfinite(gold).sum(axis=0).astype(float),\n        })\n    return pd.DataFrame(records)\n\n\ndef assign_group_balanced_folds(\n    label_table: pd.DataFrame, n_splits: int = 5, seed: int = 20260809\n) -> pd.Series:\n    \"\"\"Deterministic greedy balance with normalized-report groups kept atomic.\"\"\"\n    if n_splits < 2:\n        raise ValueError(\"n_splits must be at least two\")\n    groups = _group_statistics(label_table)\n    target_size = float(groups[\"size\"].sum()) / n_splits\n    target_pseudo = np.sum(np.stack(groups[\"pseudo_sum\"]), axis=0) / n_splits\n    total_gold_pos = np.sum(np.stack(groups[\"gold_positive\"]), axis=0)\n    target_gold_pos = total_gold_pos / n_splits\n    target_gold_known = np.sum(np.stack(groups[\"gold_known\"]), axis=0) / n_splits\n    rng = np.random.default_rng(seed)\n    rarity = 1.0 / np.maximum(total_gold_pos, 1.0)\n    priority = [\n        100.0 * row[\"expert_count\"]\n        + 10.0 * float(np.dot(row[\"gold_positive\"], rarity))\n        + math.log1p(row[\"size\"])\n        for _, row in groups.iterrows()\n    ]\n    groups = groups.assign(_priority=priority, _jitter=rng.uniform(0, 1e-6, len(groups)))\n    groups = groups.sort_values([\"_priority\", \"_jitter\", \"report_group\"], ascending=[False, False, True])\n    fold_size = np.zeros(n_splits)\n    fold_pseudo = np.zeros((n_splits, len(TARGETS)))\n    fold_gold_pos = np.zeros_like(fold_pseudo)\n    fold_gold_known = np.zeros_like(fold_pseudo)\n    assignment: Dict[str, int] = {}\n    for _, row in groups.iterrows():\n        scores = []\n        for fold in range(n_splits):\n            candidate_size = fold_size.copy()\n            candidate_pseudo = fold_pseudo.copy()\n            candidate_gold_pos = fold_gold_pos.copy()\n            candidate_gold_known = fold_gold_known.copy()\n            candidate_size[fold] += row[\"size\"]\n            candidate_pseudo[fold] += row[\"pseudo_sum\"]\n            candidate_gold_pos[fold] += row[\"gold_positive\"]\n            candidate_gold_known[fold] += row[\"gold_known\"]\n            size_cost = np.mean(((candidate_size - target_size) / max(target_size, 1.0)) ** 2)\n            pseudo_cost = np.mean(((candidate_pseudo - target_pseudo[None]) / np.maximum(target_pseudo[None], 1.0)) ** 2)\n            gold_pos_cost = np.mean(((candidate_gold_pos - target_gold_pos[None]) / np.maximum(target_gold_pos[None], 1.0)) ** 2)\n            gold_known_cost = np.mean(((candidate_gold_known - target_gold_known[None]) / np.maximum(target_gold_known[None], 1.0)) ** 2)\n            scores.append(size_cost + 0.35 * pseudo_cost + 4.0 * gold_pos_cost + 2.0 * gold_known_cost)\n        best = int(np.argmin(np.asarray(scores) + 1e-12 * np.arange(n_splits)))\n        assignment[str(row[\"report_group\"])] = best\n        fold_size[best] += row[\"size\"]\n        fold_pseudo[best] += row[\"pseudo_sum\"]\n        fold_gold_pos[best] += row[\"gold_positive\"]\n        fold_gold_known[best] += row[\"gold_known\"]\n    result = label_table[\"report_group\"].map(assignment)\n    if result.isna().any():\n        raise AssertionError(\"some report groups were not assigned\")\n    return result.astype(np.int8)\n\n\ndef pick_one_series_per_plane(series: pd.DataFrame) -> Dict[str, Optional[str]]:\n    required = {\"SeriesInstanceUID\", \"Anatomical_Plane\", \"Fluid_Sensitive\", \"Fat_Suppression\"}\n    missing = required.difference(series.columns)\n    if missing:\n        raise ValueError(f\"series table missing columns: {sorted(missing)}\")\n    chosen: Dict[str, Optional[str]] = {}\n    for plane in PLANES:\n        subset = series.loc[series[\"Anatomical_Plane\"].eq(plane)].copy()\n        if subset.empty:\n            chosen[plane] = None\n            continue\n        for column in (\"Fluid_Sensitive\", \"Fat_Suppression\"):\n            subset[column] = pd.to_numeric(subset[column], errors=\"coerce\").fillna(0)\n        subset = subset.sort_values(\n            [\"Fluid_Sensitive\", \"Fat_Suppression\", \"SeriesInstanceUID\"],\n            ascending=[False, False, True],\n        )\n        chosen[plane] = str(subset.iloc[0][\"SeriesInstanceUID\"])\n    return chosen\n\n\ndef pick_series_slots(series: pd.DataFrame, count: int = 3) -> List[Optional[str]]:\n    \"\"\"Prefer one fluid-sensitive series per plane, then fill empty slots.\"\"\"\n    required = {\"SeriesInstanceUID\", \"Anatomical_Plane\", \"Fluid_Sensitive\", \"Fat_Suppression\"}\n    missing = required.difference(series.columns)\n    if missing:\n        raise ValueError(f\"series table missing columns: {sorted(missing)}\")\n    frame = series.copy()\n    for column in (\"Fluid_Sensitive\", \"Fat_Suppression\"):\n        frame[column] = pd.to_numeric(frame[column], errors=\"coerce\").fillna(0)\n    frame[\"_plane\"] = frame[\"Anatomical_Plane\"].map({p: i for i, p in enumerate(PLANES)}).fillna(3)\n    preferred = frame.loc[frame[\"Fluid_Sensitive\"].eq(1)].sort_values(\n        [\"_plane\", \"Fat_Suppression\", \"SeriesInstanceUID\"],\n        ascending=[True, False, True],\n    )\n    selected: List[str] = []\n    seen_planes = set()\n    for _, row in preferred.iterrows():\n        plane = str(row[\"Anatomical_Plane\"])\n        if plane not in seen_planes:\n            selected.append(str(row[\"SeriesInstanceUID\"]))\n            seen_planes.add(plane)\n        if len(selected) == count:\n            break\n    ordered = frame.sort_values(\n        [\"Fluid_Sensitive\", \"Fat_Suppression\", \"_plane\", \"SeriesInstanceUID\"],\n        ascending=[False, False, True, True],\n    )\n    for series_uid in ordered[\"SeriesInstanceUID\"].astype(str):\n        if len(selected) == count:\n            break\n        if series_uid not in selected:\n            selected.append(series_uid)\n    return selected + [None] * (count - len(selected))\n\n\ndef uniform_slice_indices(length: int, count: int) -> np.ndarray:\n    if length <= 0 or count <= 0:\n        raise ValueError(\"length and count must be positive\")\n    if length <= count:\n        return np.concatenate([np.arange(length), np.full(count - length, length - 1)]).astype(np.int64)\n    return np.rint(np.linspace(0, length - 1, count)).astype(np.int64)\n\n\ndef stochastic_slice_indices(length: int, count: int, rng: np.random.Generator) -> np.ndarray:\n    if length <= count:\n        return uniform_slice_indices(length, count)\n    edges = np.linspace(0, length, count + 1)\n    values = []\n    for left, right in zip(edges[:-1], edges[1:]):\n        lo = int(math.floor(left))\n        hi = max(lo + 1, int(math.ceil(right)))\n        values.append(int(rng.integers(lo, min(hi, length))))\n    return np.asarray(values, dtype=np.int64)\n\n\ndef pooled_macro_auc(y_true: np.ndarray, y_score: np.ndarray) -> Tuple[float, Dict[str, float]]:\n    from sklearn.metrics import roc_auc_score\n    per_target: Dict[str, float] = {}\n    for j, target in enumerate(TARGETS):\n        valid = np.isfinite(y_true[:, j]) & np.isfinite(y_score[:, j])\n        per_target[target] = (\n            float(roc_auc_score(y_true[valid, j], y_score[valid, j]))\n            if valid.sum() and np.unique(y_true[valid, j]).size == 2\n            else float(\"nan\")\n        )\n    finite = [value for value in per_target.values() if np.isfinite(value)]\n    return (float(np.mean(finite)) if finite else float(\"nan\")), per_target\n\n\ndef prepare_fold_rows(\n    label_table: pd.DataFrame, fold: int, config: TrainConfig\n) -> Tuple[pd.DataFrame, pd.DataFrame]:\n    train_rows = label_table.loc[label_table[\"fold\"].ne(fold)].copy()\n    valid_rows = label_table.loc[label_table[\"fold\"].eq(fold) & label_table[\"is_expert\"].eq(1)].copy()\n    if train_rows.empty or valid_rows.empty:\n        raise ValueError(f\"empty train/validation split for fold {fold}\")\n    for target in TARGETS:\n        gold = pd.to_numeric(train_rows[f\"gold::{target}\"], errors=\"coerce\")\n        is_gold = train_rows[\"is_expert\"].eq(1) & gold.notna()\n        train_rows[f\"target::{target}\"] = train_rows[f\"pseudo::{target}\"].where(~is_gold, gold)\n        train_rows[f\"train_weight::{target}\"] = train_rows[f\"weight::{target}\"].where(~is_gold, config.gold_weight)\n        valid_rows[f\"target::{target}\"] = pd.to_numeric(valid_rows[f\"gold::{target}\"], errors=\"raise\")\n        valid_rows[f\"train_weight::{target}\"] = 1.0\n    if set(train_rows[\"report_group\"]).intersection(valid_rows[\"report_group\"]):\n        raise AssertionError(\"normalized report leakage across train and validation\")\n    return train_rows, valid_rows\n\n\ndef _training_imports():\n    try:\n        import cv2\n        import pydicom\n        import timm\n        import torch\n        import torch.nn as nn\n        from torch.utils.data import DataLoader, Dataset\n    except ImportError as exc:\n        raise RuntimeError(\"training requires torch, timm, pydicom, and opencv-python\") from exc\n    return cv2, pydicom, timm, torch, nn, DataLoader, Dataset\n\n\ndef load_dicom_volume(series_dir: Path, image_size: int, max_slices: int = 64) -> np.ndarray:\n    cv2, pydicom, _, _, _, _, _ = _training_imports()\n    slices: List[Tuple[int, np.ndarray]] = []\n    for path in sorted(series_dir.glob(\"*.dcm\")):\n        try:\n            ds = pydicom.dcmread(str(path))\n            image = ds.pixel_array.astype(np.float32)\n            image = image * float(getattr(ds, \"RescaleSlope\", 1.0) or 1.0)\n            image += float(getattr(ds, \"RescaleIntercept\", 0.0) or 0.0)\n            if str(getattr(ds, \"PhotometricInterpretation\", \"\")) == \"MONOCHROME1\":\n                image = image.max() - image\n            slices.append((int(getattr(ds, \"InstanceNumber\", 0) or 0), image))\n        except Exception:\n            continue\n    if not slices:\n        raise RuntimeError(f\"no decodable slices in {series_dir}\")\n    slices.sort(key=lambda item: item[0])\n    volume = np.stack([item[1] for item in slices])\n    lo, hi = np.percentile(volume, [1.0, 99.0])\n    hi = max(float(hi), float(lo) + 1.0)\n    volume = np.clip((volume - lo) / (hi - lo), 0.0, 1.0)\n    if len(volume) > max_slices:\n        start = (len(volume) - max_slices) // 2\n        volume = volume[start:start + max_slices]\n    resized = np.stack([\n        cv2.resize(x, (image_size, image_size), interpolation=cv2.INTER_AREA)\n        for x in volume\n    ])\n    # Match the public .903 pipeline's training-time uint8 ETL boundary.\n    return (resized * 255.0).astype(np.uint8).astype(np.float32) / 255.0\n\n\ndef make_dataset_class():\n    _, _, _, torch, _, _, Dataset = _training_imports()\n    class KneeStudyDataset(Dataset):\n        def __init__(self, rows, series, image_root, config, training, seed):\n            self.rows = rows.reset_index(drop=True)\n            self.series = {str(uid): group.copy() for uid, group in series.groupby(UID, sort=False)}\n            self.image_root, self.config, self.training, self.seed = Path(image_root), config, training, seed\n\n        def __len__(self):\n            return len(self.rows)\n\n        def __getitem__(self, index):\n            row, config = self.rows.iloc[index], self.config\n            uid = str(row[UID])\n            count = config.train_slices if self.training else config.valid_slices\n            # DataLoader seeds each worker deterministically.  Drawing once from\n            # its advancing torch RNG gives a fresh augmentation on every visit\n            # without sacrificing run-level reproducibility.\n            visit_seed = int(torch.randint(0, 2**31 - 1, (1,)).item()) if self.training else 0\n            rng = np.random.default_rng(self.seed + 1_000_003 * index + visit_seed)\n            empty = pd.DataFrame(columns=[\"SeriesInstanceUID\", \"Anatomical_Plane\", \"Fluid_Sensitive\", \"Fat_Suppression\"])\n            chosen = pick_series_slots(self.series.get(uid, empty), len(PLANES))\n            planes, masks = [], []\n            for series_uid in chosen:\n                if series_uid is None:\n                    planes.append(np.zeros((count, config.image_size, config.image_size), np.float32))\n                    masks.append(np.zeros(count, bool))\n                    continue\n                volume = load_dicom_volume(\n                    self.image_root / uid / series_uid,\n                    config.image_size,\n                    config.max_series_slices,\n                )\n                idx = stochastic_slice_indices(len(volume), count, rng) if self.training else uniform_slice_indices(len(volume), count)\n                sampled = volume[idx].astype(np.float32)\n                if self.training:\n                    sampled = np.clip(sampled * rng.uniform(0.85, 1.15), 0, 1) ** rng.uniform(0.85, 1.15)\n                    if rng.random() < 0.5:\n                        sampled = sampled[:, :, ::-1].copy()\n                sampled = (sampled - 0.449) / 0.226\n                planes.append(sampled)\n                masks.append(np.ones(count, bool))\n            return {\n                \"uid\": uid,\n                \"images\": torch.from_numpy(np.stack(planes)[:, :, None]),\n                \"mask\": torch.from_numpy(np.stack(masks)),\n                \"target\": torch.tensor([row[f\"target::{x}\"] for x in TARGETS], dtype=torch.float32),\n                \"weight\": torch.tensor([row[f\"train_weight::{x}\"] for x in TARGETS], dtype=torch.float32),\n            }\n    return KneeStudyDataset\n\n\ndef make_model_class():\n    _, _, timm, torch, nn, _, _ = _training_imports()\n    class KneeMILModel(nn.Module):\n        def __init__(self, backbone=\"efficientnet_b3\", checkpoint=None, backbone_chunk=12):\n            super().__init__()\n            self.backbone_chunk = int(backbone_chunk)\n            self.backbone = timm.create_model(backbone, pretrained=False, in_chans=1, num_classes=0, global_pool=\"avg\")\n            if checkpoint:\n                state = torch.load(str(checkpoint), map_location=\"cpu\", weights_only=False)\n                if isinstance(state, Mapping) and \"state_dict\" in state:\n                    state = state[\"state_dict\"]\n                state = {str(k).replace(\"module.\", \"\", 1): v for k, v in state.items()}\n                expected = self.backbone.state_dict()\n                if \"conv_stem.weight\" in state and state[\"conv_stem.weight\"].shape[1] == 3:\n                    state[\"conv_stem.weight\"] = state[\"conv_stem.weight\"].sum(dim=1, keepdim=True)\n                state = {key: value for key, value in state.items() if key in expected}\n                shape_errors = {\n                    key: (tuple(value.shape), tuple(expected[key].shape))\n                    for key, value in state.items()\n                    if value.shape != expected[key].shape\n                }\n                if shape_errors:\n                    raise RuntimeError(f\"backbone tensor shape mismatch: {shape_errors}\")\n                result = self.backbone.load_state_dict(state, strict=True)\n                if result.missing_keys or result.unexpected_keys:\n                    raise RuntimeError(f\"backbone state mismatch: {result}\")\n            features, hidden = int(self.backbone.num_features), 384\n            self.attn_v, self.attn_u = nn.Linear(features, hidden), nn.Linear(features, hidden)\n            self.attn_out = nn.Linear(hidden, len(TARGETS))\n            self.target_weight = nn.Parameter(torch.empty(len(TARGETS), features))\n            self.target_bias = nn.Parameter(torch.zeros(len(TARGETS)))\n            self.slice_head = nn.Linear(features, len(TARGETS))\n            self.mix_logit = nn.Parameter(torch.tensor(-1.1))\n            nn.init.xavier_uniform_(self.target_weight)\n\n        def set_backbone_trainable(self, value):\n            for parameter in self.backbone.parameters():\n                parameter.requires_grad = value\n\n        def forward(self, images, mask):\n            batch, planes, slices, channels, height, width = images.shape\n            flat = images.reshape(batch * planes * slices, channels, height, width)\n            features = []\n            train_backbone = self.training and any(p.requires_grad for p in self.backbone.parameters())\n            for start in range(0, len(flat), self.backbone_chunk):\n                chunk = flat[start:start + self.backbone_chunk]\n                if train_backbone:\n                    from torch.utils.checkpoint import checkpoint\n                    features.append(checkpoint(self.backbone, chunk, use_reentrant=False))\n                else:\n                    features.append(self.backbone(chunk))\n            features = torch.cat(features, dim=0)\n            features = features.reshape(batch, planes * slices, -1)\n            flat_mask = mask.reshape(batch, planes * slices)\n            gated = torch.tanh(self.attn_v(features)) * torch.sigmoid(self.attn_u(features))\n            attention = self.attn_out(gated).masked_fill(~flat_mask[:, :, None], -1e4).softmax(dim=1)\n            pooled = torch.einsum(\"bnt,bnf->btf\", attention, features)\n            attention_logits = torch.einsum(\"btf,tf->bt\", pooled, self.target_weight) + self.target_bias\n            max_logits = self.slice_head(features).masked_fill(~flat_mask[:, :, None], -1e4).max(dim=1).values\n            mix = torch.sigmoid(self.mix_logit)\n            return (1.0 - mix) * attention_logits + mix * max_logits\n    return KneeMILModel\n\n\ndef train_one_fold(label_table, series, image_root, output_dir, fold, backbone_checkpoint, config):\n    _, _, _, torch, nn, DataLoader, _ = _training_imports()\n    random.seed(config.seed + fold)\n    np.random.seed(config.seed + fold)\n    torch.manual_seed(config.seed + fold)\n    Dataset, Model = make_dataset_class(), make_model_class()\n    train_rows, valid_rows = prepare_fold_rows(label_table, fold, config)\n    train_loader = DataLoader(Dataset(train_rows, series, image_root, config, True, config.seed + fold), batch_size=config.batch_size, shuffle=True, num_workers=config.workers, pin_memory=True, persistent_workers=config.workers > 0)\n    valid_loader = DataLoader(Dataset(valid_rows, series, image_root, config, False, config.seed + fold), batch_size=1, shuffle=False, num_workers=max(1, config.workers // 2))\n    device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n    if device.type != \"cuda\":\n        raise RuntimeError(\"GPU is required\")\n    core_model = Model(config.backbone, backbone_checkpoint, config.backbone_chunk).to(device)\n    model = nn.DataParallel(core_model) if torch.cuda.device_count() > 1 else core_model\n    loss_fn, scaler, history = nn.BCEWithLogitsLoss(reduction=\"none\"), torch.amp.GradScaler(\"cuda\"), []\n    schedule = [(False, config.frozen_lr)] * config.frozen_epochs + [(True, config.unfrozen_lr)] * config.unfrozen_epochs\n    optimizer, active_phase = None, None\n    for epoch, (unfreeze, lr) in enumerate(schedule):\n        if active_phase != unfreeze:\n            core_model.set_backbone_trainable(unfreeze)\n            optimizer = torch.optim.AdamW(\n                [p for p in model.parameters() if p.requires_grad],\n                lr=lr,\n                weight_decay=config.weight_decay,\n            )\n            active_phase = unfreeze\n        assert optimizer is not None\n        model.train()\n        for layer in core_model.backbone.modules():\n            if isinstance(layer, nn.modules.batchnorm._BatchNorm):\n                layer.eval()\n        losses = []\n        for batch in train_loader:\n            images, mask = batch[\"images\"].to(device), batch[\"mask\"].to(device)\n            target, weight = batch[\"target\"].to(device), batch[\"weight\"].to(device)\n            optimizer.zero_grad(set_to_none=True)\n            with torch.autocast(\"cuda\", dtype=torch.float16):\n                raw = loss_fn(model(images, mask), target)\n                loss = (raw * weight).sum() / weight.sum().clamp_min(1.0)\n            scaler.scale(loss).backward()\n            scaler.unscale_(optimizer)\n            torch.nn.utils.clip_grad_norm_(model.parameters(), 5.0)\n            scaler.step(optimizer)\n            scaler.update()\n            losses.append(float(loss.detach().cpu()))\n        history.append({\n            \"epoch\": epoch,\n            \"train_loss\": float(np.mean(losses)),\n            \"backbone_trainable\": unfreeze,\n            \"learning_rate\": lr,\n        })\n    model.eval()\n    uids, truth, score = [], [], []\n    with torch.no_grad():\n        for batch in valid_loader:\n            with torch.autocast(\"cuda\", dtype=torch.float16):\n                logits = model(batch[\"images\"].to(device), batch[\"mask\"].to(device))\n            uids.extend(batch[\"uid\"])\n            truth.append(batch[\"target\"].numpy())\n            score.append(torch.sigmoid(logits.float()).cpu().numpy())\n    truth, score = np.concatenate(truth), np.concatenate(score)\n    macro, per_target = pooled_macro_auc(truth, score)\n    output_dir.mkdir(parents=True, exist_ok=True)\n    oof = pd.DataFrame({UID: uids})\n    for j, target in enumerate(TARGETS):\n        oof[f\"gold::{target}\"], oof[f\"pred::{target}\"] = truth[:, j], score[:, j]\n    oof.to_csv(output_dir / f\"fold{fold}_expert_oof.csv\", index=False)\n    destination = output_dir / f\"fold{fold}_final.pt\"\n    torch.save({\"model\": core_model.state_dict(), \"backbone\": config.backbone, \"config\": asdict(config), \"fold\": fold, \"fold_diagnostic_macro_auc\": macro, \"fold_diagnostic_per_target_auc\": per_target, \"history\": history, \"visible_gpus\": torch.cuda.device_count(), \"backbone_checkpoint_sha256\": hashlib.sha256(backbone_checkpoint.read_bytes()).hexdigest()}, destination)\n    return destination\n\n\ndef main(argv: Optional[Sequence[str]] = None) -> None:\n    parser = argparse.ArgumentParser(description=__doc__)\n    for name in (\"train-csv\", \"series-csv\", \"pilkwang-labels\", \"steven-labels\", \"lixin-labels\", \"output-dir\"):\n        parser.add_argument(f\"--{name}\", type=Path, required=True)\n    parser.add_argument(\"--image-root\", type=Path)\n    parser.add_argument(\"--backbone-checkpoint\", type=Path)\n    parser.add_argument(\"--fold\", type=int)\n    parser.add_argument(\"--prepare-only\", action=\"store_true\")\n    args = parser.parse_args(argv)\n    config = TrainConfig()\n    paths = [args.train_csv, args.series_csv, args.pilkwang_labels, args.steven_labels, args.lixin_labels]\n    frames = [pd.read_csv(path) for path in paths]\n    labels = build_public_consensus(frames[0], frames[2], frames[3], frames[4], config.pseudo_weight_floor)\n    labels[\"fold\"] = assign_group_balanced_folds(labels, config.folds, config.seed)\n    args.output_dir.mkdir(parents=True, exist_ok=True)\n    labels.to_csv(args.output_dir / \"fold_labels.csv\", index=False)\n    manifest = {\n        \"config\": asdict(config), \"rows\": len(labels), \"expert_rows\": int(labels[\"is_expert\"].sum()),\n        \"fold_counts\": labels[\"fold\"].value_counts().sort_index().to_dict(),\n        \"fold_expert_counts\": labels.loc[labels[\"is_expert\"].eq(1), \"fold\"].value_counts().sort_index().to_dict(),\n        \"inputs\": {str(path): hashlib.sha256(path.read_bytes()).hexdigest() for path in paths},\n    }\n    (args.output_dir / \"manifest.json\").write_text(json.dumps(manifest, indent=2, sort_keys=True) + \"\\n\")\n    print(json.dumps(manifest, indent=2, sort_keys=True))\n    if args.prepare_only:\n        return\n    if args.fold is None or args.image_root is None or args.backbone_checkpoint is None:\n        parser.error(\"training requires --fold, --image-root, and --backbone-checkpoint\")\n    if not 0 <= args.fold < config.folds:\n        parser.error(f\"--fold must be in [0, {config.folds - 1}]\")\n    print(\"saved\", train_one_fold(labels, frames[1], args.image_root, args.output_dir, args.fold, args.backbone_checkpoint, config))\n\n\nif __name__ == \"__main__\":\n    main()\n"},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"%%writefile /kaggle/working/rsna_b3_v42_source/efficientnet_b3_public_repro_v2_anatomy.py\n#!/usr/bin/env python3\n\"\"\"Anatomy-aware EfficientNet-B3 MIL training for RSNA Knee.\n\nThis module deliberately reuses the audited v1 DICOM and label pipeline while\nchanging only three trainable assumptions that v1 could not express:\n\n* each slice feature receives its acquisition-plane and normalized stack position;\n* expert-labelled studies are sampled often enough to influence every epoch;\n* fine-tuning is restricted to the final two EfficientNet stages.\n\nThe output checkpoint contract remains compatible with the v1 inference runner,\nprovided this v2 module is supplied to ``--module`` at inference time.\n\"\"\"\n\nfrom __future__ import annotations\n\nimport argparse\nimport hashlib\nimport importlib.util\nimport json\nimport math\nimport random\nimport sys\nfrom dataclasses import asdict, dataclass, replace\nfrom pathlib import Path\nfrom typing import Mapping, Optional, Sequence\n\nimport numpy as np\nimport pandas as pd\n\n\ndef _load_base_module():\n    path = Path(__file__).with_name(\"efficientnet_b3_public_repro_v1.py\")\n    if not path.is_file():\n        raise FileNotFoundError(f\"audited v1 dependency is absent: {path}\")\n    name = \"rsna_knee_effnet_b3_public_repro_v1_dependency\"\n    spec = importlib.util.spec_from_file_location(name, path)\n    if spec is None or spec.loader is None:\n        raise RuntimeError(f\"cannot import audited v1 dependency: {path}\")\n    module = importlib.util.module_from_spec(spec)\n    sys.modules[name] = module\n    spec.loader.exec_module(module)\n    return module\n\n\nBASE = _load_base_module()\nTARGETS = BASE.TARGETS\nPLANES = BASE.PLANES\nUID = BASE.UID\nTrainConfig = BASE.TrainConfig\nload_dicom_volume = BASE.load_dicom_volume\nuniform_slice_indices = BASE.uniform_slice_indices\nstochastic_slice_indices = BASE.stochastic_slice_indices\npooled_macro_auc = BASE.pooled_macro_auc\n\n\n@dataclass(frozen=True)\nclass TrainingPolicy:\n    \"\"\"Choices specific to v2 and therefore stored separately from TrainConfig.\"\"\"\n\n    expert_sample_fraction: float = 0.12\n    unfrozen_backbone_blocks: int = 2\n    gradient_clip_norm: float = 5.0\n    model_family: str = \"efficientnet_b3_anatomy_mil_v2\"\n\n\ndef _training_imports():\n    return BASE._training_imports()\n\n\ndef pick_series_slots(series: pd.DataFrame, count: int = 3):\n    \"\"\"Return one fixed slot per anatomical plane.\n\n    The v1 selector intentionally backfills a missing plane with another useful\n    series.  That is appropriate for an anatomy-agnostic model, but would attach\n    the wrong plane embedding in v2.  Preserve empty slots instead so index zero,\n    one, and two always mean sagittal, coronal, and axial respectively.\n    \"\"\"\n    if int(count) != len(PLANES):\n        raise ValueError(f\"v2 requires exactly {len(PLANES)} anatomical plane slots\")\n    selected = BASE.pick_one_series_per_plane(series)\n    return [selected[plane] for plane in PLANES]\n\n\ndef make_dataset_class():\n    \"\"\"Build the v1 study dataset with fixed anatomical plane slots.\"\"\"\n    _, _, _, torch, _, _, Dataset = _training_imports()\n\n    class AnatomyAwareKneeStudyDataset(Dataset):\n        def __init__(self, rows, series, image_root, config, training, seed):\n            self.rows = rows.reset_index(drop=True)\n            self.series = {\n                str(uid): group.copy() for uid, group in series.groupby(UID, sort=False)\n            }\n            self.image_root = Path(image_root)\n            self.config = config\n            self.training = bool(training)\n            self.seed = int(seed)\n\n        def __len__(self):\n            return len(self.rows)\n\n        def __getitem__(self, index):\n            row, config = self.rows.iloc[index], self.config\n            uid = str(row[UID])\n            count = config.train_slices if self.training else config.valid_slices\n            visit_seed = (\n                int(torch.randint(0, 2**31 - 1, (1,)).item()) if self.training else 0\n            )\n            rng = np.random.default_rng(self.seed + 1_000_003 * index + visit_seed)\n            empty = pd.DataFrame(\n                columns=[\n                    \"SeriesInstanceUID\",\n                    \"Anatomical_Plane\",\n                    \"Fluid_Sensitive\",\n                    \"Fat_Suppression\",\n                ]\n            )\n            chosen = pick_series_slots(self.series.get(uid, empty), len(PLANES))\n            planes, masks = [], []\n            for series_uid in chosen:\n                if series_uid is None:\n                    planes.append(\n                        np.zeros((count, config.image_size, config.image_size), np.float32)\n                    )\n                    masks.append(np.zeros(count, bool))\n                    continue\n                volume = load_dicom_volume(\n                    self.image_root / uid / series_uid,\n                    config.image_size,\n                    config.max_series_slices,\n                )\n                indices = (\n                    stochastic_slice_indices(len(volume), count, rng)\n                    if self.training\n                    else uniform_slice_indices(len(volume), count)\n                )\n                sampled = volume[indices].astype(np.float32)\n                if self.training:\n                    sampled = np.clip(sampled * rng.uniform(0.85, 1.15), 0, 1)\n                    sampled = sampled ** rng.uniform(0.85, 1.15)\n                    if rng.random() < 0.5:\n                        sampled = sampled[:, :, ::-1].copy()\n                sampled = (sampled - 0.449) / 0.226\n                planes.append(sampled)\n                masks.append(np.ones(count, bool))\n            return {\n                \"uid\": uid,\n                \"images\": torch.from_numpy(np.stack(planes)[:, :, None]),\n                \"mask\": torch.from_numpy(np.stack(masks)),\n                \"target\": torch.tensor(\n                    [row[f\"target::{target}\"] for target in TARGETS], dtype=torch.float32\n                ),\n                \"weight\": torch.tensor(\n                    [row[f\"train_weight::{target}\"] for target in TARGETS],\n                    dtype=torch.float32,\n                ),\n            }\n\n    return AnatomyAwareKneeStudyDataset\n\n\ndef make_model_class():\n    \"\"\"Build a target-attention MIL model with fixed anatomical coordinates.\"\"\"\n    _, _, timm, torch, nn, _, _ = _training_imports()\n\n    class AnatomyAwareKneeMIL(nn.Module):\n        def __init__(self, backbone=\"efficientnet_b3\", checkpoint=None, backbone_chunk=12):\n            super().__init__()\n            self.backbone_chunk = int(backbone_chunk)\n            self.backbone = timm.create_model(\n                backbone,\n                pretrained=False,\n                in_chans=1,\n                num_classes=0,\n                global_pool=\"avg\",\n            )\n            if checkpoint:\n                state = torch.load(str(checkpoint), map_location=\"cpu\", weights_only=False)\n                if isinstance(state, Mapping) and \"state_dict\" in state:\n                    state = state[\"state_dict\"]\n                state = {str(key).replace(\"module.\", \"\", 1): value for key, value in state.items()}\n                expected = self.backbone.state_dict()\n                if \"conv_stem.weight\" in state and state[\"conv_stem.weight\"].shape[1] == 3:\n                    # Sum rather than mean preserves the response magnitude expected by\n                    # the pretrained filters after the one-channel ImageNet normalization.\n                    state[\"conv_stem.weight\"] = state[\"conv_stem.weight\"].sum(dim=1, keepdim=True)\n                state = {key: value for key, value in state.items() if key in expected}\n                shape_errors = {\n                    key: (tuple(value.shape), tuple(expected[key].shape))\n                    for key, value in state.items()\n                    if value.shape != expected[key].shape\n                }\n                if shape_errors:\n                    raise RuntimeError(f\"backbone tensor shape mismatch: {shape_errors}\")\n                result = self.backbone.load_state_dict(state, strict=True)\n                if result.missing_keys or result.unexpected_keys:\n                    raise RuntimeError(f\"backbone state mismatch: {result}\")\n\n            features, hidden = int(self.backbone.num_features), 384\n            self.plane_embedding = nn.Embedding(len(PLANES), features)\n            self.position_projection = nn.Linear(4, features, bias=False)\n            nn.init.normal_(self.plane_embedding.weight, std=0.01)\n            nn.init.normal_(self.position_projection.weight, std=0.01)\n\n            self.attn_v = nn.Linear(features, hidden)\n            self.attn_u = nn.Linear(features, hidden)\n            self.attn_out = nn.Linear(hidden, len(TARGETS))\n            self.target_weight = nn.Parameter(torch.empty(len(TARGETS), features))\n            self.target_bias = nn.Parameter(torch.zeros(len(TARGETS)))\n            self.slice_head = nn.Linear(features, len(TARGETS))\n            self.mix_logit = nn.Parameter(torch.tensor(-1.1))\n            nn.init.xavier_uniform_(self.target_weight)\n\n        def set_backbone_stage(self, final_blocks=0):\n            \"\"\"Freeze the backbone, optionally reopening only its final stages.\"\"\"\n            for parameter in self.backbone.parameters():\n                parameter.requires_grad_(False)\n            final_blocks = int(final_blocks)\n            if final_blocks <= 0:\n                return\n            blocks = list(self.backbone.blocks)\n            if final_blocks > len(blocks):\n                raise ValueError(\n                    f\"requested {final_blocks} trainable blocks from a {len(blocks)}-block backbone\"\n                )\n            for block in blocks[-final_blocks:]:\n                for parameter in block.parameters():\n                    parameter.requires_grad_(True)\n            for name in (\"conv_head\", \"bn2\"):\n                layer = getattr(self.backbone, name, None)\n                if layer is not None:\n                    for parameter in layer.parameters():\n                        parameter.requires_grad_(True)\n\n        @staticmethod\n        def _position_basis(slices, device, dtype):\n            position = torch.linspace(-1.0, 1.0, slices, device=device, dtype=dtype)\n            return torch.stack(\n                (\n                    position,\n                    position.square(),\n                    torch.sin(math.pi * position),\n                    torch.cos(math.pi * position),\n                ),\n                dim=-1,\n            )\n\n        def forward(self, images, mask):\n            batch, planes, slices, channels, height, width = images.shape\n            if planes != len(PLANES):\n                raise ValueError(f\"expected {len(PLANES)} plane slots, got {planes}\")\n            flat = images.reshape(batch * planes * slices, channels, height, width)\n            encoded = []\n            train_backbone = self.training and any(\n                parameter.requires_grad for parameter in self.backbone.parameters()\n            )\n            for start in range(0, len(flat), self.backbone_chunk):\n                chunk = flat[start : start + self.backbone_chunk]\n                if train_backbone:\n                    from torch.utils.checkpoint import checkpoint\n\n                    encoded.append(checkpoint(self.backbone, chunk, use_reentrant=False))\n                else:\n                    encoded.append(self.backbone(chunk))\n            features = torch.cat(encoded, dim=0).reshape(batch, planes, slices, -1)\n\n            plane_ids = torch.arange(planes, device=features.device)\n            plane_context = self.plane_embedding(plane_ids)[None, :, None, :]\n            position = self._position_basis(slices, features.device, features.dtype)\n            position_context = self.position_projection(position)[None, None, :, :]\n            features = features + plane_context + position_context\n            features = features.reshape(batch, planes * slices, -1)\n            flat_mask = mask.reshape(batch, planes * slices)\n\n            gated = torch.tanh(self.attn_v(features)) * torch.sigmoid(self.attn_u(features))\n            attention = self.attn_out(gated).masked_fill(\n                ~flat_mask[:, :, None], -1e4\n            ).softmax(dim=1)\n            pooled = torch.einsum(\"bnt,bnf->btf\", attention, features)\n            attention_logits = (\n                torch.einsum(\"btf,tf->bt\", pooled, self.target_weight) + self.target_bias\n            )\n            max_logits = self.slice_head(features).masked_fill(\n                ~flat_mask[:, :, None], -1e4\n            ).max(dim=1).values\n            mix = torch.sigmoid(self.mix_logit)\n            return (1.0 - mix) * attention_logits + mix * max_logits\n\n    return AnatomyAwareKneeMIL\n\n\ndef _make_expert_sampler(rows, fraction, seed, torch):\n    \"\"\"Sample a fixed expected expert fraction without discarding pseudo rows.\"\"\"\n    if not 0.0 < fraction < 1.0:\n        raise ValueError(\"expert_sample_fraction must be strictly between zero and one\")\n    expert = rows[\"is_expert\"].to_numpy(np.int8).astype(bool)\n    n_expert, n_pseudo = int(expert.sum()), int((~expert).sum())\n    if n_expert == 0 or n_pseudo == 0:\n        raise ValueError(\"expert-aware sampling requires both expert and pseudo-labelled rows\")\n    expert_multiplier = fraction * n_pseudo / ((1.0 - fraction) * n_expert)\n    weights = np.where(expert, expert_multiplier, 1.0).astype(np.float64)\n    generator = torch.Generator().manual_seed(int(seed))\n    from torch.utils.data import WeightedRandomSampler\n\n    sampler = WeightedRandomSampler(\n        torch.from_numpy(weights),\n        num_samples=len(rows),\n        replacement=True,\n        generator=generator,\n    )\n    return sampler, {\n        \"expert_rows\": n_expert,\n        \"pseudo_rows\": n_pseudo,\n        \"expert_multiplier\": float(expert_multiplier),\n        \"expected_expert_fraction\": float(fraction),\n    }\n\n\ndef train_one_fold(\n    label_table,\n    series,\n    image_root,\n    output_dir,\n    fold,\n    backbone_checkpoint,\n    config,\n    policy=TrainingPolicy(),\n):\n    _, _, _, torch, nn, DataLoader, _ = _training_imports()\n    random.seed(config.seed + fold)\n    np.random.seed(config.seed + fold)\n    torch.manual_seed(config.seed + fold)\n    Dataset, Model = make_dataset_class(), make_model_class()\n    train_rows, valid_rows = BASE.prepare_fold_rows(label_table, fold, config)\n    sampler, sampler_meta = _make_expert_sampler(\n        train_rows,\n        policy.expert_sample_fraction,\n        config.seed + 10_007 * fold,\n        torch,\n    )\n    train_loader = DataLoader(\n        Dataset(train_rows, series, image_root, config, True, config.seed + fold),\n        batch_size=config.batch_size,\n        sampler=sampler,\n        num_workers=config.workers,\n        pin_memory=True,\n        persistent_workers=config.workers > 0,\n    )\n    valid_loader = DataLoader(\n        Dataset(valid_rows, series, image_root, config, False, config.seed + fold),\n        batch_size=1,\n        shuffle=False,\n        num_workers=max(1, config.workers // 2),\n    )\n    device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n    if device.type != \"cuda\":\n        raise RuntimeError(\"GPU is required\")\n    core_model = Model(config.backbone, backbone_checkpoint, config.backbone_chunk).to(device)\n    model = nn.DataParallel(core_model) if torch.cuda.device_count() > 1 else core_model\n    loss_fn = nn.BCEWithLogitsLoss(reduction=\"none\")\n    scaler = torch.amp.GradScaler(\"cuda\")\n    history = []\n    schedule = (\n        [(0, config.frozen_lr)] * config.frozen_epochs\n        + [(policy.unfrozen_backbone_blocks, config.unfrozen_lr)] * config.unfrozen_epochs\n    )\n    optimizer, active_blocks = None, None\n    for epoch, (trainable_blocks, learning_rate) in enumerate(schedule):\n        if active_blocks != trainable_blocks:\n            core_model.set_backbone_stage(trainable_blocks)\n            optimizer = torch.optim.AdamW(\n                [parameter for parameter in model.parameters() if parameter.requires_grad],\n                lr=learning_rate,\n                weight_decay=config.weight_decay,\n            )\n            active_blocks = trainable_blocks\n        assert optimizer is not None\n        model.train()\n        for layer in core_model.backbone.modules():\n            if isinstance(layer, nn.modules.batchnorm._BatchNorm):\n                layer.eval()\n        losses = []\n        sampled_expert = 0\n        sampled_total = 0\n        for batch in train_loader:\n            images, mask = batch[\"images\"].to(device), batch[\"mask\"].to(device)\n            target, weight = batch[\"target\"].to(device), batch[\"weight\"].to(device)\n            # Expert rows have the same gold weight for all twelve targets.\n            sampled_expert += int((weight.min(dim=1).values >= config.gold_weight).sum().item())\n            sampled_total += int(len(weight))\n            optimizer.zero_grad(set_to_none=True)\n            with torch.autocast(\"cuda\", dtype=torch.float16):\n                raw = loss_fn(model(images, mask), target)\n                loss = (raw * weight).sum() / weight.sum().clamp_min(1.0)\n            scaler.scale(loss).backward()\n            scaler.unscale_(optimizer)\n            torch.nn.utils.clip_grad_norm_(model.parameters(), policy.gradient_clip_norm)\n            scaler.step(optimizer)\n            scaler.update()\n            losses.append(float(loss.detach().cpu()))\n        history.append(\n            {\n                \"epoch\": epoch,\n                \"train_loss\": float(np.mean(losses)),\n                \"trainable_backbone_blocks\": int(trainable_blocks),\n                \"learning_rate\": float(learning_rate),\n                \"observed_expert_fraction\": sampled_expert / max(sampled_total, 1),\n            }\n        )\n\n    model.eval()\n    uids, truth, score = [], [], []\n    with torch.no_grad():\n        for batch in valid_loader:\n            with torch.autocast(\"cuda\", dtype=torch.float16):\n                logits = model(batch[\"images\"].to(device), batch[\"mask\"].to(device))\n            uids.extend(batch[\"uid\"])\n            truth.append(batch[\"target\"].numpy())\n            score.append(torch.sigmoid(logits.float()).cpu().numpy())\n    truth, score = np.concatenate(truth), np.concatenate(score)\n    macro, per_target = BASE.pooled_macro_auc(truth, score)\n    output_dir.mkdir(parents=True, exist_ok=True)\n    oof = pd.DataFrame({UID: uids})\n    for target_index, target in enumerate(TARGETS):\n        oof[f\"gold::{target}\"] = truth[:, target_index]\n        oof[f\"pred::{target}\"] = score[:, target_index]\n    oof.to_csv(output_dir / f\"fold{fold}_expert_oof.csv\", index=False)\n    destination = output_dir / f\"fold{fold}_final.pt\"\n    torch.save(\n        {\n            \"model\": core_model.state_dict(),\n            \"model_family\": policy.model_family,\n            \"backbone\": config.backbone,\n            \"config\": asdict(config),\n            \"training_policy\": asdict(policy),\n            \"sampler\": sampler_meta,\n            \"fold\": fold,\n            \"fold_diagnostic_macro_auc\": macro,\n            \"fold_diagnostic_per_target_auc\": per_target,\n            \"history\": history,\n            \"visible_gpus\": torch.cuda.device_count(),\n            \"backbone_checkpoint_sha256\": hashlib.sha256(\n                backbone_checkpoint.read_bytes()\n            ).hexdigest(),\n        },\n        destination,\n    )\n    return destination\n\n\ndef main(argv: Optional[Sequence[str]] = None) -> None:\n    parser = argparse.ArgumentParser(description=__doc__)\n    for name in (\n        \"train-csv\",\n        \"series-csv\",\n        \"pilkwang-labels\",\n        \"steven-labels\",\n        \"lixin-labels\",\n        \"output-dir\",\n    ):\n        parser.add_argument(f\"--{name}\", type=Path, required=True)\n    parser.add_argument(\"--image-root\", type=Path)\n    parser.add_argument(\"--backbone-checkpoint\", type=Path)\n    parser.add_argument(\"--fold\", type=int)\n    parser.add_argument(\"--prepare-only\", action=\"store_true\")\n    args = parser.parse_args(argv)\n\n    # Expert oversampling replaces some of the multiplicative gold weighting;\n    # keeping both at their v1 values would overfit the 46-or-so training experts.\n    config = replace(TrainConfig(), gold_weight=2.0)\n    policy = TrainingPolicy()\n    paths = [\n        args.train_csv,\n        args.series_csv,\n        args.pilkwang_labels,\n        args.steven_labels,\n        args.lixin_labels,\n    ]\n    frames = [pd.read_csv(path) for path in paths]\n    labels = BASE.build_public_consensus(\n        frames[0], frames[2], frames[3], frames[4], config.pseudo_weight_floor\n    )\n    labels[\"fold\"] = BASE.assign_group_balanced_folds(labels, config.folds, config.seed)\n    args.output_dir.mkdir(parents=True, exist_ok=True)\n    labels.to_csv(args.output_dir / \"fold_labels.csv\", index=False)\n    manifest = {\n        \"config\": asdict(config),\n        \"training_policy\": asdict(policy),\n        \"rows\": len(labels),\n        \"expert_rows\": int(labels[\"is_expert\"].sum()),\n        \"fold_counts\": labels[\"fold\"].value_counts().sort_index().to_dict(),\n        \"fold_expert_counts\": labels.loc[\n            labels[\"is_expert\"].eq(1), \"fold\"\n        ].value_counts().sort_index().to_dict(),\n        \"inputs\": {str(path): hashlib.sha256(path.read_bytes()).hexdigest() for path in paths},\n        \"base_module_sha256\": hashlib.sha256(\n            Path(BASE.__file__).read_bytes()\n        ).hexdigest(),\n        \"module_sha256\": hashlib.sha256(Path(__file__).read_bytes()).hexdigest(),\n    }\n    (args.output_dir / \"manifest.json\").write_text(\n        json.dumps(manifest, indent=2, sort_keys=True) + \"\\n\"\n    )\n    print(json.dumps(manifest, indent=2, sort_keys=True))\n    if args.prepare_only:\n        return\n    if args.fold is None or args.image_root is None or args.backbone_checkpoint is None:\n        parser.error(\"training requires --fold, --image-root, and --backbone-checkpoint\")\n    if not 0 <= args.fold < config.folds:\n        parser.error(f\"--fold must be in [0, {config.folds - 1}]\")\n    print(\n        \"saved\",\n        train_one_fold(\n            labels,\n            frames[1],\n            args.image_root,\n            args.output_dir,\n            args.fold,\n            args.backbone_checkpoint,\n            config,\n            policy,\n        ),\n    )\n\n\nif __name__ == \"__main__\":\n    main()\n"},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"%%writefile /kaggle/working/rsna_b3_v42_source/efficientnet_b3_public_repro_v3_physical_cache.py\n#!/usr/bin/env python3\n\"\"\"Physical-space, cache-first EfficientNet-B3 MIL training for RSNA Knee.\n\nThis reproducible experiment keeps the public .903 notebook's B3,\ntarget-attention, and five-fold design. Slices are ordered in patient space,\ncropped to a constant physical field of view, normalized to a left-knee\nconvention, and encoded once into a compact float16 feature cache. The gold OOF\nartifact produced by the completed run is the promotion gate.\n\"\"\"\n\nfrom __future__ import annotations\n\nimport argparse\nimport hashlib\nimport importlib.util\nimport json\nimport math\nimport random\nimport re\nimport sys\nfrom dataclasses import asdict, dataclass, replace\nfrom pathlib import Path\nfrom typing import Optional, Sequence\n\nimport numpy as np\nimport pandas as pd\n\n\ndef _load_dependency(filename: str, module_name: str):\n    candidates = (\n        Path(__file__).with_name(filename),\n        Path(__file__).parents[1] / \"research\" / \"rsna_knee_20260808\" / filename,\n    )\n    path = next((candidate for candidate in candidates if candidate.is_file()), None)\n    if path is None:\n        raise FileNotFoundError(f\"audited dependency is absent: {filename}\")\n    spec = importlib.util.spec_from_file_location(module_name, path)\n    if spec is None or spec.loader is None:\n        raise RuntimeError(f\"cannot import audited dependency: {path}\")\n    module = importlib.util.module_from_spec(spec)\n    sys.modules[module_name] = module\n    spec.loader.exec_module(module)\n    return module\n\n\nV2 = _load_dependency(\n    \"efficientnet_b3_public_repro_v2_anatomy.py\",\n    \"rsna_knee_effnet_b3_public_repro_v2_dependency\",\n)\nBASE = V2.BASE\nTARGETS, PLANES, UID = V2.TARGETS, V2.PLANES, V2.UID\nTrainConfig = V2.TrainConfig\n\n\n@dataclass(frozen=True)\nclass PhysicalCachePolicy:\n    crop_mm: float = 130.0\n    slice_band_low: float = 0.10\n    slice_band_high: float = 0.90\n    cache_slices: int = 32\n    laterality_dead_zone_mm: float = 20.0\n    expert_sample_fraction: float = 0.12\n    head_epochs: int = 12\n    head_batch_size: int = 32\n    head_lr: float = 4e-4\n    head_weight_decay: float = 2e-4\n    feature_dropout: float = 0.10\n    gradient_clip_norm: float = 5.0\n    model_family: str = \"efficientnet_b3_physical_cached_mil_v3\"\n\n\ndef _training_imports():\n    return BASE._training_imports()\n\n\ndef sha256_file(path: Path, chunk_size: int = 8 << 20) -> str:\n    digest = hashlib.sha256()\n    with path.open(\"rb\") as handle:\n        for chunk in iter(lambda: handle.read(chunk_size), b\"\"):\n            digest.update(chunk)\n    return digest.hexdigest()\n\n\ndef natural_key(path: Path):\n    return tuple(\n        int(part) if part.isdigit() else part.casefold()\n        for part in re.split(r\"(\\d+)\", path.name)\n    )\n\n\ndef _finite_vector(value, length: int):\n    try:\n        vector = np.asarray(value[:length], dtype=np.float64)\n    except Exception:\n        return None\n    return vector if len(vector) == length and np.isfinite(vector).all() else None\n\n\ndef _read_header(path: Path, pydicom):\n    return pydicom.dcmread(\n        str(path),\n        force=True,\n        stop_before_pixels=True,\n        specific_tags=[\n            \"ImagePositionPatient\",\n            \"ImageOrientationPatient\",\n            \"InstanceNumber\",\n            \"Laterality\",\n            \"ImageLaterality\",\n            \"PixelSpacing\",\n            \"Rows\",\n            \"Columns\",\n        ],\n    )\n\n\ndef ordered_dicom_paths(series_dir: Path):\n    \"\"\"Sort a stack by signed physical position, then by safer fallbacks.\"\"\"\n    _, pydicom, _, _, _, _, _ = _training_imports()\n    files = sorted(series_dir.glob(\"*.dcm\"), key=natural_key)\n    rows = []\n    for original, path in enumerate(files):\n        projection = instance = None\n        try:\n            ds = _read_header(path, pydicom)\n            position = _finite_vector(getattr(ds, \"ImagePositionPatient\", None), 3)\n            orientation = _finite_vector(\n                getattr(ds, \"ImageOrientationPatient\", None), 6\n            )\n            if position is not None and orientation is not None:\n                projection = float(\n                    np.dot(position, np.cross(orientation[:3], orientation[3:]))\n                )\n            if getattr(ds, \"InstanceNumber\", None) is not None:\n                instance = float(ds.InstanceNumber)\n        except Exception:\n            pass\n        rows.append((path, projection, instance, original))\n    needed = max(2, int(math.ceil(0.8 * len(rows))))\n    if sum(row[1] is not None for row in rows) >= needed:\n        fallback = float(np.median([row[1] for row in rows if row[1] is not None]))\n        rows.sort(\n            key=lambda row: (\n                row[1] if row[1] is not None else fallback,\n                row[2] if row[2] is not None else float(\"inf\"),\n                row[3],\n            )\n        )\n    elif sum(row[2] is not None for row in rows) >= needed:\n        rows.sort(\n            key=lambda row: (\n                row[2] if row[2] is not None else float(\"inf\"), row[3]\n            )\n        )\n    return [row[0] for row in rows]\n\n\ndef _series_side_evidence(series_dir: Path):\n    _, pydicom, _, _, _, _, _ = _training_imports()\n    files = sorted(series_dir.glob(\"*.dcm\"), key=natural_key)\n    if not files:\n        return None, None\n    try:\n        ds = _read_header(files[len(files) // 2], pydicom)\n    except Exception:\n        return None, None\n    explicit = None\n    for name in (\"Laterality\", \"ImageLaterality\"):\n        text = str(getattr(ds, name, \"\") or \"\").strip().upper()\n        if text and text[0] in (\"L\", \"R\"):\n            explicit = text[0]\n            break\n    position = _finite_vector(getattr(ds, \"ImagePositionPatient\", None), 3)\n    orientation = _finite_vector(getattr(ds, \"ImageOrientationPatient\", None), 6)\n    spacing = _finite_vector(getattr(ds, \"PixelSpacing\", None), 2)\n    try:\n        rows, columns = float(ds.Rows), float(ds.Columns)\n    except Exception:\n        rows = columns = float(\"nan\")\n    centre_x = None\n    if (\n        position is not None\n        and orientation is not None\n        and spacing is not None\n        and np.isfinite([rows, columns]).all()\n    ):\n        centre = (\n            position\n            + orientation[:3] * spacing[1] * columns / 2.0\n            + orientation[3:] * spacing[0] * rows / 2.0\n        )\n        centre_x = float(centre[0])\n    return explicit, centre_x\n\n\ndef build_series_selection(series: pd.DataFrame):\n    empty = pd.DataFrame(\n        columns=[\n            \"SeriesInstanceUID\",\n            \"Anatomical_Plane\",\n            \"Fluid_Sensitive\",\n            \"Fat_Suppression\",\n        ]\n    )\n    grouped = {\n        str(uid): group.copy() for uid, group in series.groupby(UID, sort=False)\n    }\n    return {\n        str(uid): V2.pick_series_slots(grouped.get(str(uid), empty), len(PLANES))\n        for uid in series[UID].astype(str).drop_duplicates()\n    }\n\n\ndef build_laterality_map(\n    uids: Sequence[str], selection, image_root: Path, dead_zone_mm: float\n):\n    result = {}\n    counts = {\"tag\": 0, \"geometry\": 0, \"unresolved\": 0, \"disagreement\": 0}\n    for uid in map(str, uids):\n        tags, centres = [], []\n        for series_uid in selection.get(uid, [None] * len(PLANES)):\n            if series_uid is None:\n                continue\n            tag, centre = _series_side_evidence(image_root / uid / str(series_uid))\n            if tag is not None:\n                tags.append(tag)\n            if centre is not None and np.isfinite(centre):\n                centres.append(float(centre))\n        tagged = max(set(tags), key=lambda value: (tags.count(value), value)) if tags else None\n        geometric = None\n        if centres:\n            centre = float(np.median(centres))\n            if abs(centre) >= float(dead_zone_mm):\n                geometric = \"R\" if centre < 0 else \"L\"\n        if tagged is not None:\n            result[uid] = tagged\n            counts[\"tag\"] += 1\n            counts[\"disagreement\"] += int(geometric is not None and geometric != tagged)\n        elif geometric is not None:\n            result[uid] = geometric\n            counts[\"geometry\"] += 1\n        else:\n            result[uid] = None\n            counts[\"unresolved\"] += 1\n    return result, counts\n\n\ndef _sample_band_indices(length: int, count: int, low: float, high: float):\n    if length <= 0 or count <= 0 or not 0 <= low < high <= 1:\n        raise ValueError(\"invalid slice-band request\")\n    start, stop = int(round(low * (length - 1))), int(round(high * (length - 1)))\n    if stop <= start:\n        start = stop = length // 2\n    return np.rint(np.linspace(start, stop, count)).astype(np.int64)\n\n\ndef load_physical_volume(\n    series_dir: Path,\n    plane: str,\n    side: Optional[str],\n    image_size: int,\n    count: int,\n    policy: PhysicalCachePolicy,\n):\n    cv2, pydicom, _, _, _, _, _ = _training_imports()\n    files = ordered_dicom_paths(series_dir)\n    if not files:\n        return None\n    indices = _sample_band_indices(\n        len(files), count, policy.slice_band_low, policy.slice_band_high\n    )\n    decoded, metadata = [], []\n    for index in indices:\n        try:\n            ds = pydicom.dcmread(str(files[int(index)]), force=True)\n            image = ds.pixel_array.astype(np.float32)\n            image *= float(getattr(ds, \"RescaleSlope\", 1.0) or 1.0)\n            image += float(getattr(ds, \"RescaleIntercept\", 0.0) or 0.0)\n            if str(getattr(ds, \"PhotometricInterpretation\", \"\")) == \"MONOCHROME1\":\n                image = float(np.max(image)) - image\n            decoded.append(image)\n            metadata.append(_finite_vector(getattr(ds, \"PixelSpacing\", None), 2))\n        except Exception:\n            decoded.append(None)\n            metadata.append(None)\n    good = [index for index, image in enumerate(decoded) if image is not None]\n    if not good:\n        return None\n    for index, image in enumerate(decoded):\n        if image is None:\n            nearest = min(good, key=lambda item: abs(item - index))\n            decoded[index] = decoded[nearest].copy()\n            metadata[index] = metadata[nearest]\n    reference_shape = decoded[good[0]].shape\n    decoded = [\n        image if image.shape == reference_shape else np.zeros(reference_shape, np.float32)\n        for image in decoded\n    ]\n    volume = np.stack(decoded).astype(np.float32, copy=False)\n    spacing = next((value for value in metadata if value is not None), None)\n    if spacing is not None:\n        want_h = int(round(policy.crop_mm / float(spacing[0])))\n        want_w = int(round(policy.crop_mm / float(spacing[1])))\n        height, width = reference_shape\n        if 16 < want_h <= height and 16 < want_w <= width:\n            top, left = (height - want_h) // 2, (width - want_w) // 2\n            volume = volume[:, top : top + want_h, left : left + want_w]\n    lo, hi = np.percentile(volume, [1.0, 99.0])\n    hi = max(float(hi), float(lo) + 1.0)\n    volume = np.clip((volume - float(lo)) / (hi - float(lo)), 0.0, 1.0)\n    resized = np.stack(\n        [cv2.resize(image, (image_size, image_size), interpolation=cv2.INTER_AREA) for image in volume]\n    )\n    resized = np.rint(resized * 255.0).astype(np.uint8).astype(np.float32) / 255.0\n    if side == \"R\":\n        if plane in (\"Coronal\", \"Axial\"):\n            resized = resized[:, :, ::-1].copy()\n        elif plane == \"Sagittal\":\n            resized = resized[::-1].copy()\n    return resized\n\n\ndef make_physical_dataset_class():\n    _, _, _, torch, _, _, Dataset = _training_imports()\n\n    class PhysicalStudyDataset(Dataset):\n        def __init__(self, uids, selection, laterality, image_root, config, policy):\n            self.uids = list(map(str, uids))\n            self.selection = selection\n            self.laterality = laterality\n            self.image_root = Path(image_root)\n            self.config = config\n            self.policy = policy\n\n        def __len__(self):\n            return len(self.uids)\n\n        def __getitem__(self, index):\n            uid = self.uids[index]\n            slots, masks = [], []\n            for plane, series_uid in zip(\n                PLANES, self.selection.get(uid, [None] * len(PLANES))\n            ):\n                volume = None\n                if series_uid is not None:\n                    volume = load_physical_volume(\n                        self.image_root / uid / str(series_uid),\n                        plane,\n                        self.laterality.get(uid),\n                        self.config.image_size,\n                        self.policy.cache_slices,\n                        self.policy,\n                    )\n                if volume is None:\n                    volume = np.zeros(\n                        (self.policy.cache_slices, self.config.image_size, self.config.image_size),\n                        np.float32,\n                    )\n                    masks.append(np.zeros(self.policy.cache_slices, bool))\n                else:\n                    masks.append(np.ones(self.policy.cache_slices, bool))\n                slots.append((volume - 0.449) / 0.226)\n            return {\n                \"uid\": uid,\n                \"images\": torch.from_numpy(np.stack(slots)[:, :, None]),\n                \"mask\": torch.from_numpy(np.stack(masks)),\n            }\n\n    return PhysicalStudyDataset\n\n\ndef make_model_class(feature_dropout: float = 0.0):\n    \"\"\"Add separate image encoding and MIL-head entry points to v2.\"\"\"\n    _, _, _, torch, _, _, _ = _training_imports()\n    Parent = V2.make_model_class()\n\n    class PhysicalCachedKneeMIL(Parent):\n        def __init__(self, *args, **kwargs):\n            super().__init__(*args, **kwargs)\n            self.feature_dropout = float(feature_dropout)\n\n        def encode_images(self, images):\n            batch, planes, slices, channels, height, width = images.shape\n            if planes != len(PLANES):\n                raise ValueError(f\"expected {len(PLANES)} plane slots, got {planes}\")\n            flat = images.reshape(batch * planes * slices, channels, height, width)\n            encoded = []\n            for start in range(0, len(flat), self.backbone_chunk):\n                encoded.append(self.backbone(flat[start : start + self.backbone_chunk]))\n            return torch.cat(encoded, dim=0).reshape(batch, planes, slices, -1)\n\n        def forward_encoded(self, features, mask):\n            batch, planes, slices, width = features.shape\n            if planes != len(PLANES) or mask.shape != (batch, planes, slices):\n                raise ValueError(\"feature/mask anatomy contract mismatch\")\n            if self.training and self.feature_dropout > 0:\n                features = torch.nn.functional.dropout(\n                    features, p=self.feature_dropout, training=True\n                )\n            plane_ids = torch.arange(planes, device=features.device)\n            features = features + self.plane_embedding(plane_ids)[None, :, None, :]\n            basis = self._position_basis(slices, features.device, features.dtype)\n            features = features + self.position_projection(basis)[None, None, :, :]\n            features = features.reshape(batch, planes * slices, width)\n            flat_mask = mask.reshape(batch, planes * slices)\n            gated = torch.tanh(self.attn_v(features)) * torch.sigmoid(\n                self.attn_u(features)\n            )\n            attention = self.attn_out(gated).masked_fill(\n                ~flat_mask[:, :, None], -1e4\n            ).softmax(dim=1)\n            pooled = torch.einsum(\"bnt,bnf->btf\", attention, features)\n            attention_logits = (\n                torch.einsum(\"btf,tf->bt\", pooled, self.target_weight)\n                + self.target_bias\n            )\n            max_logits = self.slice_head(features).masked_fill(\n                ~flat_mask[:, :, None], -1e4\n            ).max(dim=1).values\n            mix = torch.sigmoid(self.mix_logit)\n            return (1.0 - mix) * attention_logits + mix * max_logits\n\n        def forward(self, images, mask):\n            return self.forward_encoded(self.encode_images(images), mask)\n\n    return PhysicalCachedKneeMIL\n\n\ndef build_feature_cache(\n    uids,\n    series,\n    image_root,\n    cache_dir,\n    backbone_checkpoint,\n    config,\n    policy,\n):\n    _, _, _, torch, _, DataLoader, _ = _training_imports()\n    if not torch.cuda.is_available():\n        raise RuntimeError(\"GPU is required to build the B3 feature cache\")\n    cache_dir.mkdir(parents=True, exist_ok=True)\n    uids = list(map(str, uids))\n    selection = build_series_selection(series)\n    laterality, laterality_meta = build_laterality_map(\n        uids, selection, Path(image_root), policy.laterality_dead_zone_mm\n    )\n    Dataset, Model = make_physical_dataset_class(), make_model_class()\n    loader = DataLoader(\n        Dataset(uids, selection, laterality, image_root, config, policy),\n        batch_size=1,\n        shuffle=False,\n        num_workers=config.workers,\n        pin_memory=True,\n        persistent_workers=config.workers > 0,\n    )\n    model = Model(config.backbone, backbone_checkpoint, 32).cuda().eval()\n    model.set_backbone_stage(0)\n    width = int(model.backbone.num_features)\n    feature_path = cache_dir / \"features.float16.npy\"\n    mask_path = cache_dir / \"mask.bool.npy\"\n    features = np.lib.format.open_memmap(\n        feature_path,\n        mode=\"w+\",\n        dtype=np.float16,\n        shape=(len(uids), len(PLANES), policy.cache_slices, width),\n    )\n    masks = np.lib.format.open_memmap(\n        mask_path,\n        mode=\"w+\",\n        dtype=np.bool_,\n        shape=(len(uids), len(PLANES), policy.cache_slices),\n    )\n    observed = []\n    with torch.inference_mode():\n        for index, batch in enumerate(loader):\n            with torch.autocast(\"cuda\", dtype=torch.float16):\n                encoded = model.encode_images(batch[\"images\"].cuda(non_blocking=True))\n            features[index] = encoded[0].float().cpu().numpy().astype(np.float16)\n            masks[index] = batch[\"mask\"][0].numpy()\n            observed.extend(batch[\"uid\"])\n            if (index + 1) % 100 == 0 or index + 1 == len(uids):\n                features.flush()\n                masks.flush()\n                print(f\"cached {index + 1}/{len(uids)} studies\", flush=True)\n    if observed != uids:\n        raise AssertionError(\"feature cache UID order drift\")\n    pd.DataFrame({UID: uids}).to_csv(cache_dir / \"uids.csv\", index=False)\n    manifest = {\n        \"status\": \"COMPLETE\",\n        \"rows\": len(uids),\n        \"shape\": list(features.shape),\n        \"dtype\": str(features.dtype),\n        \"mask_shape\": list(masks.shape),\n        \"present_plane_fraction\": float(masks[:, :, 0].mean()),\n        \"laterality\": laterality_meta,\n        \"backbone_checkpoint_sha256\": sha256_file(Path(backbone_checkpoint)),\n        \"features_sha256\": sha256_file(feature_path),\n        \"mask_sha256\": sha256_file(mask_path),\n        \"uids_sha256\": sha256_file(cache_dir / \"uids.csv\"),\n        \"policy\": asdict(policy),\n    }\n    (cache_dir / \"cache_manifest.json\").write_text(\n        json.dumps(manifest, indent=2, sort_keys=True) + \"\\n\"\n    )\n    return manifest\n\n\ndef make_feature_dataset_class():\n    _, _, _, torch, _, _, Dataset = _training_imports()\n\n    class CachedFeatureDataset(Dataset):\n        def __init__(self, rows, cache_dir):\n            self.rows = rows.reset_index(drop=True)\n            self.cache_dir = Path(cache_dir)\n            order = pd.read_csv(self.cache_dir / \"uids.csv\")[UID].astype(str).tolist()\n            lookup = {uid: index for index, uid in enumerate(order)}\n            missing = set(self.rows[UID].astype(str)).difference(lookup)\n            if missing:\n                raise ValueError(f\"feature cache misses {len(missing)} studies\")\n            self.cache_indices = np.asarray(\n                [lookup[uid] for uid in self.rows[UID].astype(str)], dtype=np.int64\n            )\n            self.features = np.load(\n                self.cache_dir / \"features.float16.npy\", mmap_mode=\"r\"\n            )\n            self.masks = np.load(self.cache_dir / \"mask.bool.npy\", mmap_mode=\"r\")\n\n        def __len__(self):\n            return len(self.rows)\n\n        def __getitem__(self, index):\n            row = self.rows.iloc[index]\n            cache_index = int(self.cache_indices[index])\n            return {\n                \"uid\": str(row[UID]),\n                \"features\": torch.from_numpy(\n                    np.array(self.features[cache_index], dtype=np.float32, copy=True)\n                ),\n                \"mask\": torch.from_numpy(\n                    np.array(self.masks[cache_index], dtype=bool, copy=True)\n                ),\n                \"target\": torch.tensor(\n                    [row[f\"target::{target}\"] for target in TARGETS],\n                    dtype=torch.float32,\n                ),\n                \"weight\": torch.tensor(\n                    [row[f\"train_weight::{target}\"] for target in TARGETS],\n                    dtype=torch.float32,\n                ),\n            }\n\n    return CachedFeatureDataset\n\n\ndef _head_state(model):\n    return {\n        name: value.detach().cpu()\n        for name, value in model.state_dict().items()\n        if not name.startswith(\"backbone.\")\n    }\n\n\ndef train_cached_fold(\n    labels,\n    cache_dir,\n    output_dir,\n    fold,\n    backbone_checkpoint,\n    config,\n    policy,\n):\n    _, _, _, torch, nn, DataLoader, _ = _training_imports()\n    if not torch.cuda.is_available():\n        raise RuntimeError(\"GPU is required for cached-head training\")\n    seed = config.seed + 10_007 * int(fold)\n    random.seed(seed)\n    np.random.seed(seed)\n    torch.manual_seed(seed)\n    train_rows, valid_rows = BASE.prepare_fold_rows(labels, fold, config)\n    Dataset = make_feature_dataset_class()\n    Model = make_model_class(policy.feature_dropout)\n    sampler, sampler_meta = V2._make_expert_sampler(\n        train_rows, policy.expert_sample_fraction, seed, torch\n    )\n    train_loader = DataLoader(\n        Dataset(train_rows, cache_dir),\n        batch_size=policy.head_batch_size,\n        sampler=sampler,\n        num_workers=0,\n        pin_memory=True,\n    )\n    valid_loader = DataLoader(\n        Dataset(valid_rows, cache_dir),\n        batch_size=policy.head_batch_size,\n        shuffle=False,\n        num_workers=0,\n        pin_memory=True,\n    )\n    model = Model(config.backbone, backbone_checkpoint, 32).cuda()\n    model.set_backbone_stage(0)\n    parameters = [\n        parameter\n        for name, parameter in model.named_parameters()\n        if not name.startswith(\"backbone.\") and parameter.requires_grad\n    ]\n    optimizer = torch.optim.AdamW(\n        parameters, lr=policy.head_lr, weight_decay=policy.head_weight_decay\n    )\n    scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(\n        optimizer, T_max=policy.head_epochs, eta_min=policy.head_lr * 0.05\n    )\n    loss_fn = nn.BCEWithLogitsLoss(reduction=\"none\")\n    scaler = torch.amp.GradScaler(\"cuda\")\n    history = []\n    last_truth = last_score = last_uids = None\n    for epoch in range(policy.head_epochs):\n        model.train()\n        losses, sampled_expert, sampled_total = [], 0, 0\n        for batch in train_loader:\n            features = batch[\"features\"].cuda(non_blocking=True)\n            mask = batch[\"mask\"].cuda(non_blocking=True)\n            target = batch[\"target\"].cuda(non_blocking=True)\n            weight = batch[\"weight\"].cuda(non_blocking=True)\n            sampled_expert += int(\n                (weight.min(dim=1).values >= config.gold_weight).sum().item()\n            )\n            sampled_total += len(weight)\n            optimizer.zero_grad(set_to_none=True)\n            with torch.autocast(\"cuda\", dtype=torch.float16):\n                raw = loss_fn(model.forward_encoded(features, mask), target)\n                loss = (raw * weight).sum() / weight.sum().clamp_min(1.0)\n            scaler.scale(loss).backward()\n            scaler.unscale_(optimizer)\n            torch.nn.utils.clip_grad_norm_(parameters, policy.gradient_clip_norm)\n            scaler.step(optimizer)\n            scaler.update()\n            losses.append(float(loss.detach().cpu()))\n        scheduler.step()\n        model.eval()\n        uids, truth, score = [], [], []\n        with torch.inference_mode():\n            for batch in valid_loader:\n                with torch.autocast(\"cuda\", dtype=torch.float16):\n                    logits = model.forward_encoded(\n                        batch[\"features\"].cuda(non_blocking=True),\n                        batch[\"mask\"].cuda(non_blocking=True),\n                    )\n                uids.extend(batch[\"uid\"])\n                truth.append(batch[\"target\"].numpy())\n                score.append(torch.sigmoid(logits.float()).cpu().numpy())\n        last_truth = np.concatenate(truth)\n        last_score = np.concatenate(score)\n        last_uids = uids\n        macro, per_target = BASE.pooled_macro_auc(last_truth, last_score)\n        history.append(\n            {\n                \"epoch\": epoch,\n                \"train_loss\": float(np.mean(losses)),\n                \"learning_rate\": float(optimizer.param_groups[0][\"lr\"]),\n                \"observed_expert_fraction\": sampled_expert / max(sampled_total, 1),\n                \"gold_validation_macro_auc\": macro,\n                \"gold_validation_per_target_auc\": per_target,\n            }\n        )\n        print(\n            f\"fold {fold} epoch {epoch + 1}/{policy.head_epochs}: \"\n            f\"loss={history[-1]['train_loss']:.5f} gold_auc={macro:.5f}\",\n            flush=True,\n        )\n    if last_truth is None or last_score is None or last_uids is None:\n        raise AssertionError(\"fold produced no validation predictions\")\n    macro, per_target = BASE.pooled_macro_auc(last_truth, last_score)\n    output_dir.mkdir(parents=True, exist_ok=True)\n    oof = pd.DataFrame({UID: last_uids, \"fold\": int(fold)})\n    for target_index, target in enumerate(TARGETS):\n        oof[f\"gold::{target}\"] = last_truth[:, target_index]\n        oof[f\"pred::{target}\"] = last_score[:, target_index]\n    oof.to_csv(output_dir / f\"fold{fold}_expert_oof.csv\", index=False)\n    destination = output_dir / f\"fold{fold}_head.pt\"\n    torch.save(\n        {\n            \"head_state\": _head_state(model),\n            \"model_family\": policy.model_family,\n            \"backbone\": config.backbone,\n            \"config\": asdict(config),\n            \"physical_cache_policy\": asdict(policy),\n            \"sampler\": sampler_meta,\n            \"fold\": int(fold),\n            \"fold_diagnostic_macro_auc\": macro,\n            \"fold_diagnostic_per_target_auc\": per_target,\n            \"history\": history,\n            \"backbone_checkpoint_sha256\": sha256_file(Path(backbone_checkpoint)),\n        },\n        destination,\n    )\n    return destination\n\n\ndef prepare_labels(args, config):\n    paths = [\n        args.train_csv,\n        args.series_csv,\n        args.pilkwang_labels,\n        args.steven_labels,\n        args.lixin_labels,\n    ]\n    frames = [pd.read_csv(path) for path in paths]\n    labels = BASE.build_public_consensus(\n        frames[0], frames[2], frames[3], frames[4], config.pseudo_weight_floor\n    )\n    labels[\"fold\"] = BASE.assign_group_balanced_folds(\n        labels, config.folds, config.seed\n    )\n    return paths, frames, labels\n\n\ndef main(argv: Optional[Sequence[str]] = None) -> None:\n    parser = argparse.ArgumentParser(description=__doc__)\n    for name in (\n        \"train-csv\",\n        \"series-csv\",\n        \"pilkwang-labels\",\n        \"steven-labels\",\n        \"lixin-labels\",\n        \"output-dir\",\n    ):\n        parser.add_argument(f\"--{name}\", type=Path, required=True)\n    parser.add_argument(\"--image-root\", type=Path)\n    parser.add_argument(\"--backbone-checkpoint\", type=Path)\n    parser.add_argument(\"--cache-dir\", type=Path)\n    parser.add_argument(\"--prepare-only\", action=\"store_true\")\n    parser.add_argument(\"--cache-only\", action=\"store_true\")\n    parser.add_argument(\"--folds\", default=\"0,1,2,3,4\")\n    args = parser.parse_args(argv)\n\n    config = replace(\n        TrainConfig(),\n        train_slices=32,\n        valid_slices=32,\n        batch_size=32,\n        workers=4,\n        frozen_epochs=0,\n        unfrozen_epochs=0,\n        gold_weight=2.0,\n    )\n    policy = PhysicalCachePolicy()\n    paths, frames, labels = prepare_labels(args, config)\n    args.output_dir.mkdir(parents=True, exist_ok=True)\n    labels.to_csv(args.output_dir / \"fold_labels.csv\", index=False)\n    manifest = {\n        \"config\": asdict(config),\n        \"physical_cache_policy\": asdict(policy),\n        \"rows\": len(labels),\n        \"expert_rows\": int(labels[\"is_expert\"].sum()),\n        \"fold_counts\": labels[\"fold\"].value_counts().sort_index().to_dict(),\n        \"fold_expert_counts\": labels.loc[\n            labels[\"is_expert\"].eq(1), \"fold\"\n        ].value_counts().sort_index().to_dict(),\n        \"inputs\": {str(path): sha256_file(path) for path in paths},\n        \"base_module_sha256\": sha256_file(Path(BASE.__file__)),\n        \"v2_module_sha256\": sha256_file(Path(V2.__file__)),\n        \"module_sha256\": sha256_file(Path(__file__)),\n    }\n    (args.output_dir / \"manifest.json\").write_text(\n        json.dumps(manifest, indent=2, sort_keys=True) + \"\\n\"\n    )\n    print(json.dumps(manifest, indent=2, sort_keys=True))\n    if args.prepare_only:\n        return\n    if args.image_root is None or args.backbone_checkpoint is None:\n        parser.error(\"training requires --image-root and --backbone-checkpoint\")\n    cache_dir = args.cache_dir or args.output_dir / \"feature_cache\"\n    cache_manifest = cache_dir / \"cache_manifest.json\"\n    if not cache_manifest.is_file():\n        build_feature_cache(\n            labels[UID].astype(str).tolist(),\n            frames[1],\n            args.image_root,\n            cache_dir,\n            args.backbone_checkpoint,\n            config,\n            policy,\n        )\n    if args.cache_only:\n        return\n    folds = [int(value) for value in args.folds.split(\",\") if value.strip()]\n    if not folds or any(fold < 0 or fold >= config.folds for fold in folds):\n        parser.error(f\"--folds must select values from [0, {config.folds - 1}]\")\n    checkpoints = [\n        train_cached_fold(\n            labels,\n            cache_dir,\n            args.output_dir,\n            fold,\n            args.backbone_checkpoint,\n            config,\n            policy,\n        )\n        for fold in folds\n    ]\n    oof = pd.concat(\n        [pd.read_csv(args.output_dir / f\"fold{fold}_expert_oof.csv\") for fold in folds],\n        ignore_index=True,\n    )\n    if len(folds) == config.folds and (\n        len(oof) != int(labels[\"is_expert\"].sum()) or oof[UID].duplicated().any()\n    ):\n        raise AssertionError(\"five-fold expert OOF coverage is incomplete\")\n    truth = oof[[f\"gold::{target}\" for target in TARGETS]].to_numpy(float)\n    score = oof[[f\"pred::{target}\" for target in TARGETS]].to_numpy(float)\n    macro, per_target = BASE.pooled_macro_auc(truth, score)\n    summary = {\n        \"status\": \"COMPLETE\",\n        \"folds\": folds,\n        \"expert_oof_rows\": len(oof),\n        \"expert_oof_macro_auc\": macro,\n        \"expert_oof_per_target_auc\": per_target,\n        \"checkpoints\": {path.name: sha256_file(path) for path in checkpoints},\n        \"cache_manifest_sha256\": sha256_file(cache_manifest),\n    }\n    oof.to_csv(args.output_dir / \"expert_oof.csv\", index=False)\n    (args.output_dir / \"training_summary.json\").write_text(\n        json.dumps(summary, indent=2, sort_keys=True) + \"\\n\"\n    )\n    print(json.dumps(summary, indent=2, sort_keys=True))\n\n\nif __name__ == \"__main__\":\n    main()\n"},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"%%writefile /kaggle/working/rsna_b3_v42_source/efficientnet_b3_public_repro_v3_infer.py\n#!/usr/bin/env python3\n\"\"\"Five-head inference for the physical cached EfficientNet-B3 v3 model.\"\"\"\n\nfrom __future__ import annotations\n\nimport argparse\nimport hashlib\nimport importlib.util\nimport json\nimport sys\nimport time\nfrom dataclasses import asdict\nfrom pathlib import Path\nfrom typing import Optional, Sequence\n\nimport numpy as np\nimport pandas as pd\n\n\ndef load_module(path: Path):\n    name = \"rsna_knee_effnet_b3_public_repro_v3_inference_dependency\"\n    spec = importlib.util.spec_from_file_location(name, path)\n    if spec is None or spec.loader is None:\n        raise RuntimeError(f\"cannot import model module: {path}\")\n    module = importlib.util.module_from_spec(spec)\n    sys.modules[name] = module\n    spec.loader.exec_module(module)\n    return module\n\n\ndef sha256_file(path: Path, chunk_size: int = 8 << 20) -> str:\n    digest = hashlib.sha256()\n    with path.open(\"rb\") as handle:\n        for chunk in iter(lambda: handle.read(chunk_size), b\"\"):\n            digest.update(chunk)\n    return digest.hexdigest()\n\n\ndef load_ensemble(module, checkpoint_paths, backbone_checkpoint, device):\n    \"\"\"Load one shared encoder and five lightweight heads with strict contracts.\"\"\"\n    _, _, _, torch, _, _, _ = module._training_imports()\n    checkpoints = [\n        torch.load(str(path), map_location=\"cpu\", weights_only=False)\n        for path in checkpoint_paths\n    ]\n    if len(checkpoints) != 5:\n        raise ValueError(f\"expected five fold heads, got {len(checkpoints)}\")\n    folds = [int(checkpoint.get(\"fold\", -1)) for checkpoint in checkpoints]\n    if sorted(folds) != list(range(5)):\n        raise ValueError(f\"fold heads must be exactly 0..4, got {folds}\")\n    configs = [checkpoint[\"config\"] for checkpoint in checkpoints]\n    policies = [checkpoint[\"physical_cache_policy\"] for checkpoint in checkpoints]\n    if any(config != configs[0] for config in configs[1:]):\n        raise ValueError(\"fold configurations disagree\")\n    if any(policy != policies[0] for policy in policies[1:]):\n        raise ValueError(\"fold physical-cache policies disagree\")\n    expected_backbone_hash = sha256_file(Path(backbone_checkpoint))\n    recorded_hashes = {\n        str(checkpoint.get(\"backbone_checkpoint_sha256\", \"\"))\n        for checkpoint in checkpoints\n    }\n    if recorded_hashes != {expected_backbone_hash}:\n        raise ValueError(\n            \"public backbone hash disagrees with one or more fold checkpoints\"\n        )\n\n    config = module.TrainConfig(**configs[0])\n    policy = module.PhysicalCachePolicy(**policies[0])\n    Model = module.make_model_class(policy.feature_dropout)\n    models = []\n    for index, checkpoint in enumerate(checkpoints):\n        model = Model(\n            config.backbone,\n            backbone_checkpoint if index == 0 else None,\n            config.backbone_chunk,\n        )\n        expected_missing = {\n            name\n            for name in model.state_dict()\n            if name.startswith(\"backbone.\")\n            and not name.endswith(\".num_batches_tracked\")\n        }\n        result = model.load_state_dict(checkpoint[\"head_state\"], strict=False)\n        if set(result.missing_keys) != expected_missing or result.unexpected_keys:\n            raise RuntimeError(\n                f\"fold {folds[index]} head-state contract mismatch: {result}\"\n            )\n        if index > 0:\n            model.backbone = torch.nn.Identity()\n        model.to(device).eval()\n        models.append(model)\n    return models[0], models, config, policy, folds, expected_backbone_hash\n\n\ndef predict(\n    module,\n    test: pd.DataFrame,\n    series: pd.DataFrame,\n    image_root: Path,\n    encoder,\n    heads,\n    config,\n    policy,\n    device,\n    checkpoint_every: int,\n):\n    _, _, _, torch, _, DataLoader, _ = module._training_imports()\n    uids = test[module.UID].astype(str).tolist()\n    if not uids or len(set(uids)) != len(uids):\n        raise ValueError(\"test StudyInstanceUID values must be nonempty and unique\")\n    selection = module.build_series_selection(series)\n    laterality, laterality_meta = module.build_laterality_map(\n        uids, selection, image_root, policy.laterality_dead_zone_mm\n    )\n    Dataset = module.make_physical_dataset_class()\n    loader = DataLoader(\n        Dataset(uids, selection, laterality, image_root, config, policy),\n        batch_size=1,\n        shuffle=False,\n        num_workers=config.workers,\n        pin_memory=True,\n        persistent_workers=config.workers > 0,\n    )\n    scores = np.empty((len(uids), len(module.TARGETS)), dtype=np.float32)\n    observed = []\n    started = time.monotonic()\n    with torch.inference_mode():\n        for index, batch in enumerate(loader):\n            images = batch[\"images\"].to(device, non_blocking=True)\n            mask = batch[\"mask\"].to(device, non_blocking=True)\n            with torch.autocast(device_type=\"cuda\", dtype=torch.float16):\n                features = encoder.encode_images(images)\n                probabilities = torch.stack(\n                    [\n                        torch.sigmoid(head.forward_encoded(features, mask).float())\n                        for head in heads\n                    ]\n                ).mean(dim=0)\n            scores[index] = probabilities[0].cpu().numpy()\n            observed.extend(batch[\"uid\"])\n            done = index + 1\n            if done % checkpoint_every == 0 or done == len(uids):\n                elapsed = time.monotonic() - started\n                projected_hours = elapsed / done * len(uids) / 3600.0\n                print(\n                    f\"predicted {done}/{len(uids)} studies; \"\n                    f\"projected={projected_hours:.2f}h\",\n                    flush=True,\n                )\n    if observed != uids:\n        raise AssertionError(\"inference UID order drift\")\n    if not np.isfinite(scores).all() or not ((0.0 <= scores) & (scores <= 1.0)).all():\n        raise AssertionError(\"inference produced invalid probabilities\")\n    return scores, laterality_meta, time.monotonic() - started\n\n\ndef main(argv: Optional[Sequence[str]] = None) -> None:\n    parser = argparse.ArgumentParser(description=__doc__)\n    parser.add_argument(\"--module\", type=Path, required=True)\n    parser.add_argument(\"--test-csv\", type=Path, required=True)\n    parser.add_argument(\"--series-csv\", type=Path, required=True)\n    parser.add_argument(\"--image-root\", type=Path, required=True)\n    parser.add_argument(\"--backbone-checkpoint\", type=Path, required=True)\n    parser.add_argument(\"--head-checkpoints\", type=Path, nargs=5, required=True)\n    parser.add_argument(\"--output-dir\", type=Path, required=True)\n    parser.add_argument(\"--checkpoint-every\", type=int, default=25)\n    args = parser.parse_args(argv)\n    if args.checkpoint_every <= 0:\n        parser.error(\"--checkpoint-every must be positive\")\n\n    module = load_module(args.module)\n    _, _, _, torch, _, _, _ = module._training_imports()\n    if not torch.cuda.is_available():\n        raise RuntimeError(\"GPU is required for EfficientNet-B3 inference\")\n    device = torch.device(\"cuda\")\n    encoder, heads, config, policy, folds, backbone_hash = load_ensemble(\n        module, args.head_checkpoints, args.backbone_checkpoint, device\n    )\n    test, series = pd.read_csv(args.test_csv), pd.read_csv(args.series_csv)\n    scores, laterality, elapsed = predict(\n        module,\n        test,\n        series,\n        args.image_root,\n        encoder,\n        heads,\n        config,\n        policy,\n        device,\n        args.checkpoint_every,\n    )\n    submission = pd.DataFrame(scores, columns=module.TARGETS)\n    submission.insert(0, module.UID, test[module.UID].astype(str).tolist())\n    if list(submission.columns) != [module.UID, *module.TARGETS]:\n        raise AssertionError(\"submission schema drift\")\n    args.output_dir.mkdir(parents=True, exist_ok=True)\n    destination = args.output_dir / \"submission.csv\"\n    submission.to_csv(destination, index=False)\n    audit = {\n        \"status\": \"COMPLETE\",\n        \"rows\": len(submission),\n        \"finite\": True,\n        \"folds\": folds,\n        \"elapsed_seconds\": elapsed,\n        \"laterality\": laterality,\n        \"config\": asdict(config),\n        \"physical_cache_policy\": asdict(policy),\n        \"backbone_checkpoint_sha256\": backbone_hash,\n        \"head_checkpoint_sha256\": {\n            path.name: sha256_file(path) for path in args.head_checkpoints\n        },\n        \"submission_sha256\": sha256_file(destination),\n        \"module_sha256\": sha256_file(args.module),\n        \"inference_runner_sha256\": sha256_file(Path(__file__)),\n    }\n    (args.output_dir / \"inference_audit.json\").write_text(\n        json.dumps(audit, indent=2, sort_keys=True) + \"\\n\"\n    )\n    print(json.dumps(audit, indent=2, sort_keys=True))\n\n\nif __name__ == \"__main__\":\n    main()\n"},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"# Guarded B3 training branch.  The V40 public specialist remains the fallback.\nimport hashlib as _b3_hashlib\nimport json as _b3_json\nimport shutil as _b3_shutil\nimport subprocess as _b3_subprocess\nimport sys as _b3_sys\nimport traceback as _b3_traceback\nfrom pathlib import Path as _B3Path\n\n_B3_SOURCE = _B3Path(\"/kaggle/working/rsna_b3_v42_source\")\n_B3_TRAIN = _B3Path(\"/kaggle/working/rsna_b3_v42_training\")\n_B3_INFER = _B3Path(\"/kaggle/working/rsna_b3_v42_inference\")\n_B3_CACHE = _B3Path(\"/kaggle/temp/rsna_b3_v42_cache\")\n_B3_COMP = _B3Path(\"/kaggle/input/rsna-knee-abnormality-detection\")\n_B3_BASELINE = _B3Path(\"/kaggle/working/submission_public_0899.csv\")\n_B3_COMPETITION_RUNTIME_LIMIT_SECONDS = 9 * 60 * 60\n_B3_TRAIN_TIMEOUT_SECONDS = 5 * 60 * 60\n_B3_INFER_TIMEOUT_SECONDS = 2 * 60 * 60\n\n_B3_EXPECTED = {\n    _B3_SOURCE / \"efficientnet_b3_public_repro_v1.py\": \"2548480302b175aa15700888902835c91c377deec935a4edfe2a6a978e3ac232\",\n    _B3_SOURCE / \"efficientnet_b3_public_repro_v2_anatomy.py\": \"17a8a8c99afc02e6e75928cf7db2d014609d31d1280a1f05b07f15564c198696\",\n    _B3_SOURCE / \"efficientnet_b3_public_repro_v3_physical_cache.py\": \"bcd3f3193a75a6138fb0d229649872af85ffe7b0676eb8c6725fcc112fbbe2d5\",\n    _B3_SOURCE / \"efficientnet_b3_public_repro_v3_infer.py\": \"350c3a3c39201e8b26534fd311a5c2c306cd7dbed15392784b05f4d0bfb3ba69\",\n    _B3_COMP / \"train.csv\": \"8ca2203c0e9d61c080c7a314c7cdb51c1b03a1d9eb4770819f7f34af53ef4e33\",\n    _B3_COMP / \"train_series.csv\": \"573c1d80772bf41211c91b149c95677385a1c22d63f485c347f1b46c0177aef3\",\n    _B3Path(\"/kaggle/input/rsna-knee-llm-labels/report_labels_v2.csv\"): \"6f704a7bdb2f894cc49445b19ba7c4378c3f548d3449e00361e10044bee40920\",\n    _B3Path(\"/kaggle/input/rsna-knee-llm-report-labels/llm_labels_v2.csv\"): \"3c082a987939528a55202028f655bd6f9dc5c9e28eff5058bbae7ef088a9f144\",\n    _B3Path(\"/kaggle/input/rsna-knee-llm-report-labels-sol56/labels_llm_gpt56sol.csv\"): \"a79b05e609336811d51c97f9a48f00ea8004244a1a80fe886504a62bb39867ae\",\n    _B3Path(\"/kaggle/input/timm-pretrained-weights/efficientnet_b3.pth\"): \"b2fd73dffb7386bf35ed9f571e98c1ab53647ed435268eb629c307ad221b5b31\",\n}\n\ndef _b3_sha(path):\n    digest = _b3_hashlib.sha256()\n    with open(path, \"rb\") as handle:\n        for chunk in iter(lambda: handle.read(8 << 20), b\"\"):\n            digest.update(chunk)\n    return digest.hexdigest()\n\n_b3_audit = {\n    \"status\": \"FALLBACK\",\n    \"evidence_boundary\": \"The B3 branch is experimental until an official completed submission supplies a score.\",\n    \"competition_runtime_limit_seconds\": _B3_COMPETITION_RUNTIME_LIMIT_SECONDS,\n    \"train_timeout_seconds\": _B3_TRAIN_TIMEOUT_SECONDS,\n    \"infer_timeout_seconds\": _B3_INFER_TIMEOUT_SECONDS,\n    \"expected_inputs\": {str(path): value for path, value in _B3_EXPECTED.items()},\n}\n\ntry:\n    _B3_SOURCE.mkdir(parents=True, exist_ok=True)\n    _B3_TRAIN.mkdir(parents=True, exist_ok=True)\n    _B3_INFER.mkdir(parents=True, exist_ok=True)\n    if not _B3_BASELINE.is_file():\n        raise FileNotFoundError(\"the independently reproduced public .899 fallback is absent\")\n    _b3_observed = {str(path): _b3_sha(path) for path in _B3_EXPECTED}\n    _b3_bad = {\n        str(path): {\"expected\": expected, \"observed\": _b3_observed[str(path)]}\n        for path, expected in _B3_EXPECTED.items()\n        if _b3_observed[str(path)] != expected\n    }\n    if _b3_bad:\n        raise RuntimeError(f\"pinned B3 input drift: {_b3_bad}\")\n\n    _b3_train_cmd = [\n        _b3_sys.executable,\n        str(_B3_SOURCE / \"efficientnet_b3_public_repro_v3_physical_cache.py\"),\n        \"--train-csv\", str(_B3_COMP / \"train.csv\"),\n        \"--series-csv\", str(_B3_COMP / \"train_series.csv\"),\n        \"--image-root\", str(_B3_COMP / \"train_series\"),\n        \"--pilkwang-labels\", \"/kaggle/input/rsna-knee-llm-labels/report_labels_v2.csv\",\n        \"--steven-labels\", \"/kaggle/input/rsna-knee-llm-report-labels/llm_labels_v2.csv\",\n        \"--lixin-labels\", \"/kaggle/input/rsna-knee-llm-report-labels-sol56/labels_llm_gpt56sol.csv\",\n        \"--backbone-checkpoint\", \"/kaggle/input/timm-pretrained-weights/efficientnet_b3.pth\",\n        \"--cache-dir\", str(_B3_CACHE),\n        \"--output-dir\", str(_B3_TRAIN),\n        \"--folds\", \"0,1,2,3,4\",\n    ]\n    print(\"[B3 V42] building one frozen B3 feature cache and five MIL heads\", flush=True)\n    _b3_subprocess.run(\n        _b3_train_cmd,\n        check=True,\n        timeout=_B3_TRAIN_TIMEOUT_SECONDS,\n    )\n    _b3_summary = _b3_json.loads((_B3_TRAIN / \"training_summary.json\").read_text())\n    if _b3_summary.get(\"status\") != \"COMPLETE\":\n        raise RuntimeError(f\"B3 training is not terminal: {_b3_summary}\")\n    if int(_b3_summary.get(\"expert_oof_rows\", -1)) != 58:\n        raise RuntimeError(\"B3 expert OOF coverage is not exactly 58 unique studies\")\n    _b3_oof_auc = float(_b3_summary[\"expert_oof_macro_auc\"])\n\n    _b3_infer_cmd = [\n        _b3_sys.executable,\n        str(_B3_SOURCE / \"efficientnet_b3_public_repro_v3_infer.py\"),\n        \"--module\", str(_B3_SOURCE / \"efficientnet_b3_public_repro_v3_physical_cache.py\"),\n        \"--test-csv\", str(_B3_COMP / \"test.csv\"),\n        \"--series-csv\", str(_B3_COMP / \"test_series.csv\"),\n        \"--image-root\", str(_B3_COMP / \"test_series\"),\n        \"--backbone-checkpoint\", \"/kaggle/input/timm-pretrained-weights/efficientnet_b3.pth\",\n        \"--head-checkpoints\",\n        *[str(_B3_TRAIN / f\"fold{fold}_head.pt\") for fold in range(5)],\n        \"--output-dir\", str(_B3_INFER),\n    ]\n    print(f\"[B3 V42] honest 58-study OOF macro AUC={_b3_oof_auc:.6f}; inferring\", flush=True)\n    _b3_subprocess.run(\n        _b3_infer_cmd,\n        check=True,\n        timeout=_B3_INFER_TIMEOUT_SECONDS,\n    )\n\n    _b3_base = pd.read_csv(_B3_BASELINE)\n    _b3_pred = pd.read_csv(_B3_INFER / \"submission.csv\")\n    _b3_cols = list(_b3_base.columns)\n    if _b3_cols != [UID, *TARGETS] or list(_b3_pred.columns) != _b3_cols:\n        raise RuntimeError(\"B3/baseline submission schema drift\")\n    if _b3_base[UID].astype(str).tolist() != _b3_pred[UID].astype(str).tolist():\n        raise RuntimeError(\"B3/baseline UID order drift\")\n    _b3_raw = _b3_pred[TARGETS].to_numpy(float)\n    _b3_base_raw = _b3_base[TARGETS].to_numpy(float)\n    if not (np.isfinite(_b3_raw).all() and np.isfinite(_b3_base_raw).all()):\n        raise RuntimeError(\"B3/baseline contains non-finite probabilities\")\n    # Macro AUC depends on ordering, not calibration.  Rank both parents target by\n    # target before mixing so a compressed soft-label head cannot be numerically\n    # drowned out by a more dispersed parent with the same ordering quality.\n    _b3_rank = _b3_pred[TARGETS].rank(method=\"average\", pct=True).to_numpy(float)\n    _b3_base_rank = _b3_base[TARGETS].rank(method=\"average\", pct=True).to_numpy(float)\n\n    _b3_standalone = _b3_pred.copy()\n    _b3_standalone.to_csv(\"/kaggle/working/submission_b3_v42.csv\", index=False)\n    _b3_variants = {}\n    for _b3_alpha in (0.25, 0.50, 0.75):\n        _b3_blend = _b3_base.copy()\n        _b3_blend[TARGETS] = (1.0 - _b3_alpha) * _b3_base_rank + _b3_alpha * _b3_rank\n        _b3_name = f\"submission_b3_blend_{int(100 * _b3_alpha):02d}.csv\"\n        _b3_blend.to_csv(_B3Path(\"/kaggle/working\") / _b3_name, index=False)\n        _b3_variants[_b3_name] = _b3_sha(_B3Path(\"/kaggle/working\") / _b3_name)\n\n    # The scored reference reached .903 with a B3 specialist.  Use a conservative\n    # 75% B3 / 25% independently reproduced public specialist rank blend only when the honest\n    # expert OOF clears a prespecified 0.84 gate; otherwise preserve the fallback.\n    if _b3_oof_auc >= 0.84:\n        _b3_shutil.copy2(\n            \"/kaggle/working/submission_b3_blend_75.csv\",\n            \"/kaggle/working/submission.csv\",\n        )\n        _b3_promoted = \"submission_b3_blend_75.csv\"\n    else:\n        _b3_shutil.copy2(_B3_BASELINE, \"/kaggle/working/submission.csv\")\n        _b3_promoted = _B3_BASELINE.name\n    _b3_audit.update({\n        \"status\": \"COMPLETE\",\n        \"expert_oof_macro_auc\": _b3_oof_auc,\n        \"promotion_gate\": 0.84,\n        \"promoted\": _b3_promoted,\n        \"primary_sha256\": _b3_sha(\"/kaggle/working/submission.csv\"),\n        \"standalone_sha256\": _b3_sha(\"/kaggle/working/submission_b3_v42.csv\"),\n        \"blend_sha256\": _b3_variants,\n        \"observed_inputs\": _b3_observed,\n    })\nexcept Exception as _b3_error:\n    print(f\"[B3 V42] rejected safely: {type(_b3_error).__name__}: {_b3_error}\", flush=True)\n    _b3_traceback.print_exc()\n    if _B3_BASELINE.is_file():\n        _b3_shutil.copy2(_B3_BASELINE, \"/kaggle/working/submission.csv\")\n        _b3_audit[\"primary_sha256\"] = _b3_sha(\"/kaggle/working/submission.csv\")\n    _b3_audit[\"error\"] = f\"{type(_b3_error).__name__}: {_b3_error}\"\n\n_B3Path(\"/kaggle/working/b3_v42_audit.json\").write_text(\n    _b3_json.dumps(_b3_audit, indent=2, sort_keys=True) + \"\\n\"\n)\nprint(_b3_json.dumps(_b3_audit, indent=2, sort_keys=True), flush=True)\n"},{"cell_type":"markdown","metadata":{},"source":"## 11. Running B3 when Kaggle assigns an incompatible P100\n\nThe `.903` reference is not a reusable output file: its decisive five checkpoints are private. This retry keeps the same independent public inputs but probes actual CUDA execution and uses a bounded 192px, four-slice-per-plane CPU cache when the assigned GPU is incompatible. It encodes each study once with a frozen public EfficientNet-B3, trains five target-attention MIL heads on three-source soft-label consensus, and checks all 58 expert studies strictly out of fold. Physical DICOM ordering, a 130 mm crop, laterality normalization, three planes, and 32 slices are explicit contracts.\n\nThe branch is fail-safe. The independently reproduced public `.899` artifact is produced first and remains `submission.csv` if any input hash, training contract, 58-study OOF audit, or inference schema fails. Even a successful local OOF result is diagnostic evidence only; only a completed competition row can establish a public score."},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"from pathlib import Path as _B3SourcePath\n_B3SourcePath('/kaggle/working/rsna_b3_v43_source').mkdir(parents=True, exist_ok=True)\n"},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"%%writefile /kaggle/working/rsna_b3_v43_source/efficientnet_b3_public_repro_v1.py\n#!/usr/bin/env python3\n\"\"\"Leakage-controlled EfficientNet-B3 MIL training for RSNA Knee.\"\"\"\n\nfrom __future__ import annotations\n\nimport argparse\nimport hashlib\nimport json\nimport math\nimport random\nimport re\nimport unicodedata\nfrom dataclasses import asdict, dataclass\nfrom pathlib import Path\nfrom typing import Dict, List, Mapping, Optional, Sequence, Tuple\n\nimport numpy as np\nimport pandas as pd\n\nTARGETS: Tuple[str, ...] = (\n    \"ACL\", \"MCL\", \"Medial Meniscus\", \"Lateral Meniscus\", \"Medial OA\",\n    \"Lateral OA\", \"PF OA\", \"Effusion\", \"Synovitis\", \"Baker's\",\n    \"Contusion\", \"Fracture\",\n)\nPLANES: Tuple[str, ...] = (\"Sagittal\", \"Coronal\", \"Axial\")\nUID = \"StudyInstanceUID\"\n\n\n@dataclass(frozen=True)\nclass TrainConfig:\n    image_size: int = 288\n    max_series_slices: int = 64\n    train_slices: int = 20\n    valid_slices: int = 32\n    folds: int = 5\n    seed: int = 20260809\n    batch_size: int = 2\n    workers: int = 4\n    backbone_chunk: int = 12\n    frozen_epochs: int = 1\n    unfrozen_epochs: int = 2\n    frozen_lr: float = 8e-4\n    unfrozen_lr: float = 8e-5\n    weight_decay: float = 1e-4\n    gold_weight: float = 3.0\n    pseudo_weight_floor: float = 0.20\n    backbone: str = \"efficientnet_b3\"\n\n\n_DATE = re.compile(\n    r\"\\b(?:19|20)\\d{2}[-/.]\\d{1,2}[-/.]\\d{1,2}\\b|\"\n    r\"\\b\\d{1,2}[-/.]\\d{1,2}[-/.](?:\\d{2}|\\d{4})\\b\"\n)\n_NUMBER = re.compile(r\"\\b\\d+(?:[.,]\\d+)?\\b\")\n_SPACE = re.compile(r\"\\s+\")\n\n\ndef normalize_report_for_group(value: object) -> str:\n    text = \"\" if value is None else str(value)\n    text = unicodedata.normalize(\"NFKC\", text).casefold()\n    text = _DATE.sub(\" <date> \", text)\n    text = _NUMBER.sub(\" <num> \", text)\n    return _SPACE.sub(\" \", text).strip()\n\n\ndef report_group_id(value: object) -> str:\n    return hashlib.sha256(normalize_report_for_group(value).encode()).hexdigest()[:24]\n\n\ndef _indexed_labels(frame: pd.DataFrame, name: str) -> pd.DataFrame:\n    required = {UID, *TARGETS}\n    missing = required.difference(frame.columns)\n    if missing:\n        raise ValueError(f\"{name} is missing columns: {sorted(missing)}\")\n    if frame[UID].duplicated().any():\n        raise ValueError(f\"{name} contains duplicate {UID} rows\")\n    return frame[[UID, *TARGETS]].copy().set_index(UID).apply(pd.to_numeric, errors=\"coerce\")\n\n\ndef build_public_consensus(\n    train: pd.DataFrame,\n    pilkwang: pd.DataFrame,\n    steven: pd.DataFrame,\n    lixin: pd.DataFrame,\n    pseudo_weight_floor: float = 0.20,\n) -> pd.DataFrame:\n    \"\"\"Equal-source soft labels; weights come only from agreement/certainty.\"\"\"\n    if train[UID].duplicated().any():\n        raise ValueError(f\"train contains duplicate {UID} rows\")\n    base = train[[UID, \"Report\", *TARGETS]].copy().set_index(UID)\n    sources = [\n        _indexed_labels(pilkwang, \"pilkwang\"),\n        _indexed_labels(steven, \"steven\"),\n        _indexed_labels(lixin, \"lixin\"),\n    ]\n    out = base.reset_index()[[UID, \"Report\"]].copy()\n    out[\"report_group\"] = out[\"Report\"].map(report_group_id)\n    out = out.drop(columns=\"Report\")\n    cube = np.stack([source.reindex(base.index).to_numpy(float) for source in sources])\n    if np.nanmin(cube) < 0.0 or np.nanmax(cube) > 1.0:\n        raise ValueError(\"public labels must be in [0, 1]\")\n    available = np.isfinite(cube).sum(axis=0)\n    if np.any(available < 2):\n        row, target = np.argwhere(available < 2)[0]\n        raise ValueError(f\"fewer than two public labels at row={row}, target={TARGETS[target]}\")\n    consensus = np.nanmean(cube, axis=0)\n    disagreement = np.nanmean(np.abs(cube - consensus[None]), axis=0)\n    agreement = np.clip(1.0 - 2.0 * disagreement, 0.0, 1.0)\n    certainty = np.clip(2.0 * np.abs(consensus - 0.5), 0.0, 1.0)\n    weight = pseudo_weight_floor + (1.0 - pseudo_weight_floor) * (\n        0.65 * agreement + 0.35 * certainty\n    )\n    out[\"is_expert\"] = base[list(TARGETS)].notna().all(axis=1).astype(np.int8).to_numpy()\n    for j, target in enumerate(TARGETS):\n        out[f\"pseudo::{target}\"] = consensus[:, j].astype(np.float32)\n        out[f\"weight::{target}\"] = weight[:, j].astype(np.float32)\n        out[f\"gold::{target}\"] = pd.to_numeric(base[target], errors=\"coerce\").to_numpy()\n    return out\n\n\ndef _group_statistics(label_table: pd.DataFrame) -> pd.DataFrame:\n    pseudo_cols = [f\"pseudo::{target}\" for target in TARGETS]\n    gold_cols = [f\"gold::{target}\" for target in TARGETS]\n    records = []\n    for group_id, group in label_table.groupby(\"report_group\", sort=True):\n        gold = group[gold_cols].to_numpy(float)\n        records.append({\n            \"report_group\": group_id,\n            \"size\": len(group),\n            \"expert_count\": int(group[\"is_expert\"].sum()),\n            \"pseudo_sum\": group[pseudo_cols].to_numpy(float).sum(axis=0),\n            \"gold_positive\": np.nansum(gold, axis=0),\n            \"gold_known\": np.isfinite(gold).sum(axis=0).astype(float),\n        })\n    return pd.DataFrame(records)\n\n\ndef assign_group_balanced_folds(\n    label_table: pd.DataFrame, n_splits: int = 5, seed: int = 20260809\n) -> pd.Series:\n    \"\"\"Deterministic greedy balance with normalized-report groups kept atomic.\"\"\"\n    if n_splits < 2:\n        raise ValueError(\"n_splits must be at least two\")\n    groups = _group_statistics(label_table)\n    target_size = float(groups[\"size\"].sum()) / n_splits\n    target_pseudo = np.sum(np.stack(groups[\"pseudo_sum\"]), axis=0) / n_splits\n    total_gold_pos = np.sum(np.stack(groups[\"gold_positive\"]), axis=0)\n    target_gold_pos = total_gold_pos / n_splits\n    target_gold_known = np.sum(np.stack(groups[\"gold_known\"]), axis=0) / n_splits\n    rng = np.random.default_rng(seed)\n    rarity = 1.0 / np.maximum(total_gold_pos, 1.0)\n    priority = [\n        100.0 * row[\"expert_count\"]\n        + 10.0 * float(np.dot(row[\"gold_positive\"], rarity))\n        + math.log1p(row[\"size\"])\n        for _, row in groups.iterrows()\n    ]\n    groups = groups.assign(_priority=priority, _jitter=rng.uniform(0, 1e-6, len(groups)))\n    groups = groups.sort_values([\"_priority\", \"_jitter\", \"report_group\"], ascending=[False, False, True])\n    fold_size = np.zeros(n_splits)\n    fold_pseudo = np.zeros((n_splits, len(TARGETS)))\n    fold_gold_pos = np.zeros_like(fold_pseudo)\n    fold_gold_known = np.zeros_like(fold_pseudo)\n    assignment: Dict[str, int] = {}\n    for _, row in groups.iterrows():\n        scores = []\n        for fold in range(n_splits):\n            candidate_size = fold_size.copy()\n            candidate_pseudo = fold_pseudo.copy()\n            candidate_gold_pos = fold_gold_pos.copy()\n            candidate_gold_known = fold_gold_known.copy()\n            candidate_size[fold] += row[\"size\"]\n            candidate_pseudo[fold] += row[\"pseudo_sum\"]\n            candidate_gold_pos[fold] += row[\"gold_positive\"]\n            candidate_gold_known[fold] += row[\"gold_known\"]\n            size_cost = np.mean(((candidate_size - target_size) / max(target_size, 1.0)) ** 2)\n            pseudo_cost = np.mean(((candidate_pseudo - target_pseudo[None]) / np.maximum(target_pseudo[None], 1.0)) ** 2)\n            gold_pos_cost = np.mean(((candidate_gold_pos - target_gold_pos[None]) / np.maximum(target_gold_pos[None], 1.0)) ** 2)\n            gold_known_cost = np.mean(((candidate_gold_known - target_gold_known[None]) / np.maximum(target_gold_known[None], 1.0)) ** 2)\n            scores.append(size_cost + 0.35 * pseudo_cost + 4.0 * gold_pos_cost + 2.0 * gold_known_cost)\n        best = int(np.argmin(np.asarray(scores) + 1e-12 * np.arange(n_splits)))\n        assignment[str(row[\"report_group\"])] = best\n        fold_size[best] += row[\"size\"]\n        fold_pseudo[best] += row[\"pseudo_sum\"]\n        fold_gold_pos[best] += row[\"gold_positive\"]\n        fold_gold_known[best] += row[\"gold_known\"]\n    result = label_table[\"report_group\"].map(assignment)\n    if result.isna().any():\n        raise AssertionError(\"some report groups were not assigned\")\n    return result.astype(np.int8)\n\n\ndef pick_one_series_per_plane(series: pd.DataFrame) -> Dict[str, Optional[str]]:\n    required = {\"SeriesInstanceUID\", \"Anatomical_Plane\", \"Fluid_Sensitive\", \"Fat_Suppression\"}\n    missing = required.difference(series.columns)\n    if missing:\n        raise ValueError(f\"series table missing columns: {sorted(missing)}\")\n    chosen: Dict[str, Optional[str]] = {}\n    for plane in PLANES:\n        subset = series.loc[series[\"Anatomical_Plane\"].eq(plane)].copy()\n        if subset.empty:\n            chosen[plane] = None\n            continue\n        for column in (\"Fluid_Sensitive\", \"Fat_Suppression\"):\n            subset[column] = pd.to_numeric(subset[column], errors=\"coerce\").fillna(0)\n        subset = subset.sort_values(\n            [\"Fluid_Sensitive\", \"Fat_Suppression\", \"SeriesInstanceUID\"],\n            ascending=[False, False, True],\n        )\n        chosen[plane] = str(subset.iloc[0][\"SeriesInstanceUID\"])\n    return chosen\n\n\ndef pick_series_slots(series: pd.DataFrame, count: int = 3) -> List[Optional[str]]:\n    \"\"\"Prefer one fluid-sensitive series per plane, then fill empty slots.\"\"\"\n    required = {\"SeriesInstanceUID\", \"Anatomical_Plane\", \"Fluid_Sensitive\", \"Fat_Suppression\"}\n    missing = required.difference(series.columns)\n    if missing:\n        raise ValueError(f\"series table missing columns: {sorted(missing)}\")\n    frame = series.copy()\n    for column in (\"Fluid_Sensitive\", \"Fat_Suppression\"):\n        frame[column] = pd.to_numeric(frame[column], errors=\"coerce\").fillna(0)\n    frame[\"_plane\"] = frame[\"Anatomical_Plane\"].map({p: i for i, p in enumerate(PLANES)}).fillna(3)\n    preferred = frame.loc[frame[\"Fluid_Sensitive\"].eq(1)].sort_values(\n        [\"_plane\", \"Fat_Suppression\", \"SeriesInstanceUID\"],\n        ascending=[True, False, True],\n    )\n    selected: List[str] = []\n    seen_planes = set()\n    for _, row in preferred.iterrows():\n        plane = str(row[\"Anatomical_Plane\"])\n        if plane not in seen_planes:\n            selected.append(str(row[\"SeriesInstanceUID\"]))\n            seen_planes.add(plane)\n        if len(selected) == count:\n            break\n    ordered = frame.sort_values(\n        [\"Fluid_Sensitive\", \"Fat_Suppression\", \"_plane\", \"SeriesInstanceUID\"],\n        ascending=[False, False, True, True],\n    )\n    for series_uid in ordered[\"SeriesInstanceUID\"].astype(str):\n        if len(selected) == count:\n            break\n        if series_uid not in selected:\n            selected.append(series_uid)\n    return selected + [None] * (count - len(selected))\n\n\ndef uniform_slice_indices(length: int, count: int) -> np.ndarray:\n    if length <= 0 or count <= 0:\n        raise ValueError(\"length and count must be positive\")\n    if length <= count:\n        return np.concatenate([np.arange(length), np.full(count - length, length - 1)]).astype(np.int64)\n    return np.rint(np.linspace(0, length - 1, count)).astype(np.int64)\n\n\ndef stochastic_slice_indices(length: int, count: int, rng: np.random.Generator) -> np.ndarray:\n    if length <= count:\n        return uniform_slice_indices(length, count)\n    edges = np.linspace(0, length, count + 1)\n    values = []\n    for left, right in zip(edges[:-1], edges[1:]):\n        lo = int(math.floor(left))\n        hi = max(lo + 1, int(math.ceil(right)))\n        values.append(int(rng.integers(lo, min(hi, length))))\n    return np.asarray(values, dtype=np.int64)\n\n\ndef pooled_macro_auc(y_true: np.ndarray, y_score: np.ndarray) -> Tuple[float, Dict[str, float]]:\n    from sklearn.metrics import roc_auc_score\n    per_target: Dict[str, float] = {}\n    for j, target in enumerate(TARGETS):\n        valid = np.isfinite(y_true[:, j]) & np.isfinite(y_score[:, j])\n        per_target[target] = (\n            float(roc_auc_score(y_true[valid, j], y_score[valid, j]))\n            if valid.sum() and np.unique(y_true[valid, j]).size == 2\n            else float(\"nan\")\n        )\n    finite = [value for value in per_target.values() if np.isfinite(value)]\n    return (float(np.mean(finite)) if finite else float(\"nan\")), per_target\n\n\ndef prepare_fold_rows(\n    label_table: pd.DataFrame, fold: int, config: TrainConfig\n) -> Tuple[pd.DataFrame, pd.DataFrame]:\n    train_rows = label_table.loc[label_table[\"fold\"].ne(fold)].copy()\n    valid_rows = label_table.loc[label_table[\"fold\"].eq(fold) & label_table[\"is_expert\"].eq(1)].copy()\n    if train_rows.empty or valid_rows.empty:\n        raise ValueError(f\"empty train/validation split for fold {fold}\")\n    for target in TARGETS:\n        gold = pd.to_numeric(train_rows[f\"gold::{target}\"], errors=\"coerce\")\n        is_gold = train_rows[\"is_expert\"].eq(1) & gold.notna()\n        train_rows[f\"target::{target}\"] = train_rows[f\"pseudo::{target}\"].where(~is_gold, gold)\n        train_rows[f\"train_weight::{target}\"] = train_rows[f\"weight::{target}\"].where(~is_gold, config.gold_weight)\n        valid_rows[f\"target::{target}\"] = pd.to_numeric(valid_rows[f\"gold::{target}\"], errors=\"raise\")\n        valid_rows[f\"train_weight::{target}\"] = 1.0\n    if set(train_rows[\"report_group\"]).intersection(valid_rows[\"report_group\"]):\n        raise AssertionError(\"normalized report leakage across train and validation\")\n    return train_rows, valid_rows\n\n\ndef _training_imports():\n    try:\n        import cv2\n        import pydicom\n        import timm\n        import torch\n        import torch.nn as nn\n        from torch.utils.data import DataLoader, Dataset\n    except ImportError as exc:\n        raise RuntimeError(\"training requires torch, timm, pydicom, and opencv-python\") from exc\n    return cv2, pydicom, timm, torch, nn, DataLoader, Dataset\n\n\ndef load_dicom_volume(series_dir: Path, image_size: int, max_slices: int = 64) -> np.ndarray:\n    cv2, pydicom, _, _, _, _, _ = _training_imports()\n    slices: List[Tuple[int, np.ndarray]] = []\n    for path in sorted(series_dir.glob(\"*.dcm\")):\n        try:\n            ds = pydicom.dcmread(str(path))\n            image = ds.pixel_array.astype(np.float32)\n            image = image * float(getattr(ds, \"RescaleSlope\", 1.0) or 1.0)\n            image += float(getattr(ds, \"RescaleIntercept\", 0.0) or 0.0)\n            if str(getattr(ds, \"PhotometricInterpretation\", \"\")) == \"MONOCHROME1\":\n                image = image.max() - image\n            slices.append((int(getattr(ds, \"InstanceNumber\", 0) or 0), image))\n        except Exception:\n            continue\n    if not slices:\n        raise RuntimeError(f\"no decodable slices in {series_dir}\")\n    slices.sort(key=lambda item: item[0])\n    volume = np.stack([item[1] for item in slices])\n    lo, hi = np.percentile(volume, [1.0, 99.0])\n    hi = max(float(hi), float(lo) + 1.0)\n    volume = np.clip((volume - lo) / (hi - lo), 0.0, 1.0)\n    if len(volume) > max_slices:\n        start = (len(volume) - max_slices) // 2\n        volume = volume[start:start + max_slices]\n    resized = np.stack([\n        cv2.resize(x, (image_size, image_size), interpolation=cv2.INTER_AREA)\n        for x in volume\n    ])\n    # Match the public .903 pipeline's training-time uint8 ETL boundary.\n    return (resized * 255.0).astype(np.uint8).astype(np.float32) / 255.0\n\n\ndef make_dataset_class():\n    _, _, _, torch, _, _, Dataset = _training_imports()\n    class KneeStudyDataset(Dataset):\n        def __init__(self, rows, series, image_root, config, training, seed):\n            self.rows = rows.reset_index(drop=True)\n            self.series = {str(uid): group.copy() for uid, group in series.groupby(UID, sort=False)}\n            self.image_root, self.config, self.training, self.seed = Path(image_root), config, training, seed\n\n        def __len__(self):\n            return len(self.rows)\n\n        def __getitem__(self, index):\n            row, config = self.rows.iloc[index], self.config\n            uid = str(row[UID])\n            count = config.train_slices if self.training else config.valid_slices\n            # DataLoader seeds each worker deterministically.  Drawing once from\n            # its advancing torch RNG gives a fresh augmentation on every visit\n            # without sacrificing run-level reproducibility.\n            visit_seed = int(torch.randint(0, 2**31 - 1, (1,)).item()) if self.training else 0\n            rng = np.random.default_rng(self.seed + 1_000_003 * index + visit_seed)\n            empty = pd.DataFrame(columns=[\"SeriesInstanceUID\", \"Anatomical_Plane\", \"Fluid_Sensitive\", \"Fat_Suppression\"])\n            chosen = pick_series_slots(self.series.get(uid, empty), len(PLANES))\n            planes, masks = [], []\n            for series_uid in chosen:\n                if series_uid is None:\n                    planes.append(np.zeros((count, config.image_size, config.image_size), np.float32))\n                    masks.append(np.zeros(count, bool))\n                    continue\n                volume = load_dicom_volume(\n                    self.image_root / uid / series_uid,\n                    config.image_size,\n                    config.max_series_slices,\n                )\n                idx = stochastic_slice_indices(len(volume), count, rng) if self.training else uniform_slice_indices(len(volume), count)\n                sampled = volume[idx].astype(np.float32)\n                if self.training:\n                    sampled = np.clip(sampled * rng.uniform(0.85, 1.15), 0, 1) ** rng.uniform(0.85, 1.15)\n                    if rng.random() < 0.5:\n                        sampled = sampled[:, :, ::-1].copy()\n                sampled = (sampled - 0.449) / 0.226\n                planes.append(sampled)\n                masks.append(np.ones(count, bool))\n            return {\n                \"uid\": uid,\n                \"images\": torch.from_numpy(np.stack(planes)[:, :, None]),\n                \"mask\": torch.from_numpy(np.stack(masks)),\n                \"target\": torch.tensor([row[f\"target::{x}\"] for x in TARGETS], dtype=torch.float32),\n                \"weight\": torch.tensor([row[f\"train_weight::{x}\"] for x in TARGETS], dtype=torch.float32),\n            }\n    return KneeStudyDataset\n\n\ndef make_model_class():\n    _, _, timm, torch, nn, _, _ = _training_imports()\n    class KneeMILModel(nn.Module):\n        def __init__(self, backbone=\"efficientnet_b3\", checkpoint=None, backbone_chunk=12):\n            super().__init__()\n            self.backbone_chunk = int(backbone_chunk)\n            self.backbone = timm.create_model(backbone, pretrained=False, in_chans=1, num_classes=0, global_pool=\"avg\")\n            if checkpoint:\n                state = torch.load(str(checkpoint), map_location=\"cpu\", weights_only=False)\n                if isinstance(state, Mapping) and \"state_dict\" in state:\n                    state = state[\"state_dict\"]\n                state = {str(k).replace(\"module.\", \"\", 1): v for k, v in state.items()}\n                expected = self.backbone.state_dict()\n                if \"conv_stem.weight\" in state and state[\"conv_stem.weight\"].shape[1] == 3:\n                    state[\"conv_stem.weight\"] = state[\"conv_stem.weight\"].sum(dim=1, keepdim=True)\n                state = {key: value for key, value in state.items() if key in expected}\n                shape_errors = {\n                    key: (tuple(value.shape), tuple(expected[key].shape))\n                    for key, value in state.items()\n                    if value.shape != expected[key].shape\n                }\n                if shape_errors:\n                    raise RuntimeError(f\"backbone tensor shape mismatch: {shape_errors}\")\n                result = self.backbone.load_state_dict(state, strict=True)\n                if result.missing_keys or result.unexpected_keys:\n                    raise RuntimeError(f\"backbone state mismatch: {result}\")\n            features, hidden = int(self.backbone.num_features), 384\n            self.attn_v, self.attn_u = nn.Linear(features, hidden), nn.Linear(features, hidden)\n            self.attn_out = nn.Linear(hidden, len(TARGETS))\n            self.target_weight = nn.Parameter(torch.empty(len(TARGETS), features))\n            self.target_bias = nn.Parameter(torch.zeros(len(TARGETS)))\n            self.slice_head = nn.Linear(features, len(TARGETS))\n            self.mix_logit = nn.Parameter(torch.tensor(-1.1))\n            nn.init.xavier_uniform_(self.target_weight)\n\n        def set_backbone_trainable(self, value):\n            for parameter in self.backbone.parameters():\n                parameter.requires_grad = value\n\n        def forward(self, images, mask):\n            batch, planes, slices, channels, height, width = images.shape\n            flat = images.reshape(batch * planes * slices, channels, height, width)\n            features = []\n            train_backbone = self.training and any(p.requires_grad for p in self.backbone.parameters())\n            for start in range(0, len(flat), self.backbone_chunk):\n                chunk = flat[start:start + self.backbone_chunk]\n                if train_backbone:\n                    from torch.utils.checkpoint import checkpoint\n                    features.append(checkpoint(self.backbone, chunk, use_reentrant=False))\n                else:\n                    features.append(self.backbone(chunk))\n            features = torch.cat(features, dim=0)\n            features = features.reshape(batch, planes * slices, -1)\n            flat_mask = mask.reshape(batch, planes * slices)\n            gated = torch.tanh(self.attn_v(features)) * torch.sigmoid(self.attn_u(features))\n            attention = self.attn_out(gated).masked_fill(~flat_mask[:, :, None], -1e4).softmax(dim=1)\n            pooled = torch.einsum(\"bnt,bnf->btf\", attention, features)\n            attention_logits = torch.einsum(\"btf,tf->bt\", pooled, self.target_weight) + self.target_bias\n            max_logits = self.slice_head(features).masked_fill(~flat_mask[:, :, None], -1e4).max(dim=1).values\n            mix = torch.sigmoid(self.mix_logit)\n            return (1.0 - mix) * attention_logits + mix * max_logits\n    return KneeMILModel\n\n\ndef train_one_fold(label_table, series, image_root, output_dir, fold, backbone_checkpoint, config):\n    _, _, _, torch, nn, DataLoader, _ = _training_imports()\n    random.seed(config.seed + fold)\n    np.random.seed(config.seed + fold)\n    torch.manual_seed(config.seed + fold)\n    Dataset, Model = make_dataset_class(), make_model_class()\n    train_rows, valid_rows = prepare_fold_rows(label_table, fold, config)\n    train_loader = DataLoader(Dataset(train_rows, series, image_root, config, True, config.seed + fold), batch_size=config.batch_size, shuffle=True, num_workers=config.workers, pin_memory=True, persistent_workers=config.workers > 0)\n    valid_loader = DataLoader(Dataset(valid_rows, series, image_root, config, False, config.seed + fold), batch_size=1, shuffle=False, num_workers=max(1, config.workers // 2))\n    device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n    if device.type != \"cuda\":\n        raise RuntimeError(\"GPU is required\")\n    core_model = Model(config.backbone, backbone_checkpoint, config.backbone_chunk).to(device)\n    model = nn.DataParallel(core_model) if torch.cuda.device_count() > 1 else core_model\n    loss_fn, scaler, history = nn.BCEWithLogitsLoss(reduction=\"none\"), torch.amp.GradScaler(\"cuda\"), []\n    schedule = [(False, config.frozen_lr)] * config.frozen_epochs + [(True, config.unfrozen_lr)] * config.unfrozen_epochs\n    optimizer, active_phase = None, None\n    for epoch, (unfreeze, lr) in enumerate(schedule):\n        if active_phase != unfreeze:\n            core_model.set_backbone_trainable(unfreeze)\n            optimizer = torch.optim.AdamW(\n                [p for p in model.parameters() if p.requires_grad],\n                lr=lr,\n                weight_decay=config.weight_decay,\n            )\n            active_phase = unfreeze\n        assert optimizer is not None\n        model.train()\n        for layer in core_model.backbone.modules():\n            if isinstance(layer, nn.modules.batchnorm._BatchNorm):\n                layer.eval()\n        losses = []\n        for batch in train_loader:\n            images, mask = batch[\"images\"].to(device), batch[\"mask\"].to(device)\n            target, weight = batch[\"target\"].to(device), batch[\"weight\"].to(device)\n            optimizer.zero_grad(set_to_none=True)\n            with torch.autocast(\"cuda\", dtype=torch.float16):\n                raw = loss_fn(model(images, mask), target)\n                loss = (raw * weight).sum() / weight.sum().clamp_min(1.0)\n            scaler.scale(loss).backward()\n            scaler.unscale_(optimizer)\n            torch.nn.utils.clip_grad_norm_(model.parameters(), 5.0)\n            scaler.step(optimizer)\n            scaler.update()\n            losses.append(float(loss.detach().cpu()))\n        history.append({\n            \"epoch\": epoch,\n            \"train_loss\": float(np.mean(losses)),\n            \"backbone_trainable\": unfreeze,\n            \"learning_rate\": lr,\n        })\n    model.eval()\n    uids, truth, score = [], [], []\n    with torch.no_grad():\n        for batch in valid_loader:\n            with torch.autocast(\"cuda\", dtype=torch.float16):\n                logits = model(batch[\"images\"].to(device), batch[\"mask\"].to(device))\n            uids.extend(batch[\"uid\"])\n            truth.append(batch[\"target\"].numpy())\n            score.append(torch.sigmoid(logits.float()).cpu().numpy())\n    truth, score = np.concatenate(truth), np.concatenate(score)\n    macro, per_target = pooled_macro_auc(truth, score)\n    output_dir.mkdir(parents=True, exist_ok=True)\n    oof = pd.DataFrame({UID: uids})\n    for j, target in enumerate(TARGETS):\n        oof[f\"gold::{target}\"], oof[f\"pred::{target}\"] = truth[:, j], score[:, j]\n    oof.to_csv(output_dir / f\"fold{fold}_expert_oof.csv\", index=False)\n    destination = output_dir / f\"fold{fold}_final.pt\"\n    torch.save({\"model\": core_model.state_dict(), \"backbone\": config.backbone, \"config\": asdict(config), \"fold\": fold, \"fold_diagnostic_macro_auc\": macro, \"fold_diagnostic_per_target_auc\": per_target, \"history\": history, \"visible_gpus\": torch.cuda.device_count(), \"backbone_checkpoint_sha256\": hashlib.sha256(backbone_checkpoint.read_bytes()).hexdigest()}, destination)\n    return destination\n\n\ndef main(argv: Optional[Sequence[str]] = None) -> None:\n    parser = argparse.ArgumentParser(description=__doc__)\n    for name in (\"train-csv\", \"series-csv\", \"pilkwang-labels\", \"steven-labels\", \"lixin-labels\", \"output-dir\"):\n        parser.add_argument(f\"--{name}\", type=Path, required=True)\n    parser.add_argument(\"--image-root\", type=Path)\n    parser.add_argument(\"--backbone-checkpoint\", type=Path)\n    parser.add_argument(\"--fold\", type=int)\n    parser.add_argument(\"--prepare-only\", action=\"store_true\")\n    args = parser.parse_args(argv)\n    config = TrainConfig()\n    paths = [args.train_csv, args.series_csv, args.pilkwang_labels, args.steven_labels, args.lixin_labels]\n    frames = [pd.read_csv(path) for path in paths]\n    labels = build_public_consensus(frames[0], frames[2], frames[3], frames[4], config.pseudo_weight_floor)\n    labels[\"fold\"] = assign_group_balanced_folds(labels, config.folds, config.seed)\n    args.output_dir.mkdir(parents=True, exist_ok=True)\n    labels.to_csv(args.output_dir / \"fold_labels.csv\", index=False)\n    manifest = {\n        \"config\": asdict(config), \"rows\": len(labels), \"expert_rows\": int(labels[\"is_expert\"].sum()),\n        \"fold_counts\": labels[\"fold\"].value_counts().sort_index().to_dict(),\n        \"fold_expert_counts\": labels.loc[labels[\"is_expert\"].eq(1), \"fold\"].value_counts().sort_index().to_dict(),\n        \"inputs\": {str(path): hashlib.sha256(path.read_bytes()).hexdigest() for path in paths},\n    }\n    (args.output_dir / \"manifest.json\").write_text(json.dumps(manifest, indent=2, sort_keys=True) + \"\\n\")\n    print(json.dumps(manifest, indent=2, sort_keys=True))\n    if args.prepare_only:\n        return\n    if args.fold is None or args.image_root is None or args.backbone_checkpoint is None:\n        parser.error(\"training requires --fold, --image-root, and --backbone-checkpoint\")\n    if not 0 <= args.fold < config.folds:\n        parser.error(f\"--fold must be in [0, {config.folds - 1}]\")\n    print(\"saved\", train_one_fold(labels, frames[1], args.image_root, args.output_dir, args.fold, args.backbone_checkpoint, config))\n\n\nif __name__ == \"__main__\":\n    main()\n"},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"%%writefile /kaggle/working/rsna_b3_v43_source/efficientnet_b3_public_repro_v2_anatomy.py\n#!/usr/bin/env python3\n\"\"\"Anatomy-aware EfficientNet-B3 MIL training for RSNA Knee.\n\nThis module deliberately reuses the audited v1 DICOM and label pipeline while\nchanging only three trainable assumptions that v1 could not express:\n\n* each slice feature receives its acquisition-plane and normalized stack position;\n* expert-labelled studies are sampled often enough to influence every epoch;\n* fine-tuning is restricted to the final two EfficientNet stages.\n\nThe output checkpoint contract remains compatible with the v1 inference runner,\nprovided this v2 module is supplied to ``--module`` at inference time.\n\"\"\"\n\nfrom __future__ import annotations\n\nimport argparse\nimport hashlib\nimport importlib.util\nimport json\nimport math\nimport random\nimport sys\nfrom dataclasses import asdict, dataclass, replace\nfrom pathlib import Path\nfrom typing import Mapping, Optional, Sequence\n\nimport numpy as np\nimport pandas as pd\n\n\ndef _load_base_module():\n    path = Path(__file__).with_name(\"efficientnet_b3_public_repro_v1.py\")\n    if not path.is_file():\n        raise FileNotFoundError(f\"audited v1 dependency is absent: {path}\")\n    name = \"rsna_knee_effnet_b3_public_repro_v1_dependency\"\n    spec = importlib.util.spec_from_file_location(name, path)\n    if spec is None or spec.loader is None:\n        raise RuntimeError(f\"cannot import audited v1 dependency: {path}\")\n    module = importlib.util.module_from_spec(spec)\n    sys.modules[name] = module\n    spec.loader.exec_module(module)\n    return module\n\n\nBASE = _load_base_module()\nTARGETS = BASE.TARGETS\nPLANES = BASE.PLANES\nUID = BASE.UID\nTrainConfig = BASE.TrainConfig\nload_dicom_volume = BASE.load_dicom_volume\nuniform_slice_indices = BASE.uniform_slice_indices\nstochastic_slice_indices = BASE.stochastic_slice_indices\npooled_macro_auc = BASE.pooled_macro_auc\n\n\n@dataclass(frozen=True)\nclass TrainingPolicy:\n    \"\"\"Choices specific to v2 and therefore stored separately from TrainConfig.\"\"\"\n\n    expert_sample_fraction: float = 0.12\n    unfrozen_backbone_blocks: int = 2\n    gradient_clip_norm: float = 5.0\n    model_family: str = \"efficientnet_b3_anatomy_mil_v2\"\n\n\ndef _training_imports():\n    return BASE._training_imports()\n\n\ndef pick_series_slots(series: pd.DataFrame, count: int = 3):\n    \"\"\"Return one fixed slot per anatomical plane.\n\n    The v1 selector intentionally backfills a missing plane with another useful\n    series.  That is appropriate for an anatomy-agnostic model, but would attach\n    the wrong plane embedding in v2.  Preserve empty slots instead so index zero,\n    one, and two always mean sagittal, coronal, and axial respectively.\n    \"\"\"\n    if int(count) != len(PLANES):\n        raise ValueError(f\"v2 requires exactly {len(PLANES)} anatomical plane slots\")\n    selected = BASE.pick_one_series_per_plane(series)\n    return [selected[plane] for plane in PLANES]\n\n\ndef make_dataset_class():\n    \"\"\"Build the v1 study dataset with fixed anatomical plane slots.\"\"\"\n    _, _, _, torch, _, _, Dataset = _training_imports()\n\n    class AnatomyAwareKneeStudyDataset(Dataset):\n        def __init__(self, rows, series, image_root, config, training, seed):\n            self.rows = rows.reset_index(drop=True)\n            self.series = {\n                str(uid): group.copy() for uid, group in series.groupby(UID, sort=False)\n            }\n            self.image_root = Path(image_root)\n            self.config = config\n            self.training = bool(training)\n            self.seed = int(seed)\n\n        def __len__(self):\n            return len(self.rows)\n\n        def __getitem__(self, index):\n            row, config = self.rows.iloc[index], self.config\n            uid = str(row[UID])\n            count = config.train_slices if self.training else config.valid_slices\n            visit_seed = (\n                int(torch.randint(0, 2**31 - 1, (1,)).item()) if self.training else 0\n            )\n            rng = np.random.default_rng(self.seed + 1_000_003 * index + visit_seed)\n            empty = pd.DataFrame(\n                columns=[\n                    \"SeriesInstanceUID\",\n                    \"Anatomical_Plane\",\n                    \"Fluid_Sensitive\",\n                    \"Fat_Suppression\",\n                ]\n            )\n            chosen = pick_series_slots(self.series.get(uid, empty), len(PLANES))\n            planes, masks = [], []\n            for series_uid in chosen:\n                if series_uid is None:\n                    planes.append(\n                        np.zeros((count, config.image_size, config.image_size), np.float32)\n                    )\n                    masks.append(np.zeros(count, bool))\n                    continue\n                volume = load_dicom_volume(\n                    self.image_root / uid / series_uid,\n                    config.image_size,\n                    config.max_series_slices,\n                )\n                indices = (\n                    stochastic_slice_indices(len(volume), count, rng)\n                    if self.training\n                    else uniform_slice_indices(len(volume), count)\n                )\n                sampled = volume[indices].astype(np.float32)\n                if self.training:\n                    sampled = np.clip(sampled * rng.uniform(0.85, 1.15), 0, 1)\n                    sampled = sampled ** rng.uniform(0.85, 1.15)\n                    if rng.random() < 0.5:\n                        sampled = sampled[:, :, ::-1].copy()\n                sampled = (sampled - 0.449) / 0.226\n                planes.append(sampled)\n                masks.append(np.ones(count, bool))\n            return {\n                \"uid\": uid,\n                \"images\": torch.from_numpy(np.stack(planes)[:, :, None]),\n                \"mask\": torch.from_numpy(np.stack(masks)),\n                \"target\": torch.tensor(\n                    [row[f\"target::{target}\"] for target in TARGETS], dtype=torch.float32\n                ),\n                \"weight\": torch.tensor(\n                    [row[f\"train_weight::{target}\"] for target in TARGETS],\n                    dtype=torch.float32,\n                ),\n            }\n\n    return AnatomyAwareKneeStudyDataset\n\n\ndef make_model_class():\n    \"\"\"Build a target-attention MIL model with fixed anatomical coordinates.\"\"\"\n    _, _, timm, torch, nn, _, _ = _training_imports()\n\n    class AnatomyAwareKneeMIL(nn.Module):\n        def __init__(self, backbone=\"efficientnet_b3\", checkpoint=None, backbone_chunk=12):\n            super().__init__()\n            self.backbone_chunk = int(backbone_chunk)\n            self.backbone = timm.create_model(\n                backbone,\n                pretrained=False,\n                in_chans=1,\n                num_classes=0,\n                global_pool=\"avg\",\n            )\n            if checkpoint:\n                state = torch.load(str(checkpoint), map_location=\"cpu\", weights_only=False)\n                if isinstance(state, Mapping) and \"state_dict\" in state:\n                    state = state[\"state_dict\"]\n                state = {str(key).replace(\"module.\", \"\", 1): value for key, value in state.items()}\n                expected = self.backbone.state_dict()\n                if \"conv_stem.weight\" in state and state[\"conv_stem.weight\"].shape[1] == 3:\n                    # Sum rather than mean preserves the response magnitude expected by\n                    # the pretrained filters after the one-channel ImageNet normalization.\n                    state[\"conv_stem.weight\"] = state[\"conv_stem.weight\"].sum(dim=1, keepdim=True)\n                state = {key: value for key, value in state.items() if key in expected}\n                shape_errors = {\n                    key: (tuple(value.shape), tuple(expected[key].shape))\n                    for key, value in state.items()\n                    if value.shape != expected[key].shape\n                }\n                if shape_errors:\n                    raise RuntimeError(f\"backbone tensor shape mismatch: {shape_errors}\")\n                result = self.backbone.load_state_dict(state, strict=True)\n                if result.missing_keys or result.unexpected_keys:\n                    raise RuntimeError(f\"backbone state mismatch: {result}\")\n\n            features, hidden = int(self.backbone.num_features), 384\n            self.plane_embedding = nn.Embedding(len(PLANES), features)\n            self.position_projection = nn.Linear(4, features, bias=False)\n            nn.init.normal_(self.plane_embedding.weight, std=0.01)\n            nn.init.normal_(self.position_projection.weight, std=0.01)\n\n            self.attn_v = nn.Linear(features, hidden)\n            self.attn_u = nn.Linear(features, hidden)\n            self.attn_out = nn.Linear(hidden, len(TARGETS))\n            self.target_weight = nn.Parameter(torch.empty(len(TARGETS), features))\n            self.target_bias = nn.Parameter(torch.zeros(len(TARGETS)))\n            self.slice_head = nn.Linear(features, len(TARGETS))\n            self.mix_logit = nn.Parameter(torch.tensor(-1.1))\n            nn.init.xavier_uniform_(self.target_weight)\n\n        def set_backbone_stage(self, final_blocks=0):\n            \"\"\"Freeze the backbone, optionally reopening only its final stages.\"\"\"\n            for parameter in self.backbone.parameters():\n                parameter.requires_grad_(False)\n            final_blocks = int(final_blocks)\n            if final_blocks <= 0:\n                return\n            blocks = list(self.backbone.blocks)\n            if final_blocks > len(blocks):\n                raise ValueError(\n                    f\"requested {final_blocks} trainable blocks from a {len(blocks)}-block backbone\"\n                )\n            for block in blocks[-final_blocks:]:\n                for parameter in block.parameters():\n                    parameter.requires_grad_(True)\n            for name in (\"conv_head\", \"bn2\"):\n                layer = getattr(self.backbone, name, None)\n                if layer is not None:\n                    for parameter in layer.parameters():\n                        parameter.requires_grad_(True)\n\n        @staticmethod\n        def _position_basis(slices, device, dtype):\n            position = torch.linspace(-1.0, 1.0, slices, device=device, dtype=dtype)\n            return torch.stack(\n                (\n                    position,\n                    position.square(),\n                    torch.sin(math.pi * position),\n                    torch.cos(math.pi * position),\n                ),\n                dim=-1,\n            )\n\n        def forward(self, images, mask):\n            batch, planes, slices, channels, height, width = images.shape\n            if planes != len(PLANES):\n                raise ValueError(f\"expected {len(PLANES)} plane slots, got {planes}\")\n            flat = images.reshape(batch * planes * slices, channels, height, width)\n            encoded = []\n            train_backbone = self.training and any(\n                parameter.requires_grad for parameter in self.backbone.parameters()\n            )\n            for start in range(0, len(flat), self.backbone_chunk):\n                chunk = flat[start : start + self.backbone_chunk]\n                if train_backbone:\n                    from torch.utils.checkpoint import checkpoint\n\n                    encoded.append(checkpoint(self.backbone, chunk, use_reentrant=False))\n                else:\n                    encoded.append(self.backbone(chunk))\n            features = torch.cat(encoded, dim=0).reshape(batch, planes, slices, -1)\n\n            plane_ids = torch.arange(planes, device=features.device)\n            plane_context = self.plane_embedding(plane_ids)[None, :, None, :]\n            position = self._position_basis(slices, features.device, features.dtype)\n            position_context = self.position_projection(position)[None, None, :, :]\n            features = features + plane_context + position_context\n            features = features.reshape(batch, planes * slices, -1)\n            flat_mask = mask.reshape(batch, planes * slices)\n\n            gated = torch.tanh(self.attn_v(features)) * torch.sigmoid(self.attn_u(features))\n            attention = self.attn_out(gated).masked_fill(\n                ~flat_mask[:, :, None], -1e4\n            ).softmax(dim=1)\n            pooled = torch.einsum(\"bnt,bnf->btf\", attention, features)\n            attention_logits = (\n                torch.einsum(\"btf,tf->bt\", pooled, self.target_weight) + self.target_bias\n            )\n            max_logits = self.slice_head(features).masked_fill(\n                ~flat_mask[:, :, None], -1e4\n            ).max(dim=1).values\n            mix = torch.sigmoid(self.mix_logit)\n            return (1.0 - mix) * attention_logits + mix * max_logits\n\n    return AnatomyAwareKneeMIL\n\n\ndef _make_expert_sampler(rows, fraction, seed, torch):\n    \"\"\"Sample a fixed expected expert fraction without discarding pseudo rows.\"\"\"\n    if not 0.0 < fraction < 1.0:\n        raise ValueError(\"expert_sample_fraction must be strictly between zero and one\")\n    expert = rows[\"is_expert\"].to_numpy(np.int8).astype(bool)\n    n_expert, n_pseudo = int(expert.sum()), int((~expert).sum())\n    if n_expert == 0 or n_pseudo == 0:\n        raise ValueError(\"expert-aware sampling requires both expert and pseudo-labelled rows\")\n    expert_multiplier = fraction * n_pseudo / ((1.0 - fraction) * n_expert)\n    weights = np.where(expert, expert_multiplier, 1.0).astype(np.float64)\n    generator = torch.Generator().manual_seed(int(seed))\n    from torch.utils.data import WeightedRandomSampler\n\n    sampler = WeightedRandomSampler(\n        torch.from_numpy(weights),\n        num_samples=len(rows),\n        replacement=True,\n        generator=generator,\n    )\n    return sampler, {\n        \"expert_rows\": n_expert,\n        \"pseudo_rows\": n_pseudo,\n        \"expert_multiplier\": float(expert_multiplier),\n        \"expected_expert_fraction\": float(fraction),\n    }\n\n\ndef train_one_fold(\n    label_table,\n    series,\n    image_root,\n    output_dir,\n    fold,\n    backbone_checkpoint,\n    config,\n    policy=TrainingPolicy(),\n):\n    _, _, _, torch, nn, DataLoader, _ = _training_imports()\n    random.seed(config.seed + fold)\n    np.random.seed(config.seed + fold)\n    torch.manual_seed(config.seed + fold)\n    Dataset, Model = make_dataset_class(), make_model_class()\n    train_rows, valid_rows = BASE.prepare_fold_rows(label_table, fold, config)\n    sampler, sampler_meta = _make_expert_sampler(\n        train_rows,\n        policy.expert_sample_fraction,\n        config.seed + 10_007 * fold,\n        torch,\n    )\n    train_loader = DataLoader(\n        Dataset(train_rows, series, image_root, config, True, config.seed + fold),\n        batch_size=config.batch_size,\n        sampler=sampler,\n        num_workers=config.workers,\n        pin_memory=True,\n        persistent_workers=config.workers > 0,\n    )\n    valid_loader = DataLoader(\n        Dataset(valid_rows, series, image_root, config, False, config.seed + fold),\n        batch_size=1,\n        shuffle=False,\n        num_workers=max(1, config.workers // 2),\n    )\n    device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n    if device.type != \"cuda\":\n        raise RuntimeError(\"GPU is required\")\n    core_model = Model(config.backbone, backbone_checkpoint, config.backbone_chunk).to(device)\n    model = nn.DataParallel(core_model) if torch.cuda.device_count() > 1 else core_model\n    loss_fn = nn.BCEWithLogitsLoss(reduction=\"none\")\n    scaler = torch.amp.GradScaler(\"cuda\")\n    history = []\n    schedule = (\n        [(0, config.frozen_lr)] * config.frozen_epochs\n        + [(policy.unfrozen_backbone_blocks, config.unfrozen_lr)] * config.unfrozen_epochs\n    )\n    optimizer, active_blocks = None, None\n    for epoch, (trainable_blocks, learning_rate) in enumerate(schedule):\n        if active_blocks != trainable_blocks:\n            core_model.set_backbone_stage(trainable_blocks)\n            optimizer = torch.optim.AdamW(\n                [parameter for parameter in model.parameters() if parameter.requires_grad],\n                lr=learning_rate,\n                weight_decay=config.weight_decay,\n            )\n            active_blocks = trainable_blocks\n        assert optimizer is not None\n        model.train()\n        for layer in core_model.backbone.modules():\n            if isinstance(layer, nn.modules.batchnorm._BatchNorm):\n                layer.eval()\n        losses = []\n        sampled_expert = 0\n        sampled_total = 0\n        for batch in train_loader:\n            images, mask = batch[\"images\"].to(device), batch[\"mask\"].to(device)\n            target, weight = batch[\"target\"].to(device), batch[\"weight\"].to(device)\n            # Expert rows have the same gold weight for all twelve targets.\n            sampled_expert += int((weight.min(dim=1).values >= config.gold_weight).sum().item())\n            sampled_total += int(len(weight))\n            optimizer.zero_grad(set_to_none=True)\n            with torch.autocast(\"cuda\", dtype=torch.float16):\n                raw = loss_fn(model(images, mask), target)\n                loss = (raw * weight).sum() / weight.sum().clamp_min(1.0)\n            scaler.scale(loss).backward()\n            scaler.unscale_(optimizer)\n            torch.nn.utils.clip_grad_norm_(model.parameters(), policy.gradient_clip_norm)\n            scaler.step(optimizer)\n            scaler.update()\n            losses.append(float(loss.detach().cpu()))\n        history.append(\n            {\n                \"epoch\": epoch,\n                \"train_loss\": float(np.mean(losses)),\n                \"trainable_backbone_blocks\": int(trainable_blocks),\n                \"learning_rate\": float(learning_rate),\n                \"observed_expert_fraction\": sampled_expert / max(sampled_total, 1),\n            }\n        )\n\n    model.eval()\n    uids, truth, score = [], [], []\n    with torch.no_grad():\n        for batch in valid_loader:\n            with torch.autocast(\"cuda\", dtype=torch.float16):\n                logits = model(batch[\"images\"].to(device), batch[\"mask\"].to(device))\n            uids.extend(batch[\"uid\"])\n            truth.append(batch[\"target\"].numpy())\n            score.append(torch.sigmoid(logits.float()).cpu().numpy())\n    truth, score = np.concatenate(truth), np.concatenate(score)\n    macro, per_target = BASE.pooled_macro_auc(truth, score)\n    output_dir.mkdir(parents=True, exist_ok=True)\n    oof = pd.DataFrame({UID: uids})\n    for target_index, target in enumerate(TARGETS):\n        oof[f\"gold::{target}\"] = truth[:, target_index]\n        oof[f\"pred::{target}\"] = score[:, target_index]\n    oof.to_csv(output_dir / f\"fold{fold}_expert_oof.csv\", index=False)\n    destination = output_dir / f\"fold{fold}_final.pt\"\n    torch.save(\n        {\n            \"model\": core_model.state_dict(),\n            \"model_family\": policy.model_family,\n            \"backbone\": config.backbone,\n            \"config\": asdict(config),\n            \"training_policy\": asdict(policy),\n            \"sampler\": sampler_meta,\n            \"fold\": fold,\n            \"fold_diagnostic_macro_auc\": macro,\n            \"fold_diagnostic_per_target_auc\": per_target,\n            \"history\": history,\n            \"visible_gpus\": torch.cuda.device_count(),\n            \"backbone_checkpoint_sha256\": hashlib.sha256(\n                backbone_checkpoint.read_bytes()\n            ).hexdigest(),\n        },\n        destination,\n    )\n    return destination\n\n\ndef main(argv: Optional[Sequence[str]] = None) -> None:\n    parser = argparse.ArgumentParser(description=__doc__)\n    for name in (\n        \"train-csv\",\n        \"series-csv\",\n        \"pilkwang-labels\",\n        \"steven-labels\",\n        \"lixin-labels\",\n        \"output-dir\",\n    ):\n        parser.add_argument(f\"--{name}\", type=Path, required=True)\n    parser.add_argument(\"--image-root\", type=Path)\n    parser.add_argument(\"--backbone-checkpoint\", type=Path)\n    parser.add_argument(\"--fold\", type=int)\n    parser.add_argument(\"--prepare-only\", action=\"store_true\")\n    args = parser.parse_args(argv)\n\n    # Expert oversampling replaces some of the multiplicative gold weighting;\n    # keeping both at their v1 values would overfit the 46-or-so training experts.\n    config = replace(TrainConfig(), gold_weight=2.0)\n    policy = TrainingPolicy()\n    paths = [\n        args.train_csv,\n        args.series_csv,\n        args.pilkwang_labels,\n        args.steven_labels,\n        args.lixin_labels,\n    ]\n    frames = [pd.read_csv(path) for path in paths]\n    labels = BASE.build_public_consensus(\n        frames[0], frames[2], frames[3], frames[4], config.pseudo_weight_floor\n    )\n    labels[\"fold\"] = BASE.assign_group_balanced_folds(labels, config.folds, config.seed)\n    args.output_dir.mkdir(parents=True, exist_ok=True)\n    labels.to_csv(args.output_dir / \"fold_labels.csv\", index=False)\n    manifest = {\n        \"config\": asdict(config),\n        \"training_policy\": asdict(policy),\n        \"rows\": len(labels),\n        \"expert_rows\": int(labels[\"is_expert\"].sum()),\n        \"fold_counts\": labels[\"fold\"].value_counts().sort_index().to_dict(),\n        \"fold_expert_counts\": labels.loc[\n            labels[\"is_expert\"].eq(1), \"fold\"\n        ].value_counts().sort_index().to_dict(),\n        \"inputs\": {str(path): hashlib.sha256(path.read_bytes()).hexdigest() for path in paths},\n        \"base_module_sha256\": hashlib.sha256(\n            Path(BASE.__file__).read_bytes()\n        ).hexdigest(),\n        \"module_sha256\": hashlib.sha256(Path(__file__).read_bytes()).hexdigest(),\n    }\n    (args.output_dir / \"manifest.json\").write_text(\n        json.dumps(manifest, indent=2, sort_keys=True) + \"\\n\"\n    )\n    print(json.dumps(manifest, indent=2, sort_keys=True))\n    if args.prepare_only:\n        return\n    if args.fold is None or args.image_root is None or args.backbone_checkpoint is None:\n        parser.error(\"training requires --fold, --image-root, and --backbone-checkpoint\")\n    if not 0 <= args.fold < config.folds:\n        parser.error(f\"--fold must be in [0, {config.folds - 1}]\")\n    print(\n        \"saved\",\n        train_one_fold(\n            labels,\n            frames[1],\n            args.image_root,\n            args.output_dir,\n            args.fold,\n            args.backbone_checkpoint,\n            config,\n            policy,\n        ),\n    )\n\n\nif __name__ == \"__main__\":\n    main()\n"},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"%%writefile /kaggle/working/rsna_b3_v43_source/efficientnet_b3_public_repro_v3_physical_cache.py\n#!/usr/bin/env python3\n\"\"\"Physical-space, cache-first EfficientNet-B3 MIL training for RSNA Knee.\n\nThis reproducible experiment keeps the public .903 notebook's B3,\ntarget-attention, and five-fold design. Slices are ordered in patient space,\ncropped to a constant physical field of view, normalized to a left-knee\nconvention, and encoded once into a compact float16 feature cache. The gold OOF\nartifact produced by the completed run is the promotion gate.\n\"\"\"\n\nfrom __future__ import annotations\n\nimport argparse\nimport hashlib\nimport importlib.util\nimport json\nimport math\nimport random\nimport re\nimport sys\nfrom dataclasses import asdict, dataclass, replace\nfrom pathlib import Path\nfrom typing import Optional, Sequence\n\nimport numpy as np\nimport pandas as pd\n\n\ndef _load_dependency(filename: str, module_name: str):\n    candidates = (\n        Path(__file__).with_name(filename),\n        Path(__file__).parents[1] / \"research\" / \"rsna_knee_20260808\" / filename,\n    )\n    path = next((candidate for candidate in candidates if candidate.is_file()), None)\n    if path is None:\n        raise FileNotFoundError(f\"audited dependency is absent: {filename}\")\n    spec = importlib.util.spec_from_file_location(module_name, path)\n    if spec is None or spec.loader is None:\n        raise RuntimeError(f\"cannot import audited dependency: {path}\")\n    module = importlib.util.module_from_spec(spec)\n    sys.modules[module_name] = module\n    spec.loader.exec_module(module)\n    return module\n\n\nV2 = _load_dependency(\n    \"efficientnet_b3_public_repro_v2_anatomy.py\",\n    \"rsna_knee_effnet_b3_public_repro_v2_dependency\",\n)\nBASE = V2.BASE\nTARGETS, PLANES, UID = V2.TARGETS, V2.PLANES, V2.UID\nTrainConfig = V2.TrainConfig\n\n_SELECTED_DEVICE_TYPE = None\n\n\ndef select_torch_device(torch):\n    \"\"\"Use CUDA only after a real kernel probe; otherwise fail over to CPU.\"\"\"\n    global _SELECTED_DEVICE_TYPE\n    if _SELECTED_DEVICE_TYPE is None:\n        _SELECTED_DEVICE_TYPE = \"cpu\"\n        if torch.cuda.is_available():\n            try:\n                probe = torch.ones(8, device=\"cuda\")\n                observed = float((probe * probe).sum().cpu())\n                if observed != 8.0:\n                    raise RuntimeError(f\"unexpected CUDA probe result: {observed}\")\n                torch.cuda.synchronize()\n                _SELECTED_DEVICE_TYPE = \"cuda\"\n            except Exception as error:\n                print(\n                    f\"CUDA execution probe failed ({type(error).__name__}: {error}); \"\n                    \"using bounded CPU fallback\",\n                    flush=True,\n                )\n        print(f\"selected torch device: {_SELECTED_DEVICE_TYPE}\", flush=True)\n    return torch.device(_SELECTED_DEVICE_TYPE)\n\n\n@dataclass(frozen=True)\nclass PhysicalCachePolicy:\n    crop_mm: float = 130.0\n    slice_band_low: float = 0.10\n    slice_band_high: float = 0.90\n    cache_slices: int = 32\n    laterality_dead_zone_mm: float = 20.0\n    expert_sample_fraction: float = 0.12\n    head_epochs: int = 12\n    head_batch_size: int = 32\n    head_lr: float = 4e-4\n    head_weight_decay: float = 2e-4\n    feature_dropout: float = 0.10\n    gradient_clip_norm: float = 5.0\n    model_family: str = \"efficientnet_b3_physical_cached_mil_v3\"\n\n\ndef _training_imports():\n    return BASE._training_imports()\n\n\ndef sha256_file(path: Path, chunk_size: int = 8 << 20) -> str:\n    digest = hashlib.sha256()\n    with path.open(\"rb\") as handle:\n        for chunk in iter(lambda: handle.read(chunk_size), b\"\"):\n            digest.update(chunk)\n    return digest.hexdigest()\n\n\ndef natural_key(path: Path):\n    return tuple(\n        int(part) if part.isdigit() else part.casefold()\n        for part in re.split(r\"(\\d+)\", path.name)\n    )\n\n\ndef _finite_vector(value, length: int):\n    try:\n        vector = np.asarray(value[:length], dtype=np.float64)\n    except Exception:\n        return None\n    return vector if len(vector) == length and np.isfinite(vector).all() else None\n\n\ndef _read_header(path: Path, pydicom):\n    return pydicom.dcmread(\n        str(path),\n        force=True,\n        stop_before_pixels=True,\n        specific_tags=[\n            \"ImagePositionPatient\",\n            \"ImageOrientationPatient\",\n            \"InstanceNumber\",\n            \"Laterality\",\n            \"ImageLaterality\",\n            \"PixelSpacing\",\n            \"Rows\",\n            \"Columns\",\n        ],\n    )\n\n\ndef ordered_dicom_paths(series_dir: Path):\n    \"\"\"Sort a stack by signed physical position, then by safer fallbacks.\"\"\"\n    _, pydicom, _, _, _, _, _ = _training_imports()\n    files = sorted(series_dir.glob(\"*.dcm\"), key=natural_key)\n    rows = []\n    for original, path in enumerate(files):\n        projection = instance = None\n        try:\n            ds = _read_header(path, pydicom)\n            position = _finite_vector(getattr(ds, \"ImagePositionPatient\", None), 3)\n            orientation = _finite_vector(\n                getattr(ds, \"ImageOrientationPatient\", None), 6\n            )\n            if position is not None and orientation is not None:\n                projection = float(\n                    np.dot(position, np.cross(orientation[:3], orientation[3:]))\n                )\n            if getattr(ds, \"InstanceNumber\", None) is not None:\n                instance = float(ds.InstanceNumber)\n        except Exception:\n            pass\n        rows.append((path, projection, instance, original))\n    needed = max(2, int(math.ceil(0.8 * len(rows))))\n    if sum(row[1] is not None for row in rows) >= needed:\n        fallback = float(np.median([row[1] for row in rows if row[1] is not None]))\n        rows.sort(\n            key=lambda row: (\n                row[1] if row[1] is not None else fallback,\n                row[2] if row[2] is not None else float(\"inf\"),\n                row[3],\n            )\n        )\n    elif sum(row[2] is not None for row in rows) >= needed:\n        rows.sort(\n            key=lambda row: (\n                row[2] if row[2] is not None else float(\"inf\"), row[3]\n            )\n        )\n    return [row[0] for row in rows]\n\n\ndef _series_side_evidence(series_dir: Path):\n    _, pydicom, _, _, _, _, _ = _training_imports()\n    files = sorted(series_dir.glob(\"*.dcm\"), key=natural_key)\n    if not files:\n        return None, None\n    try:\n        ds = _read_header(files[len(files) // 2], pydicom)\n    except Exception:\n        return None, None\n    explicit = None\n    for name in (\"Laterality\", \"ImageLaterality\"):\n        text = str(getattr(ds, name, \"\") or \"\").strip().upper()\n        if text and text[0] in (\"L\", \"R\"):\n            explicit = text[0]\n            break\n    position = _finite_vector(getattr(ds, \"ImagePositionPatient\", None), 3)\n    orientation = _finite_vector(getattr(ds, \"ImageOrientationPatient\", None), 6)\n    spacing = _finite_vector(getattr(ds, \"PixelSpacing\", None), 2)\n    try:\n        rows, columns = float(ds.Rows), float(ds.Columns)\n    except Exception:\n        rows = columns = float(\"nan\")\n    centre_x = None\n    if (\n        position is not None\n        and orientation is not None\n        and spacing is not None\n        and np.isfinite([rows, columns]).all()\n    ):\n        centre = (\n            position\n            + orientation[:3] * spacing[1] * columns / 2.0\n            + orientation[3:] * spacing[0] * rows / 2.0\n        )\n        centre_x = float(centre[0])\n    return explicit, centre_x\n\n\ndef build_series_selection(series: pd.DataFrame):\n    empty = pd.DataFrame(\n        columns=[\n            \"SeriesInstanceUID\",\n            \"Anatomical_Plane\",\n            \"Fluid_Sensitive\",\n            \"Fat_Suppression\",\n        ]\n    )\n    grouped = {\n        str(uid): group.copy() for uid, group in series.groupby(UID, sort=False)\n    }\n    return {\n        str(uid): V2.pick_series_slots(grouped.get(str(uid), empty), len(PLANES))\n        for uid in series[UID].astype(str).drop_duplicates()\n    }\n\n\ndef build_laterality_map(\n    uids: Sequence[str], selection, image_root: Path, dead_zone_mm: float\n):\n    result = {}\n    counts = {\"tag\": 0, \"geometry\": 0, \"unresolved\": 0, \"disagreement\": 0}\n    for uid in map(str, uids):\n        tags, centres = [], []\n        for series_uid in selection.get(uid, [None] * len(PLANES)):\n            if series_uid is None:\n                continue\n            tag, centre = _series_side_evidence(image_root / uid / str(series_uid))\n            if tag is not None:\n                tags.append(tag)\n            if centre is not None and np.isfinite(centre):\n                centres.append(float(centre))\n        tagged = max(set(tags), key=lambda value: (tags.count(value), value)) if tags else None\n        geometric = None\n        if centres:\n            centre = float(np.median(centres))\n            if abs(centre) >= float(dead_zone_mm):\n                geometric = \"R\" if centre < 0 else \"L\"\n        if tagged is not None:\n            result[uid] = tagged\n            counts[\"tag\"] += 1\n            counts[\"disagreement\"] += int(geometric is not None and geometric != tagged)\n        elif geometric is not None:\n            result[uid] = geometric\n            counts[\"geometry\"] += 1\n        else:\n            result[uid] = None\n            counts[\"unresolved\"] += 1\n    return result, counts\n\n\ndef _sample_band_indices(length: int, count: int, low: float, high: float):\n    if length <= 0 or count <= 0 or not 0 <= low < high <= 1:\n        raise ValueError(\"invalid slice-band request\")\n    start, stop = int(round(low * (length - 1))), int(round(high * (length - 1)))\n    if stop <= start:\n        start = stop = length // 2\n    return np.rint(np.linspace(start, stop, count)).astype(np.int64)\n\n\ndef load_physical_volume(\n    series_dir: Path,\n    plane: str,\n    side: Optional[str],\n    image_size: int,\n    count: int,\n    policy: PhysicalCachePolicy,\n):\n    cv2, pydicom, _, _, _, _, _ = _training_imports()\n    files = ordered_dicom_paths(series_dir)\n    if not files:\n        return None\n    indices = _sample_band_indices(\n        len(files), count, policy.slice_band_low, policy.slice_band_high\n    )\n    decoded, metadata = [], []\n    for index in indices:\n        try:\n            ds = pydicom.dcmread(str(files[int(index)]), force=True)\n            image = ds.pixel_array.astype(np.float32)\n            image *= float(getattr(ds, \"RescaleSlope\", 1.0) or 1.0)\n            image += float(getattr(ds, \"RescaleIntercept\", 0.0) or 0.0)\n            if str(getattr(ds, \"PhotometricInterpretation\", \"\")) == \"MONOCHROME1\":\n                image = float(np.max(image)) - image\n            decoded.append(image)\n            metadata.append(_finite_vector(getattr(ds, \"PixelSpacing\", None), 2))\n        except Exception:\n            decoded.append(None)\n            metadata.append(None)\n    good = [index for index, image in enumerate(decoded) if image is not None]\n    if not good:\n        return None\n    for index, image in enumerate(decoded):\n        if image is None:\n            nearest = min(good, key=lambda item: abs(item - index))\n            decoded[index] = decoded[nearest].copy()\n            metadata[index] = metadata[nearest]\n    reference_shape = decoded[good[0]].shape\n    decoded = [\n        image if image.shape == reference_shape else np.zeros(reference_shape, np.float32)\n        for image in decoded\n    ]\n    volume = np.stack(decoded).astype(np.float32, copy=False)\n    spacing = next((value for value in metadata if value is not None), None)\n    if spacing is not None:\n        want_h = int(round(policy.crop_mm / float(spacing[0])))\n        want_w = int(round(policy.crop_mm / float(spacing[1])))\n        height, width = reference_shape\n        if 16 < want_h <= height and 16 < want_w <= width:\n            top, left = (height - want_h) // 2, (width - want_w) // 2\n            volume = volume[:, top : top + want_h, left : left + want_w]\n    lo, hi = np.percentile(volume, [1.0, 99.0])\n    hi = max(float(hi), float(lo) + 1.0)\n    volume = np.clip((volume - float(lo)) / (hi - float(lo)), 0.0, 1.0)\n    resized = np.stack(\n        [cv2.resize(image, (image_size, image_size), interpolation=cv2.INTER_AREA) for image in volume]\n    )\n    resized = np.rint(resized * 255.0).astype(np.uint8).astype(np.float32) / 255.0\n    if side == \"R\":\n        if plane in (\"Coronal\", \"Axial\"):\n            resized = resized[:, :, ::-1].copy()\n        elif plane == \"Sagittal\":\n            resized = resized[::-1].copy()\n    return resized\n\n\ndef make_physical_dataset_class():\n    _, _, _, torch, _, _, Dataset = _training_imports()\n\n    class PhysicalStudyDataset(Dataset):\n        def __init__(self, uids, selection, laterality, image_root, config, policy):\n            self.uids = list(map(str, uids))\n            self.selection = selection\n            self.laterality = laterality\n            self.image_root = Path(image_root)\n            self.config = config\n            self.policy = policy\n\n        def __len__(self):\n            return len(self.uids)\n\n        def __getitem__(self, index):\n            uid = self.uids[index]\n            slots, masks = [], []\n            for plane, series_uid in zip(\n                PLANES, self.selection.get(uid, [None] * len(PLANES))\n            ):\n                volume = None\n                if series_uid is not None:\n                    volume = load_physical_volume(\n                        self.image_root / uid / str(series_uid),\n                        plane,\n                        self.laterality.get(uid),\n                        self.config.image_size,\n                        self.policy.cache_slices,\n                        self.policy,\n                    )\n                if volume is None:\n                    volume = np.zeros(\n                        (self.policy.cache_slices, self.config.image_size, self.config.image_size),\n                        np.float32,\n                    )\n                    masks.append(np.zeros(self.policy.cache_slices, bool))\n                else:\n                    masks.append(np.ones(self.policy.cache_slices, bool))\n                slots.append((volume - 0.449) / 0.226)\n            return {\n                \"uid\": uid,\n                \"images\": torch.from_numpy(np.stack(slots)[:, :, None]),\n                \"mask\": torch.from_numpy(np.stack(masks)),\n            }\n\n    return PhysicalStudyDataset\n\n\ndef make_model_class(feature_dropout: float = 0.0):\n    \"\"\"Add separate image encoding and MIL-head entry points to v2.\"\"\"\n    _, _, _, torch, _, _, _ = _training_imports()\n    Parent = V2.make_model_class()\n\n    class PhysicalCachedKneeMIL(Parent):\n        def __init__(self, *args, **kwargs):\n            super().__init__(*args, **kwargs)\n            self.feature_dropout = float(feature_dropout)\n\n        def encode_images(self, images):\n            batch, planes, slices, channels, height, width = images.shape\n            if planes != len(PLANES):\n                raise ValueError(f\"expected {len(PLANES)} plane slots, got {planes}\")\n            flat = images.reshape(batch * planes * slices, channels, height, width)\n            encoded = []\n            for start in range(0, len(flat), self.backbone_chunk):\n                encoded.append(self.backbone(flat[start : start + self.backbone_chunk]))\n            return torch.cat(encoded, dim=0).reshape(batch, planes, slices, -1)\n\n        def forward_encoded(self, features, mask):\n            batch, planes, slices, width = features.shape\n            if planes != len(PLANES) or mask.shape != (batch, planes, slices):\n                raise ValueError(\"feature/mask anatomy contract mismatch\")\n            if self.training and self.feature_dropout > 0:\n                features = torch.nn.functional.dropout(\n                    features, p=self.feature_dropout, training=True\n                )\n            plane_ids = torch.arange(planes, device=features.device)\n            features = features + self.plane_embedding(plane_ids)[None, :, None, :]\n            basis = self._position_basis(slices, features.device, features.dtype)\n            features = features + self.position_projection(basis)[None, None, :, :]\n            features = features.reshape(batch, planes * slices, width)\n            flat_mask = mask.reshape(batch, planes * slices)\n            gated = torch.tanh(self.attn_v(features)) * torch.sigmoid(\n                self.attn_u(features)\n            )\n            attention = self.attn_out(gated).masked_fill(\n                ~flat_mask[:, :, None], -1e4\n            ).softmax(dim=1)\n            pooled = torch.einsum(\"bnt,bnf->btf\", attention, features)\n            attention_logits = (\n                torch.einsum(\"btf,tf->bt\", pooled, self.target_weight)\n                + self.target_bias\n            )\n            max_logits = self.slice_head(features).masked_fill(\n                ~flat_mask[:, :, None], -1e4\n            ).max(dim=1).values\n            mix = torch.sigmoid(self.mix_logit)\n            return (1.0 - mix) * attention_logits + mix * max_logits\n\n        def forward(self, images, mask):\n            return self.forward_encoded(self.encode_images(images), mask)\n\n    return PhysicalCachedKneeMIL\n\n\ndef build_feature_cache(\n    uids,\n    series,\n    image_root,\n    cache_dir,\n    backbone_checkpoint,\n    config,\n    policy,\n):\n    _, _, _, torch, _, DataLoader, _ = _training_imports()\n    device = select_torch_device(torch)\n    use_cuda = device.type == \"cuda\"\n    cache_dir.mkdir(parents=True, exist_ok=True)\n    uids = list(map(str, uids))\n    selection = build_series_selection(series)\n    laterality, laterality_meta = build_laterality_map(\n        uids, selection, Path(image_root), policy.laterality_dead_zone_mm\n    )\n    Dataset, Model = make_physical_dataset_class(), make_model_class()\n    loader = DataLoader(\n        Dataset(uids, selection, laterality, image_root, config, policy),\n        batch_size=1,\n        shuffle=False,\n        num_workers=config.workers,\n        pin_memory=use_cuda,\n        persistent_workers=config.workers > 0,\n    )\n    model = Model(config.backbone, backbone_checkpoint, 32).to(device).eval()\n    model.set_backbone_stage(0)\n    width = int(model.backbone.num_features)\n    feature_path = cache_dir / \"features.float16.npy\"\n    mask_path = cache_dir / \"mask.bool.npy\"\n    features = np.lib.format.open_memmap(\n        feature_path,\n        mode=\"w+\",\n        dtype=np.float16,\n        shape=(len(uids), len(PLANES), policy.cache_slices, width),\n    )\n    masks = np.lib.format.open_memmap(\n        mask_path,\n        mode=\"w+\",\n        dtype=np.bool_,\n        shape=(len(uids), len(PLANES), policy.cache_slices),\n    )\n    observed = []\n    with torch.inference_mode():\n        for index, batch in enumerate(loader):\n            with torch.autocast(\n                device_type=device.type, dtype=torch.float16, enabled=use_cuda\n            ):\n                encoded = model.encode_images(\n                    batch[\"images\"].to(device, non_blocking=use_cuda)\n                )\n            features[index] = encoded[0].float().cpu().numpy().astype(np.float16)\n            masks[index] = batch[\"mask\"][0].numpy()\n            observed.extend(batch[\"uid\"])\n            if (index + 1) % 100 == 0 or index + 1 == len(uids):\n                features.flush()\n                masks.flush()\n                print(f\"cached {index + 1}/{len(uids)} studies\", flush=True)\n    if observed != uids:\n        raise AssertionError(\"feature cache UID order drift\")\n    pd.DataFrame({UID: uids}).to_csv(cache_dir / \"uids.csv\", index=False)\n    manifest = {\n        \"status\": \"COMPLETE\",\n        \"rows\": len(uids),\n        \"shape\": list(features.shape),\n        \"dtype\": str(features.dtype),\n        \"mask_shape\": list(masks.shape),\n        \"present_plane_fraction\": float(masks[:, :, 0].mean()),\n        \"laterality\": laterality_meta,\n        \"backbone_checkpoint_sha256\": sha256_file(Path(backbone_checkpoint)),\n        \"features_sha256\": sha256_file(feature_path),\n        \"mask_sha256\": sha256_file(mask_path),\n        \"uids_sha256\": sha256_file(cache_dir / \"uids.csv\"),\n        \"policy\": asdict(policy),\n    }\n    (cache_dir / \"cache_manifest.json\").write_text(\n        json.dumps(manifest, indent=2, sort_keys=True) + \"\\n\"\n    )\n    return manifest\n\n\ndef make_feature_dataset_class():\n    _, _, _, torch, _, _, Dataset = _training_imports()\n\n    class CachedFeatureDataset(Dataset):\n        def __init__(self, rows, cache_dir):\n            self.rows = rows.reset_index(drop=True)\n            self.cache_dir = Path(cache_dir)\n            order = pd.read_csv(self.cache_dir / \"uids.csv\")[UID].astype(str).tolist()\n            lookup = {uid: index for index, uid in enumerate(order)}\n            missing = set(self.rows[UID].astype(str)).difference(lookup)\n            if missing:\n                raise ValueError(f\"feature cache misses {len(missing)} studies\")\n            self.cache_indices = np.asarray(\n                [lookup[uid] for uid in self.rows[UID].astype(str)], dtype=np.int64\n            )\n            self.features = np.load(\n                self.cache_dir / \"features.float16.npy\", mmap_mode=\"r\"\n            )\n            self.masks = np.load(self.cache_dir / \"mask.bool.npy\", mmap_mode=\"r\")\n\n        def __len__(self):\n            return len(self.rows)\n\n        def __getitem__(self, index):\n            row = self.rows.iloc[index]\n            cache_index = int(self.cache_indices[index])\n            return {\n                \"uid\": str(row[UID]),\n                \"features\": torch.from_numpy(\n                    np.array(self.features[cache_index], dtype=np.float32, copy=True)\n                ),\n                \"mask\": torch.from_numpy(\n                    np.array(self.masks[cache_index], dtype=bool, copy=True)\n                ),\n                \"target\": torch.tensor(\n                    [row[f\"target::{target}\"] for target in TARGETS],\n                    dtype=torch.float32,\n                ),\n                \"weight\": torch.tensor(\n                    [row[f\"train_weight::{target}\"] for target in TARGETS],\n                    dtype=torch.float32,\n                ),\n            }\n\n    return CachedFeatureDataset\n\n\ndef _head_state(model):\n    return {\n        name: value.detach().cpu()\n        for name, value in model.state_dict().items()\n        if not name.startswith(\"backbone.\")\n    }\n\n\ndef train_cached_fold(\n    labels,\n    cache_dir,\n    output_dir,\n    fold,\n    backbone_checkpoint,\n    config,\n    policy,\n):\n    _, _, _, torch, nn, DataLoader, _ = _training_imports()\n    device = select_torch_device(torch)\n    use_cuda = device.type == \"cuda\"\n    seed = config.seed + 10_007 * int(fold)\n    random.seed(seed)\n    np.random.seed(seed)\n    torch.manual_seed(seed)\n    train_rows, valid_rows = BASE.prepare_fold_rows(labels, fold, config)\n    Dataset = make_feature_dataset_class()\n    Model = make_model_class(policy.feature_dropout)\n    sampler, sampler_meta = V2._make_expert_sampler(\n        train_rows, policy.expert_sample_fraction, seed, torch\n    )\n    train_loader = DataLoader(\n        Dataset(train_rows, cache_dir),\n        batch_size=policy.head_batch_size,\n        sampler=sampler,\n        num_workers=0,\n        pin_memory=use_cuda,\n    )\n    valid_loader = DataLoader(\n        Dataset(valid_rows, cache_dir),\n        batch_size=policy.head_batch_size,\n        shuffle=False,\n        num_workers=0,\n        pin_memory=use_cuda,\n    )\n    model = Model(config.backbone, backbone_checkpoint, 32).to(device)\n    model.set_backbone_stage(0)\n    parameters = [\n        parameter\n        for name, parameter in model.named_parameters()\n        if not name.startswith(\"backbone.\") and parameter.requires_grad\n    ]\n    optimizer = torch.optim.AdamW(\n        parameters, lr=policy.head_lr, weight_decay=policy.head_weight_decay\n    )\n    scheduler = torch.optim.lr_scheduler.CosineAnnealingLR(\n        optimizer, T_max=policy.head_epochs, eta_min=policy.head_lr * 0.05\n    )\n    loss_fn = nn.BCEWithLogitsLoss(reduction=\"none\")\n    scaler = torch.amp.GradScaler(\"cuda\", enabled=use_cuda)\n    history = []\n    last_truth = last_score = last_uids = None\n    for epoch in range(policy.head_epochs):\n        model.train()\n        losses, sampled_expert, sampled_total = [], 0, 0\n        for batch in train_loader:\n            features = batch[\"features\"].to(device, non_blocking=use_cuda)\n            mask = batch[\"mask\"].to(device, non_blocking=use_cuda)\n            target = batch[\"target\"].to(device, non_blocking=use_cuda)\n            weight = batch[\"weight\"].to(device, non_blocking=use_cuda)\n            sampled_expert += int(\n                (weight.min(dim=1).values >= config.gold_weight).sum().item()\n            )\n            sampled_total += len(weight)\n            optimizer.zero_grad(set_to_none=True)\n            with torch.autocast(\n                device_type=device.type, dtype=torch.float16, enabled=use_cuda\n            ):\n                raw = loss_fn(model.forward_encoded(features, mask), target)\n                loss = (raw * weight).sum() / weight.sum().clamp_min(1.0)\n            scaler.scale(loss).backward()\n            scaler.unscale_(optimizer)\n            torch.nn.utils.clip_grad_norm_(parameters, policy.gradient_clip_norm)\n            scaler.step(optimizer)\n            scaler.update()\n            losses.append(float(loss.detach().cpu()))\n        scheduler.step()\n        model.eval()\n        uids, truth, score = [], [], []\n        with torch.inference_mode():\n            for batch in valid_loader:\n                with torch.autocast(\n                    device_type=device.type, dtype=torch.float16, enabled=use_cuda\n                ):\n                    logits = model.forward_encoded(\n                        batch[\"features\"].to(device, non_blocking=use_cuda),\n                        batch[\"mask\"].to(device, non_blocking=use_cuda),\n                    )\n                uids.extend(batch[\"uid\"])\n                truth.append(batch[\"target\"].numpy())\n                score.append(torch.sigmoid(logits.float()).cpu().numpy())\n        last_truth = np.concatenate(truth)\n        last_score = np.concatenate(score)\n        last_uids = uids\n        macro, per_target = BASE.pooled_macro_auc(last_truth, last_score)\n        history.append(\n            {\n                \"epoch\": epoch,\n                \"train_loss\": float(np.mean(losses)),\n                \"learning_rate\": float(optimizer.param_groups[0][\"lr\"]),\n                \"observed_expert_fraction\": sampled_expert / max(sampled_total, 1),\n                \"gold_validation_macro_auc\": macro,\n                \"gold_validation_per_target_auc\": per_target,\n            }\n        )\n        print(\n            f\"fold {fold} epoch {epoch + 1}/{policy.head_epochs}: \"\n            f\"loss={history[-1]['train_loss']:.5f} gold_auc={macro:.5f}\",\n            flush=True,\n        )\n    if last_truth is None or last_score is None or last_uids is None:\n        raise AssertionError(\"fold produced no validation predictions\")\n    macro, per_target = BASE.pooled_macro_auc(last_truth, last_score)\n    output_dir.mkdir(parents=True, exist_ok=True)\n    oof = pd.DataFrame({UID: last_uids, \"fold\": int(fold)})\n    for target_index, target in enumerate(TARGETS):\n        oof[f\"gold::{target}\"] = last_truth[:, target_index]\n        oof[f\"pred::{target}\"] = last_score[:, target_index]\n    oof.to_csv(output_dir / f\"fold{fold}_expert_oof.csv\", index=False)\n    destination = output_dir / f\"fold{fold}_head.pt\"\n    torch.save(\n        {\n            \"head_state\": _head_state(model),\n            \"model_family\": policy.model_family,\n            \"backbone\": config.backbone,\n            \"config\": asdict(config),\n            \"physical_cache_policy\": asdict(policy),\n            \"sampler\": sampler_meta,\n            \"fold\": int(fold),\n            \"fold_diagnostic_macro_auc\": macro,\n            \"fold_diagnostic_per_target_auc\": per_target,\n            \"history\": history,\n            \"backbone_checkpoint_sha256\": sha256_file(Path(backbone_checkpoint)),\n        },\n        destination,\n    )\n    return destination\n\n\ndef prepare_labels(args, config):\n    paths = [\n        args.train_csv,\n        args.series_csv,\n        args.pilkwang_labels,\n        args.steven_labels,\n        args.lixin_labels,\n    ]\n    frames = [pd.read_csv(path) for path in paths]\n    labels = BASE.build_public_consensus(\n        frames[0], frames[2], frames[3], frames[4], config.pseudo_weight_floor\n    )\n    labels[\"fold\"] = BASE.assign_group_balanced_folds(\n        labels, config.folds, config.seed\n    )\n    return paths, frames, labels\n\n\ndef main(argv: Optional[Sequence[str]] = None) -> None:\n    parser = argparse.ArgumentParser(description=__doc__)\n    for name in (\n        \"train-csv\",\n        \"series-csv\",\n        \"pilkwang-labels\",\n        \"steven-labels\",\n        \"lixin-labels\",\n        \"output-dir\",\n    ):\n        parser.add_argument(f\"--{name}\", type=Path, required=True)\n    parser.add_argument(\"--image-root\", type=Path)\n    parser.add_argument(\"--backbone-checkpoint\", type=Path)\n    parser.add_argument(\"--cache-dir\", type=Path)\n    parser.add_argument(\"--prepare-only\", action=\"store_true\")\n    parser.add_argument(\"--cache-only\", action=\"store_true\")\n    parser.add_argument(\"--folds\", default=\"0,1,2,3,4\")\n    args = parser.parse_args(argv)\n\n    _, _, _, torch, _, _, _ = _training_imports()\n    device = select_torch_device(torch)\n    cpu_fallback = device.type == \"cpu\"\n    config = replace(\n        TrainConfig(),\n        image_size=192 if cpu_fallback else 288,\n        train_slices=4 if cpu_fallback else 32,\n        valid_slices=4 if cpu_fallback else 32,\n        batch_size=32,\n        workers=4,\n        backbone_chunk=8 if cpu_fallback else 12,\n        frozen_epochs=0,\n        unfrozen_epochs=0,\n        gold_weight=2.0,\n    )\n    policy = replace(\n        PhysicalCachePolicy(),\n        cache_slices=4 if cpu_fallback else 32,\n        head_epochs=8 if cpu_fallback else 12,\n        head_batch_size=64 if cpu_fallback else 32,\n    )\n    paths, frames, labels = prepare_labels(args, config)\n    args.output_dir.mkdir(parents=True, exist_ok=True)\n    labels.to_csv(args.output_dir / \"fold_labels.csv\", index=False)\n    manifest = {\n        \"config\": asdict(config),\n        \"physical_cache_policy\": asdict(policy),\n        \"rows\": len(labels),\n        \"expert_rows\": int(labels[\"is_expert\"].sum()),\n        \"fold_counts\": labels[\"fold\"].value_counts().sort_index().to_dict(),\n        \"fold_expert_counts\": labels.loc[\n            labels[\"is_expert\"].eq(1), \"fold\"\n        ].value_counts().sort_index().to_dict(),\n        \"inputs\": {str(path): sha256_file(path) for path in paths},\n        \"base_module_sha256\": sha256_file(Path(BASE.__file__)),\n        \"v2_module_sha256\": sha256_file(Path(V2.__file__)),\n        \"module_sha256\": sha256_file(Path(__file__)),\n    }\n    (args.output_dir / \"manifest.json\").write_text(\n        json.dumps(manifest, indent=2, sort_keys=True) + \"\\n\"\n    )\n    print(json.dumps(manifest, indent=2, sort_keys=True))\n    if args.prepare_only:\n        return\n    if args.image_root is None or args.backbone_checkpoint is None:\n        parser.error(\"training requires --image-root and --backbone-checkpoint\")\n    cache_dir = args.cache_dir or args.output_dir / \"feature_cache\"\n    cache_manifest = cache_dir / \"cache_manifest.json\"\n    if not cache_manifest.is_file():\n        build_feature_cache(\n            labels[UID].astype(str).tolist(),\n            frames[1],\n            args.image_root,\n            cache_dir,\n            args.backbone_checkpoint,\n            config,\n            policy,\n        )\n    if args.cache_only:\n        return\n    folds = [int(value) for value in args.folds.split(\",\") if value.strip()]\n    if not folds or any(fold < 0 or fold >= config.folds for fold in folds):\n        parser.error(f\"--folds must select values from [0, {config.folds - 1}]\")\n    checkpoints = [\n        train_cached_fold(\n            labels,\n            cache_dir,\n            args.output_dir,\n            fold,\n            args.backbone_checkpoint,\n            config,\n            policy,\n        )\n        for fold in folds\n    ]\n    oof = pd.concat(\n        [pd.read_csv(args.output_dir / f\"fold{fold}_expert_oof.csv\") for fold in folds],\n        ignore_index=True,\n    )\n    if len(folds) == config.folds and (\n        len(oof) != int(labels[\"is_expert\"].sum()) or oof[UID].duplicated().any()\n    ):\n        raise AssertionError(\"five-fold expert OOF coverage is incomplete\")\n    truth = oof[[f\"gold::{target}\" for target in TARGETS]].to_numpy(float)\n    score = oof[[f\"pred::{target}\" for target in TARGETS]].to_numpy(float)\n    macro, per_target = BASE.pooled_macro_auc(truth, score)\n    summary = {\n        \"status\": \"COMPLETE\",\n        \"folds\": folds,\n        \"expert_oof_rows\": len(oof),\n        \"expert_oof_macro_auc\": macro,\n        \"expert_oof_per_target_auc\": per_target,\n        \"checkpoints\": {path.name: sha256_file(path) for path in checkpoints},\n        \"cache_manifest_sha256\": sha256_file(cache_manifest),\n    }\n    oof.to_csv(args.output_dir / \"expert_oof.csv\", index=False)\n    (args.output_dir / \"training_summary.json\").write_text(\n        json.dumps(summary, indent=2, sort_keys=True) + \"\\n\"\n    )\n    print(json.dumps(summary, indent=2, sort_keys=True))\n\n\nif __name__ == \"__main__\":\n    main()\n"},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"%%writefile /kaggle/working/rsna_b3_v43_source/efficientnet_b3_public_repro_v3_infer.py\n#!/usr/bin/env python3\n\"\"\"Five-head inference for the physical cached EfficientNet-B3 v3 model.\"\"\"\n\nfrom __future__ import annotations\n\nimport argparse\nimport hashlib\nimport importlib.util\nimport json\nimport sys\nimport time\nfrom dataclasses import asdict\nfrom pathlib import Path\nfrom typing import Optional, Sequence\n\nimport numpy as np\nimport pandas as pd\n\n\ndef load_module(path: Path):\n    name = \"rsna_knee_effnet_b3_public_repro_v3_inference_dependency\"\n    spec = importlib.util.spec_from_file_location(name, path)\n    if spec is None or spec.loader is None:\n        raise RuntimeError(f\"cannot import model module: {path}\")\n    module = importlib.util.module_from_spec(spec)\n    sys.modules[name] = module\n    spec.loader.exec_module(module)\n    return module\n\n\ndef sha256_file(path: Path, chunk_size: int = 8 << 20) -> str:\n    digest = hashlib.sha256()\n    with path.open(\"rb\") as handle:\n        for chunk in iter(lambda: handle.read(chunk_size), b\"\"):\n            digest.update(chunk)\n    return digest.hexdigest()\n\n\ndef load_ensemble(module, checkpoint_paths, backbone_checkpoint, device):\n    \"\"\"Load one shared encoder and five lightweight heads with strict contracts.\"\"\"\n    _, _, _, torch, _, _, _ = module._training_imports()\n    checkpoints = [\n        torch.load(str(path), map_location=\"cpu\", weights_only=False)\n        for path in checkpoint_paths\n    ]\n    if len(checkpoints) != 5:\n        raise ValueError(f\"expected five fold heads, got {len(checkpoints)}\")\n    folds = [int(checkpoint.get(\"fold\", -1)) for checkpoint in checkpoints]\n    if sorted(folds) != list(range(5)):\n        raise ValueError(f\"fold heads must be exactly 0..4, got {folds}\")\n    configs = [checkpoint[\"config\"] for checkpoint in checkpoints]\n    policies = [checkpoint[\"physical_cache_policy\"] for checkpoint in checkpoints]\n    if any(config != configs[0] for config in configs[1:]):\n        raise ValueError(\"fold configurations disagree\")\n    if any(policy != policies[0] for policy in policies[1:]):\n        raise ValueError(\"fold physical-cache policies disagree\")\n    expected_backbone_hash = sha256_file(Path(backbone_checkpoint))\n    recorded_hashes = {\n        str(checkpoint.get(\"backbone_checkpoint_sha256\", \"\"))\n        for checkpoint in checkpoints\n    }\n    if recorded_hashes != {expected_backbone_hash}:\n        raise ValueError(\n            \"public backbone hash disagrees with one or more fold checkpoints\"\n        )\n\n    config = module.TrainConfig(**configs[0])\n    policy = module.PhysicalCachePolicy(**policies[0])\n    Model = module.make_model_class(policy.feature_dropout)\n    models = []\n    for index, checkpoint in enumerate(checkpoints):\n        model = Model(\n            config.backbone,\n            backbone_checkpoint if index == 0 else None,\n            config.backbone_chunk,\n        )\n        expected_missing = {\n            name\n            for name in model.state_dict()\n            if name.startswith(\"backbone.\")\n            and not name.endswith(\".num_batches_tracked\")\n        }\n        result = model.load_state_dict(checkpoint[\"head_state\"], strict=False)\n        if set(result.missing_keys) != expected_missing or result.unexpected_keys:\n            raise RuntimeError(\n                f\"fold {folds[index]} head-state contract mismatch: {result}\"\n            )\n        if index > 0:\n            model.backbone = torch.nn.Identity()\n        model.to(device).eval()\n        models.append(model)\n    return models[0], models, config, policy, folds, expected_backbone_hash\n\n\ndef predict(\n    module,\n    test: pd.DataFrame,\n    series: pd.DataFrame,\n    image_root: Path,\n    encoder,\n    heads,\n    config,\n    policy,\n    device,\n    checkpoint_every: int,\n):\n    _, _, _, torch, _, DataLoader, _ = module._training_imports()\n    use_cuda = device.type == \"cuda\"\n    uids = test[module.UID].astype(str).tolist()\n    if not uids or len(set(uids)) != len(uids):\n        raise ValueError(\"test StudyInstanceUID values must be nonempty and unique\")\n    selection = module.build_series_selection(series)\n    laterality, laterality_meta = module.build_laterality_map(\n        uids, selection, image_root, policy.laterality_dead_zone_mm\n    )\n    Dataset = module.make_physical_dataset_class()\n    loader = DataLoader(\n        Dataset(uids, selection, laterality, image_root, config, policy),\n        batch_size=1,\n        shuffle=False,\n        num_workers=config.workers,\n        pin_memory=use_cuda,\n        persistent_workers=config.workers > 0,\n    )\n    scores = np.empty((len(uids), len(module.TARGETS)), dtype=np.float32)\n    observed = []\n    started = time.monotonic()\n    with torch.inference_mode():\n        for index, batch in enumerate(loader):\n            images = batch[\"images\"].to(device, non_blocking=use_cuda)\n            mask = batch[\"mask\"].to(device, non_blocking=use_cuda)\n            with torch.autocast(\n                device_type=device.type, dtype=torch.float16, enabled=use_cuda\n            ):\n                features = encoder.encode_images(images)\n                probabilities = torch.stack(\n                    [\n                        torch.sigmoid(head.forward_encoded(features, mask).float())\n                        for head in heads\n                    ]\n                ).mean(dim=0)\n            scores[index] = probabilities[0].cpu().numpy()\n            observed.extend(batch[\"uid\"])\n            done = index + 1\n            if done % checkpoint_every == 0 or done == len(uids):\n                elapsed = time.monotonic() - started\n                projected_hours = elapsed / done * len(uids) / 3600.0\n                print(\n                    f\"predicted {done}/{len(uids)} studies; \"\n                    f\"projected={projected_hours:.2f}h\",\n                    flush=True,\n                )\n    if observed != uids:\n        raise AssertionError(\"inference UID order drift\")\n    if not np.isfinite(scores).all() or not ((0.0 <= scores) & (scores <= 1.0)).all():\n        raise AssertionError(\"inference produced invalid probabilities\")\n    return scores, laterality_meta, time.monotonic() - started\n\n\ndef main(argv: Optional[Sequence[str]] = None) -> None:\n    parser = argparse.ArgumentParser(description=__doc__)\n    parser.add_argument(\"--module\", type=Path, required=True)\n    parser.add_argument(\"--test-csv\", type=Path, required=True)\n    parser.add_argument(\"--series-csv\", type=Path, required=True)\n    parser.add_argument(\"--image-root\", type=Path, required=True)\n    parser.add_argument(\"--backbone-checkpoint\", type=Path, required=True)\n    parser.add_argument(\"--head-checkpoints\", type=Path, nargs=5, required=True)\n    parser.add_argument(\"--output-dir\", type=Path, required=True)\n    parser.add_argument(\"--checkpoint-every\", type=int, default=25)\n    args = parser.parse_args(argv)\n    if args.checkpoint_every <= 0:\n        parser.error(\"--checkpoint-every must be positive\")\n\n    module = load_module(args.module)\n    _, _, _, torch, _, _, _ = module._training_imports()\n    device = module.select_torch_device(torch)\n    encoder, heads, config, policy, folds, backbone_hash = load_ensemble(\n        module, args.head_checkpoints, args.backbone_checkpoint, device\n    )\n    test, series = pd.read_csv(args.test_csv), pd.read_csv(args.series_csv)\n    scores, laterality, elapsed = predict(\n        module,\n        test,\n        series,\n        args.image_root,\n        encoder,\n        heads,\n        config,\n        policy,\n        device,\n        args.checkpoint_every,\n    )\n    submission = pd.DataFrame(scores, columns=module.TARGETS)\n    submission.insert(0, module.UID, test[module.UID].astype(str).tolist())\n    if list(submission.columns) != [module.UID, *module.TARGETS]:\n        raise AssertionError(\"submission schema drift\")\n    args.output_dir.mkdir(parents=True, exist_ok=True)\n    destination = args.output_dir / \"submission.csv\"\n    submission.to_csv(destination, index=False)\n    audit = {\n        \"status\": \"COMPLETE\",\n        \"rows\": len(submission),\n        \"finite\": True,\n        \"folds\": folds,\n        \"elapsed_seconds\": elapsed,\n        \"laterality\": laterality,\n        \"config\": asdict(config),\n        \"physical_cache_policy\": asdict(policy),\n        \"backbone_checkpoint_sha256\": backbone_hash,\n        \"head_checkpoint_sha256\": {\n            path.name: sha256_file(path) for path in args.head_checkpoints\n        },\n        \"submission_sha256\": sha256_file(destination),\n        \"module_sha256\": sha256_file(args.module),\n        \"inference_runner_sha256\": sha256_file(Path(__file__)),\n    }\n    (args.output_dir / \"inference_audit.json\").write_text(\n        json.dumps(audit, indent=2, sort_keys=True) + \"\\n\"\n    )\n    print(json.dumps(audit, indent=2, sort_keys=True))\n\n\nif __name__ == \"__main__\":\n    main()\n"},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"# Guarded B3 training branch.  The V40 public specialist remains the fallback.\nimport hashlib as _b3_hashlib\nimport json as _b3_json\nimport shutil as _b3_shutil\nimport subprocess as _b3_subprocess\nimport sys as _b3_sys\nimport traceback as _b3_traceback\nfrom pathlib import Path as _B3Path\n\n_B3_SOURCE = _B3Path(\"/kaggle/working/rsna_b3_v43_source\")\n_B3_TRAIN = _B3Path(\"/kaggle/working/rsna_b3_v43_training\")\n_B3_INFER = _B3Path(\"/kaggle/working/rsna_b3_v43_inference\")\n_B3_CACHE = _B3Path(\"/kaggle/temp/rsna_b3_v43_cache\")\n_B3_COMP_CANDIDATES = (\n    _B3Path(\"/kaggle/input/competitions/rsna-knee-abnormality-detection\"),\n    _B3Path(\"/kaggle/input/rsna-knee-abnormality-detection\"),\n)\n_B3_COMP = next(\n    (_path for _path in _B3_COMP_CANDIDATES if (_path / \"train.csv\").is_file()),\n    _B3_COMP_CANDIDATES[0],\n)\n_B3_BASELINE = _B3Path(\"/kaggle/working/submission_public_0899.csv\")\n_B3_COMPETITION_RUNTIME_LIMIT_SECONDS = 9 * 60 * 60\n_B3_TRAIN_TIMEOUT_SECONDS = 5 * 60 * 60\n_B3_INFER_TIMEOUT_SECONDS = 2 * 60 * 60\n\n_B3_EXPECTED = {\n    _B3_SOURCE / \"efficientnet_b3_public_repro_v1.py\": \"2548480302b175aa15700888902835c91c377deec935a4edfe2a6a978e3ac232\",\n    _B3_SOURCE / \"efficientnet_b3_public_repro_v2_anatomy.py\": \"17a8a8c99afc02e6e75928cf7db2d014609d31d1280a1f05b07f15564c198696\",\n    _B3_SOURCE / \"efficientnet_b3_public_repro_v3_physical_cache.py\": \"c1cd1721d26e068834f1a367c96ca0ef8cff9ad1381312c72169404a168fe9c8\",\n    _B3_SOURCE / \"efficientnet_b3_public_repro_v3_infer.py\": \"a6f98f3827ad9e84613f155f7127b0069f237387aa1fee46fe93e35be09548e9\",\n    _B3_COMP / \"train.csv\": \"8ca2203c0e9d61c080c7a314c7cdb51c1b03a1d9eb4770819f7f34af53ef4e33\",\n    _B3_COMP / \"train_series.csv\": \"573c1d80772bf41211c91b149c95677385a1c22d63f485c347f1b46c0177aef3\",\n    _B3Path(\"/kaggle/input/rsna-knee-llm-labels/report_labels_v2.csv\"): \"6f704a7bdb2f894cc49445b19ba7c4378c3f548d3449e00361e10044bee40920\",\n    _B3Path(\"/kaggle/input/rsna-knee-llm-report-labels/llm_labels_v2.csv\"): \"3c082a987939528a55202028f655bd6f9dc5c9e28eff5058bbae7ef088a9f144\",\n    _B3Path(\"/kaggle/input/rsna-knee-llm-report-labels-sol56/labels_llm_gpt56sol.csv\"): \"a79b05e609336811d51c97f9a48f00ea8004244a1a80fe886504a62bb39867ae\",\n    _B3Path(\"/kaggle/input/timm-pretrained-weights/efficientnet_b3.pth\"): \"b2fd73dffb7386bf35ed9f571e98c1ab53647ed435268eb629c307ad221b5b31\",\n}\n\ndef _b3_sha(path):\n    digest = _b3_hashlib.sha256()\n    with open(path, \"rb\") as handle:\n        for chunk in iter(lambda: handle.read(8 << 20), b\"\"):\n            digest.update(chunk)\n    return digest.hexdigest()\n\n_b3_audit = {\n    \"status\": \"FALLBACK\",\n    \"evidence_boundary\": \"The B3 branch is experimental until an official completed submission supplies a score.\",\n    \"competition_runtime_limit_seconds\": _B3_COMPETITION_RUNTIME_LIMIT_SECONDS,\n    \"train_timeout_seconds\": _B3_TRAIN_TIMEOUT_SECONDS,\n    \"infer_timeout_seconds\": _B3_INFER_TIMEOUT_SECONDS,\n    \"expected_inputs\": {str(path): value for path, value in _B3_EXPECTED.items()},\n}\n\ntry:\n    _B3_SOURCE.mkdir(parents=True, exist_ok=True)\n    _B3_TRAIN.mkdir(parents=True, exist_ok=True)\n    _B3_INFER.mkdir(parents=True, exist_ok=True)\n    if not _B3_BASELINE.is_file():\n        raise FileNotFoundError(\"the independently reproduced public .899 fallback is absent\")\n    _b3_observed = {str(path): _b3_sha(path) for path in _B3_EXPECTED}\n    _b3_bad = {\n        str(path): {\"expected\": expected, \"observed\": _b3_observed[str(path)]}\n        for path, expected in _B3_EXPECTED.items()\n        if _b3_observed[str(path)] != expected\n    }\n    if _b3_bad:\n        raise RuntimeError(f\"pinned B3 input drift: {_b3_bad}\")\n\n    _b3_train_cmd = [\n        _b3_sys.executable,\n        str(_B3_SOURCE / \"efficientnet_b3_public_repro_v3_physical_cache.py\"),\n        \"--train-csv\", str(_B3_COMP / \"train.csv\"),\n        \"--series-csv\", str(_B3_COMP / \"train_series.csv\"),\n        \"--image-root\", str(_B3_COMP / \"train_series\"),\n        \"--pilkwang-labels\", \"/kaggle/input/rsna-knee-llm-labels/report_labels_v2.csv\",\n        \"--steven-labels\", \"/kaggle/input/rsna-knee-llm-report-labels/llm_labels_v2.csv\",\n        \"--lixin-labels\", \"/kaggle/input/rsna-knee-llm-report-labels-sol56/labels_llm_gpt56sol.csv\",\n        \"--backbone-checkpoint\", \"/kaggle/input/timm-pretrained-weights/efficientnet_b3.pth\",\n        \"--cache-dir\", str(_B3_CACHE),\n        \"--output-dir\", str(_B3_TRAIN),\n        \"--folds\", \"0,1,2,3,4\",\n    ]\n    print(\"[B3 V43] building one frozen B3 feature cache and five MIL heads\", flush=True)\n    _b3_subprocess.run(\n        _b3_train_cmd,\n        check=True,\n        timeout=_B3_TRAIN_TIMEOUT_SECONDS,\n    )\n    _b3_summary = _b3_json.loads((_B3_TRAIN / \"training_summary.json\").read_text())\n    if _b3_summary.get(\"status\") != \"COMPLETE\":\n        raise RuntimeError(f\"B3 training is not terminal: {_b3_summary}\")\n    if int(_b3_summary.get(\"expert_oof_rows\", -1)) != 58:\n        raise RuntimeError(\"B3 expert OOF coverage is not exactly 58 unique studies\")\n    _b3_oof_auc = float(_b3_summary[\"expert_oof_macro_auc\"])\n\n    _b3_infer_cmd = [\n        _b3_sys.executable,\n        str(_B3_SOURCE / \"efficientnet_b3_public_repro_v3_infer.py\"),\n        \"--module\", str(_B3_SOURCE / \"efficientnet_b3_public_repro_v3_physical_cache.py\"),\n        \"--test-csv\", str(_B3_COMP / \"test.csv\"),\n        \"--series-csv\", str(_B3_COMP / \"test_series.csv\"),\n        \"--image-root\", str(_B3_COMP / \"test_series\"),\n        \"--backbone-checkpoint\", \"/kaggle/input/timm-pretrained-weights/efficientnet_b3.pth\",\n        \"--head-checkpoints\",\n        *[str(_B3_TRAIN / f\"fold{fold}_head.pt\") for fold in range(5)],\n        \"--output-dir\", str(_B3_INFER),\n    ]\n    print(f\"[B3 V43] honest 58-study OOF macro AUC={_b3_oof_auc:.6f}; inferring\", flush=True)\n    _b3_subprocess.run(\n        _b3_infer_cmd,\n        check=True,\n        timeout=_B3_INFER_TIMEOUT_SECONDS,\n    )\n\n    _b3_base = pd.read_csv(_B3_BASELINE)\n    _b3_pred = pd.read_csv(_B3_INFER / \"submission.csv\")\n    _b3_cols = list(_b3_base.columns)\n    if _b3_cols != [UID, *TARGETS] or list(_b3_pred.columns) != _b3_cols:\n        raise RuntimeError(\"B3/baseline submission schema drift\")\n    if _b3_base[UID].astype(str).tolist() != _b3_pred[UID].astype(str).tolist():\n        raise RuntimeError(\"B3/baseline UID order drift\")\n    _b3_raw = _b3_pred[TARGETS].to_numpy(float)\n    _b3_base_raw = _b3_base[TARGETS].to_numpy(float)\n    if not (np.isfinite(_b3_raw).all() and np.isfinite(_b3_base_raw).all()):\n        raise RuntimeError(\"B3/baseline contains non-finite probabilities\")\n    # Macro AUC depends on ordering, not calibration.  Rank both parents target by\n    # target before mixing so a compressed soft-label head cannot be numerically\n    # drowned out by a more dispersed parent with the same ordering quality.\n    _b3_rank = _b3_pred[TARGETS].rank(method=\"average\", pct=True).to_numpy(float)\n    _b3_base_rank = _b3_base[TARGETS].rank(method=\"average\", pct=True).to_numpy(float)\n\n    _b3_standalone = _b3_pred.copy()\n    _b3_standalone.to_csv(\"/kaggle/working/submission_b3_v43.csv\", index=False)\n    _b3_variants = {}\n    for _b3_alpha in (0.25, 0.50, 0.75):\n        _b3_blend = _b3_base.copy()\n        _b3_blend[TARGETS] = (1.0 - _b3_alpha) * _b3_base_rank + _b3_alpha * _b3_rank\n        _b3_name = f\"submission_b3_blend_{int(100 * _b3_alpha):02d}.csv\"\n        _b3_blend.to_csv(_B3Path(\"/kaggle/working\") / _b3_name, index=False)\n        _b3_variants[_b3_name] = _b3_sha(_B3Path(\"/kaggle/working\") / _b3_name)\n\n    # The scored reference reached .903 with a B3 specialist.  Use a conservative\n    # 75% B3 / 25% independently reproduced public specialist rank blend only when the honest\n    # expert OOF clears a prespecified 0.84 gate; otherwise preserve the fallback.\n    if _b3_oof_auc >= 0.84:\n        _b3_shutil.copy2(\n            \"/kaggle/working/submission_b3_blend_75.csv\",\n            \"/kaggle/working/submission.csv\",\n        )\n        _b3_promoted = \"submission_b3_blend_75.csv\"\n    else:\n        _b3_shutil.copy2(_B3_BASELINE, \"/kaggle/working/submission.csv\")\n        _b3_promoted = _B3_BASELINE.name\n    _b3_audit.update({\n        \"status\": \"COMPLETE\",\n        \"expert_oof_macro_auc\": _b3_oof_auc,\n        \"promotion_gate\": 0.84,\n        \"promoted\": _b3_promoted,\n        \"primary_sha256\": _b3_sha(\"/kaggle/working/submission.csv\"),\n        \"standalone_sha256\": _b3_sha(\"/kaggle/working/submission_b3_v43.csv\"),\n        \"blend_sha256\": _b3_variants,\n        \"observed_inputs\": _b3_observed,\n    })\nexcept Exception as _b3_error:\n    print(f\"[B3 V43] rejected safely: {type(_b3_error).__name__}: {_b3_error}\", flush=True)\n    _b3_traceback.print_exc()\n    if _B3_BASELINE.is_file():\n        _b3_shutil.copy2(_B3_BASELINE, \"/kaggle/working/submission.csv\")\n        _b3_audit[\"primary_sha256\"] = _b3_sha(\"/kaggle/working/submission.csv\")\n    _b3_audit[\"error\"] = f\"{type(_b3_error).__name__}: {_b3_error}\"\n\n_B3Path(\"/kaggle/working/b3_v43_audit.json\").write_text(\n    _b3_json.dumps(_b3_audit, indent=2, sort_keys=True) + \"\\n\"\n)\nprint(_b3_json.dumps(_b3_audit, indent=2, sort_keys=True), flush=True)\n"},{"cell_type":"markdown","metadata":{},"source":"## 12. Two-T4 final-block fine-tuning from the public `.903` recipe\n\nThe scored `.903` notebook discloses its full pipeline but not its five private fold weights. This independent branch retains its decisive public choices: EfficientNet-B3, one fluid-sensitive series per plane, InstanceNumber ordering, 1st–99th percentile windowing, square padding, uint8 roundtrip, five group-aware folds, no horizontal flip, and max-sensitive multi-instance pooling. The official versioned timm backbone is hash-pinned, and folds are trained in waves on two T4s.\n\nAll 58 expert studies are scored strictly out of fold. The established `.899` public-specialist artifact remains `submission.csv`; standalone and rank-blended V47 candidates are separate outputs until an official completed competition row can establish their score."},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"from pathlib import Path as _B3SourcePath\n_B3SourcePath('/kaggle/working/rsna_b3_v47_source').mkdir(parents=True, exist_ok=True)\n"},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"%%writefile /kaggle/working/rsna_b3_v47_source/efficientnet_b3_public_repro_v1.py\n#!/usr/bin/env python3\n\"\"\"Leakage-controlled EfficientNet-B3 MIL training for RSNA Knee.\"\"\"\n\nfrom __future__ import annotations\n\nimport argparse\nimport hashlib\nimport json\nimport math\nimport random\nimport re\nimport unicodedata\nfrom dataclasses import asdict, dataclass\nfrom pathlib import Path\nfrom typing import Dict, List, Mapping, Optional, Sequence, Tuple\n\nimport numpy as np\nimport pandas as pd\n\nTARGETS: Tuple[str, ...] = (\n    \"ACL\", \"MCL\", \"Medial Meniscus\", \"Lateral Meniscus\", \"Medial OA\",\n    \"Lateral OA\", \"PF OA\", \"Effusion\", \"Synovitis\", \"Baker's\",\n    \"Contusion\", \"Fracture\",\n)\nPLANES: Tuple[str, ...] = (\"Sagittal\", \"Coronal\", \"Axial\")\nUID = \"StudyInstanceUID\"\n\n\n@dataclass(frozen=True)\nclass TrainConfig:\n    image_size: int = 288\n    max_series_slices: int = 64\n    train_slices: int = 20\n    valid_slices: int = 32\n    folds: int = 5\n    seed: int = 20260809\n    batch_size: int = 2\n    workers: int = 4\n    backbone_chunk: int = 12\n    frozen_epochs: int = 1\n    unfrozen_epochs: int = 2\n    frozen_lr: float = 8e-4\n    unfrozen_lr: float = 8e-5\n    weight_decay: float = 1e-4\n    gold_weight: float = 3.0\n    pseudo_weight_floor: float = 0.20\n    backbone: str = \"efficientnet_b3\"\n\n\n_DATE = re.compile(\n    r\"\\b(?:19|20)\\d{2}[-/.]\\d{1,2}[-/.]\\d{1,2}\\b|\"\n    r\"\\b\\d{1,2}[-/.]\\d{1,2}[-/.](?:\\d{2}|\\d{4})\\b\"\n)\n_NUMBER = re.compile(r\"\\b\\d+(?:[.,]\\d+)?\\b\")\n_SPACE = re.compile(r\"\\s+\")\n\n\ndef normalize_report_for_group(value: object) -> str:\n    text = \"\" if value is None else str(value)\n    text = unicodedata.normalize(\"NFKC\", text).casefold()\n    text = _DATE.sub(\" <date> \", text)\n    text = _NUMBER.sub(\" <num> \", text)\n    return _SPACE.sub(\" \", text).strip()\n\n\ndef report_group_id(value: object) -> str:\n    return hashlib.sha256(normalize_report_for_group(value).encode()).hexdigest()[:24]\n\n\ndef _indexed_labels(frame: pd.DataFrame, name: str) -> pd.DataFrame:\n    required = {UID, *TARGETS}\n    missing = required.difference(frame.columns)\n    if missing:\n        raise ValueError(f\"{name} is missing columns: {sorted(missing)}\")\n    if frame[UID].duplicated().any():\n        raise ValueError(f\"{name} contains duplicate {UID} rows\")\n    return frame[[UID, *TARGETS]].copy().set_index(UID).apply(pd.to_numeric, errors=\"coerce\")\n\n\ndef build_public_consensus(\n    train: pd.DataFrame,\n    pilkwang: pd.DataFrame,\n    steven: pd.DataFrame,\n    lixin: pd.DataFrame,\n    pseudo_weight_floor: float = 0.20,\n) -> pd.DataFrame:\n    \"\"\"Equal-source soft labels; weights come only from agreement/certainty.\"\"\"\n    if train[UID].duplicated().any():\n        raise ValueError(f\"train contains duplicate {UID} rows\")\n    base = train[[UID, \"Report\", *TARGETS]].copy().set_index(UID)\n    sources = [\n        _indexed_labels(pilkwang, \"pilkwang\"),\n        _indexed_labels(steven, \"steven\"),\n        _indexed_labels(lixin, \"lixin\"),\n    ]\n    out = base.reset_index()[[UID, \"Report\"]].copy()\n    out[\"report_group\"] = out[\"Report\"].map(report_group_id)\n    out = out.drop(columns=\"Report\")\n    cube = np.stack([source.reindex(base.index).to_numpy(float) for source in sources])\n    if np.nanmin(cube) < 0.0 or np.nanmax(cube) > 1.0:\n        raise ValueError(\"public labels must be in [0, 1]\")\n    available = np.isfinite(cube).sum(axis=0)\n    if np.any(available < 2):\n        row, target = np.argwhere(available < 2)[0]\n        raise ValueError(f\"fewer than two public labels at row={row}, target={TARGETS[target]}\")\n    consensus = np.nanmean(cube, axis=0)\n    disagreement = np.nanmean(np.abs(cube - consensus[None]), axis=0)\n    agreement = np.clip(1.0 - 2.0 * disagreement, 0.0, 1.0)\n    certainty = np.clip(2.0 * np.abs(consensus - 0.5), 0.0, 1.0)\n    weight = pseudo_weight_floor + (1.0 - pseudo_weight_floor) * (\n        0.65 * agreement + 0.35 * certainty\n    )\n    out[\"is_expert\"] = base[list(TARGETS)].notna().all(axis=1).astype(np.int8).to_numpy()\n    for j, target in enumerate(TARGETS):\n        out[f\"pseudo::{target}\"] = consensus[:, j].astype(np.float32)\n        out[f\"weight::{target}\"] = weight[:, j].astype(np.float32)\n        out[f\"gold::{target}\"] = pd.to_numeric(base[target], errors=\"coerce\").to_numpy()\n    return out\n\n\ndef _group_statistics(label_table: pd.DataFrame) -> pd.DataFrame:\n    pseudo_cols = [f\"pseudo::{target}\" for target in TARGETS]\n    gold_cols = [f\"gold::{target}\" for target in TARGETS]\n    records = []\n    for group_id, group in label_table.groupby(\"report_group\", sort=True):\n        gold = group[gold_cols].to_numpy(float)\n        records.append({\n            \"report_group\": group_id,\n            \"size\": len(group),\n            \"expert_count\": int(group[\"is_expert\"].sum()),\n            \"pseudo_sum\": group[pseudo_cols].to_numpy(float).sum(axis=0),\n            \"gold_positive\": np.nansum(gold, axis=0),\n            \"gold_known\": np.isfinite(gold).sum(axis=0).astype(float),\n        })\n    return pd.DataFrame(records)\n\n\ndef assign_group_balanced_folds(\n    label_table: pd.DataFrame, n_splits: int = 5, seed: int = 20260809\n) -> pd.Series:\n    \"\"\"Deterministic greedy balance with normalized-report groups kept atomic.\"\"\"\n    if n_splits < 2:\n        raise ValueError(\"n_splits must be at least two\")\n    groups = _group_statistics(label_table)\n    target_size = float(groups[\"size\"].sum()) / n_splits\n    target_pseudo = np.sum(np.stack(groups[\"pseudo_sum\"]), axis=0) / n_splits\n    total_gold_pos = np.sum(np.stack(groups[\"gold_positive\"]), axis=0)\n    target_gold_pos = total_gold_pos / n_splits\n    target_gold_known = np.sum(np.stack(groups[\"gold_known\"]), axis=0) / n_splits\n    rng = np.random.default_rng(seed)\n    rarity = 1.0 / np.maximum(total_gold_pos, 1.0)\n    priority = [\n        100.0 * row[\"expert_count\"]\n        + 10.0 * float(np.dot(row[\"gold_positive\"], rarity))\n        + math.log1p(row[\"size\"])\n        for _, row in groups.iterrows()\n    ]\n    groups = groups.assign(_priority=priority, _jitter=rng.uniform(0, 1e-6, len(groups)))\n    groups = groups.sort_values([\"_priority\", \"_jitter\", \"report_group\"], ascending=[False, False, True])\n    fold_size = np.zeros(n_splits)\n    fold_pseudo = np.zeros((n_splits, len(TARGETS)))\n    fold_gold_pos = np.zeros_like(fold_pseudo)\n    fold_gold_known = np.zeros_like(fold_pseudo)\n    assignment: Dict[str, int] = {}\n    for _, row in groups.iterrows():\n        scores = []\n        for fold in range(n_splits):\n            candidate_size = fold_size.copy()\n            candidate_pseudo = fold_pseudo.copy()\n            candidate_gold_pos = fold_gold_pos.copy()\n            candidate_gold_known = fold_gold_known.copy()\n            candidate_size[fold] += row[\"size\"]\n            candidate_pseudo[fold] += row[\"pseudo_sum\"]\n            candidate_gold_pos[fold] += row[\"gold_positive\"]\n            candidate_gold_known[fold] += row[\"gold_known\"]\n            size_cost = np.mean(((candidate_size - target_size) / max(target_size, 1.0)) ** 2)\n            pseudo_cost = np.mean(((candidate_pseudo - target_pseudo[None]) / np.maximum(target_pseudo[None], 1.0)) ** 2)\n            gold_pos_cost = np.mean(((candidate_gold_pos - target_gold_pos[None]) / np.maximum(target_gold_pos[None], 1.0)) ** 2)\n            gold_known_cost = np.mean(((candidate_gold_known - target_gold_known[None]) / np.maximum(target_gold_known[None], 1.0)) ** 2)\n            scores.append(size_cost + 0.35 * pseudo_cost + 4.0 * gold_pos_cost + 2.0 * gold_known_cost)\n        best = int(np.argmin(np.asarray(scores) + 1e-12 * np.arange(n_splits)))\n        assignment[str(row[\"report_group\"])] = best\n        fold_size[best] += row[\"size\"]\n        fold_pseudo[best] += row[\"pseudo_sum\"]\n        fold_gold_pos[best] += row[\"gold_positive\"]\n        fold_gold_known[best] += row[\"gold_known\"]\n    result = label_table[\"report_group\"].map(assignment)\n    if result.isna().any():\n        raise AssertionError(\"some report groups were not assigned\")\n    return result.astype(np.int8)\n\n\ndef pick_one_series_per_plane(series: pd.DataFrame) -> Dict[str, Optional[str]]:\n    required = {\"SeriesInstanceUID\", \"Anatomical_Plane\", \"Fluid_Sensitive\", \"Fat_Suppression\"}\n    missing = required.difference(series.columns)\n    if missing:\n        raise ValueError(f\"series table missing columns: {sorted(missing)}\")\n    chosen: Dict[str, Optional[str]] = {}\n    for plane in PLANES:\n        subset = series.loc[series[\"Anatomical_Plane\"].eq(plane)].copy()\n        if subset.empty:\n            chosen[plane] = None\n            continue\n        for column in (\"Fluid_Sensitive\", \"Fat_Suppression\"):\n            subset[column] = pd.to_numeric(subset[column], errors=\"coerce\").fillna(0)\n        subset = subset.sort_values(\n            [\"Fluid_Sensitive\", \"Fat_Suppression\", \"SeriesInstanceUID\"],\n            ascending=[False, False, True],\n        )\n        chosen[plane] = str(subset.iloc[0][\"SeriesInstanceUID\"])\n    return chosen\n\n\ndef pick_series_slots(series: pd.DataFrame, count: int = 3) -> List[Optional[str]]:\n    \"\"\"Prefer one fluid-sensitive series per plane, then fill empty slots.\"\"\"\n    required = {\"SeriesInstanceUID\", \"Anatomical_Plane\", \"Fluid_Sensitive\", \"Fat_Suppression\"}\n    missing = required.difference(series.columns)\n    if missing:\n        raise ValueError(f\"series table missing columns: {sorted(missing)}\")\n    frame = series.copy()\n    for column in (\"Fluid_Sensitive\", \"Fat_Suppression\"):\n        frame[column] = pd.to_numeric(frame[column], errors=\"coerce\").fillna(0)\n    frame[\"_plane\"] = frame[\"Anatomical_Plane\"].map({p: i for i, p in enumerate(PLANES)}).fillna(3)\n    preferred = frame.loc[frame[\"Fluid_Sensitive\"].eq(1)].sort_values(\n        [\"_plane\", \"Fat_Suppression\", \"SeriesInstanceUID\"],\n        ascending=[True, False, True],\n    )\n    selected: List[str] = []\n    seen_planes = set()\n    for _, row in preferred.iterrows():\n        plane = str(row[\"Anatomical_Plane\"])\n        if plane not in seen_planes:\n            selected.append(str(row[\"SeriesInstanceUID\"]))\n            seen_planes.add(plane)\n        if len(selected) == count:\n            break\n    ordered = frame.sort_values(\n        [\"Fluid_Sensitive\", \"Fat_Suppression\", \"_plane\", \"SeriesInstanceUID\"],\n        ascending=[False, False, True, True],\n    )\n    for series_uid in ordered[\"SeriesInstanceUID\"].astype(str):\n        if len(selected) == count:\n            break\n        if series_uid not in selected:\n            selected.append(series_uid)\n    return selected + [None] * (count - len(selected))\n\n\ndef uniform_slice_indices(length: int, count: int) -> np.ndarray:\n    if length <= 0 or count <= 0:\n        raise ValueError(\"length and count must be positive\")\n    if length <= count:\n        return np.concatenate([np.arange(length), np.full(count - length, length - 1)]).astype(np.int64)\n    return np.rint(np.linspace(0, length - 1, count)).astype(np.int64)\n\n\ndef stochastic_slice_indices(length: int, count: int, rng: np.random.Generator) -> np.ndarray:\n    if length <= count:\n        return uniform_slice_indices(length, count)\n    edges = np.linspace(0, length, count + 1)\n    values = []\n    for left, right in zip(edges[:-1], edges[1:]):\n        lo = int(math.floor(left))\n        hi = max(lo + 1, int(math.ceil(right)))\n        values.append(int(rng.integers(lo, min(hi, length))))\n    return np.asarray(values, dtype=np.int64)\n\n\ndef pooled_macro_auc(y_true: np.ndarray, y_score: np.ndarray) -> Tuple[float, Dict[str, float]]:\n    from sklearn.metrics import roc_auc_score\n    per_target: Dict[str, float] = {}\n    for j, target in enumerate(TARGETS):\n        valid = np.isfinite(y_true[:, j]) & np.isfinite(y_score[:, j])\n        per_target[target] = (\n            float(roc_auc_score(y_true[valid, j], y_score[valid, j]))\n            if valid.sum() and np.unique(y_true[valid, j]).size == 2\n            else float(\"nan\")\n        )\n    finite = [value for value in per_target.values() if np.isfinite(value)]\n    return (float(np.mean(finite)) if finite else float(\"nan\")), per_target\n\n\ndef prepare_fold_rows(\n    label_table: pd.DataFrame, fold: int, config: TrainConfig\n) -> Tuple[pd.DataFrame, pd.DataFrame]:\n    train_rows = label_table.loc[label_table[\"fold\"].ne(fold)].copy()\n    valid_rows = label_table.loc[label_table[\"fold\"].eq(fold) & label_table[\"is_expert\"].eq(1)].copy()\n    if train_rows.empty or valid_rows.empty:\n        raise ValueError(f\"empty train/validation split for fold {fold}\")\n    for target in TARGETS:\n        gold = pd.to_numeric(train_rows[f\"gold::{target}\"], errors=\"coerce\")\n        is_gold = train_rows[\"is_expert\"].eq(1) & gold.notna()\n        train_rows[f\"target::{target}\"] = train_rows[f\"pseudo::{target}\"].where(~is_gold, gold)\n        train_rows[f\"train_weight::{target}\"] = train_rows[f\"weight::{target}\"].where(~is_gold, config.gold_weight)\n        valid_rows[f\"target::{target}\"] = pd.to_numeric(valid_rows[f\"gold::{target}\"], errors=\"raise\")\n        valid_rows[f\"train_weight::{target}\"] = 1.0\n    if set(train_rows[\"report_group\"]).intersection(valid_rows[\"report_group\"]):\n        raise AssertionError(\"normalized report leakage across train and validation\")\n    return train_rows, valid_rows\n\n\ndef _training_imports():\n    try:\n        import cv2\n        import pydicom\n        import timm\n        import torch\n        import torch.nn as nn\n        from torch.utils.data import DataLoader, Dataset\n    except ImportError as exc:\n        raise RuntimeError(\"training requires torch, timm, pydicom, and opencv-python\") from exc\n    return cv2, pydicom, timm, torch, nn, DataLoader, Dataset\n\n\ndef load_dicom_volume(series_dir: Path, image_size: int, max_slices: int = 64) -> np.ndarray:\n    cv2, pydicom, _, _, _, _, _ = _training_imports()\n    slices: List[Tuple[int, np.ndarray]] = []\n    for path in sorted(series_dir.glob(\"*.dcm\")):\n        try:\n            ds = pydicom.dcmread(str(path))\n            image = ds.pixel_array.astype(np.float32)\n            image = image * float(getattr(ds, \"RescaleSlope\", 1.0) or 1.0)\n            image += float(getattr(ds, \"RescaleIntercept\", 0.0) or 0.0)\n            if str(getattr(ds, \"PhotometricInterpretation\", \"\")) == \"MONOCHROME1\":\n                image = image.max() - image\n            slices.append((int(getattr(ds, \"InstanceNumber\", 0) or 0), image))\n        except Exception:\n            continue\n    if not slices:\n        raise RuntimeError(f\"no decodable slices in {series_dir}\")\n    slices.sort(key=lambda item: item[0])\n    volume = np.stack([item[1] for item in slices])\n    lo, hi = np.percentile(volume, [1.0, 99.0])\n    hi = max(float(hi), float(lo) + 1.0)\n    volume = np.clip((volume - lo) / (hi - lo), 0.0, 1.0)\n    if len(volume) > max_slices:\n        start = (len(volume) - max_slices) // 2\n        volume = volume[start:start + max_slices]\n    resized = np.stack([\n        cv2.resize(x, (image_size, image_size), interpolation=cv2.INTER_AREA)\n        for x in volume\n    ])\n    # Match the public .903 pipeline's training-time uint8 ETL boundary.\n    return (resized * 255.0).astype(np.uint8).astype(np.float32) / 255.0\n\n\ndef make_dataset_class():\n    _, _, _, torch, _, _, Dataset = _training_imports()\n    class KneeStudyDataset(Dataset):\n        def __init__(self, rows, series, image_root, config, training, seed):\n            self.rows = rows.reset_index(drop=True)\n            self.series = {str(uid): group.copy() for uid, group in series.groupby(UID, sort=False)}\n            self.image_root, self.config, self.training, self.seed = Path(image_root), config, training, seed\n\n        def __len__(self):\n            return len(self.rows)\n\n        def __getitem__(self, index):\n            row, config = self.rows.iloc[index], self.config\n            uid = str(row[UID])\n            count = config.train_slices if self.training else config.valid_slices\n            # DataLoader seeds each worker deterministically.  Drawing once from\n            # its advancing torch RNG gives a fresh augmentation on every visit\n            # without sacrificing run-level reproducibility.\n            visit_seed = int(torch.randint(0, 2**31 - 1, (1,)).item()) if self.training else 0\n            rng = np.random.default_rng(self.seed + 1_000_003 * index + visit_seed)\n            empty = pd.DataFrame(columns=[\"SeriesInstanceUID\", \"Anatomical_Plane\", \"Fluid_Sensitive\", \"Fat_Suppression\"])\n            chosen = pick_series_slots(self.series.get(uid, empty), len(PLANES))\n            planes, masks = [], []\n            for series_uid in chosen:\n                if series_uid is None:\n                    planes.append(np.zeros((count, config.image_size, config.image_size), np.float32))\n                    masks.append(np.zeros(count, bool))\n                    continue\n                volume = load_dicom_volume(\n                    self.image_root / uid / series_uid,\n                    config.image_size,\n                    config.max_series_slices,\n                )\n                idx = stochastic_slice_indices(len(volume), count, rng) if self.training else uniform_slice_indices(len(volume), count)\n                sampled = volume[idx].astype(np.float32)\n                if self.training:\n                    sampled = np.clip(sampled * rng.uniform(0.85, 1.15), 0, 1) ** rng.uniform(0.85, 1.15)\n                    if rng.random() < 0.5:\n                        sampled = sampled[:, :, ::-1].copy()\n                sampled = (sampled - 0.449) / 0.226\n                planes.append(sampled)\n                masks.append(np.ones(count, bool))\n            return {\n                \"uid\": uid,\n                \"images\": torch.from_numpy(np.stack(planes)[:, :, None]),\n                \"mask\": torch.from_numpy(np.stack(masks)),\n                \"target\": torch.tensor([row[f\"target::{x}\"] for x in TARGETS], dtype=torch.float32),\n                \"weight\": torch.tensor([row[f\"train_weight::{x}\"] for x in TARGETS], dtype=torch.float32),\n            }\n    return KneeStudyDataset\n\n\ndef make_model_class():\n    _, _, timm, torch, nn, _, _ = _training_imports()\n    class KneeMILModel(nn.Module):\n        def __init__(self, backbone=\"efficientnet_b3\", checkpoint=None, backbone_chunk=12):\n            super().__init__()\n            self.backbone_chunk = int(backbone_chunk)\n            self.backbone = timm.create_model(backbone, pretrained=False, in_chans=1, num_classes=0, global_pool=\"avg\")\n            if checkpoint:\n                state = torch.load(str(checkpoint), map_location=\"cpu\", weights_only=False)\n                if isinstance(state, Mapping) and \"state_dict\" in state:\n                    state = state[\"state_dict\"]\n                state = {str(k).replace(\"module.\", \"\", 1): v for k, v in state.items()}\n                expected = self.backbone.state_dict()\n                if \"conv_stem.weight\" in state and state[\"conv_stem.weight\"].shape[1] == 3:\n                    state[\"conv_stem.weight\"] = state[\"conv_stem.weight\"].sum(dim=1, keepdim=True)\n                state = {key: value for key, value in state.items() if key in expected}\n                shape_errors = {\n                    key: (tuple(value.shape), tuple(expected[key].shape))\n                    for key, value in state.items()\n                    if value.shape != expected[key].shape\n                }\n                if shape_errors:\n                    raise RuntimeError(f\"backbone tensor shape mismatch: {shape_errors}\")\n                result = self.backbone.load_state_dict(state, strict=True)\n                if result.missing_keys or result.unexpected_keys:\n                    raise RuntimeError(f\"backbone state mismatch: {result}\")\n            features, hidden = int(self.backbone.num_features), 384\n            self.attn_v, self.attn_u = nn.Linear(features, hidden), nn.Linear(features, hidden)\n            self.attn_out = nn.Linear(hidden, len(TARGETS))\n            self.target_weight = nn.Parameter(torch.empty(len(TARGETS), features))\n            self.target_bias = nn.Parameter(torch.zeros(len(TARGETS)))\n            self.slice_head = nn.Linear(features, len(TARGETS))\n            self.mix_logit = nn.Parameter(torch.tensor(-1.1))\n            nn.init.xavier_uniform_(self.target_weight)\n\n        def set_backbone_trainable(self, value):\n            for parameter in self.backbone.parameters():\n                parameter.requires_grad = value\n\n        def forward(self, images, mask):\n            batch, planes, slices, channels, height, width = images.shape\n            flat = images.reshape(batch * planes * slices, channels, height, width)\n            features = []\n            train_backbone = self.training and any(p.requires_grad for p in self.backbone.parameters())\n            for start in range(0, len(flat), self.backbone_chunk):\n                chunk = flat[start:start + self.backbone_chunk]\n                if train_backbone:\n                    from torch.utils.checkpoint import checkpoint\n                    features.append(checkpoint(self.backbone, chunk, use_reentrant=False))\n                else:\n                    features.append(self.backbone(chunk))\n            features = torch.cat(features, dim=0)\n            features = features.reshape(batch, planes * slices, -1)\n            flat_mask = mask.reshape(batch, planes * slices)\n            gated = torch.tanh(self.attn_v(features)) * torch.sigmoid(self.attn_u(features))\n            attention = self.attn_out(gated).masked_fill(~flat_mask[:, :, None], -1e4).softmax(dim=1)\n            pooled = torch.einsum(\"bnt,bnf->btf\", attention, features)\n            attention_logits = torch.einsum(\"btf,tf->bt\", pooled, self.target_weight) + self.target_bias\n            max_logits = self.slice_head(features).masked_fill(~flat_mask[:, :, None], -1e4).max(dim=1).values\n            mix = torch.sigmoid(self.mix_logit)\n            return (1.0 - mix) * attention_logits + mix * max_logits\n    return KneeMILModel\n\n\ndef train_one_fold(label_table, series, image_root, output_dir, fold, backbone_checkpoint, config):\n    _, _, _, torch, nn, DataLoader, _ = _training_imports()\n    random.seed(config.seed + fold)\n    np.random.seed(config.seed + fold)\n    torch.manual_seed(config.seed + fold)\n    Dataset, Model = make_dataset_class(), make_model_class()\n    train_rows, valid_rows = prepare_fold_rows(label_table, fold, config)\n    train_loader = DataLoader(Dataset(train_rows, series, image_root, config, True, config.seed + fold), batch_size=config.batch_size, shuffle=True, num_workers=config.workers, pin_memory=True, persistent_workers=config.workers > 0)\n    valid_loader = DataLoader(Dataset(valid_rows, series, image_root, config, False, config.seed + fold), batch_size=1, shuffle=False, num_workers=max(1, config.workers // 2))\n    device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n    if device.type != \"cuda\":\n        raise RuntimeError(\"GPU is required\")\n    core_model = Model(config.backbone, backbone_checkpoint, config.backbone_chunk).to(device)\n    model = nn.DataParallel(core_model) if torch.cuda.device_count() > 1 else core_model\n    loss_fn, scaler, history = nn.BCEWithLogitsLoss(reduction=\"none\"), torch.amp.GradScaler(\"cuda\"), []\n    schedule = [(False, config.frozen_lr)] * config.frozen_epochs + [(True, config.unfrozen_lr)] * config.unfrozen_epochs\n    optimizer, active_phase = None, None\n    for epoch, (unfreeze, lr) in enumerate(schedule):\n        if active_phase != unfreeze:\n            core_model.set_backbone_trainable(unfreeze)\n            optimizer = torch.optim.AdamW(\n                [p for p in model.parameters() if p.requires_grad],\n                lr=lr,\n                weight_decay=config.weight_decay,\n            )\n            active_phase = unfreeze\n        assert optimizer is not None\n        model.train()\n        for layer in core_model.backbone.modules():\n            if isinstance(layer, nn.modules.batchnorm._BatchNorm):\n                layer.eval()\n        losses = []\n        for batch in train_loader:\n            images, mask = batch[\"images\"].to(device), batch[\"mask\"].to(device)\n            target, weight = batch[\"target\"].to(device), batch[\"weight\"].to(device)\n            optimizer.zero_grad(set_to_none=True)\n            with torch.autocast(\"cuda\", dtype=torch.float16):\n                raw = loss_fn(model(images, mask), target)\n                loss = (raw * weight).sum() / weight.sum().clamp_min(1.0)\n            scaler.scale(loss).backward()\n            scaler.unscale_(optimizer)\n            torch.nn.utils.clip_grad_norm_(model.parameters(), 5.0)\n            scaler.step(optimizer)\n            scaler.update()\n            losses.append(float(loss.detach().cpu()))\n        history.append({\n            \"epoch\": epoch,\n            \"train_loss\": float(np.mean(losses)),\n            \"backbone_trainable\": unfreeze,\n            \"learning_rate\": lr,\n        })\n    model.eval()\n    uids, truth, score = [], [], []\n    with torch.no_grad():\n        for batch in valid_loader:\n            with torch.autocast(\"cuda\", dtype=torch.float16):\n                logits = model(batch[\"images\"].to(device), batch[\"mask\"].to(device))\n            uids.extend(batch[\"uid\"])\n            truth.append(batch[\"target\"].numpy())\n            score.append(torch.sigmoid(logits.float()).cpu().numpy())\n    truth, score = np.concatenate(truth), np.concatenate(score)\n    macro, per_target = pooled_macro_auc(truth, score)\n    output_dir.mkdir(parents=True, exist_ok=True)\n    oof = pd.DataFrame({UID: uids})\n    for j, target in enumerate(TARGETS):\n        oof[f\"gold::{target}\"], oof[f\"pred::{target}\"] = truth[:, j], score[:, j]\n    oof.to_csv(output_dir / f\"fold{fold}_expert_oof.csv\", index=False)\n    destination = output_dir / f\"fold{fold}_final.pt\"\n    torch.save({\"model\": core_model.state_dict(), \"backbone\": config.backbone, \"config\": asdict(config), \"fold\": fold, \"fold_diagnostic_macro_auc\": macro, \"fold_diagnostic_per_target_auc\": per_target, \"history\": history, \"visible_gpus\": torch.cuda.device_count(), \"backbone_checkpoint_sha256\": hashlib.sha256(backbone_checkpoint.read_bytes()).hexdigest()}, destination)\n    return destination\n\n\ndef main(argv: Optional[Sequence[str]] = None) -> None:\n    parser = argparse.ArgumentParser(description=__doc__)\n    for name in (\"train-csv\", \"series-csv\", \"pilkwang-labels\", \"steven-labels\", \"lixin-labels\", \"output-dir\"):\n        parser.add_argument(f\"--{name}\", type=Path, required=True)\n    parser.add_argument(\"--image-root\", type=Path)\n    parser.add_argument(\"--backbone-checkpoint\", type=Path)\n    parser.add_argument(\"--fold\", type=int)\n    parser.add_argument(\"--prepare-only\", action=\"store_true\")\n    args = parser.parse_args(argv)\n    config = TrainConfig()\n    paths = [args.train_csv, args.series_csv, args.pilkwang_labels, args.steven_labels, args.lixin_labels]\n    frames = [pd.read_csv(path) for path in paths]\n    labels = build_public_consensus(frames[0], frames[2], frames[3], frames[4], config.pseudo_weight_floor)\n    labels[\"fold\"] = assign_group_balanced_folds(labels, config.folds, config.seed)\n    args.output_dir.mkdir(parents=True, exist_ok=True)\n    labels.to_csv(args.output_dir / \"fold_labels.csv\", index=False)\n    manifest = {\n        \"config\": asdict(config), \"rows\": len(labels), \"expert_rows\": int(labels[\"is_expert\"].sum()),\n        \"fold_counts\": labels[\"fold\"].value_counts().sort_index().to_dict(),\n        \"fold_expert_counts\": labels.loc[labels[\"is_expert\"].eq(1), \"fold\"].value_counts().sort_index().to_dict(),\n        \"inputs\": {str(path): hashlib.sha256(path.read_bytes()).hexdigest() for path in paths},\n    }\n    (args.output_dir / \"manifest.json\").write_text(json.dumps(manifest, indent=2, sort_keys=True) + \"\\n\")\n    print(json.dumps(manifest, indent=2, sort_keys=True))\n    if args.prepare_only:\n        return\n    if args.fold is None or args.image_root is None or args.backbone_checkpoint is None:\n        parser.error(\"training requires --fold, --image-root, and --backbone-checkpoint\")\n    if not 0 <= args.fold < config.folds:\n        parser.error(f\"--fold must be in [0, {config.folds - 1}]\")\n    print(\"saved\", train_one_fold(labels, frames[1], args.image_root, args.output_dir, args.fold, args.backbone_checkpoint, config))\n\n\nif __name__ == \"__main__\":\n    main()\n"},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"%%writefile /kaggle/working/rsna_b3_v47_source/efficientnet_b3_public_repro_v2_anatomy.py\n#!/usr/bin/env python3\n\"\"\"Anatomy-aware EfficientNet-B3 MIL training for RSNA Knee.\n\nThis module deliberately reuses the audited v1 DICOM and label pipeline while\nchanging only three trainable assumptions that v1 could not express:\n\n* each slice feature receives its acquisition-plane and normalized stack position;\n* expert-labelled studies are sampled often enough to influence every epoch;\n* fine-tuning is restricted to the final two EfficientNet stages.\n\nThe output checkpoint contract remains compatible with the v1 inference runner,\nprovided this v2 module is supplied to ``--module`` at inference time.\n\"\"\"\n\nfrom __future__ import annotations\n\nimport argparse\nimport hashlib\nimport importlib.util\nimport json\nimport math\nimport random\nimport sys\nfrom dataclasses import asdict, dataclass, replace\nfrom pathlib import Path\nfrom typing import Mapping, Optional, Sequence\n\nimport numpy as np\nimport pandas as pd\n\n\ndef _load_base_module():\n    path = Path(__file__).with_name(\"efficientnet_b3_public_repro_v1.py\")\n    if not path.is_file():\n        raise FileNotFoundError(f\"audited v1 dependency is absent: {path}\")\n    name = \"rsna_knee_effnet_b3_public_repro_v1_dependency\"\n    spec = importlib.util.spec_from_file_location(name, path)\n    if spec is None or spec.loader is None:\n        raise RuntimeError(f\"cannot import audited v1 dependency: {path}\")\n    module = importlib.util.module_from_spec(spec)\n    sys.modules[name] = module\n    spec.loader.exec_module(module)\n    return module\n\n\nBASE = _load_base_module()\nTARGETS = BASE.TARGETS\nPLANES = BASE.PLANES\nUID = BASE.UID\nTrainConfig = BASE.TrainConfig\nload_dicom_volume = BASE.load_dicom_volume\nuniform_slice_indices = BASE.uniform_slice_indices\nstochastic_slice_indices = BASE.stochastic_slice_indices\npooled_macro_auc = BASE.pooled_macro_auc\n\n\n@dataclass(frozen=True)\nclass TrainingPolicy:\n    \"\"\"Choices specific to v2 and therefore stored separately from TrainConfig.\"\"\"\n\n    expert_sample_fraction: float = 0.12\n    unfrozen_backbone_blocks: int = 2\n    gradient_clip_norm: float = 5.0\n    model_family: str = \"efficientnet_b3_anatomy_mil_v2\"\n\n\ndef _training_imports():\n    return BASE._training_imports()\n\n\ndef pick_series_slots(series: pd.DataFrame, count: int = 3):\n    \"\"\"Return one fixed slot per anatomical plane.\n\n    The v1 selector intentionally backfills a missing plane with another useful\n    series.  That is appropriate for an anatomy-agnostic model, but would attach\n    the wrong plane embedding in v2.  Preserve empty slots instead so index zero,\n    one, and two always mean sagittal, coronal, and axial respectively.\n    \"\"\"\n    if int(count) != len(PLANES):\n        raise ValueError(f\"v2 requires exactly {len(PLANES)} anatomical plane slots\")\n    selected = BASE.pick_one_series_per_plane(series)\n    return [selected[plane] for plane in PLANES]\n\n\ndef make_dataset_class():\n    \"\"\"Build the v1 study dataset with fixed anatomical plane slots.\"\"\"\n    _, _, _, torch, _, _, Dataset = _training_imports()\n\n    class AnatomyAwareKneeStudyDataset(Dataset):\n        def __init__(self, rows, series, image_root, config, training, seed):\n            self.rows = rows.reset_index(drop=True)\n            self.series = {\n                str(uid): group.copy() for uid, group in series.groupby(UID, sort=False)\n            }\n            self.image_root = Path(image_root)\n            self.config = config\n            self.training = bool(training)\n            self.seed = int(seed)\n\n        def __len__(self):\n            return len(self.rows)\n\n        def __getitem__(self, index):\n            row, config = self.rows.iloc[index], self.config\n            uid = str(row[UID])\n            count = config.train_slices if self.training else config.valid_slices\n            visit_seed = (\n                int(torch.randint(0, 2**31 - 1, (1,)).item()) if self.training else 0\n            )\n            rng = np.random.default_rng(self.seed + 1_000_003 * index + visit_seed)\n            empty = pd.DataFrame(\n                columns=[\n                    \"SeriesInstanceUID\",\n                    \"Anatomical_Plane\",\n                    \"Fluid_Sensitive\",\n                    \"Fat_Suppression\",\n                ]\n            )\n            chosen = pick_series_slots(self.series.get(uid, empty), len(PLANES))\n            planes, masks = [], []\n            for series_uid in chosen:\n                if series_uid is None:\n                    planes.append(\n                        np.zeros((count, config.image_size, config.image_size), np.float32)\n                    )\n                    masks.append(np.zeros(count, bool))\n                    continue\n                volume = load_dicom_volume(\n                    self.image_root / uid / series_uid,\n                    config.image_size,\n                    config.max_series_slices,\n                )\n                indices = (\n                    stochastic_slice_indices(len(volume), count, rng)\n                    if self.training\n                    else uniform_slice_indices(len(volume), count)\n                )\n                sampled = volume[indices].astype(np.float32)\n                if self.training:\n                    sampled = np.clip(sampled * rng.uniform(0.85, 1.15), 0, 1)\n                    sampled = sampled ** rng.uniform(0.85, 1.15)\n                    if rng.random() < 0.5:\n                        sampled = sampled[:, :, ::-1].copy()\n                sampled = (sampled - 0.449) / 0.226\n                planes.append(sampled)\n                masks.append(np.ones(count, bool))\n            return {\n                \"uid\": uid,\n                \"images\": torch.from_numpy(np.stack(planes)[:, :, None]),\n                \"mask\": torch.from_numpy(np.stack(masks)),\n                \"target\": torch.tensor(\n                    [row[f\"target::{target}\"] for target in TARGETS], dtype=torch.float32\n                ),\n                \"weight\": torch.tensor(\n                    [row[f\"train_weight::{target}\"] for target in TARGETS],\n                    dtype=torch.float32,\n                ),\n            }\n\n    return AnatomyAwareKneeStudyDataset\n\n\ndef make_model_class():\n    \"\"\"Build a target-attention MIL model with fixed anatomical coordinates.\"\"\"\n    _, _, timm, torch, nn, _, _ = _training_imports()\n\n    class AnatomyAwareKneeMIL(nn.Module):\n        def __init__(self, backbone=\"efficientnet_b3\", checkpoint=None, backbone_chunk=12):\n            super().__init__()\n            self.backbone_chunk = int(backbone_chunk)\n            self.backbone = timm.create_model(\n                backbone,\n                pretrained=False,\n                in_chans=1,\n                num_classes=0,\n                global_pool=\"avg\",\n            )\n            if checkpoint:\n                state = torch.load(str(checkpoint), map_location=\"cpu\", weights_only=False)\n                if isinstance(state, Mapping) and \"state_dict\" in state:\n                    state = state[\"state_dict\"]\n                state = {str(key).replace(\"module.\", \"\", 1): value for key, value in state.items()}\n                expected = self.backbone.state_dict()\n                if \"conv_stem.weight\" in state and state[\"conv_stem.weight\"].shape[1] == 3:\n                    # Sum rather than mean preserves the response magnitude expected by\n                    # the pretrained filters after the one-channel ImageNet normalization.\n                    state[\"conv_stem.weight\"] = state[\"conv_stem.weight\"].sum(dim=1, keepdim=True)\n                state = {key: value for key, value in state.items() if key in expected}\n                shape_errors = {\n                    key: (tuple(value.shape), tuple(expected[key].shape))\n                    for key, value in state.items()\n                    if value.shape != expected[key].shape\n                }\n                if shape_errors:\n                    raise RuntimeError(f\"backbone tensor shape mismatch: {shape_errors}\")\n                result = self.backbone.load_state_dict(state, strict=True)\n                if result.missing_keys or result.unexpected_keys:\n                    raise RuntimeError(f\"backbone state mismatch: {result}\")\n\n            features, hidden = int(self.backbone.num_features), 384\n            self.plane_embedding = nn.Embedding(len(PLANES), features)\n            self.position_projection = nn.Linear(4, features, bias=False)\n            nn.init.normal_(self.plane_embedding.weight, std=0.01)\n            nn.init.normal_(self.position_projection.weight, std=0.01)\n\n            self.attn_v = nn.Linear(features, hidden)\n            self.attn_u = nn.Linear(features, hidden)\n            self.attn_out = nn.Linear(hidden, len(TARGETS))\n            self.target_weight = nn.Parameter(torch.empty(len(TARGETS), features))\n            self.target_bias = nn.Parameter(torch.zeros(len(TARGETS)))\n            self.slice_head = nn.Linear(features, len(TARGETS))\n            self.mix_logit = nn.Parameter(torch.tensor(-1.1))\n            nn.init.xavier_uniform_(self.target_weight)\n\n        def set_backbone_stage(self, final_blocks=0):\n            \"\"\"Freeze the backbone, optionally reopening only its final stages.\"\"\"\n            for parameter in self.backbone.parameters():\n                parameter.requires_grad_(False)\n            final_blocks = int(final_blocks)\n            if final_blocks <= 0:\n                return\n            blocks = list(self.backbone.blocks)\n            if final_blocks > len(blocks):\n                raise ValueError(\n                    f\"requested {final_blocks} trainable blocks from a {len(blocks)}-block backbone\"\n                )\n            for block in blocks[-final_blocks:]:\n                for parameter in block.parameters():\n                    parameter.requires_grad_(True)\n            for name in (\"conv_head\", \"bn2\"):\n                layer = getattr(self.backbone, name, None)\n                if layer is not None:\n                    for parameter in layer.parameters():\n                        parameter.requires_grad_(True)\n\n        @staticmethod\n        def _position_basis(slices, device, dtype):\n            position = torch.linspace(-1.0, 1.0, slices, device=device, dtype=dtype)\n            return torch.stack(\n                (\n                    position,\n                    position.square(),\n                    torch.sin(math.pi * position),\n                    torch.cos(math.pi * position),\n                ),\n                dim=-1,\n            )\n\n        def forward(self, images, mask):\n            batch, planes, slices, channels, height, width = images.shape\n            if planes != len(PLANES):\n                raise ValueError(f\"expected {len(PLANES)} plane slots, got {planes}\")\n            flat = images.reshape(batch * planes * slices, channels, height, width)\n            encoded = []\n            train_backbone = self.training and any(\n                parameter.requires_grad for parameter in self.backbone.parameters()\n            )\n            for start in range(0, len(flat), self.backbone_chunk):\n                chunk = flat[start : start + self.backbone_chunk]\n                if train_backbone:\n                    from torch.utils.checkpoint import checkpoint\n\n                    encoded.append(checkpoint(self.backbone, chunk, use_reentrant=False))\n                else:\n                    encoded.append(self.backbone(chunk))\n            features = torch.cat(encoded, dim=0).reshape(batch, planes, slices, -1)\n\n            plane_ids = torch.arange(planes, device=features.device)\n            plane_context = self.plane_embedding(plane_ids)[None, :, None, :]\n            position = self._position_basis(slices, features.device, features.dtype)\n            position_context = self.position_projection(position)[None, None, :, :]\n            features = features + plane_context + position_context\n            features = features.reshape(batch, planes * slices, -1)\n            flat_mask = mask.reshape(batch, planes * slices)\n\n            gated = torch.tanh(self.attn_v(features)) * torch.sigmoid(self.attn_u(features))\n            attention = self.attn_out(gated).masked_fill(\n                ~flat_mask[:, :, None], -1e4\n            ).softmax(dim=1)\n            pooled = torch.einsum(\"bnt,bnf->btf\", attention, features)\n            attention_logits = (\n                torch.einsum(\"btf,tf->bt\", pooled, self.target_weight) + self.target_bias\n            )\n            max_logits = self.slice_head(features).masked_fill(\n                ~flat_mask[:, :, None], -1e4\n            ).max(dim=1).values\n            mix = torch.sigmoid(self.mix_logit)\n            return (1.0 - mix) * attention_logits + mix * max_logits\n\n    return AnatomyAwareKneeMIL\n\n\ndef _make_expert_sampler(rows, fraction, seed, torch):\n    \"\"\"Sample a fixed expected expert fraction without discarding pseudo rows.\"\"\"\n    if not 0.0 < fraction < 1.0:\n        raise ValueError(\"expert_sample_fraction must be strictly between zero and one\")\n    expert = rows[\"is_expert\"].to_numpy(np.int8).astype(bool)\n    n_expert, n_pseudo = int(expert.sum()), int((~expert).sum())\n    if n_expert == 0 or n_pseudo == 0:\n        raise ValueError(\"expert-aware sampling requires both expert and pseudo-labelled rows\")\n    expert_multiplier = fraction * n_pseudo / ((1.0 - fraction) * n_expert)\n    weights = np.where(expert, expert_multiplier, 1.0).astype(np.float64)\n    generator = torch.Generator().manual_seed(int(seed))\n    from torch.utils.data import WeightedRandomSampler\n\n    sampler = WeightedRandomSampler(\n        torch.from_numpy(weights),\n        num_samples=len(rows),\n        replacement=True,\n        generator=generator,\n    )\n    return sampler, {\n        \"expert_rows\": n_expert,\n        \"pseudo_rows\": n_pseudo,\n        \"expert_multiplier\": float(expert_multiplier),\n        \"expected_expert_fraction\": float(fraction),\n    }\n\n\ndef train_one_fold(\n    label_table,\n    series,\n    image_root,\n    output_dir,\n    fold,\n    backbone_checkpoint,\n    config,\n    policy=TrainingPolicy(),\n):\n    _, _, _, torch, nn, DataLoader, _ = _training_imports()\n    random.seed(config.seed + fold)\n    np.random.seed(config.seed + fold)\n    torch.manual_seed(config.seed + fold)\n    Dataset, Model = make_dataset_class(), make_model_class()\n    train_rows, valid_rows = BASE.prepare_fold_rows(label_table, fold, config)\n    sampler, sampler_meta = _make_expert_sampler(\n        train_rows,\n        policy.expert_sample_fraction,\n        config.seed + 10_007 * fold,\n        torch,\n    )\n    train_loader = DataLoader(\n        Dataset(train_rows, series, image_root, config, True, config.seed + fold),\n        batch_size=config.batch_size,\n        sampler=sampler,\n        num_workers=config.workers,\n        pin_memory=True,\n        persistent_workers=config.workers > 0,\n    )\n    valid_loader = DataLoader(\n        Dataset(valid_rows, series, image_root, config, False, config.seed + fold),\n        batch_size=1,\n        shuffle=False,\n        num_workers=max(1, config.workers // 2),\n    )\n    device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n    if device.type != \"cuda\":\n        raise RuntimeError(\"GPU is required\")\n    core_model = Model(config.backbone, backbone_checkpoint, config.backbone_chunk).to(device)\n    model = nn.DataParallel(core_model) if torch.cuda.device_count() > 1 else core_model\n    loss_fn = nn.BCEWithLogitsLoss(reduction=\"none\")\n    scaler = torch.amp.GradScaler(\"cuda\")\n    history = []\n    schedule = (\n        [(0, config.frozen_lr)] * config.frozen_epochs\n        + [(policy.unfrozen_backbone_blocks, config.unfrozen_lr)] * config.unfrozen_epochs\n    )\n    optimizer, active_blocks = None, None\n    for epoch, (trainable_blocks, learning_rate) in enumerate(schedule):\n        if active_blocks != trainable_blocks:\n            core_model.set_backbone_stage(trainable_blocks)\n            optimizer = torch.optim.AdamW(\n                [parameter for parameter in model.parameters() if parameter.requires_grad],\n                lr=learning_rate,\n                weight_decay=config.weight_decay,\n            )\n            active_blocks = trainable_blocks\n        assert optimizer is not None\n        model.train()\n        for layer in core_model.backbone.modules():\n            if isinstance(layer, nn.modules.batchnorm._BatchNorm):\n                layer.eval()\n        losses = []\n        sampled_expert = 0\n        sampled_total = 0\n        for batch in train_loader:\n            images, mask = batch[\"images\"].to(device), batch[\"mask\"].to(device)\n            target, weight = batch[\"target\"].to(device), batch[\"weight\"].to(device)\n            # Expert rows have the same gold weight for all twelve targets.\n            sampled_expert += int((weight.min(dim=1).values >= config.gold_weight).sum().item())\n            sampled_total += int(len(weight))\n            optimizer.zero_grad(set_to_none=True)\n            with torch.autocast(\"cuda\", dtype=torch.float16):\n                raw = loss_fn(model(images, mask), target)\n                loss = (raw * weight).sum() / weight.sum().clamp_min(1.0)\n            scaler.scale(loss).backward()\n            scaler.unscale_(optimizer)\n            torch.nn.utils.clip_grad_norm_(model.parameters(), policy.gradient_clip_norm)\n            scaler.step(optimizer)\n            scaler.update()\n            losses.append(float(loss.detach().cpu()))\n        history.append(\n            {\n                \"epoch\": epoch,\n                \"train_loss\": float(np.mean(losses)),\n                \"trainable_backbone_blocks\": int(trainable_blocks),\n                \"learning_rate\": float(learning_rate),\n                \"observed_expert_fraction\": sampled_expert / max(sampled_total, 1),\n            }\n        )\n\n    model.eval()\n    uids, truth, score = [], [], []\n    with torch.no_grad():\n        for batch in valid_loader:\n            with torch.autocast(\"cuda\", dtype=torch.float16):\n                logits = model(batch[\"images\"].to(device), batch[\"mask\"].to(device))\n            uids.extend(batch[\"uid\"])\n            truth.append(batch[\"target\"].numpy())\n            score.append(torch.sigmoid(logits.float()).cpu().numpy())\n    truth, score = np.concatenate(truth), np.concatenate(score)\n    macro, per_target = BASE.pooled_macro_auc(truth, score)\n    output_dir.mkdir(parents=True, exist_ok=True)\n    oof = pd.DataFrame({UID: uids})\n    for target_index, target in enumerate(TARGETS):\n        oof[f\"gold::{target}\"] = truth[:, target_index]\n        oof[f\"pred::{target}\"] = score[:, target_index]\n    oof.to_csv(output_dir / f\"fold{fold}_expert_oof.csv\", index=False)\n    destination = output_dir / f\"fold{fold}_final.pt\"\n    torch.save(\n        {\n            \"model\": core_model.state_dict(),\n            \"model_family\": policy.model_family,\n            \"backbone\": config.backbone,\n            \"config\": asdict(config),\n            \"training_policy\": asdict(policy),\n            \"sampler\": sampler_meta,\n            \"fold\": fold,\n            \"fold_diagnostic_macro_auc\": macro,\n            \"fold_diagnostic_per_target_auc\": per_target,\n            \"history\": history,\n            \"visible_gpus\": torch.cuda.device_count(),\n            \"backbone_checkpoint_sha256\": hashlib.sha256(\n                backbone_checkpoint.read_bytes()\n            ).hexdigest(),\n        },\n        destination,\n    )\n    return destination\n\n\ndef main(argv: Optional[Sequence[str]] = None) -> None:\n    parser = argparse.ArgumentParser(description=__doc__)\n    for name in (\n        \"train-csv\",\n        \"series-csv\",\n        \"pilkwang-labels\",\n        \"steven-labels\",\n        \"lixin-labels\",\n        \"output-dir\",\n    ):\n        parser.add_argument(f\"--{name}\", type=Path, required=True)\n    parser.add_argument(\"--image-root\", type=Path)\n    parser.add_argument(\"--backbone-checkpoint\", type=Path)\n    parser.add_argument(\"--fold\", type=int)\n    parser.add_argument(\"--prepare-only\", action=\"store_true\")\n    args = parser.parse_args(argv)\n\n    # Expert oversampling replaces some of the multiplicative gold weighting;\n    # keeping both at their v1 values would overfit the 46-or-so training experts.\n    config = replace(TrainConfig(), gold_weight=2.0)\n    policy = TrainingPolicy()\n    paths = [\n        args.train_csv,\n        args.series_csv,\n        args.pilkwang_labels,\n        args.steven_labels,\n        args.lixin_labels,\n    ]\n    frames = [pd.read_csv(path) for path in paths]\n    labels = BASE.build_public_consensus(\n        frames[0], frames[2], frames[3], frames[4], config.pseudo_weight_floor\n    )\n    labels[\"fold\"] = BASE.assign_group_balanced_folds(labels, config.folds, config.seed)\n    args.output_dir.mkdir(parents=True, exist_ok=True)\n    labels.to_csv(args.output_dir / \"fold_labels.csv\", index=False)\n    manifest = {\n        \"config\": asdict(config),\n        \"training_policy\": asdict(policy),\n        \"rows\": len(labels),\n        \"expert_rows\": int(labels[\"is_expert\"].sum()),\n        \"fold_counts\": labels[\"fold\"].value_counts().sort_index().to_dict(),\n        \"fold_expert_counts\": labels.loc[\n            labels[\"is_expert\"].eq(1), \"fold\"\n        ].value_counts().sort_index().to_dict(),\n        \"inputs\": {str(path): hashlib.sha256(path.read_bytes()).hexdigest() for path in paths},\n        \"base_module_sha256\": hashlib.sha256(\n            Path(BASE.__file__).read_bytes()\n        ).hexdigest(),\n        \"module_sha256\": hashlib.sha256(Path(__file__).read_bytes()).hexdigest(),\n    }\n    (args.output_dir / \"manifest.json\").write_text(\n        json.dumps(manifest, indent=2, sort_keys=True) + \"\\n\"\n    )\n    print(json.dumps(manifest, indent=2, sort_keys=True))\n    if args.prepare_only:\n        return\n    if args.fold is None or args.image_root is None or args.backbone_checkpoint is None:\n        parser.error(\"training requires --fold, --image-root, and --backbone-checkpoint\")\n    if not 0 <= args.fold < config.folds:\n        parser.error(f\"--fold must be in [0, {config.folds - 1}]\")\n    print(\n        \"saved\",\n        train_one_fold(\n            labels,\n            frames[1],\n            args.image_root,\n            args.output_dir,\n            args.fold,\n            args.backbone_checkpoint,\n            config,\n            policy,\n        ),\n    )\n\n\nif __name__ == \"__main__\":\n    main()\n"},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"%%writefile /kaggle/working/rsna_b3_v47_source/efficientnet_b3_public_repro_v4_t4.py\n#!/usr/bin/env python3\n\"\"\"T4-bounded, no-flip EfficientNet-B3 training for RSNA Knee.\n\nThis is an independent reproduction of the public .903 recipe.  It keeps the\naudited v2 model and report-label contracts, but corrects two important input\ndetails disclosed by that notebook: non-square slices are padded before resize,\nand no horizontal flip is ever applied to laterality-specific targets.  A\nsmaller 224px / six-slice-per-plane schedule makes five-fold final-block\nfine-tuning feasible inside Kaggle's nine-hour code-competition budget.\n\"\"\"\n\nfrom __future__ import annotations\n\nimport argparse\nimport hashlib\nimport importlib.util\nimport json\nimport math\nimport random\nimport sys\nfrom dataclasses import asdict, replace\nfrom pathlib import Path\nfrom typing import Optional, Sequence\n\nimport numpy as np\nimport pandas as pd\n\n\ndef _load_v2():\n    path = Path(__file__).with_name(\"efficientnet_b3_public_repro_v2_anatomy.py\")\n    if not path.is_file():\n        raise FileNotFoundError(f\"audited v2 dependency is absent: {path}\")\n    name = \"rsna_knee_effnet_b3_public_repro_v2_t4_dependency\"\n    spec = importlib.util.spec_from_file_location(name, path)\n    if spec is None or spec.loader is None:\n        raise RuntimeError(f\"cannot import audited v2 dependency: {path}\")\n    module = importlib.util.module_from_spec(spec)\n    sys.modules[name] = module\n    spec.loader.exec_module(module)\n    return module\n\n\nV2 = _load_v2()\nBASE = V2.BASE\nTARGETS, PLANES, UID = V2.TARGETS, V2.PLANES, V2.UID\nTrainConfig = V2.TrainConfig\nTrainingPolicy = V2.TrainingPolicy\npick_series_slots = V2.pick_series_slots\nuniform_slice_indices = V2.uniform_slice_indices\nstochastic_slice_indices = V2.stochastic_slice_indices\nmake_model_class = V2.make_model_class\n_training_imports = V2._training_imports\n\n\ndef _natural_key(path: Path):\n    import re\n\n    return tuple(\n        int(part) if part.isdigit() else part.casefold()\n        for part in re.split(r\"(\\d+)\", path.name)\n    )\n\n\ndef load_dicom_volume(series_dir: Path, image_size: int, max_slices: int = 64):\n    \"\"\"Decode like the .903 inference path, including square padding.\"\"\"\n    cv2, pydicom, _, _, _, _, _ = _training_imports()\n    slices = []\n    for path in sorted(Path(series_dir).glob(\"*.dcm\"), key=_natural_key):\n        try:\n            ds = pydicom.dcmread(str(path), force=True)\n            image = ds.pixel_array.astype(np.float32)\n            image *= float(getattr(ds, \"RescaleSlope\", 1.0) or 1.0)\n            image += float(getattr(ds, \"RescaleIntercept\", 0.0) or 0.0)\n            if str(getattr(ds, \"PhotometricInterpretation\", \"\")) == \"MONOCHROME1\":\n                image = float(np.max(image)) - image\n            instance = getattr(ds, \"InstanceNumber\", None)\n            try:\n                instance = int(instance) if instance is not None else None\n            except (TypeError, ValueError):\n                instance = None\n            location = getattr(ds, \"SliceLocation\", None)\n            if location is None:\n                position = getattr(ds, \"ImagePositionPatient\", None)\n                location = position[2] if position is not None and len(position) >= 3 else None\n            try:\n                location = float(location) if location is not None else None\n            except (TypeError, ValueError):\n                location = None\n            slices.append((instance, location, path.name, image))\n        except Exception:\n            continue\n    if not slices:\n        raise RuntimeError(f\"no decodable slices in {series_dir}\")\n\n    slices.sort(\n        key=lambda item: (\n            0 if item[0] is not None else 1 if item[1] is not None else 2,\n            item[0] if item[0] is not None else 0,\n            item[1] if item[1] is not None else 0.0,\n            item[2],\n        )\n    )\n    volume = np.stack([item[3] for item in slices]).astype(np.float32, copy=False)\n    lo, hi = np.percentile(volume, [1.0, 99.0])\n    hi = max(float(hi), float(lo) + 1.0)\n    volume = np.clip((volume - float(lo)) / (hi - float(lo)), 0.0, 1.0)\n    if len(volume) > int(max_slices):\n        start = (len(volume) - int(max_slices)) // 2\n        volume = volume[start : start + int(max_slices)]\n\n    resized = []\n    for image in volume:\n        height, width = image.shape\n        if height != width:\n            side = max(height, width)\n            padded = np.zeros((side, side), dtype=image.dtype)\n            top, left = (side - height) // 2, (side - width) // 2\n            padded[top : top + height, left : left + width] = image\n            image = padded\n        resized.append(\n            cv2.resize(image, (int(image_size), int(image_size)), interpolation=cv2.INTER_AREA)\n        )\n    return np.rint(np.stack(resized) * 255.0).astype(np.uint8).astype(np.float32) / 255.0\n\n\ndef make_dataset_class():\n    \"\"\"Fixed-plane dataset with intensity jitter and explicitly no flip.\"\"\"\n    _, _, _, torch, _, _, Dataset = _training_imports()\n\n    class NoFlipKneeStudyDataset(Dataset):\n        def __init__(self, rows, series, image_root, config, training, seed):\n            self.rows = rows.reset_index(drop=True)\n            self.series = {\n                str(uid): group.copy() for uid, group in series.groupby(UID, sort=False)\n            }\n            self.image_root = Path(image_root)\n            self.config = config\n            self.training = bool(training)\n            self.seed = int(seed)\n\n        def __len__(self):\n            return len(self.rows)\n\n        def __getitem__(self, index):\n            row, config = self.rows.iloc[index], self.config\n            uid = str(row[UID])\n            count = config.train_slices if self.training else config.valid_slices\n            visit_seed = (\n                int(torch.randint(0, 2**31 - 1, (1,)).item()) if self.training else 0\n            )\n            rng = np.random.default_rng(self.seed + 1_000_003 * index + visit_seed)\n            empty = pd.DataFrame(\n                columns=[\n                    \"SeriesInstanceUID\",\n                    \"Anatomical_Plane\",\n                    \"Fluid_Sensitive\",\n                    \"Fat_Suppression\",\n                ]\n            )\n            selected = pick_series_slots(self.series.get(uid, empty), len(PLANES))\n            planes, masks = [], []\n            for series_uid in selected:\n                if series_uid is None:\n                    planes.append(\n                        np.zeros((count, config.image_size, config.image_size), np.float32)\n                    )\n                    masks.append(np.zeros(count, bool))\n                    continue\n                volume = load_dicom_volume(\n                    self.image_root / uid / str(series_uid),\n                    config.image_size,\n                    config.max_series_slices,\n                )\n                indices = (\n                    stochastic_slice_indices(len(volume), count, rng)\n                    if self.training\n                    else uniform_slice_indices(len(volume), count)\n                )\n                sampled = volume[indices].astype(np.float32)\n                if self.training:\n                    sampled = np.clip(sampled * rng.uniform(0.88, 1.12), 0.0, 1.0)\n                    sampled = sampled ** rng.uniform(0.88, 1.12)\n                planes.append((sampled - 0.449) / 0.226)\n                masks.append(np.ones(count, bool))\n            return {\n                \"uid\": uid,\n                \"images\": torch.from_numpy(np.stack(planes)[:, :, None]),\n                \"mask\": torch.from_numpy(np.stack(masks)),\n                \"target\": torch.tensor(\n                    [row[f\"target::{target}\"] for target in TARGETS], dtype=torch.float32\n                ),\n                \"weight\": torch.tensor(\n                    [row[f\"train_weight::{target}\"] for target in TARGETS],\n                    dtype=torch.float32,\n                ),\n            }\n\n    return NoFlipKneeStudyDataset\n\n\n# V2.train_one_fold resolves these names from its own module globals.\nV2.load_dicom_volume = load_dicom_volume\nV2.make_dataset_class = make_dataset_class\n\n\ndef main(argv: Optional[Sequence[str]] = None) -> None:\n    parser = argparse.ArgumentParser(description=__doc__)\n    for name in (\n        \"train-csv\",\n        \"series-csv\",\n        \"pilkwang-labels\",\n        \"steven-labels\",\n        \"lixin-labels\",\n        \"output-dir\",\n    ):\n        parser.add_argument(f\"--{name}\", type=Path, required=True)\n    parser.add_argument(\"--image-root\", type=Path)\n    parser.add_argument(\"--backbone-checkpoint\", type=Path)\n    parser.add_argument(\"--fold\", type=int)\n    parser.add_argument(\"--prepare-only\", action=\"store_true\")\n    args = parser.parse_args(argv)\n\n    config = replace(\n        TrainConfig(),\n        image_size=224,\n        train_slices=6,\n        valid_slices=20,\n        batch_size=2,\n        workers=4,\n        backbone_chunk=16,\n        frozen_epochs=1,\n        unfrozen_epochs=2,\n        frozen_lr=8e-4,\n        unfrozen_lr=6e-5,\n        gold_weight=2.0,\n        backbone=\"efficientnet_b3\",\n    )\n    policy = replace(\n        TrainingPolicy(),\n        expert_sample_fraction=0.15,\n        unfrozen_backbone_blocks=3,\n        model_family=\"efficientnet_b3_noflip_t4_v4\",\n    )\n    paths = [\n        args.train_csv,\n        args.series_csv,\n        args.pilkwang_labels,\n        args.steven_labels,\n        args.lixin_labels,\n    ]\n    frames = [pd.read_csv(path) for path in paths]\n    labels = BASE.build_public_consensus(\n        frames[0], frames[2], frames[3], frames[4], config.pseudo_weight_floor\n    )\n    labels[\"fold\"] = BASE.assign_group_balanced_folds(labels, config.folds, config.seed)\n    args.output_dir.mkdir(parents=True, exist_ok=True)\n    labels.to_csv(args.output_dir / \"fold_labels.csv\", index=False)\n    manifest = {\n        \"status\": \"PREPARED\",\n        \"config\": asdict(config),\n        \"training_policy\": asdict(policy),\n        \"rows\": len(labels),\n        \"expert_rows\": int(labels[\"is_expert\"].sum()),\n        \"fold_counts\": labels[\"fold\"].value_counts().sort_index().to_dict(),\n        \"fold_expert_counts\": labels.loc[\n            labels[\"is_expert\"].eq(1), \"fold\"\n        ].value_counts().sort_index().to_dict(),\n        \"inputs\": {str(path): hashlib.sha256(path.read_bytes()).hexdigest() for path in paths},\n        \"module_sha256\": hashlib.sha256(Path(__file__).read_bytes()).hexdigest(),\n        \"no_horizontal_flip\": True,\n        \"square_padding_before_resize\": True,\n    }\n    (args.output_dir / \"manifest.json\").write_text(\n        json.dumps(manifest, indent=2, sort_keys=True) + \"\\n\"\n    )\n    print(json.dumps(manifest, indent=2, sort_keys=True), flush=True)\n    if args.prepare_only:\n        return\n    if args.fold is None or args.image_root is None or args.backbone_checkpoint is None:\n        parser.error(\"training requires --fold, --image-root, and --backbone-checkpoint\")\n    if not 0 <= args.fold < config.folds:\n        parser.error(f\"--fold must be in [0, {config.folds - 1}]\")\n    print(\n        \"saved\",\n        V2.train_one_fold(\n            labels,\n            frames[1],\n            args.image_root,\n            args.output_dir,\n            args.fold,\n            args.backbone_checkpoint,\n            config,\n            policy,\n        ),\n        flush=True,\n    )\n\n\nif __name__ == \"__main__\":\n    main()\n"},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"%%writefile /kaggle/working/rsna_b3_v47_source/efficientnet_b3_public_repro_v1_infer.py\n#!/usr/bin/env python3\n\"\"\"Five-fold inference for the public EfficientNet-B3 reproduction package.\"\"\"\n\nfrom __future__ import annotations\n\nimport argparse\nimport contextlib\nimport importlib.util\nimport json\nimport time\nfrom pathlib import Path\n\nimport numpy as np\nimport pandas as pd\n\n\ndef load_module(path: Path):\n    spec = importlib.util.spec_from_file_location(\"effnet_public_repro\", path)\n    if spec is None or spec.loader is None:\n        raise RuntimeError(f\"cannot import {path}\")\n    module = importlib.util.module_from_spec(spec)\n    import sys\n    sys.modules[spec.name] = module\n    spec.loader.exec_module(module)\n    return module\n\n\ndef write_submission(path: Path, uids, targets, predictions) -> None:\n    rows = []\n    for uid in uids:\n        values = predictions.get(str(uid), np.full(len(targets), 0.5, dtype=float))\n        rows.append({\"StudyInstanceUID\": uid, **dict(zip(targets, map(float, values)))})\n    pd.DataFrame(rows, columns=[\"StudyInstanceUID\", *targets]).to_csv(path, index=False)\n\n\ndef main() -> None:\n    parser = argparse.ArgumentParser(description=__doc__)\n    parser.add_argument(\"--module\", type=Path, required=True)\n    parser.add_argument(\"--test-csv\", type=Path, required=True)\n    parser.add_argument(\"--series-csv\", type=Path, required=True)\n    parser.add_argument(\"--image-root\", type=Path, required=True)\n    parser.add_argument(\"--checkpoints\", type=Path, nargs=5, required=True)\n    parser.add_argument(\"--output-dir\", type=Path, required=True)\n    parser.add_argument(\"--budget-hours\", type=float, default=8.0)\n    parser.add_argument(\"--checkpoint-every\", type=int, default=25)\n    parser.add_argument(\"--allow-cpu-smoke\", action=\"store_true\")\n    args = parser.parse_args()\n\n    module = load_module(args.module)\n    _, _, _, torch, nn, _, _ = module._training_imports()\n    if not torch.cuda.is_available() and not args.allow_cpu_smoke:\n        raise RuntimeError(\"GPU is required\")\n    device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n    amp_dtype = (\n        torch.bfloat16 if device.type == \"cuda\" and torch.cuda.get_device_capability(0) >= (8, 0)\n        else torch.float16\n    )\n    Model = module.make_model_class()\n    models, configs, folds = [], [], []\n    for path in args.checkpoints:\n        checkpoint = torch.load(path, map_location=\"cpu\", weights_only=False)\n        folds.append(int(checkpoint.get(\"fold\", -1)))\n        config = module.TrainConfig(**checkpoint[\"config\"])\n        model = Model(config.backbone, checkpoint=None, backbone_chunk=config.backbone_chunk)\n        model.load_state_dict(checkpoint[\"model\"], strict=True)\n        model.eval().to(device)\n        models.append(model)\n        configs.append(config)\n    if set(folds) != set(range(5)):\n        raise ValueError(f\"expected exactly folds 0..4, got {folds}\")\n    if len({json.dumps(vars(config), sort_keys=True) for config in configs}) != 1:\n        raise ValueError(\"fold checkpoint configs disagree\")\n    config = configs[0]\n\n    test = pd.read_csv(args.test_csv)\n    series = pd.read_csv(args.series_csv)\n    if test[module.UID].duplicated().any():\n        raise ValueError(\"test contains duplicate study IDs\")\n    grouped = {str(uid): group.copy() for uid, group in series.groupby(module.UID, sort=False)}\n    empty = pd.DataFrame(columns=[\n        \"SeriesInstanceUID\", \"Anatomical_Plane\", \"Fluid_Sensitive\", \"Fat_Suppression\"\n    ])\n    uids = test[module.UID].astype(str).tolist()\n    args.output_dir.mkdir(parents=True, exist_ok=True)\n    submission_path = args.output_dir / \"submission.csv\"\n    predictions, errors, series_errors = {}, {}, {}\n    eval_slices, active_models = config.valid_slices, len(models)\n    started = time.time()\n\n    for index, uid in enumerate(uids):\n        try:\n            slots = module.pick_series_slots(grouped.get(uid, empty), len(module.PLANES))\n            images, masks = [], []\n            for series_uid in slots:\n                if series_uid is None:\n                    images.append(np.zeros((eval_slices, config.image_size, config.image_size), np.float32))\n                    masks.append(np.zeros(eval_slices, bool))\n                    continue\n                try:\n                    volume = module.load_dicom_volume(\n                        args.image_root / uid / series_uid,\n                        config.image_size,\n                        config.max_series_slices,\n                    )\n                except Exception as exc:\n                    series_errors[f\"{uid}/{series_uid}\"] = repr(exc)\n                    images.append(np.zeros((eval_slices, config.image_size, config.image_size), np.float32))\n                    masks.append(np.zeros(eval_slices, bool))\n                    continue\n                chosen = module.uniform_slice_indices(len(volume), eval_slices)\n                sampled = (volume[chosen].astype(np.float32) - 0.449) / 0.226\n                images.append(sampled)\n                masks.append(np.ones(eval_slices, bool))\n            mask = np.stack(masks)\n            if not mask.any():\n                raise RuntimeError(\"no decodable series\")\n            image_tensor = torch.from_numpy(np.stack(images)[:, :, None])[None].to(device)\n            mask_tensor = torch.from_numpy(mask)[None].to(device)\n            probability = torch.zeros(len(module.TARGETS), device=\"cpu\")\n            with torch.no_grad():\n                for model in models[:active_models]:\n                    amp = (\n                        torch.autocast(\"cuda\", dtype=amp_dtype)\n                        if device.type == \"cuda\" else contextlib.nullcontext()\n                    )\n                    with amp:\n                        logits = model(image_tensor, mask_tensor)\n                    probability += torch.sigmoid(logits.float()).cpu().squeeze(0)\n            value = (probability / active_models).numpy()\n            if not np.isfinite(value).all():\n                raise RuntimeError(\"non-finite prediction\")\n            predictions[uid] = value\n        except Exception as exc:\n            errors[uid] = repr(exc)\n            predictions[uid] = np.full(len(module.TARGETS), 0.5, dtype=float)\n\n        done = index + 1\n        if done % args.checkpoint_every == 0 or done == len(uids):\n            write_submission(submission_path, uids, module.TARGETS, predictions)\n            projected = (time.time() - started) / done * len(uids)\n            if done >= args.checkpoint_every and projected > args.budget_hours * 3600:\n                if eval_slices > 16:\n                    eval_slices = 16\n                elif active_models > 3:\n                    active_models = 3\n            print(\n                f\"{done}/{len(uids)} projected_hours={projected / 3600:.2f} \"\n                f\"slices={eval_slices} folds={active_models} errors={len(errors)}\",\n                flush=True,\n            )\n\n    write_submission(submission_path, uids, module.TARGETS, predictions)\n    meta = {\n        \"status\": \"PASS\" if not errors else \"PASS_WITH_FALLBACKS\",\n        \"studies\": len(uids),\n        \"errors\": errors,\n        \"series_errors\": series_errors,\n        \"wall_seconds\": time.time() - started,\n        \"eval_slices_final\": eval_slices,\n        \"active_models_final\": active_models,\n        \"checkpoints\": [str(path) for path in args.checkpoints],\n        \"folds\": folds,\n    }\n    (args.output_dir / \"infer_meta.json\").write_text(json.dumps(meta, indent=2, sort_keys=True) + \"\\n\")\n    print(json.dumps(meta, indent=2, sort_keys=True))\n\n\nif __name__ == \"__main__\":\n    main()\n"},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"# V49: inference-only five-fold B3 diversity blend. The independently completed\n# public .899 family remains the fallback until every pinned artifact and output\n# check passes. OOF selection used only official train.csv labels.\nimport hashlib as _v49_hashlib\nimport json as _v49_json\nimport math as _v49_math\nimport shutil as _v49_shutil\nimport subprocess as _v49_subprocess\nimport sys as _v49_sys\nimport time as _v49_time\nimport traceback as _v49_traceback\nfrom pathlib import Path as _V49Path\n\nimport numpy as _v49_np\nimport pandas as _v49_pd\n\n_V49_DATA = _V49Path(\"/kaggle/input/rsna-knee-b3-v47-folds-0-3\")\n_V49_COMP = _V49Path(\"/kaggle/input/competitions/rsna-knee-abnormality-detection\")\n_V49_BASELINE = _V49Path(\"/kaggle/working/submission_public_0899.csv\")\n_V49_OUTPUT = _V49Path(\"/kaggle/working/rsna_b3_v49_inference\")\n_V49_PRIMARY = _V49Path(\"/kaggle/working/submission.csv\")\n_V49_GLOBAL_ALPHA = 0.10\n_V49_TOTAL_LIMIT_SECONDS = 8.70 * 3600\n_V49_MIN_START_SECONDS = 12 * 60\n_V49_TARGETS = [\n    \"ACL\", \"MCL\", \"Medial Meniscus\", \"Lateral Meniscus\",\n    \"Medial OA\", \"Lateral OA\", \"PF OA\", \"Effusion\",\n    \"Synovitis\", \"Baker's\", \"Contusion\", \"Fracture\",\n]\n_V49_TARGET_ALPHAS = {\n    \"ACL\": 0.00,\n    \"MCL\": 0.10,\n    \"Medial Meniscus\": 0.00,\n    \"Lateral Meniscus\": 0.35,\n    \"Medial OA\": 0.15,\n    \"Lateral OA\": 0.35,\n    \"PF OA\": 0.35,\n    \"Effusion\": 0.25,\n    \"Synovitis\": 0.35,\n    \"Baker's\": 0.35,\n    \"Contusion\": 0.00,\n    \"Fracture\": 0.00,\n}\n_V49_EXPECTED = {\n    _V49_DATA / \"source/efficientnet_b3_public_repro_v4_t4.py\":\n        \"825b82656eea4ab3dd53c103fd04ce9ead803451486d29f61febdfc73e3751ec\",\n    _V49_DATA / \"source/efficientnet_b3_public_repro_v1_infer.py\":\n        \"42a53bb80a7cece4d06a5e4ddc1f887e900a3adc4991e51ebfbbec58476438f5\",\n    _V49_DATA / \"fold0/fold0_final.pt\":\n        \"71a4cafb8f13341c4ca8a50b20bcc1ca3ad63c88b721d632865e2305cbb282e3\",\n    _V49_DATA / \"fold1/fold1_final.pt\":\n        \"088065c483017a7497a770c73530c8850467422738ddff3a6ca111a3f1749948\",\n    _V49_DATA / \"fold2/fold2_final.pt\":\n        \"58482553af61a9afdfaccd7b7319a0af81046421054af0cf995c1da70dc36486\",\n    _V49_DATA / \"fold3/fold3_final.pt\":\n        \"6b60f18a28c6fc80ab04939ae25be2358705013e4779706bbd9534c52fec4472\",\n    _V49_DATA / \"fold4/fold4_final.pt\":\n        \"ccb18c38000fbdb152c2717be2f340c446978cad40f2db7667f15a1a49e31249\",\n    _V49_DATA / \"audit/audit.json\":\n        \"67fb65145da05f3fe8e8ab5fb49166a0550f87fe6a0805f8ebb3adb0a04d108f\",\n}\n\n\ndef _v49_sha256(path):\n    digest = _v49_hashlib.sha256()\n    with open(path, \"rb\") as handle:\n        for chunk in iter(lambda: handle.read(8 << 20), b\"\"):\n            digest.update(chunk)\n    return digest.hexdigest()\n\n\n_v49_audit = {\n    \"status\": \"FALLBACK\",\n    \"evidence_boundary\": (\n        \"The .899 fallback is an official completed Kaggle score. V49 OOF values \"\n        \"are local diagnostics on 58 official expert labels, not competition scores.\"\n    ),\n    \"selection\": {\n        \"primary\": \"global rank blend\",\n        \"b3_weight\": _V49_GLOBAL_ALPHA,\n        \"official_label_oof\": {\n            \"exact_public\": 0.8434849072528627,\n            \"b3\": 0.7792484680871908,\n            \"global_10_percent\": 0.8473465265579451,\n            \"fold_nested\": 0.8463056657160722,\n            \"foldwise_selected_weights\": [0.10, 0.10, 0.10, 0.10, 0.20],\n        },\n    },\n    \"expected_artifacts\": {str(path): digest for path, digest in _V49_EXPECTED.items()},\n}\n\n# Establish the independently completed fallback before any optional work.\nif _V49_BASELINE.is_file():\n    _v49_shutil.copy2(_V49_BASELINE, _V49_PRIMARY)\n\ntry:\n    if not _V49_BASELINE.is_file():\n        raise FileNotFoundError(\"independently completed public .899 fallback is absent\")\n    _V49_OUTPUT.mkdir(parents=True, exist_ok=True)\n\n    _v49_observed = {str(path): _v49_sha256(path) for path in _V49_EXPECTED}\n    _v49_bad = {\n        str(path): {\"expected\": expected, \"observed\": _v49_observed[str(path)]}\n        for path, expected in _V49_EXPECTED.items()\n        if _v49_observed[str(path)] != expected\n    }\n    if _v49_bad:\n        raise RuntimeError(f\"pinned V49 artifact drift: {_v49_bad}\")\n\n    _v49_official_audit = _v49_json.loads(\n        (_V49_DATA / \"audit/audit.json\").read_text()\n    )\n    _v49_mismatches = _v49_official_audit.get(\n        \"package_y_vs_official_label_mismatches\", {}\n    )\n    if not _v49_mismatches or not all(int(value) > 0 for value in _v49_mismatches.values()):\n        raise RuntimeError(\"official-label audit provenance is absent or changed\")\n    _v49_nested = float(\n        _v49_official_audit[\"selection\"][\"global_nested_macro_auc\"]\n    )\n    _v49_base_oof = float(\n        _v49_official_audit[\"arms\"][\"exact_public_macro_auc\"]\n    )\n    if _v49_nested <= _v49_base_oof:\n        raise RuntimeError(\"fold-nested official-label OOF does not support the blend\")\n\n    import torch as _v49_torch\n\n    if not _v49_torch.cuda.is_available():\n        raise RuntimeError(\"V49 five-fold inference requires CUDA\")\n    _v49_elapsed_before = max(0.0, _v49_time.time() - float(globals().get(\"T0\", _v49_time.time())))\n    _v49_available = _V49_TOTAL_LIMIT_SECONDS - _v49_elapsed_before\n    if _v49_available < _V49_MIN_START_SECONDS:\n        raise TimeoutError(\n            f\"only {_v49_available / 60:.1f} minutes remain; preserving .899 fallback\"\n        )\n    _v49_budget_hours = min(1.75, max(0.20, 0.90 * _v49_available / 3600.0))\n    _v49_checkpoints = [\n        _V49_DATA / f\"fold{fold}\" / f\"fold{fold}_final.pt\" for fold in range(5)\n    ]\n    _v49_cmd = [\n        _v49_sys.executable,\n        str(_V49_DATA / \"source/efficientnet_b3_public_repro_v1_infer.py\"),\n        \"--module\", str(_V49_DATA / \"source/efficientnet_b3_public_repro_v4_t4.py\"),\n        \"--test-csv\", str(_V49_COMP / \"test.csv\"),\n        \"--series-csv\", str(_V49_COMP / \"test_series.csv\"),\n        \"--image-root\", str(_V49_COMP / \"test_series\"),\n        \"--checkpoints\", *map(str, _v49_checkpoints),\n        \"--output-dir\", str(_V49_OUTPUT),\n        \"--budget-hours\", f\"{_v49_budget_hours:.6f}\",\n        \"--checkpoint-every\", \"10\",\n    ]\n    print(\n        f\"[B3 V49] inference-only candidate; available={_v49_available / 3600:.2f}h \"\n        f\"adaptive-budget={_v49_budget_hours:.2f}h\",\n        flush=True,\n    )\n    _v49_started = _v49_time.monotonic()\n    _v49_result = _v49_subprocess.run(\n        _v49_cmd,\n        timeout=max(60.0, 0.98 * _v49_available),\n        check=False,\n    )\n    if _v49_result.returncode != 0:\n        raise RuntimeError(f\"V49 inference exit={_v49_result.returncode}\")\n\n    _v49_b3_path = _V49_OUTPUT / \"submission.csv\"\n    _v49_meta_path = _V49_OUTPUT / \"infer_meta.json\"\n    if not _v49_b3_path.is_file() or not _v49_meta_path.is_file():\n        raise FileNotFoundError(\"V49 inference outputs are incomplete\")\n    _v49_meta = _v49_json.loads(_v49_meta_path.read_text())\n    _v49_studies = int(_v49_meta.get(\"studies\", 0))\n    _v49_error_count = len(_v49_meta.get(\"errors\", {}))\n    _v49_error_cap = max(2, int(_v49_math.ceil(0.01 * _v49_studies)))\n    if _v49_studies < 1 or _v49_error_count > _v49_error_cap:\n        raise RuntimeError(\n            f\"study fallback count {_v49_error_count} exceeds cap {_v49_error_cap}\"\n        )\n    if int(_v49_meta.get(\"active_models_final\", 0)) < 3:\n        raise RuntimeError(\"fewer than three B3 folds remained active\")\n    if int(_v49_meta.get(\"eval_slices_final\", 0)) < 16:\n        raise RuntimeError(\"B3 inference slice count fell below the audited floor\")\n\n    _v49_base = _v49_pd.read_csv(_V49_BASELINE, dtype={\"StudyInstanceUID\": str})\n    _v49_b3 = _v49_pd.read_csv(_v49_b3_path, dtype={\"StudyInstanceUID\": str})\n    _v49_columns = [\"StudyInstanceUID\", *_V49_TARGETS]\n    if _v49_base.columns.tolist() != _v49_columns or _v49_b3.columns.tolist() != _v49_columns:\n        raise RuntimeError(\"V49 submission schema drift\")\n    if _v49_base[\"StudyInstanceUID\"].tolist() != _v49_b3[\"StudyInstanceUID\"].tolist():\n        raise RuntimeError(\"V49 study order drift\")\n    if _v49_base[\"StudyInstanceUID\"].duplicated().any():\n        raise RuntimeError(\"V49 duplicate study IDs\")\n    if not (\n        _v49_np.isfinite(_v49_base[_V49_TARGETS].to_numpy()).all()\n        and _v49_np.isfinite(_v49_b3[_V49_TARGETS].to_numpy()).all()\n    ):\n        raise RuntimeError(\"V49 non-finite prediction\")\n\n    _v49_base_rank = _v49_base[_V49_TARGETS].rank(method=\"average\", pct=True)\n    _v49_b3_rank = _v49_b3[_V49_TARGETS].rank(method=\"average\", pct=True)\n    for _v49_alpha in (0.05, 0.10, 0.15, 0.20, 0.25):\n        _v49_candidate = _v49_base.copy()\n        _v49_candidate[_V49_TARGETS] = (\n            (1.0 - _v49_alpha) * _v49_base_rank + _v49_alpha * _v49_b3_rank\n        )\n        _v49_candidate.to_csv(\n            f\"/kaggle/working/submission_b3_v49_global_{int(100 * _v49_alpha):02d}.csv\",\n            index=False,\n        )\n\n    _v49_target_candidate = _v49_base.copy()\n    for _v49_target in _V49_TARGETS:\n        _v49_alpha = _V49_TARGET_ALPHAS[_v49_target]\n        _v49_target_candidate[_v49_target] = (\n            (1.0 - _v49_alpha) * _v49_base_rank[_v49_target]\n            + _v49_alpha * _v49_b3_rank[_v49_target]\n        )\n    _v49_target_candidate.to_csv(\n        \"/kaggle/working/submission_b3_v49_target_nested.csv\", index=False\n    )\n\n    _v49_selected = _V49Path(\"/kaggle/working/submission_b3_v49_global_10.csv\")\n    _v49_selected_frame = _v49_pd.read_csv(\n        _v49_selected, dtype={\"StudyInstanceUID\": str}\n    )\n    if _v49_selected_frame.shape != _v49_base.shape:\n        raise RuntimeError(\"V49 selected candidate shape drift\")\n    _v49_shutil.copy2(_v49_selected, _V49_PRIMARY)\n    _v49_audit.update(\n        {\n            \"status\": \"CANDIDATE_SELECTED\",\n            \"elapsed_before_b3_seconds\": _v49_elapsed_before,\n            \"b3_wall_seconds\": _v49_time.monotonic() - _v49_started,\n            \"adaptive_budget_hours\": _v49_budget_hours,\n            \"infer_meta\": _v49_meta,\n            \"selected_sha256\": _v49_sha256(_V49_PRIMARY),\n            \"fallback_sha256\": _v49_sha256(_V49_BASELINE),\n        }\n    )\n    print(\n        f\"[B3 V49] selected global 10% rank blend; sha={_v49_audit['selected_sha256']}\",\n        flush=True,\n    )\nexcept Exception as _v49_error:\n    _v49_audit[\"status\"] = \"FALLBACK\"\n    _v49_audit[\"error\"] = f\"{type(_v49_error).__name__}: {_v49_error}\"\n    _v49_audit[\"traceback\"] = _v49_traceback.format_exc()\n    if _V49_BASELINE.is_file():\n        _v49_shutil.copy2(_V49_BASELINE, _V49_PRIMARY)\n    print(\n        f\"[B3 V49] preserving .899 fallback: {_v49_audit['error']}\",\n        flush=True,\n    )\nfinally:\n    _V49Path(\"/kaggle/working/b3_v49_audit.json\").write_text(\n        _v49_json.dumps(_v49_audit, indent=2, sort_keys=True) + \"\\n\"\n    )\n"},{"cell_type":"code","execution_count":null,"id":"v50-orthofoundation","metadata":{},"outputs":[],"source":"# V52: official RadImageNet ResNet-50 frozen MRI slice encoder.\n# The grouped cross-fit protocol, official-label gate, and V49 fallback remain unchanged.\n# V49 is preserved unless exact 58-label outer-fold evidence supports this branch.\nSLOTS = [\n    (\"SAG_FS\", \"Sagittal\", None, True),\n    (\"COR_FS\", \"Coronal\", None, True),\n    (\"AX_FS\", \"Axial\", None, True),\n]\nN_SLOT = len(SLOTS)\nCACHE_SLICES = 8\nTIME_BUDGET = 8.72 * 3600\nIMG = CACHE_IMG = 224\n# Match the released RadImageNet model's full-frame pretraining. Setting a crop\n# larger than every acquisition disables the optional physical crop in read_slot.\nCROP_MM = 10_000.0\nSLICE_BAND = (0.12, 0.88)\nTOKEN_DIM = 2048             # official ResNet-50 global-average feature\nHEAD_DIM = 512\n\n\ndef _v52_as_bool(value):\n    if pd.isna(value):\n        return None\n    text = str(value).strip().upper()\n    if text in {\"1\", \"TRUE\", \"T\", \"YES\", \"Y\"}:\n        return True\n    if text in {\"0\", \"FALSE\", \"F\", \"NO\", \"N\"}:\n        return False\n    try:\n        number = float(text)\n        return True if number == 1 else False if number == 0 else None\n    except Exception:\n        return None\n\n\ndef audit_official_sequence_metadata(inferred, official):\n    \"\"\"Audit metadata agreement without changing the checkpoint pixel contract.\"\"\"\n    needed = {\"SeriesInstanceUID\", \"Fluid_Sensitive\", \"Fat_Suppression\"}\n    if inferred.empty or official.empty or not needed.issubset(official.columns):\n        return\n    inferred_flags = inferred[[\"SeriesInstanceUID\", \"fluid\", \"fatsat\"]].copy()\n    official_flags = official[\n        [\"SeriesInstanceUID\", \"Fluid_Sensitive\", \"Fat_Suppression\"]\n    ].copy()\n    official_flags[\"official_fluid\"] = official_flags[\"Fluid_Sensitive\"].map(\n        _v52_as_bool\n    )\n    official_flags[\"official_fatsat\"] = official_flags[\"Fat_Suppression\"].map(\n        _v52_as_bool\n    )\n    merged = inferred_flags.merge(\n        official_flags[[\"SeriesInstanceUID\", \"official_fluid\", \"official_fatsat\"]],\n        on=\"SeriesInstanceUID\",\n        how=\"inner\",\n    )\n    for inferred_col, official_col, name in [\n        (\"fluid\", \"official_fluid\", \"Fluid_Sensitive\"),\n        (\"fatsat\", \"official_fatsat\", \"Fat_Suppression\"),\n    ]:\n        valid = merged[official_col].notna() & merged[inferred_col].notna()\n        if valid.any():\n            agreement = (\n                merged.loc[valid, inferred_col].astype(bool).to_numpy()\n                == merged.loc[valid, official_col].astype(bool).to_numpy()\n            ).mean()\n            log(\n                f\"V52 metadata audit {name}: {agreement:.1%} agreement \"\n                f\"on {int(valid.sum())} series\"\n            )\n\n\ndef find_input_file(name):\n    for root, dirs, files in os.walk(\"/kaggle/input\"):\n        dirs[:] = [d for d in dirs if d not in (\"train_series\", \"test_series\")]\n        if name in files:\n            return Path(root) / name\n    raise FileNotFoundError(name)\n\n\ndef find_input_dir(name):\n    for root, dirs, files in os.walk(\"/kaggle/input\"):\n        if Path(root).name == name:\n            return Path(root)\n    raise FileNotFoundError(name)\n\n\ndef make_targets(train):\n    \"\"\"Three independent public report teachers; image-read gold always wins.\"\"\"\n    uid = \"StudyInstanceUID\"\n    sources = [\n        pd.read_csv(find_input_file(\"report_labels_v2.csv\")),\n        pd.read_csv(find_input_file(\"llm_labels_v2.csv\")),\n        pd.read_csv(find_input_file(\"labels_llm_gpt56sol.csv\")),\n    ]\n    cube = []\n    for frame in sources:\n        if frame[uid].duplicated().any():\n            raise ValueError(\"duplicate study in report-label source\")\n        aligned = train[[uid]].merge(frame[[uid] + TARGETS], on=uid, how=\"left\")\n        cube.append(aligned[TARGETS].to_numpy(float))\n    cube = np.stack(cube)\n    available = np.isfinite(cube).sum(0)\n    if np.any(available < 2):\n        raise ValueError(\"fewer than two report teachers for a study/target\")\n    y = np.nanmean(cube, axis=0).astype(np.float32)\n    disagreement = np.nanmean(np.abs(cube - y[None]), axis=0)\n    agreement = np.clip(1.0 - 2.0 * disagreement, 0, 1)\n    certainty = np.clip(2.0 * np.abs(y - .5), 0, 1)\n    w = (.15 + .85 * (.65 * agreement + .35 * certainty)).astype(np.float32)\n    gold = train[TARGETS].notna().all(axis=1).to_numpy()\n    y[gold] = train.loc[gold, TARGETS].to_numpy(np.float32)\n    w[gold] = 3.0\n    return y, w, gold\n\n\ndef report_groups(train):\n    report = (train.Report.fillna(\"\").astype(str).str.lower()\n              .str.replace(r\"\\s+\", \" \", regex=True).str.strip())\n    return np.array([hashlib.sha256(x.encode()).hexdigest()[:24] for x in report])\n\n\ndef _v52_sha256(path):\n    digest = hashlib.sha256()\n    with open(path, \"rb\") as handle:\n        for chunk in iter(lambda: handle.read(8 << 20), b\"\"):\n            digest.update(chunk)\n    return digest.hexdigest()\n\n\ndef load_radimagenet(device):\n    \"\"\"Strictly load the official RadImageNet ResNet-50 PyTorch checkpoint.\"\"\"\n    from torchvision.models import resnet50\n\n    checkpoint = find_input_file(\"ResNet50.pt\")\n    expected_checkpoint = \"08629f7e7bd3e29b8ee9522ca3f65ce4d010a7ddf74f0ea3c7e3f3d0bbab0734\"\n    observed_checkpoint = _v52_sha256(checkpoint)\n    if observed_checkpoint != expected_checkpoint:\n        raise RuntimeError(f\"RadImageNet checkpoint drift: {observed_checkpoint}\")\n\n    class RadImageNetEncoder(nn.Module):\n        def __init__(self):\n            super().__init__()\n            self.backbone = nn.Sequential(\n                *list(resnet50(weights=None).children())[:-2]\n            )\n\n        def forward(self, image):\n            return self.backbone(image).mean(dim=(2, 3))\n\n    model = RadImageNetEncoder()\n    state = torch.load(checkpoint, map_location=\"cpu\", weights_only=True)\n    if not state or not all(str(key).startswith(\"backbone.\") for key in state):\n        raise RuntimeError(\"unexpected RadImageNet state-dict namespace\")\n    model.load_state_dict(state, strict=True)\n    parameter_count = sum(parameter.numel() for parameter in model.parameters())\n    if parameter_count != 23_508_032:\n        raise RuntimeError(f\"unexpected RadImageNet parameter count {parameter_count}\")\n    model.eval().to(device)\n    for parameter in model.parameters():\n        parameter.requires_grad_(False)\n    gpu_count = torch.cuda.device_count() if device.type == \"cuda\" else 0\n    if gpu_count > 1:\n        model = nn.DataParallel(model, device_ids=list(range(gpu_count)))\n    log(\n        f\"RadImageNet strict load: {parameter_count:,} params; \"\n        f\"inference GPUs={max(1, gpu_count)}\"\n    )\n    return model\n\n\n@torch.inference_mode()\ndef encode_radimagenet(cache, slot_mask, device):\n    \"\"\"Encode acquired slices with the official [-1, 1] RadImageNet contract.\"\"\"\n    n, slots, slices, h, w = cache.shape\n    features = np.zeros((n, slots * slices, TOKEN_DIM), np.float16)\n    token_mask = np.repeat(slot_mask[:, :, None], slices, axis=2).reshape(n, -1)\n    valid = np.flatnonzero(token_mask.reshape(-1) > 0)\n    flat = cache.reshape(-1, h, w)\n    model = load_radimagenet(device)\n    if device.type == \"cuda\":\n        batch = 192 if torch.cuda.device_count() > 1 else 96\n    else:\n        batch = 8\n    for b0 in range(0, len(valid), batch):\n        ix = valid[b0:b0 + batch]\n        x = torch.from_numpy(flat[ix]).to(device).float().div_(127.5).sub_(1.0)\n        x = x.unsqueeze(1).expand(-1, 3, -1, -1).contiguous()\n        with torch.autocast(\"cuda\", enabled=device.type == \"cuda\"):\n            feat = model(x)\n        if feat.shape[1:] != (TOKEN_DIM,):\n            raise RuntimeError(f\"unexpected RadImageNet feature shape {tuple(feat.shape)}\")\n        features.reshape(-1, TOKEN_DIM)[ix] = (\n            feat.float().cpu().numpy().astype(np.float16)\n        )\n        if b0 % (batch * 100) == 0:\n            log(f\"RadImageNet encoded {b0}/{len(valid)} acquired slices\")\n    del model\n    if device.type == \"cuda\":\n        torch.cuda.empty_cache()\n    return features, token_mask.astype(np.float32)\n\n\nclass FoundationQueryHead(nn.Module):\n    def __init__(self):\n        super().__init__()\n        self.project = nn.Sequential(nn.LayerNorm(TOKEN_DIM),\n                                     nn.Linear(TOKEN_DIM, HEAD_DIM), nn.GELU())\n        self.plane = nn.Parameter(torch.randn(N_SLOT, HEAD_DIM) * .01)\n        self.position = nn.Parameter(torch.randn(CACHE_SLICES, HEAD_DIM) * .01)\n        self.query = nn.Parameter(torch.randn(len(TARGETS), HEAD_DIM) * .02)\n        self.attn = nn.MultiheadAttention(HEAD_DIM, 8, dropout=.10, batch_first=True)\n        self.fuse = nn.Sequential(\n            nn.LayerNorm(HEAD_DIM * 4), nn.Linear(HEAD_DIM * 4, HEAD_DIM),\n            nn.GELU(), nn.Dropout(.15),\n        )\n        self.weight = nn.Parameter(torch.randn(len(TARGETS), HEAD_DIM) * .02)\n        self.bias = nn.Parameter(torch.zeros(len(TARGETS)))\n\n    def forward(self, feature, mask):\n        token = self.project(feature.float())\n        token = token.view(len(token), N_SLOT, CACHE_SLICES, HEAD_DIM)\n        token = token + self.plane[None, :, None] + self.position[None, None]\n        token = token.flatten(1, 2)\n        key_padding = mask <= 0\n        # No study should be empty, but keep MHA numerically defined if one is.\n        all_empty = key_padding.all(1)\n        if all_empty.any():\n            key_padding = key_padding.clone()\n            key_padding[all_empty, 0] = False\n        query = self.query.unsqueeze(0).expand(len(token), -1, -1)\n        attended = query + self.attn(query, token, token,\n                                     key_padding_mask=key_padding,\n                                     need_weights=False)[0]\n        denom = mask.sum(1, keepdim=True).clamp_min(1).unsqueeze(-1)\n        mean = (token * mask.unsqueeze(-1)).sum(1, keepdims=True) / denom\n        mean = mean.expand(-1, len(TARGETS), -1)\n        fused = self.fuse(torch.cat(\n            [attended, mean, torch.abs(attended - mean), attended * mean], -1))\n        return (fused * self.weight.unsqueeze(0)).sum(-1) + self.bias\n\n\ndef macro_auc(y, pred):\n    from sklearn.metrics import roc_auc_score\n    hard = (np.asarray(y) >= .5).astype(np.uint8)\n    values = [roc_auc_score(hard[:, j], pred[:, j])\n              for j in range(hard.shape[1]) if np.unique(hard[:, j]).size == 2]\n    return float(np.mean(values))\n\n\n@torch.inference_mode()\ndef predict_head(model, features, masks, indices, device, batch=64):\n    model.eval()\n    pred = []\n    for b0 in range(0, len(indices), batch):\n        ix = indices[b0:b0 + batch]\n        x = torch.from_numpy(features[ix]).to(device)\n        m = torch.from_numpy(masks[ix]).to(device)\n        with torch.autocast(\"cuda\", enabled=device.type == \"cuda\"):\n            pred.append(torch.sigmoid(model(x, m)).float().cpu())\n    return torch.cat(pred).numpy()\n\n\ndef train_fold(features, masks, y, weights, train_idx, val_idx, fold, device):\n    from torch.utils.data import DataLoader, Dataset\n    class Rows(Dataset):\n        def __init__(self, indices): self.indices = np.asarray(indices)\n        def __len__(self): return len(self.indices)\n        def __getitem__(self, k):\n            i = self.indices[k]\n            return features[i], masks[i], y[i], weights[i]\n    model = FoundationQueryHead().to(device)\n    optimizer = torch.optim.AdamW(model.parameters(), lr=2e-4, weight_decay=3e-3)\n    generator = torch.Generator().manual_seed(SEED + 100 + fold)\n    loader = DataLoader(Rows(train_idx), batch_size=48, shuffle=True,\n                        generator=generator, num_workers=2, pin_memory=True,\n                        persistent_workers=True)\n    best, best_auc, stale = None, -1.0, 0\n    for epoch in range(24):\n        model.train()\n        for x, m, target, weight in loader:\n            x, m = x.to(device), m.to(device)\n            target, weight = target.to(device), weight.to(device)\n            with torch.autocast(\"cuda\", enabled=device.type == \"cuda\"):\n                logits = model(x, m)\n                raw = F.binary_cross_entropy_with_logits(logits, target,\n                                                          reduction=\"none\")\n                loss = (raw * weight).sum() / weight.sum().clamp_min(1)\n            optimizer.zero_grad(set_to_none=True)\n            loss.backward()\n            nn.utils.clip_grad_norm_(model.parameters(), 1.0)\n            optimizer.step()\n        pred = predict_head(model, features, masks, val_idx, device)\n        score = macro_auc(y[val_idx], pred)\n        log(f\"fold {fold} epoch {epoch}: grouped weak-val AUC {score:.5f}\")\n        if score > best_auc + 2e-4:\n            best_auc, stale = score, 0\n            best = {k: v.detach().cpu() for k, v in model.state_dict().items()}\n        else:\n            stale += 1\n            if stale >= 5: break\n    return best, best_auc\n\n\ndef _v52_rank_columns(values):\n    frame = pd.DataFrame(np.asarray(values, dtype=np.float64))\n    return frame.rank(method=\"average\", pct=True).to_numpy(np.float64)\n\n\ndef _v52_validate_submission(frame, expected_ids):\n    expected_columns = [\"StudyInstanceUID\", *TARGETS]\n    if frame.columns.tolist() != expected_columns:\n        raise RuntimeError(\"V52 submission schema drift\")\n    ids = frame[\"StudyInstanceUID\"].astype(str).tolist()\n    if ids != list(map(str, expected_ids)) or len(ids) != len(set(ids)):\n        raise RuntimeError(\"V52 submission study identity/order drift\")\n    values = frame[TARGETS].to_numpy(np.float64)\n    if not np.isfinite(values).all() or values.min() < 0 or values.max() > 1:\n        raise RuntimeError(\"V52 submission values are invalid\")\n\n\ndef main_v52():\n    import shutil\n    from sklearn.model_selection import GroupKFold\n\n    output = Path(\"/kaggle/working/rsna_rad_v52\")\n    output.mkdir(parents=True, exist_ok=True)\n    primary = Path(\"/kaggle/working/submission.csv\")\n    preserved = Path(\"/kaggle/working/submission_v49_preserved.csv\")\n    audit_path = Path(\"/kaggle/working/rad_v52_audit.json\")\n    audit = {\n        \"status\": \"V49_PRESERVED\",\n        \"evidence_boundary\": (\n            \"All OOF values are local diagnostics on 58 official image labels; \"\n            \"they are not Kaggle competition scores. V49 remains the primary unless \"\n            \"strict artifact, OOF, inference, and submission gates all pass.\"\n        ),\n        \"encoder\": \"RadImageNet ResNet-50 official PyTorch release\",\n        \"encoder_license\": \"CC-BY-NC-SA-4.0 (Kaggle-hosted weight metadata)\",\n        \"encoder_sha256\": \"08629f7e7bd3e29b8ee9522ca3f65ce4d010a7ddf74f0ea3c7e3f3d0bbab0734\",\n        \"encoder_source_commit\": \"0ce16f7375db4236e646829d1eca61cdb4282133\",\n    }\n    if not primary.is_file():\n        raise FileNotFoundError(\"V49 primary submission is absent\")\n    shutil.copy2(primary, preserved)\n\n    try:\n        device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n        if device.type != \"cuda\":\n            raise RuntimeError(\"V52 RadImageNet experiment requires CUDA\")\n        elapsed = max(0.0, time.time() - float(globals().get(\"T0\", time.time())))\n        available = 8.72 * 3600 - elapsed\n        audit[\"elapsed_before_v52_seconds\"] = elapsed\n        audit[\"available_at_start_seconds\"] = available\n        if available < 2.0 * 3600:\n            raise TimeoutError(f\"only {available / 60:.1f} minutes remain\")\n\n        train = pd.read_csv(ROOT / \"train.csv\", dtype={\"StudyInstanceUID\": str})\n        train_series = pd.read_csv(\n            ROOT / \"train_series.csv\",\n            dtype={\"StudyInstanceUID\": str, \"SeriesInstanceUID\": str},\n        )\n        if len(train) != 4407:\n            raise RuntimeError(f\"unexpected train study count {len(train)}\")\n        plane = dict(zip(train_series.SeriesInstanceUID, train_series.Anatomical_Plane))\n        headers = annotate(walk(\"train_series\"))\n        audit_official_sequence_metadata(headers, train_series)\n        studies, pixels, slot_mask = build_cache(\n            pick_slots(headers, plane), plane, lat_of(headers, \"train-v52 \"), \"train-v52\"\n        )\n        by_uid = {str(uid): i for i, uid in enumerate(studies)}\n        missing = [uid for uid in train.StudyInstanceUID if uid not in by_uid]\n        if missing:\n            raise RuntimeError(f\"{len(missing)} train studies absent from cache\")\n        order = np.array([by_uid[uid] for uid in train.StudyInstanceUID], dtype=np.int64)\n        pixels, slot_mask = pixels[order], slot_mask[order]\n        train_token_count = int(np.repeat(slot_mask[:, :, None], CACHE_SLICES, 2).sum())\n        if train_token_count < int(0.90 * len(train) * N_SLOT * CACHE_SLICES):\n            raise RuntimeError(f\"insufficient acquired train slices: {train_token_count}\")\n        features, token_mask = encode_radimagenet(pixels, slot_mask, device)\n        del pixels, slot_mask, headers\n        gc.collect()\n\n        y, weights, gold = make_targets(train)\n        if int(gold.sum()) != 58:\n            raise RuntimeError(f\"expected 58 fully gold studies, observed {int(gold.sum())}\")\n        groups = report_groups(train)\n        if len(np.unique(groups)) < 4000:\n            raise RuntimeError(\"unexpected report-group collapse\")\n\n        splits = list(GroupKFold(5).split(features, groups=groups))\n        fold_id = np.full(len(train), -1, dtype=np.int8)\n        folds = []\n        oof = np.zeros_like(y, dtype=np.float32)\n        for fold, (tr, va) in enumerate(splits):\n            if set(groups[tr]).intersection(groups[va]):\n                raise RuntimeError(f\"report leakage in fold {fold}\")\n            fold_id[va] = fold\n            state, score = train_fold(\n                features, token_mask, y, weights, tr, va, fold, device\n            )\n            if state is None:\n                raise RuntimeError(f\"fold {fold} produced no checkpoint\")\n            head = FoundationQueryHead().to(device)\n            head.load_state_dict(state, strict=True)\n            oof[va] = predict_head(head, features, token_mask, va, device)\n            folds.append({\"fold\": fold, \"weak_auc\": float(score), \"state_dict\": state})\n            del head\n            torch.cuda.empty_cache()\n        if (fold_id < 0).any() or not np.isfinite(oof).all():\n            raise RuntimeError(\"incomplete V52 OOF\")\n\n        weak_auc = macro_auc(y, oof)\n        gold_auc = macro_auc(y[gold], oof[gold])\n        log(f\"V52 RadImageNet OOF weak macro AUC {weak_auc:.5f}\")\n        log(f\"V52 RadImageNet OOF gold macro AUC {gold_auc:.5f} on 58 studies\")\n        torch.save(\n            {\n                \"version\": \"v52-radimagenet-resnet50-official-1\",\n                \"targets\": TARGETS,\n                \"encoder_sha256\": audit[\"encoder_sha256\"],\n                \"encoder_source_commit\": audit[\"encoder_source_commit\"],\n                \"img\": IMG,\n                \"slices_per_plane\": CACHE_SLICES,\n                \"feature\": \"global_average_pool\",\n                \"folds\": folds,\n                \"weak_oof_auc\": weak_auc,\n                \"gold_oof_auc\": gold_auc,\n            },\n            output / \"v52_radimagenet_heads.pt\",\n        )\n        oof_frame = pd.DataFrame(oof, columns=TARGETS)\n        oof_frame.insert(0, \"StudyInstanceUID\", train.StudyInstanceUID)\n        oof_frame[\"fold\"] = fold_id\n        oof_frame[\"is_gold\"] = gold.astype(np.uint8)\n        oof_frame.to_csv(output / \"v52_oof.csv\", index=False)\n\n        exact_npz = find_input_file(\"exact_public_family_oof.npz\")\n        expert_csv = find_input_file(\"expert_oof_all.csv\")\n        expected_oof_hashes = {\n            str(exact_npz): \"d8af61726d9ec07abba19cabfa9c548de05bdb11206c9442a3d72ede737eb004\",\n            str(expert_csv): \"c4c7cd8cadc67b2e88619ba10db355b5760c4a065b73a5445c4ef0caf9a86e0d\",\n        }\n        observed_oof_hashes = {path: _v52_sha256(path) for path in expected_oof_hashes}\n        if observed_oof_hashes != expected_oof_hashes:\n            raise RuntimeError(f\"official-label OOF artifact drift: {observed_oof_hashes}\")\n        exact = np.load(exact_npz, allow_pickle=False)\n        expert = pd.read_csv(expert_csv, dtype={\"StudyInstanceUID\": str})\n        if len(expert) != 58 or expert.StudyInstanceUID.duplicated().any():\n            raise RuntimeError(\"expert OOF identity drift\")\n        train_index = {uid: i for i, uid in enumerate(train.StudyInstanceUID)}\n        exact_index = {str(uid): i for i, uid in enumerate(exact[\"ids\"].tolist())}\n        if not set(expert.StudyInstanceUID).issubset(train_index):\n            raise RuntimeError(\"expert UID absent from train.csv\")\n        if not set(expert.StudyInstanceUID).issubset(exact_index):\n            raise RuntimeError(\"expert UID absent from exact public OOF\")\n        train_rows = np.array([train_index[uid] for uid in expert.StudyInstanceUID])\n        exact_rows = np.array([exact_index[uid] for uid in expert.StudyInstanceUID])\n        gold_y = expert[[f\"gold::{target}\" for target in TARGETS]].to_numpy(np.float64)\n        train_gold_y = train.loc[train_rows, TARGETS].to_numpy(np.float64)\n        if not np.array_equal(gold_y, train_gold_y):\n            raise RuntimeError(\"expert labels do not exactly match train.csv\")\n        exact_public = exact[\"ours\"][exact_rows].astype(np.float64)\n        b3 = expert[[f\"pred::{target}\" for target in TARGETS]].to_numpy(np.float64)\n        rad = oof[train_rows].astype(np.float64)\n        if not all(np.isfinite(x).all() for x in (gold_y, exact_public, b3, rad)):\n            raise RuntimeError(\"non-finite aligned OOF value\")\n\n        v49_raw = 0.90 * _v52_rank_columns(exact_public) + 0.10 * _v52_rank_columns(b3)\n        v49_rank = _v52_rank_columns(v49_raw)\n        rad_rank = _v52_rank_columns(rad)\n        base_score = macro_auc(gold_y, v49_rank)\n        rad_score = macro_auc(gold_y, rad_rank)\n        alpha_grid = np.array([0.0, 0.025, 0.05, 0.10, 0.15, 0.20, 0.25])\n        gold_folds = fold_id[train_rows]\n        if sorted(np.unique(gold_folds).tolist()) != [0, 1, 2, 3, 4]:\n            raise RuntimeError(\"gold rows do not cover all five grouped folds\")\n        nested = np.zeros_like(v49_rank)\n        choices = []\n        outer_train_scores = []\n        for outer in range(5):\n            tr = gold_folds != outer\n            va = ~tr\n            scored = []\n            for alpha in alpha_grid:\n                blend = (1.0 - alpha) * v49_rank[tr] + alpha * rad_rank[tr]\n                score = macro_auc(gold_y[tr], blend) - 0.01 * float(alpha)\n                scored.append(float(score))\n            best = max(range(len(alpha_grid)), key=lambda i: (scored[i], -alpha_grid[i]))\n            alpha = float(alpha_grid[best])\n            choices.append(alpha)\n            outer_train_scores.append(scored)\n            nested[va] = (1.0 - alpha) * v49_rank[va] + alpha * rad_rank[va]\n        nested_score = macro_auc(gold_y, nested)\n        final_alpha = min(0.20, float(np.median(np.asarray(choices))))\n        final_oof = (1.0 - final_alpha) * v49_rank + final_alpha * rad_rank\n        final_score = macro_auc(gold_y, final_oof)\n        grid_scores = {\n            f\"{alpha:.3f}\": macro_auc(\n                gold_y, (1.0 - alpha) * v49_rank + alpha * rad_rank\n            )\n            for alpha in alpha_grid\n        }\n        positive_folds = int(sum(alpha > 0 for alpha in choices))\n        supported = bool(\n            final_alpha > 0\n            and positive_folds >= 3\n            and nested_score >= base_score + 0.001\n            and final_score >= base_score + 0.001\n        )\n        audit[\"oof\"] = {\n            \"rows\": 58,\n            \"weak_macro_auc\": weak_auc,\n            \"rad_gold_macro_auc\": rad_score,\n            \"v49_global10_macro_auc\": base_score,\n            \"outer_fold_choices\": choices,\n            \"outer_fold_penalized_train_scores\": outer_train_scores,\n            \"nested_blend_macro_auc\": nested_score,\n            \"final_alpha\": final_alpha,\n            \"final_descriptive_macro_auc\": final_score,\n            \"full_grid_macro_auc\": grid_scores,\n            \"positive_outer_folds\": positive_folds,\n            \"gold_fold_counts\": {\n                str(fold): int((gold_folds == fold).sum()) for fold in range(5)\n            },\n            \"selection_supported\": supported,\n        }\n        audit[\"train_available_slice_tokens\"] = train_token_count\n        audit[\"head_count\"] = len(folds)\n        if not supported:\n            audit[\"status\"] = \"OOF_REJECTED_V49_PRESERVED\"\n            log(\n                f\"V52 rejected by nested OOF: base={base_score:.5f}, \"\n                f\"nested={nested_score:.5f}, final={final_score:.5f}, choices={choices}\"\n            )\n            return\n\n        del features, token_mask\n        gc.collect()\n        test = pd.read_csv(ROOT / \"test.csv\", dtype={\"StudyInstanceUID\": str})\n        test_series = pd.read_csv(\n            ROOT / \"test_series.csv\",\n            dtype={\"StudyInstanceUID\": str, \"SeriesInstanceUID\": str},\n        )\n        test_plane = dict(zip(test_series.SeriesInstanceUID, test_series.Anatomical_Plane))\n        test_headers = annotate(walk(\"test_series\"))\n        audit_official_sequence_metadata(test_headers, test_series)\n        test_studies, test_pixels, test_slot_mask = build_cache(\n            pick_slots(test_headers, test_plane),\n            test_plane,\n            lat_of(test_headers, \"test-v52 \"),\n            \"test-v52\",\n        )\n        test_by_uid = {str(uid): i for i, uid in enumerate(test_studies)}\n        test_missing = [uid for uid in test.StudyInstanceUID if uid not in test_by_uid]\n        if test_missing:\n            raise RuntimeError(f\"{len(test_missing)} test studies absent from cache\")\n        test_order = np.array([test_by_uid[uid] for uid in test.StudyInstanceUID])\n        test_pixels = test_pixels[test_order]\n        test_slot_mask = test_slot_mask[test_order]\n        test_token_count = int(\n            np.repeat(test_slot_mask[:, :, None], CACHE_SLICES, 2).sum()\n        )\n        if test_token_count < int(0.85 * len(test) * N_SLOT * CACHE_SLICES):\n            raise RuntimeError(f\"insufficient acquired test slices: {test_token_count}\")\n        test_features, test_token_mask = encode_radimagenet(\n            test_pixels, test_slot_mask, device\n        )\n        del test_pixels, test_slot_mask, test_headers\n        gc.collect()\n\n        fold_predictions = []\n        all_test = np.arange(len(test), dtype=np.int64)\n        for record in folds:\n            head = FoundationQueryHead().to(device)\n            head.load_state_dict(record[\"state_dict\"], strict=True)\n            fold_predictions.append(\n                predict_head(head, test_features, test_token_mask, all_test, device)\n            )\n            del head\n            torch.cuda.empty_cache()\n        if len(fold_predictions) != 5:\n            raise RuntimeError(\"test inference did not use all five heads\")\n        rad_test = np.mean(np.stack(fold_predictions), axis=0)\n        if not np.isfinite(rad_test).all():\n            raise RuntimeError(\"non-finite RadImageNet test prediction\")\n\n        baseline = pd.read_csv(preserved, dtype={\"StudyInstanceUID\": str})\n        _v52_validate_submission(baseline, test.StudyInstanceUID)\n        rad_frame = pd.DataFrame(rad_test, columns=TARGETS)\n        rad_frame.insert(0, \"StudyInstanceUID\", test.StudyInstanceUID)\n        _v52_validate_submission(rad_frame, test.StudyInstanceUID)\n        rad_frame.to_csv(output / \"submission_rad_only.csv\", index=False)\n        baseline_rank = _v52_rank_columns(baseline[TARGETS].to_numpy())\n        rad_test_rank = _v52_rank_columns(rad_test)\n        selected_path = None\n        for alpha in alpha_grid[1:]:\n            candidate = baseline.copy()\n            candidate[TARGETS] = (\n                (1.0 - alpha) * baseline_rank + alpha * rad_test_rank\n            )\n            _v52_validate_submission(candidate, test.StudyInstanceUID)\n            path = output / f\"submission_v49_rad_{int(round(1000 * alpha)):03d}.csv\"\n            candidate.to_csv(path, index=False)\n            if abs(float(alpha) - final_alpha) < 1e-12:\n                selected_path = path\n        if selected_path is None or not selected_path.is_file():\n            raise RuntimeError(\"selected V52 blend artifact is absent\")\n        selected = pd.read_csv(selected_path, dtype={\"StudyInstanceUID\": str})\n        _v52_validate_submission(selected, test.StudyInstanceUID)\n        audit[\"test_studies\"] = len(test)\n        audit[\"test_available_slice_tokens\"] = test_token_count\n        audit[\"test_head_count\"] = len(fold_predictions)\n        audit[\"selected_path\"] = str(selected_path)\n        audit[\"selected_sha256\"] = _v52_sha256(selected_path)\n        audit[\"fallback_sha256\"] = _v52_sha256(preserved)\n        shutil.copy2(selected_path, primary)\n        if _v52_sha256(primary) != audit[\"selected_sha256\"]:\n            raise RuntimeError(\"primary V52 copy hash mismatch\")\n        audit[\"status\"] = \"CANDIDATE_SELECTED\"\n        log(\n            f\"V52 selected alpha={final_alpha:.3f}; \"\n            f\"nested={nested_score:.5f} vs V49 OOF={base_score:.5f}\"\n        )\n    except Exception as error:\n        audit[\"status\"] = \"ERROR_V49_PRESERVED\"\n        audit[\"error\"] = f\"{type(error).__name__}: {error}\"\n        audit[\"traceback\"] = traceback.format_exc()\n        log(f\"V52 preserves V49: {audit['error']}\")\n    finally:\n        if audit.get(\"status\") != \"CANDIDATE_SELECTED\" and preserved.is_file():\n            shutil.copy2(preserved, primary)\n        audit[\"primary_sha256\"] = _v52_sha256(primary) if primary.is_file() else None\n        audit_path.write_text(json.dumps(audit, indent=2, sort_keys=True) + \"\\n\")\n\n\nmain_v52()\n\n\n# V58: Yash-inspired EfficientNet-B3 inference-contract sweep.\n#\n# The public .903 reference uses 288px inputs and 32 slices per series.  Our\n# independently trained five-fold B3 bundle used 224px and 20 validation slices.\n# This cell changes inference only: 224px x 32 and 288px x 32 are recomputed for\n# every one of the 58 official image-labelled studies with that study's own OOF\n# fold checkpoint, then compared against the already selected V52 RadImageNet\n# blend under a five-fold nested gate.  The incoming submission is always the\n# fallback and is changed only if the strict gate passes.\nimport contextlib as _v58_contextlib\nimport gc as _v58_gc\nimport hashlib as _v58_hashlib\nimport importlib.util as _v58_importlib_util\nimport json as _v58_json\nimport shutil as _v58_shutil\nimport sys as _v58_sys\nimport time as _v58_time\nimport traceback as _v58_traceback\nfrom pathlib import Path as _V58Path\n\nimport numpy as _v58_np\nimport pandas as _v58_pd\n\n_V58_DATA = _V58Path(\"/kaggle/input/rsna-knee-b3-v47-folds-0-3\")\n_V58_COMP = _V58Path(\"/kaggle/input/competitions/rsna-knee-abnormality-detection\")\n_V58_RAD_DIR = _V58Path(\"/kaggle/working/rsna_rad_v52\")\n_V58_OUT = _V58Path(\"/kaggle/working/rsna_b3_v58_resolution_sweep\")\n_V58_PRIMARY = _V58Path(\"/kaggle/working/submission.csv\")\n_V58_FALLBACK = _V58Path(\"/kaggle/working/submission_parent_v58.csv\")\n_V58_AUDIT = _V58Path(\"/kaggle/working/b3_v58_resolution_audit.json\")\n_V58_TARGETS = [\n    \"ACL\", \"MCL\", \"Medial Meniscus\", \"Lateral Meniscus\",\n    \"Medial OA\", \"Lateral OA\", \"PF OA\", \"Effusion\",\n    \"Synovitis\", \"Baker's\", \"Contusion\", \"Fracture\",\n]\n_V58_UID = \"StudyInstanceUID\"\n_V58_ARMS = {\n    \"b3_224px_32s\": (224, 32),\n    \"b3_288px_32s\": (288, 32),\n}\n_V58_BETAS = (0.05, 0.10, 0.15)\n_V58_RAD_ALPHA = 0.20\n_V58_MIN_NESTED_GAIN = 0.001\n_V58_MAX_TEST_ERROR_FRACTION = 0.01\n_V58_EXPECTED = {\n    _V58_DATA / \"source/efficientnet_b3_public_repro_v4_t4.py\":\n        \"825b82656eea4ab3dd53c103fd04ce9ead803451486d29f61febdfc73e3751ec\",\n    _V58_DATA / \"source/efficientnet_b3_public_repro_v1.py\":\n        \"2548480302b175aa15700888902835c91c377deec935a4edfe2a6a978e3ac232\",\n    _V58_DATA / \"source/efficientnet_b3_public_repro_v2_anatomy.py\":\n        \"17a8a8c99afc02e6e75928cf7db2d014609d31d1280a1f05b07f15564c198696\",\n    _V58_DATA / \"audit/expert_oof_all.csv\":\n        \"c4c7cd8cadc67b2e88619ba10db355b5760c4a065b73a5445c4ef0caf9a86e0d\",\n    _V58_DATA / \"exact_public_family_oof.npz\":\n        \"d8af61726d9ec07abba19cabfa9c548de05bdb11206c9442a3d72ede737eb004\",\n    _V58_DATA / \"fold0/fold0_final.pt\":\n        \"71a4cafb8f13341c4ca8a50b20bcc1ca3ad63c88b721d632865e2305cbb282e3\",\n    _V58_DATA / \"fold1/fold1_final.pt\":\n        \"088065c483017a7497a770c73530c8850467422738ddff3a6ca111a3f1749948\",\n    _V58_DATA / \"fold2/fold2_final.pt\":\n        \"58482553af61a9afdfaccd7b7319a0af81046421054af0cf995c1da70dc36486\",\n    _V58_DATA / \"fold3/fold3_final.pt\":\n        \"6b60f18a28c6fc80ab04939ae25be2358705013e4779706bbd9534c52fec4472\",\n    _V58_DATA / \"fold4/fold4_final.pt\":\n        \"ccb18c38000fbdb152c2717be2f340c446978cad40f2db7667f15a1a49e31249\",\n}\n\n\ndef _v58_sha256(path):\n    digest = _v58_hashlib.sha256()\n    with open(path, \"rb\") as handle:\n        for chunk in iter(lambda: handle.read(8 << 20), b\"\"):\n            digest.update(chunk)\n    return digest.hexdigest()\n\n\ndef _v58_rank(array):\n    frame = _v58_pd.DataFrame(array)\n    return frame.rank(axis=0, method=\"average\", pct=True).to_numpy(_v58_np.float64)\n\n\ndef _v58_auc(y_true, y_score):\n    from sklearn.metrics import roc_auc_score as _v58_roc_auc_score\n\n    values = []\n    for column in range(y_true.shape[1]):\n        truth = y_true[:, column]\n        if _v58_np.unique(truth).size != 2:\n            raise RuntimeError(f\"target {column} lacks both classes\")\n        values.append(float(_v58_roc_auc_score(truth, y_score[:, column])))\n    return float(_v58_np.mean(values))\n\n\ndef _v58_load_module(path):\n    name = \"rsna_knee_v58_b3_module\"\n    spec = _v58_importlib_util.spec_from_file_location(name, path)\n    if spec is None or spec.loader is None:\n        raise RuntimeError(f\"cannot import {path}\")\n    module = _v58_importlib_util.module_from_spec(spec)\n    _v58_sys.modules[name] = module\n    spec.loader.exec_module(module)\n    return module\n\n\ndef _v58_validate_submission(frame, expected_uids):\n    expected_columns = [_V58_UID, *_V58_TARGETS]\n    if frame.columns.tolist() != expected_columns:\n        raise RuntimeError(f\"submission schema drift: {frame.columns.tolist()}\")\n    if frame[_V58_UID].astype(str).tolist() != list(map(str, expected_uids)):\n        raise RuntimeError(\"submission study order drift\")\n    if frame[_V58_UID].astype(str).duplicated().any():\n        raise RuntimeError(\"duplicate submission study ID\")\n    values = frame[_V58_TARGETS].to_numpy(_v58_np.float64)\n    if not _v58_np.isfinite(values).all():\n        raise RuntimeError(\"submission contains non-finite predictions\")\n\n\ndef _v58_build_tensor(module, torch, series_group, image_root, uid, image_size, slices):\n    empty = _v58_pd.DataFrame(\n        columns=[\n            \"SeriesInstanceUID\", \"Anatomical_Plane\",\n            \"Fluid_Sensitive\", \"Fat_Suppression\",\n        ]\n    )\n    selected = module.pick_series_slots(series_group.get(str(uid), empty), len(module.PLANES))\n    planes, masks = [], []\n    for series_uid in selected:\n        if series_uid is None:\n            planes.append(_v58_np.zeros((slices, image_size, image_size), _v58_np.float32))\n            masks.append(_v58_np.zeros(slices, bool))\n            continue\n        volume = module.load_dicom_volume(\n            image_root / str(uid) / str(series_uid), image_size, 64\n        )\n        indices = module.uniform_slice_indices(len(volume), slices)\n        sampled = (volume[indices].astype(_v58_np.float32) - 0.449) / 0.226\n        planes.append(sampled)\n        masks.append(_v58_np.ones(slices, bool))\n    mask = _v58_np.stack(masks)\n    if not mask.any():\n        raise RuntimeError(f\"no decodable series for {uid}\")\n    image_tensor = torch.from_numpy(_v58_np.stack(planes)[:, :, None])[None]\n    mask_tensor = torch.from_numpy(mask)[None]\n    return image_tensor, mask_tensor\n\n\ndef _v58_predict_one(model, image_tensor, mask_tensor, torch, device, amp_dtype):\n    image_tensor = image_tensor.to(device, non_blocking=True)\n    mask_tensor = mask_tensor.to(device, non_blocking=True)\n    amp = (\n        torch.autocast(\"cuda\", dtype=amp_dtype)\n        if device.type == \"cuda\" else _v58_contextlib.nullcontext()\n    )\n    with torch.no_grad(), amp:\n        logits = model(image_tensor, mask_tensor)\n    result = torch.sigmoid(logits.float()).cpu().numpy()[0]\n    if not _v58_np.isfinite(result).all():\n        raise RuntimeError(\"non-finite model output\")\n    return result.astype(_v58_np.float64)\n\n\ndef _v58_infer_arm(\n    name, image_size, slices, module, torch, models, device, amp_dtype,\n    expert, train_groups, test_groups, train_root, test_root, test_uids,\n):\n    gold_pred = _v58_np.zeros((len(expert), len(_V58_TARGETS)), _v58_np.float64)\n    for row_index, row in expert.iterrows():\n        uid, fold = str(row[_V58_UID]), int(row[\"fold\"])\n        images, mask = _v58_build_tensor(\n            module, torch, train_groups, train_root, uid, image_size, slices\n        )\n        gold_pred[row_index] = _v58_predict_one(\n            models[fold], images, mask, torch, device, amp_dtype\n        )\n        if (row_index + 1) % 10 == 0 or row_index + 1 == len(expert):\n            print(f\"[B3 V58] {name} OOF {row_index + 1}/{len(expert)}\", flush=True)\n\n    test_pred = _v58_np.zeros((len(test_uids), len(_V58_TARGETS)), _v58_np.float64)\n    test_errors = {}\n    for row_index, uid in enumerate(test_uids):\n        try:\n            images, mask = _v58_build_tensor(\n                module, torch, test_groups, test_root, uid, image_size, slices\n            )\n            fold_values = [\n                _v58_predict_one(model, images, mask, torch, device, amp_dtype)\n                for model in models\n            ]\n            test_pred[row_index] = _v58_np.mean(fold_values, axis=0)\n        except Exception as error:\n            test_errors[str(uid)] = f\"{type(error).__name__}: {error}\"\n            test_pred[row_index] = 0.5\n        if (row_index + 1) % 10 == 0 or row_index + 1 == len(test_uids):\n            print(\n                f\"[B3 V58] {name} test {row_index + 1}/{len(test_uids)} \"\n                f\"errors={len(test_errors)}\",\n                flush=True,\n            )\n    return gold_pred, test_pred, test_errors\n\n\ndef _v58_candidate(exact_rank, old_b3_rank, new_b3_rank, rad_rank, beta):\n    # Preserve V52's topology: first build and rank the V49 family, then add\n    # the fixed RadImageNet rank arm selected by V52.\n    b3_family = (1.0 - float(beta)) * exact_rank + float(beta) * new_b3_rank\n    return (1.0 - _V58_RAD_ALPHA) * _v58_rank(b3_family) + _V58_RAD_ALPHA * rad_rank\n\n\ndef _v58_main():\n    started = _v58_time.monotonic()\n    audit = {\n        \"status\": \"FALLBACK\",\n        \"evidence_boundary\": (\n            \"All 58-study AUC values are local official-label diagnostics. Only a \"\n            \"completed Kaggle competition row with a nonempty score is leaderboard evidence.\"\n        ),\n        \"arms\": {name: {\"image_size\": size, \"slices_per_plane\": slices}\n                 for name, (size, slices) in _V58_ARMS.items()},\n        \"incoming_primary_sha256\": None,\n    }\n    if not _V58_PRIMARY.is_file():\n        raise FileNotFoundError(\"incoming parent submission is absent\")\n    _V58_OUT.mkdir(parents=True, exist_ok=True)\n    _v58_shutil.copy2(_V58_PRIMARY, _V58_FALLBACK)\n    audit[\"incoming_primary_sha256\"] = _v58_sha256(_V58_FALLBACK)\n\n    # If the preceding ViTDet experiment selected itself, it has already passed\n    # its own stricter gate. Never replace it using a comparison against V52.\n    vit_audit_path = _V58Path(\"/kaggle/working/vitdet_fpn_v57_audit.json\")\n    if vit_audit_path.is_file():\n        vit_audit = _v58_json.loads(vit_audit_path.read_text())\n        audit[\"preceding_vitdet_status\"] = vit_audit.get(\"status\")\n        if vit_audit.get(\"status\") == \"CANDIDATE_SELECTED\":\n            audit[\"status\"] = \"PARENT_VITDET_SELECTED_PRESERVED\"\n            return audit\n\n    observed = {str(path): _v58_sha256(path) for path in _V58_EXPECTED}\n    drift = {\n        str(path): {\"expected\": expected, \"observed\": observed[str(path)]}\n        for path, expected in _V58_EXPECTED.items()\n        if observed[str(path)] != expected\n    }\n    if drift:\n        raise RuntimeError(f\"pinned B3 artifact drift: {drift}\")\n    audit[\"observed_inputs\"] = observed\n\n    rad_oof_path = _V58_RAD_DIR / \"v52_oof.csv\"\n    rad_audit_path = _V58Path(\"/kaggle/working/rad_v52_audit.json\")\n    if not rad_oof_path.is_file() or not rad_audit_path.is_file():\n        raise FileNotFoundError(\"V52 OOF contract is absent\")\n    rad_audit = _v58_json.loads(rad_audit_path.read_text())\n    if (\n        rad_audit.get(\"status\") != \"CANDIDATE_SELECTED\"\n        or abs(float(rad_audit.get(\"oof\", {}).get(\"final_alpha\", -1)) - _V58_RAD_ALPHA) > 1e-12\n        or not bool(rad_audit.get(\"oof\", {}).get(\"selection_supported\"))\n    ):\n        raise RuntimeError(\"V52 strict gate was not selected at the pinned 20% weight\")\n\n    expert = _v58_pd.read_csv(\n        _V58_DATA / \"audit/expert_oof_all.csv\", dtype={_V58_UID: str}\n    ).reset_index(drop=True)\n    if len(expert) != 58 or expert[_V58_UID].duplicated().any():\n        raise RuntimeError(\"expert OOF identity drift\")\n    if sorted(expert[\"fold\"].astype(int).unique().tolist()) != list(range(5)):\n        raise RuntimeError(\"B3 OOF fold coverage drift\")\n    truth = expert[[f\"gold::{target}\" for target in _V58_TARGETS]].to_numpy(_v58_np.float64)\n    old_b3 = expert[[f\"pred::{target}\" for target in _V58_TARGETS]].to_numpy(_v58_np.float64)\n\n    exact = _v58_np.load(_V58_DATA / \"exact_public_family_oof.npz\", allow_pickle=False)\n    exact_index = {str(uid): index for index, uid in enumerate(exact[\"ids\"].tolist())}\n    if not set(expert[_V58_UID]).issubset(exact_index):\n        raise RuntimeError(\"expert UID absent from exact-public OOF\")\n    exact_rows = _v58_np.asarray([exact_index[uid] for uid in expert[_V58_UID]], int)\n    exact_public = exact[\"ours\"][exact_rows].astype(_v58_np.float64)\n\n    rad_frame = _v58_pd.read_csv(rad_oof_path, dtype={_V58_UID: str})\n    if rad_frame[_V58_UID].duplicated().any():\n        raise RuntimeError(\"V52 OOF contains duplicate studies\")\n    rad_aligned = expert[[_V58_UID]].merge(\n        rad_frame[[_V58_UID, *_V58_TARGETS, \"fold\", \"is_gold\"]],\n        on=_V58_UID, how=\"left\", validate=\"one_to_one\",\n    )\n    if not rad_aligned[\"is_gold\"].eq(1).all():\n        raise RuntimeError(\"expert/V52 gold alignment drift\")\n    outer_folds = rad_aligned[\"fold\"].to_numpy(int)\n    if sorted(_v58_np.unique(outer_folds).tolist()) != list(range(5)):\n        raise RuntimeError(\"V52 outer-fold coverage drift\")\n    rad = rad_aligned[_V58_TARGETS].to_numpy(_v58_np.float64)\n    if not all(_v58_np.isfinite(x).all() for x in (truth, old_b3, exact_public, rad)):\n        raise RuntimeError(\"non-finite OOF input\")\n\n    module = _v58_load_module(\n        _V58_DATA / \"source/efficientnet_b3_public_repro_v4_t4.py\"\n    )\n    _, _, _, torch, _, _, _ = module._training_imports()\n    if not torch.cuda.is_available():\n        raise RuntimeError(\"V58 requires CUDA\")\n    device = torch.device(\"cuda:0\")\n    amp_dtype = (\n        torch.bfloat16 if torch.cuda.get_device_capability(0) >= (8, 0)\n        else torch.float16\n    )\n    _v58_gc.collect()\n    torch.cuda.empty_cache()\n    Model = module.make_model_class()\n    models = []\n    checkpoint_configs = []\n    for fold in range(5):\n        path = _V58_DATA / f\"fold{fold}/fold{fold}_final.pt\"\n        checkpoint = torch.load(path, map_location=\"cpu\", weights_only=False)\n        if int(checkpoint.get(\"fold\", -1)) != fold:\n            raise RuntimeError(f\"checkpoint fold drift at {fold}\")\n        config = module.TrainConfig(**checkpoint[\"config\"])\n        model = Model(\n            config.backbone, checkpoint=None, backbone_chunk=config.backbone_chunk\n        )\n        model.load_state_dict(checkpoint[\"model\"], strict=True)\n        model.eval().to(device)\n        models.append(model)\n        checkpoint_configs.append(checkpoint[\"config\"])\n    if len({_v58_json.dumps(value, sort_keys=True) for value in checkpoint_configs}) != 1:\n        raise RuntimeError(\"fold checkpoint configs disagree\")\n\n    train_series = _v58_pd.read_csv(_V58_COMP / \"train_series.csv\", dtype={_V58_UID: str})\n    test = _v58_pd.read_csv(_V58_COMP / \"test.csv\", dtype={_V58_UID: str})\n    test_series = _v58_pd.read_csv(_V58_COMP / \"test_series.csv\", dtype={_V58_UID: str})\n    if test[_V58_UID].duplicated().any():\n        raise RuntimeError(\"test contains duplicate studies\")\n    train_groups = {\n        str(uid): group.copy() for uid, group in train_series.groupby(_V58_UID, sort=False)\n    }\n    test_groups = {\n        str(uid): group.copy() for uid, group in test_series.groupby(_V58_UID, sort=False)\n    }\n    test_uids = test[_V58_UID].astype(str).tolist()\n\n    arm_gold, arm_test, arm_errors = {}, {}, {}\n    for name, (image_size, slices) in _V58_ARMS.items():\n        values, test_values, errors = _v58_infer_arm(\n            name, image_size, slices, module, torch, models, device, amp_dtype,\n            expert, train_groups, test_groups,\n            _V58_COMP / \"train_series\", _V58_COMP / \"test_series\", test_uids,\n        )\n        if not _v58_np.isfinite(values).all():\n            raise RuntimeError(f\"{name} OOF contains non-finite values\")\n        error_cap = max(1, int(_v58_np.ceil(_V58_MAX_TEST_ERROR_FRACTION * len(test_uids))))\n        if len(errors) > error_cap:\n            raise RuntimeError(f\"{name} test errors {len(errors)} exceed cap {error_cap}\")\n        arm_gold[name] = values\n        arm_test[name] = test_values\n        arm_errors[name] = errors\n        gold_frame = _v58_pd.DataFrame(values, columns=_V58_TARGETS)\n        gold_frame.insert(0, _V58_UID, expert[_V58_UID])\n        gold_frame[\"b3_fold\"] = expert[\"fold\"].astype(int)\n        gold_frame[\"outer_fold\"] = outer_folds\n        gold_frame.to_csv(_V58_OUT / f\"{name}_gold_oof.csv\", index=False)\n        test_frame = _v58_pd.DataFrame(test_values, columns=_V58_TARGETS)\n        test_frame.insert(0, _V58_UID, test_uids)\n        test_frame.to_csv(_V58_OUT / f\"{name}_test.csv\", index=False)\n\n    exact_rank = _v58_rank(exact_public)\n    old_b3_rank = _v58_rank(old_b3)\n    rad_rank = _v58_rank(rad)\n    baseline_v49 = 0.90 * exact_rank + 0.10 * old_b3_rank\n    baseline_v52 = 0.80 * _v58_rank(baseline_v49) + 0.20 * rad_rank\n    baseline_score = _v58_auc(truth, baseline_v52)\n    expected_baseline_score = float(rad_audit[\"oof\"][\"final_descriptive_macro_auc\"])\n    if abs(baseline_score - expected_baseline_score) > 1e-10:\n        raise RuntimeError(\n            f\"V52 OOF reproduction drift: {baseline_score} != {expected_baseline_score}\"\n        )\n\n    candidate_oof = {\"v52_baseline\": baseline_v52}\n    candidate_meta = {\"v52_baseline\": {\"arm\": \"old_b3_224px_20s\", \"beta\": 0.10}}\n    for name, values in arm_gold.items():\n        new_rank = _v58_rank(values)\n        for beta in _V58_BETAS:\n            key = f\"{name}_beta_{int(round(100 * beta)):02d}\"\n            candidate_oof[key] = _v58_candidate(\n                exact_rank, old_b3_rank, new_rank, rad_rank, beta\n            )\n            candidate_meta[key] = {\"arm\": name, \"beta\": float(beta)}\n\n    candidate_scores = {\n        key: _v58_auc(truth, values) for key, values in candidate_oof.items()\n    }\n    nested = _v58_np.zeros_like(baseline_v52)\n    outer_choices, outer_train_scores, outer_held_gains = [], [], []\n    keys = list(candidate_oof)\n    for outer in range(5):\n        train_mask = outer_folds != outer\n        held_mask = ~train_mask\n        scored = {}\n        for key in keys:\n            raw_score = _v58_auc(truth[train_mask], candidate_oof[key][train_mask])\n            beta = float(candidate_meta[key][\"beta\"])\n            # Penalize moving the B3 weight away from the already validated 10%.\n            penalty = 0.005 * abs(beta - 0.10)\n            scored[key] = float(raw_score - penalty)\n        best = max(keys, key=lambda key: (scored[key], key == \"v52_baseline\", key))\n        outer_choices.append(best)\n        outer_train_scores.append(scored)\n        nested[held_mask] = candidate_oof[best][held_mask]\n        outer_held_gains.append(\n            _v58_auc(truth[held_mask], candidate_oof[best][held_mask])\n            - _v58_auc(truth[held_mask], baseline_v52[held_mask])\n        )\n    nested_score = _v58_auc(truth, nested)\n    positive_outer_folds = int(sum(choice != \"v52_baseline\" for choice in outer_choices))\n\n    # Deploy only an arm that was independently selected in at least two outer folds.\n    eligible = [\n        key for key in keys if key != \"v52_baseline\" and outer_choices.count(key) >= 2\n    ]\n    final_key = (\n        max(eligible, key=lambda key: (candidate_scores[key], -abs(candidate_meta[key][\"beta\"] - 0.10), key))\n        if eligible else \"v52_baseline\"\n    )\n    final_score = candidate_scores[final_key]\n    supported = bool(\n        final_key != \"v52_baseline\"\n        and positive_outer_folds >= 3\n        and nested_score >= baseline_score + _V58_MIN_NESTED_GAIN\n        and final_score >= baseline_score + _V58_MIN_NESTED_GAIN\n        and sum(gain > 0 for gain in outer_held_gains) >= 3\n    )\n    audit[\"oof\"] = {\n        \"rows\": 58,\n        \"baseline_v52_macro_auc\": baseline_score,\n        \"candidate_macro_auc\": candidate_scores,\n        \"outer_choices\": outer_choices,\n        \"outer_penalized_train_scores\": outer_train_scores,\n        \"outer_held_gains\": outer_held_gains,\n        \"positive_outer_folds\": positive_outer_folds,\n        \"nested_macro_auc\": nested_score,\n        \"selected_key\": final_key,\n        \"selected_descriptive_macro_auc\": final_score,\n        \"selection_supported\": supported,\n        \"b3_fold_matches_v52_outer_fold\": int(\n            (expert[\"fold\"].to_numpy(int) == outer_folds).sum()\n        ),\n    }\n    audit[\"test_errors\"] = arm_errors\n\n    fallback = _v58_pd.read_csv(_V58_FALLBACK, dtype={_V58_UID: str})\n    _v58_validate_submission(fallback, test_uids)\n    if supported:\n        selected = candidate_meta[final_key]\n        selected_name = selected[\"arm\"]\n        selected_beta = float(selected[\"beta\"])\n        exact_test_path = _V58Path(\"/kaggle/working/submission_public_0899.csv\")\n        rad_test_path = _V58_RAD_DIR / \"submission_rad_only.csv\"\n        if not exact_test_path.is_file() or not rad_test_path.is_file():\n            raise FileNotFoundError(\"test-time exact-public or Rad arm is absent\")\n        exact_test = _v58_pd.read_csv(exact_test_path, dtype={_V58_UID: str})\n        rad_test = _v58_pd.read_csv(rad_test_path, dtype={_V58_UID: str})\n        _v58_validate_submission(exact_test, test_uids)\n        _v58_validate_submission(rad_test, test_uids)\n        new_test = arm_test[selected_name]\n        test_v49_family = (\n            (1.0 - selected_beta) * _v58_rank(exact_test[_V58_TARGETS].to_numpy(float))\n            + selected_beta * _v58_rank(new_test)\n        )\n        test_candidate_values = (\n            0.80 * _v58_rank(test_v49_family)\n            + 0.20 * _v58_rank(rad_test[_V58_TARGETS].to_numpy(float))\n        )\n        selected_frame = fallback.copy()\n        selected_frame[_V58_TARGETS] = test_candidate_values\n        _v58_validate_submission(selected_frame, test_uids)\n        selected_path = _V58_OUT / \"submission_v58_selected.csv\"\n        selected_frame.to_csv(selected_path, index=False)\n        _v58_shutil.copy2(selected_path, _V58_PRIMARY)\n        audit[\"status\"] = \"CANDIDATE_SELECTED\"\n        audit[\"selected_path\"] = str(selected_path)\n        audit[\"selected_sha256\"] = _v58_sha256(_V58_PRIMARY)\n        audit[\"selected_arm\"] = selected_name\n        audit[\"selected_beta\"] = selected_beta\n    else:\n        _v58_shutil.copy2(_V58_FALLBACK, _V58_PRIMARY)\n        audit[\"status\"] = \"OOF_REJECTED_PARENT_PRESERVED\"\n\n    audit[\"fallback_sha256\"] = _v58_sha256(_V58_FALLBACK)\n    audit[\"primary_sha256\"] = _v58_sha256(_V58_PRIMARY)\n    audit[\"wall_seconds\"] = _v58_time.monotonic() - started\n    return audit\n\n\n_v58_result = None\ntry:\n    _v58_result = _v58_main()\nexcept Exception as _v58_error:\n    print(\n        f\"[B3 V58] rejected safely: {type(_v58_error).__name__}: {_v58_error}\",\n        flush=True,\n    )\n    _v58_traceback.print_exc()\n    if _V58_FALLBACK.is_file():\n        _v58_shutil.copy2(_V58_FALLBACK, _V58_PRIMARY)\n    _v58_result = {\n        \"status\": \"ERROR_PARENT_PRESERVED\",\n        \"error\": f\"{type(_v58_error).__name__}: {_v58_error}\",\n        \"traceback\": _v58_traceback.format_exc(),\n        \"fallback_sha256\": _v58_sha256(_V58_FALLBACK) if _V58_FALLBACK.is_file() else None,\n        \"primary_sha256\": _v58_sha256(_V58_PRIMARY) if _V58_PRIMARY.is_file() else None,\n    }\nfinally:\n    _V58_AUDIT.write_text(_v58_json.dumps(_v58_result, indent=2, sort_keys=True) + \"\\n\")\n    print(_v58_json.dumps(_v58_result, indent=2, sort_keys=True), flush=True)\n"},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"\"\"\"Leakage-safe DICOM-header branch for the RSNA knee notebook.\n\nThis module is intended to be appended as the final notebook cell.  It treats the\nincoming submission as immutable parent evidence, builds DICOM-header features without\nreading pixels, and may replace the parent only after a five-fold nested official-label\ngate.  All training labels are the same three report teachers used by V52; held-out gold\nrows are excluded from the corresponding outer-fold fit.\n\"\"\"\n\nfrom __future__ import annotations\n\nimport hashlib as _v60_hashlib\nimport json as _v60_json\nimport math as _v60_math\nimport re as _v60_re\nimport shutil as _v60_shutil\nimport time as _v60_time\nimport traceback as _v60_traceback\nfrom collections import Counter as _V60Counter\nfrom pathlib import Path as _V60Path\n\nimport numpy as _v60_np\nimport pandas as _v60_pd\n\n\n_V60_UID = \"StudyInstanceUID\"\n_V60_TARGETS = [\n    \"ACL\", \"MCL\", \"Medial Meniscus\", \"Lateral Meniscus\",\n    \"Medial OA\", \"Lateral OA\", \"PF OA\", \"Effusion\",\n    \"Synovitis\", \"Baker's\", \"Contusion\", \"Fracture\",\n]\n_V60_COMP = _V60Path(\"/kaggle/input/competitions/rsna-knee-abnormality-detection\")\n_V60_DATA = _V60Path(\"/kaggle/input/rsna-knee-b3-v47-folds-0-3\")\n_V60_RAD_DIR = _V60Path(\"/kaggle/working/rsna_rad_v52\")\n_V60_V58_DIR = _V60Path(\"/kaggle/working/rsna_b3_v58_resolution_sweep\")\n_V60_V58_AUDIT = _V60Path(\"/kaggle/working/b3_v58_resolution_audit.json\")\n_V60_RAD_AUDIT = _V60Path(\"/kaggle/working/rad_v52_audit.json\")\n_V60_PRIMARY = _V60Path(\"/kaggle/working/submission.csv\")\n_V60_OUT = _V60Path(\"/kaggle/working/rsna_dicom_metadata_v60\")\n_V60_PARENT = _V60_OUT / \"submission_parent_before_metadata.csv\"\n_V60_AUDIT = _V60Path(\"/kaggle/working/dicom_metadata_v60_audit.json\")\n_V60_MIN_GAIN = 0.0015\n_V60_GAMMAS = (0.05, 0.10)\n_V60_ALPHAS = (30.0, 100.0)\n_V60_TAGS = [\n    \"PatientSex\", \"Manufacturer\", \"ManufacturerModelName\",\n    \"MagneticFieldStrength\", \"SeriesDescription\", \"ProtocolName\",\n    \"SequenceName\", \"Rows\", \"Columns\", \"PixelSpacing\",\n    \"SliceThickness\", \"SpacingBetweenSlices\",\n]\n\n\ndef _v60_sha256(path):\n    digest = _v60_hashlib.sha256()\n    with open(path, \"rb\") as handle:\n        for chunk in iter(lambda: handle.read(8 << 20), b\"\"):\n            digest.update(chunk)\n    return digest.hexdigest()\n\n\ndef _v60_rank(values):\n    from scipy.stats import rankdata\n\n    values = _v60_np.asarray(values, _v60_np.float64)\n    ranked = _v60_np.empty_like(values)\n    for column in range(values.shape[1]):\n        ranked[:, column] = rankdata(values[:, column], method=\"average\")\n    return ranked / max(1, values.shape[0])\n\n\ndef _v60_auc(truth, pred):\n    from sklearn.metrics import roc_auc_score\n\n    truth = _v60_np.asarray(truth)\n    pred = _v60_np.asarray(pred)\n    values = []\n    for column in range(truth.shape[1]):\n        hard = (truth[:, column] >= 0.5).astype(_v60_np.uint8)\n        if _v60_np.unique(hard).size == 2:\n            values.append(roc_auc_score(hard, pred[:, column]))\n    if not values:\n        raise RuntimeError(\"macro AUC has no non-constant target\")\n    return float(_v60_np.mean(values))\n\n\ndef _v60_validate_submission(frame, expected_uids):\n    expected_columns = [_V60_UID, *_V60_TARGETS]\n    if frame.columns.tolist() != expected_columns:\n        raise RuntimeError(f\"submission schema drift: {frame.columns.tolist()}\")\n    observed = frame[_V60_UID].astype(str).tolist()\n    expected = list(map(str, expected_uids))\n    if observed != expected:\n        raise RuntimeError(\"submission study order drift\")\n    if frame[_V60_UID].astype(str).duplicated().any():\n        raise RuntimeError(\"duplicate submission study ID\")\n    values = frame[_V60_TARGETS].to_numpy(_v60_np.float64)\n    if not _v60_np.isfinite(values).all():\n        raise RuntimeError(\"submission contains non-finite predictions\")\n\n\ndef _v60_find_input_file(name):\n    base = _V60Path(\"/kaggle/input\")\n    for root, dirs, files in __import__(\"os\").walk(base):\n        dirs[:] = [item for item in dirs if item not in (\"train_series\", \"test_series\")]\n        if name in files:\n            return _V60Path(root) / name\n    raise FileNotFoundError(name)\n\n\ndef _v60_parent_oof():\n    \"\"\"Reproduce the exact incoming V52 or selected V58 OOF topology.\"\"\"\n    if not _V60_V58_AUDIT.is_file() or not _V60_RAD_AUDIT.is_file():\n        raise FileNotFoundError(\"V52/V58 audit contract is absent\")\n    v58_audit = _v60_json.loads(_V60_V58_AUDIT.read_text())\n    rad_audit = _v60_json.loads(_V60_RAD_AUDIT.read_text())\n    if rad_audit.get(\"status\") != \"CANDIDATE_SELECTED\":\n        raise RuntimeError(\"V52 RadImageNet parent was not selected\")\n    if abs(float(rad_audit.get(\"oof\", {}).get(\"final_alpha\", -1)) - 0.20) > 1e-12:\n        raise RuntimeError(\"V52 RadImageNet alpha drift\")\n    if not bool(rad_audit.get(\"oof\", {}).get(\"selection_supported\")):\n        raise RuntimeError(\"V52 RadImageNet support gate drift\")\n    expected_primary = v58_audit.get(\"primary_sha256\")\n    if expected_primary and _v60_sha256(_V60_PRIMARY) != expected_primary:\n        raise RuntimeError(\"incoming V58 primary SHA drift\")\n\n    special_parent = v58_audit.get(\"status\") == \"PARENT_VITDET_SELECTED_PRESERVED\"\n    if special_parent:\n        return {\"special_parent\": True, \"v58_audit\": v58_audit}\n\n    expert = _v60_pd.read_csv(\n        _V60_DATA / \"audit/expert_oof_all.csv\", dtype={_V60_UID: str}\n    ).reset_index(drop=True)\n    if len(expert) != 58 or expert[_V60_UID].duplicated().any():\n        raise RuntimeError(\"expert OOF identity drift\")\n    truth = expert[[f\"gold::{target}\" for target in _V60_TARGETS]].to_numpy(\n        _v60_np.float64\n    )\n    old_b3 = expert[[f\"pred::{target}\" for target in _V60_TARGETS]].to_numpy(\n        _v60_np.float64\n    )\n\n    exact = _v60_np.load(\n        _V60_DATA / \"exact_public_family_oof.npz\", allow_pickle=False\n    )\n    exact_index = {str(uid): index for index, uid in enumerate(exact[\"ids\"].tolist())}\n    if not set(expert[_V60_UID]).issubset(exact_index):\n        raise RuntimeError(\"expert UID absent from exact-public OOF\")\n    exact_rows = _v60_np.asarray([exact_index[uid] for uid in expert[_V60_UID]], int)\n    exact_public = exact[\"ours\"][exact_rows].astype(_v60_np.float64)\n\n    rad_frame = _v60_pd.read_csv(\n        _V60_RAD_DIR / \"v52_oof.csv\", dtype={_V60_UID: str}\n    )\n    aligned = expert[[_V60_UID]].merge(\n        rad_frame[[_V60_UID, *_V60_TARGETS, \"fold\", \"is_gold\"]],\n        on=_V60_UID, how=\"left\", validate=\"one_to_one\",\n    )\n    if not aligned[\"is_gold\"].eq(1).all():\n        raise RuntimeError(\"expert/V52 gold alignment drift\")\n    outer_folds = aligned[\"fold\"].to_numpy(int)\n    if sorted(_v60_np.unique(outer_folds).tolist()) != list(range(5)):\n        raise RuntimeError(\"outer-fold coverage drift\")\n    rad = aligned[_V60_TARGETS].to_numpy(_v60_np.float64)\n\n    exact_rank = _v60_rank(exact_public)\n    old_rank = _v60_rank(old_b3)\n    rad_rank = _v60_rank(rad)\n    v49 = 0.90 * exact_rank + 0.10 * old_rank\n    baseline = 0.80 * _v60_rank(v49) + 0.20 * rad_rank\n    parent_name = \"v52\"\n    expected_score = float(rad_audit[\"oof\"][\"final_descriptive_macro_auc\"])\n\n    if v58_audit.get(\"status\") == \"CANDIDATE_SELECTED\":\n        arm = str(v58_audit.get(\"selected_arm\"))\n        beta = float(v58_audit.get(\"selected_beta\"))\n        arm_path = _V60_V58_DIR / f\"{arm}_gold_oof.csv\"\n        arm_frame = _v60_pd.read_csv(arm_path, dtype={_V60_UID: str})\n        arm_aligned = expert[[_V60_UID]].merge(\n            arm_frame[[_V60_UID, *_V60_TARGETS]],\n            on=_V60_UID, how=\"left\", validate=\"one_to_one\",\n        )\n        new_b3 = arm_aligned[_V60_TARGETS].to_numpy(_v60_np.float64)\n        b3_family = (1.0 - beta) * exact_rank + beta * _v60_rank(new_b3)\n        baseline = 0.80 * _v60_rank(b3_family) + 0.20 * rad_rank\n        parent_name = f\"v58::{arm}::beta={beta:.3f}\"\n        expected_score = float(\n            v58_audit[\"oof\"][\"selected_descriptive_macro_auc\"]\n        )\n    elif v58_audit.get(\"status\") not in {\n        \"OOF_REJECTED_PARENT_PRESERVED\", \"ERROR_PARENT_PRESERVED\",\n        \"FALLBACK\", \"PARENT_VITDET_SELECTED_PRESERVED\",\n    }:\n        raise RuntimeError(f\"unexpected V58 status {v58_audit.get('status')!r}\")\n\n    observed_score = _v60_auc(truth, baseline)\n    if abs(observed_score - expected_score) > 1e-10:\n        raise RuntimeError(\n            f\"parent OOF reproduction drift: {observed_score} != {expected_score}\"\n        )\n    return {\n        \"special_parent\": False,\n        \"parent_name\": parent_name,\n        \"baseline\": baseline,\n        \"baseline_score\": observed_score,\n        \"truth\": truth,\n        \"expert\": expert,\n        \"outer_folds\": outer_folds,\n        \"v58_audit\": v58_audit,\n    }\n\n\ndef _v60_targets(train):\n    \"\"\"Reproduce V52's three-teacher target and weight contract.\"\"\"\n    sources = [\n        _v60_pd.read_csv(_v60_find_input_file(\"report_labels_v2.csv\"),\n                         dtype={_V60_UID: str}),\n        _v60_pd.read_csv(_v60_find_input_file(\"llm_labels_v2.csv\"),\n                         dtype={_V60_UID: str}),\n        _v60_pd.read_csv(_v60_find_input_file(\"labels_llm_gpt56sol.csv\"),\n                         dtype={_V60_UID: str}),\n    ]\n    cube = []\n    for frame in sources:\n        if frame[_V60_UID].duplicated().any():\n            raise RuntimeError(\"duplicate study in report-label source\")\n        aligned = train[[_V60_UID]].merge(\n            frame[[_V60_UID, *_V60_TARGETS]], on=_V60_UID, how=\"left\",\n            validate=\"one_to_one\",\n        )\n        cube.append(aligned[_V60_TARGETS].to_numpy(_v60_np.float64))\n    cube = _v60_np.stack(cube)\n    if _v60_np.any(_v60_np.isfinite(cube).sum(0) < 2):\n        raise RuntimeError(\"fewer than two report teachers for a study/target\")\n    target = _v60_np.nanmean(cube, axis=0).astype(_v60_np.float64)\n    disagreement = _v60_np.nanmean(_v60_np.abs(cube - target[None]), axis=0)\n    agreement = _v60_np.clip(1.0 - 2.0 * disagreement, 0, 1)\n    certainty = _v60_np.clip(2.0 * _v60_np.abs(target - 0.5), 0, 1)\n    weight = 0.15 + 0.85 * (0.65 * agreement + 0.35 * certainty)\n    gold = train[_V60_TARGETS].notna().all(axis=1).to_numpy()\n    target[gold] = train.loc[gold, _V60_TARGETS].to_numpy(_v60_np.float64)\n    weight[gold] = 3.0\n    return target, weight, gold\n\n\ndef _v60_clean_text(value):\n    text = \"\" if value is None else str(value)\n    text = text.lower().strip()\n    text = _v60_re.sub(r\"[^a-z0-9]+\", \" \", text)\n    return _v60_re.sub(r\"\\s+\", \" \", text).strip()\n\n\ndef _v60_vendor(value):\n    text = _v60_clean_text(value).upper()\n    if \"SIEMENS\" in text:\n        return \"SIEMENS\"\n    if \"PHILIPS\" in text:\n        return \"PHILIPS\"\n    if \"GENERAL ELECTRIC\" in text or text.startswith(\"GE \") or text == \"GE\":\n        return \"GE\"\n    if \"TOSHIBA\" in text or \"CANON\" in text:\n        return \"CANON_TOSHIBA\"\n    return text or \"UNKNOWN\"\n\n\ndef _v60_sex(value):\n    text = str(value or \"\").strip().upper()\n    return text if text in {\"M\", \"F\"} else \"U\"\n\n\ndef _v60_float(value):\n    try:\n        if value is None:\n            return _v60_np.nan\n        if hasattr(value, \"__len__\") and not isinstance(value, (str, bytes)):\n            value = value[0]\n        return float(value)\n    except Exception:\n        return _v60_np.nan\n\n\ndef _v60_bool(value):\n    text = str(value).strip().upper()\n    if text in {\"1\", \"TRUE\", \"T\", \"YES\", \"Y\"}:\n        return 1.0\n    if text in {\"0\", \"FALSE\", \"F\", \"NO\", \"N\"}:\n        return 0.0\n    try:\n        number = float(text)\n        return 1.0 if number == 1 else 0.0 if number == 0 else _v60_np.nan\n    except Exception:\n        return _v60_np.nan\n\n\ndef _v60_mode(values, default=\"UNKNOWN\"):\n    clean = [str(value) for value in values if str(value) not in {\"\", \"nan\", \"None\"}]\n    if not clean:\n        return default\n    counts = _V60Counter(clean)\n    return max(counts, key=lambda key: (counts[key], key))\n\n\ndef _v60_first_dicom(series_dir):\n    direct = sorted(series_dir.glob(\"*.dcm\"))\n    if direct:\n        return direct[0]\n    nested = sorted(series_dir.glob(\"*/*.dcm\"))\n    return nested[0] if nested else None\n\n\ndef _v60_metadata(split, studies, series):\n    \"\"\"Read one header per series and aggregate it to one study row.\"\"\"\n    import pydicom\n\n    split_root = _V60_COMP / f\"{split}_series\"\n    rows = []\n    errors = {}\n    started = _v60_time.monotonic()\n    for index, row in enumerate(series.itertuples(index=False), start=1):\n        uid = str(getattr(row, _V60_UID))\n        series_uid = str(getattr(row, \"SeriesInstanceUID\"))\n        path = _v60_first_dicom(split_root / uid / series_uid)\n        try:\n            if path is None:\n                raise FileNotFoundError(\"series has no DICOM\")\n            ds = pydicom.dcmread(\n                path, stop_before_pixels=True, force=True, specific_tags=_V60_TAGS,\n            )\n            pixel_spacing = getattr(ds, \"PixelSpacing\", None)\n            pixel0 = _v60_float(pixel_spacing)\n            field = _v60_float(getattr(ds, \"MagneticFieldStrength\", None))\n            description = \" \".join(\n                part for part in [\n                    _v60_clean_text(getattr(ds, \"SeriesDescription\", \"\")),\n                    _v60_clean_text(getattr(ds, \"ProtocolName\", \"\")),\n                    _v60_clean_text(getattr(ds, \"SequenceName\", \"\")),\n                    f\"plane_{_v60_clean_text(getattr(row, 'Anatomical_Plane', ''))}\",\n                    f\"fluid_{int(_v60_bool(getattr(row, 'Fluid_Sensitive', 0)) == 1)}\",\n                    f\"fatsup_{int(_v60_bool(getattr(row, 'Fat_Suppression', 0)) == 1)}\",\n                ] if part\n            )\n            rows.append({\n                _V60_UID: uid,\n                \"sex\": _v60_sex(getattr(ds, \"PatientSex\", \"\")),\n                \"vendor\": _v60_vendor(getattr(ds, \"Manufacturer\", \"\")),\n                \"model\": _v60_clean_text(\n                    getattr(ds, \"ManufacturerModelName\", \"\")\n                ).upper() or \"UNKNOWN\",\n                \"field_strength\": field,\n                \"rows\": _v60_float(getattr(ds, \"Rows\", None)),\n                \"columns\": _v60_float(getattr(ds, \"Columns\", None)),\n                \"pixel_spacing\": pixel0,\n                \"slice_thickness\": _v60_float(\n                    getattr(ds, \"SliceThickness\", None)\n                ),\n                \"spacing_between\": _v60_float(\n                    getattr(ds, \"SpacingBetweenSlices\", None)\n                ),\n                \"protocol_piece\": description,\n                \"plane\": _v60_clean_text(getattr(row, \"Anatomical_Plane\", \"\")),\n                \"fluid\": _v60_bool(getattr(row, \"Fluid_Sensitive\", None)),\n                \"fatsup\": _v60_bool(getattr(row, \"Fat_Suppression\", None)),\n            })\n        except Exception as error:\n            errors[f\"{uid}/{series_uid}\"] = f\"{type(error).__name__}: {error}\"\n        if index % 2500 == 0 or index == len(series):\n            print(\n                f\"[DICOM V60] {split} headers {index}/{len(series)} \"\n                f\"errors={len(errors)} elapsed={_v60_time.monotonic()-started:.1f}s\",\n                flush=True,\n            )\n    if not rows:\n        raise RuntimeError(f\"no {split} DICOM metadata was decoded\")\n    if len(errors) > max(10, int(_v60_math.ceil(0.05 * len(series)))):\n        raise RuntimeError(f\"{split} DICOM metadata errors exceed 5%: {len(errors)}\")\n\n    raw = _v60_pd.DataFrame(rows)\n    aggregated = []\n    for uid, group in raw.groupby(_V60_UID, sort=False):\n        fields = group[\"field_strength\"].to_numpy(_v60_np.float64)\n        finite_field = fields[_v60_np.isfinite(fields)]\n        field = float(_v60_np.median(finite_field)) if len(finite_field) else _v60_np.nan\n        bucket = (\n            \"1.5T\" if _v60_np.isfinite(field) and abs(field - 1.5) <= 0.3\n            else \"3T\" if _v60_np.isfinite(field) and abs(field - 3.0) <= 0.5\n            else \"OTHER\"\n        )\n\n        def median(column):\n            values = group[column].to_numpy(_v60_np.float64)\n            values = values[_v60_np.isfinite(values)]\n            return float(_v60_np.median(values)) if len(values) else _v60_np.nan\n\n        n_series = len(group)\n        aggregated.append({\n            _V60_UID: str(uid),\n            \"sex\": _v60_mode(group[\"sex\"], \"U\"),\n            \"vendor\": _v60_mode(group[\"vendor\"]),\n            \"model\": _v60_mode(group[\"model\"]),\n            \"field_bucket\": bucket,\n            \"field_strength\": field,\n            \"n_series\": float(n_series),\n            \"n_sagittal\": float((group[\"plane\"] == \"sagittal\").sum()),\n            \"n_coronal\": float((group[\"plane\"] == \"coronal\").sum()),\n            \"n_axial\": float((group[\"plane\"] == \"axial\").sum()),\n            \"fraction_fluid\": float(_v60_np.nanmean(group[\"fluid\"])),\n            \"fraction_fatsup\": float(_v60_np.nanmean(group[\"fatsup\"])),\n            \"median_rows\": median(\"rows\"),\n            \"median_columns\": median(\"columns\"),\n            \"median_pixel_spacing\": median(\"pixel_spacing\"),\n            \"median_slice_thickness\": median(\"slice_thickness\"),\n            \"median_spacing_between\": median(\"spacing_between\"),\n            \"protocol_text\": \" \".join(group[\"protocol_piece\"].astype(str)),\n        })\n    result = studies[[_V60_UID]].merge(\n        _v60_pd.DataFrame(aggregated), on=_V60_UID, how=\"left\", validate=\"one_to_one\"\n    )\n    for column in [\"sex\", \"vendor\", \"model\", \"field_bucket\", \"protocol_text\"]:\n        result[column] = result[column].fillna(\"UNKNOWN\").astype(str)\n    return result, errors\n\n\n_V60_SPEC = {\n    \"demographic\": {\n        \"categorical\": [\"sex\"],\n        \"numeric\": [],\n        \"text\": False,\n        \"target_indices\": [0, 4],  # ACL and Medial OA: prespecified public claim.\n    },\n    \"scanner\": {\n        \"categorical\": [\"sex\", \"vendor\", \"model\", \"field_bucket\"],\n        \"numeric\": [\n            \"field_strength\", \"n_series\", \"n_sagittal\", \"n_coronal\", \"n_axial\",\n            \"fraction_fluid\", \"fraction_fatsup\", \"median_rows\", \"median_columns\",\n            \"median_pixel_spacing\", \"median_slice_thickness\",\n            \"median_spacing_between\",\n        ],\n        \"text\": False,\n        \"target_indices\": list(range(12)),\n    },\n    \"scanner_protocol\": {\n        \"categorical\": [\"sex\", \"vendor\", \"model\", \"field_bucket\"],\n        \"numeric\": [\n            \"field_strength\", \"n_series\", \"n_sagittal\", \"n_coronal\", \"n_axial\",\n            \"fraction_fluid\", \"fraction_fatsup\", \"median_rows\", \"median_columns\",\n            \"median_pixel_spacing\", \"median_slice_thickness\",\n            \"median_spacing_between\",\n        ],\n        \"text\": True,\n        \"target_indices\": list(range(12)),\n    },\n}\n\n\ndef _v60_preprocessor(spec):\n    from sklearn.compose import ColumnTransformer\n    from sklearn.feature_extraction.text import TfidfVectorizer\n    from sklearn.impute import SimpleImputer\n    from sklearn.pipeline import Pipeline\n    from sklearn.preprocessing import OneHotEncoder, StandardScaler\n\n    transformers = []\n    if spec[\"categorical\"]:\n        transformers.append((\n            \"categorical\",\n            OneHotEncoder(handle_unknown=\"ignore\", dtype=_v60_np.float64),\n            spec[\"categorical\"],\n        ))\n    if spec[\"numeric\"]:\n        transformers.append((\n            \"numeric\",\n            Pipeline([\n                (\"imputer\", SimpleImputer(strategy=\"median\", add_indicator=True)),\n                (\"scale\", StandardScaler()),\n            ]),\n            spec[\"numeric\"],\n        ))\n    if spec[\"text\"]:\n        transformers.append((\n            \"protocol\",\n            TfidfVectorizer(\n                ngram_range=(1, 2), min_df=5, max_features=512,\n                sublinear_tf=True, strip_accents=\"unicode\",\n                token_pattern=r\"(?u)\\b[a-z0-9][a-z0-9_]+\\b\",\n            ),\n            \"protocol_text\",\n        ))\n    return ColumnTransformer(transformers, remainder=\"drop\", sparse_threshold=0.3)\n\n\ndef _v60_crossfit(metadata, target, weight, train, expert, outer_folds):\n    from sklearn.linear_model import Ridge\n\n    train_index = {uid: index for index, uid in enumerate(train[_V60_UID])}\n    if not set(expert[_V60_UID]).issubset(train_index):\n        raise RuntimeError(\"expert UID absent from train.csv\")\n    expert_rows = _v60_np.asarray([train_index[uid] for uid in expert[_V60_UID]], int)\n    predictions = {\n        f\"{name}::alpha={alpha:g}\": _v60_np.full(\n            (len(expert), len(_V60_TARGETS)), _v60_np.nan, _v60_np.float64\n        )\n        for name in _V60_SPEC for alpha in _V60_ALPHAS\n    }\n    dimensions = {}\n    for name, spec in _V60_SPEC.items():\n        for outer in range(5):\n            held_expert = outer_folds == outer\n            held_rows = expert_rows[held_expert]\n            fit_mask = _v60_np.ones(len(train), bool)\n            fit_mask[held_rows] = False\n            preprocessor = _v60_preprocessor(spec)\n            x_fit = preprocessor.fit_transform(metadata.loc[fit_mask])\n            x_held = preprocessor.transform(metadata.iloc[held_rows])\n            dimensions[f\"{name}::fold={outer}\"] = int(x_fit.shape[1])\n            for alpha in _V60_ALPHAS:\n                key = f\"{name}::alpha={alpha:g}\"\n                for column in range(len(_V60_TARGETS)):\n                    model = Ridge(alpha=alpha, solver=\"lsqr\", fit_intercept=True)\n                    model.fit(\n                        x_fit, target[fit_mask, column],\n                        sample_weight=weight[fit_mask, column],\n                    )\n                    predictions[key][held_expert, column] = model.predict(x_held)\n            print(\n                f\"[DICOM V60] OOF spec={name} outer={outer} \"\n                f\"features={x_fit.shape[1]}\", flush=True,\n            )\n    if not all(_v60_np.isfinite(values).all() for values in predictions.values()):\n        raise RuntimeError(\"metadata OOF contains non-finite values\")\n    return predictions, dimensions\n\n\ndef _v60_candidates(baseline, metadata_oof):\n    values = {\"parent\": baseline}\n    definitions = {\n        \"parent\": {\"metadata_key\": None, \"gamma\": 0.0,\n                   \"target_indices\": list(range(12))}\n    }\n    baseline_rank = _v60_rank(baseline)\n    for metadata_key, metadata_values in metadata_oof.items():\n        spec_name = metadata_key.split(\"::\", 1)[0]\n        target_indices = list(_V60_SPEC[spec_name][\"target_indices\"])\n        metadata_rank = _v60_rank(metadata_values)\n        for gamma in _V60_GAMMAS:\n            key = f\"{metadata_key}::gamma={gamma:.2f}\"\n            candidate = baseline_rank.copy()\n            candidate[:, target_indices] = (\n                (1.0 - gamma) * baseline_rank[:, target_indices]\n                + gamma * metadata_rank[:, target_indices]\n            )\n            values[key] = candidate\n            definitions[key] = {\n                \"metadata_key\": metadata_key,\n                \"gamma\": float(gamma),\n                \"target_indices\": target_indices,\n            }\n    return values, definitions\n\n\ndef _v60_select(truth, outer_folds, candidates, definitions, baseline_score):\n    keys = list(candidates)\n    nested = _v60_np.zeros_like(candidates[\"parent\"])\n    choices, train_scores, held_gains = [], [], []\n    for outer in range(5):\n        fit = outer_folds != outer\n        held = ~fit\n        scored = {}\n        for key in keys:\n            definition = definitions[key]\n            fraction = len(definition[\"target_indices\"]) / len(_V60_TARGETS)\n            penalty = 0.006 * float(definition[\"gamma\"]) * fraction\n            if key != \"parent\" and key.startswith(\"scanner_protocol\"):\n                penalty += 0.0002\n            scored[key] = _v60_auc(truth[fit], candidates[key][fit]) - penalty\n        best = max(keys, key=lambda key: (scored[key], key == \"parent\", key))\n        choices.append(best)\n        train_scores.append(scored)\n        nested[held] = candidates[best][held]\n        held_gains.append(\n            _v60_auc(truth[held], candidates[best][held])\n            - _v60_auc(truth[held], candidates[\"parent\"][held])\n        )\n    nested_score = _v60_auc(truth, nested)\n    descriptive = {key: _v60_auc(truth, value) for key, value in candidates.items()}\n    eligible = [\n        key for key in keys if key != \"parent\" and choices.count(key) >= 2\n    ]\n    final_key = (\n        max(\n            eligible,\n            key=lambda key: (\n                descriptive[key], -definitions[key][\"gamma\"], key\n            ),\n        ) if eligible else \"parent\"\n    )\n    final_score = descriptive[final_key]\n    non_parent_folds = sum(choice != \"parent\" for choice in choices)\n    supported = bool(\n        final_key != \"parent\"\n        and non_parent_folds >= 4\n        and nested_score >= baseline_score + _V60_MIN_GAIN\n        and final_score >= baseline_score + _V60_MIN_GAIN\n        and sum(gain > 0 for gain in held_gains) >= 4\n    )\n    return {\n        \"candidate_macro_auc\": descriptive,\n        \"outer_choices\": choices,\n        \"outer_penalized_train_scores\": train_scores,\n        \"outer_held_gains\": held_gains,\n        \"nested_macro_auc\": nested_score,\n        \"selected_key\": final_key,\n        \"selected_descriptive_macro_auc\": final_score,\n        \"selection_supported\": supported,\n        \"non_parent_outer_folds\": non_parent_folds,\n    }\n\n\ndef _v60_fit_test(spec_name, alpha, metadata_train, metadata_test,\n                  target, weight):\n    from sklearn.linear_model import Ridge\n\n    spec = _V60_SPEC[spec_name]\n    preprocessor = _v60_preprocessor(spec)\n    x_train = preprocessor.fit_transform(metadata_train)\n    x_test = preprocessor.transform(metadata_test)\n    result = _v60_np.zeros((len(metadata_test), len(_V60_TARGETS)), _v60_np.float64)\n    for column in range(len(_V60_TARGETS)):\n        model = Ridge(alpha=alpha, solver=\"lsqr\", fit_intercept=True)\n        model.fit(x_train, target[:, column], sample_weight=weight[:, column])\n        result[:, column] = model.predict(x_test)\n    if not _v60_np.isfinite(result).all():\n        raise RuntimeError(\"metadata test prediction contains non-finite values\")\n    return result, int(x_train.shape[1])\n\n\ndef _v60_main():\n    audit = {\n        \"status\": \"PARENT_PRESERVED\",\n        \"evidence_boundary\": (\n            \"All 58-study AUC values are local official-label diagnostics. Only a \"\n            \"completed Kaggle competition row with a nonempty score is leaderboard evidence.\"\n        ),\n        \"incoming_primary_sha256\": _v60_sha256(_V60_PRIMARY),\n        \"min_nested_gain\": _V60_MIN_GAIN,\n    }\n    parent = _v60_parent_oof()\n    audit[\"v58_status\"] = parent[\"v58_audit\"].get(\"status\")\n    if parent[\"special_parent\"]:\n        audit[\"status\"] = \"PARENT_SPECIALIST_SELECTED_PRESERVED\"\n        return audit\n\n    audit[\"parent_name\"] = parent[\"parent_name\"]\n    audit[\"parent_oof_macro_auc\"] = parent[\"baseline_score\"]\n    train = _v60_pd.read_csv(_V60_COMP / \"train.csv\", dtype={_V60_UID: str})\n    test = _v60_pd.read_csv(_V60_COMP / \"test.csv\", dtype={_V60_UID: str})\n    train_series = _v60_pd.read_csv(\n        _V60_COMP / \"train_series.csv\", dtype={_V60_UID: str, \"SeriesInstanceUID\": str}\n    )\n    test_series = _v60_pd.read_csv(\n        _V60_COMP / \"test_series.csv\", dtype={_V60_UID: str, \"SeriesInstanceUID\": str}\n    )\n    if train[_V60_UID].duplicated().any() or test[_V60_UID].duplicated().any():\n        raise RuntimeError(\"duplicate study in train/test CSV\")\n    target, weight, gold = _v60_targets(train)\n    if int(gold.sum()) != 58:\n        raise RuntimeError(f\"expected 58 gold studies, observed {int(gold.sum())}\")\n\n    metadata_train, train_errors = _v60_metadata(\n        \"train\", train, train_series\n    )\n    metadata_test, test_errors = _v60_metadata(\n        \"test\", test, test_series\n    )\n    metadata_train.to_csv(_V60_OUT / \"train_dicom_metadata.csv\", index=False)\n    metadata_test.to_csv(_V60_OUT / \"test_dicom_metadata.csv\", index=False)\n    audit[\"metadata\"] = {\n        \"train_studies\": len(metadata_train),\n        \"test_studies\": len(metadata_test),\n        \"train_series\": len(train_series),\n        \"test_series\": len(test_series),\n        \"train_header_errors\": train_errors,\n        \"test_header_errors\": test_errors,\n        \"train_sex_counts\": metadata_train[\"sex\"].value_counts().to_dict(),\n        \"test_sex_counts\": metadata_test[\"sex\"].value_counts().to_dict(),\n        \"train_vendor_counts\": metadata_train[\"vendor\"].value_counts().to_dict(),\n        \"test_vendor_counts\": metadata_test[\"vendor\"].value_counts().to_dict(),\n        \"train_field_counts\": metadata_train[\"field_bucket\"].value_counts().to_dict(),\n        \"test_field_counts\": metadata_test[\"field_bucket\"].value_counts().to_dict(),\n    }\n\n    metadata_oof, dimensions = _v60_crossfit(\n        metadata_train, target, weight, train,\n        parent[\"expert\"], parent[\"outer_folds\"],\n    )\n    candidates, definitions = _v60_candidates(parent[\"baseline\"], metadata_oof)\n    selection = _v60_select(\n        parent[\"truth\"], parent[\"outer_folds\"], candidates, definitions,\n        parent[\"baseline_score\"],\n    )\n    audit[\"feature_dimensions\"] = dimensions\n    audit[\"oof\"] = selection\n    oof_frame = parent[\"expert\"][[_V60_UID]].copy()\n    for key, values in metadata_oof.items():\n        safe = _v60_re.sub(r\"[^a-zA-Z0-9]+\", \"_\", key).strip(\"_\")\n        for column, target_name in enumerate(_V60_TARGETS):\n            oof_frame[f\"{safe}::{target_name}\"] = values[:, column]\n    oof_frame[\"outer_fold\"] = parent[\"outer_folds\"]\n    oof_frame.to_csv(_V60_OUT / \"metadata_gold_oof.csv\", index=False)\n\n    if not selection[\"selection_supported\"]:\n        audit[\"status\"] = \"OOF_REJECTED_PARENT_PRESERVED\"\n        return audit\n\n    selected_key = selection[\"selected_key\"]\n    definition = definitions[selected_key]\n    metadata_key = definition[\"metadata_key\"]\n    spec_name, alpha_text = metadata_key.split(\"::alpha=\")\n    alpha = float(alpha_text)\n    metadata_test_pred, feature_count = _v60_fit_test(\n        spec_name, alpha, metadata_train, metadata_test, target, weight\n    )\n    metadata_test_frame = _v60_pd.DataFrame(metadata_test_pred, columns=_V60_TARGETS)\n    metadata_test_frame.insert(0, _V60_UID, test[_V60_UID])\n    metadata_test_frame.to_csv(_V60_OUT / \"submission_metadata_only.csv\", index=False)\n\n    parent_submission = _v60_pd.read_csv(_V60_PARENT, dtype={_V60_UID: str})\n    _v60_validate_submission(parent_submission, test[_V60_UID])\n    selected = parent_submission.copy()\n    indices = definition[\"target_indices\"]\n    gamma = float(definition[\"gamma\"])\n    parent_rank = _v60_rank(parent_submission[_V60_TARGETS].to_numpy(float))\n    metadata_rank = _v60_rank(metadata_test_pred)\n    values = parent_rank.copy()\n    values[:, indices] = (\n        (1.0 - gamma) * parent_rank[:, indices]\n        + gamma * metadata_rank[:, indices]\n    )\n    selected[_V60_TARGETS] = values\n    _v60_validate_submission(selected, test[_V60_UID])\n    selected_path = _V60_OUT / \"submission_metadata_selected.csv\"\n    selected.to_csv(selected_path, index=False)\n    _v60_shutil.copy2(selected_path, _V60_PRIMARY)\n    if _v60_sha256(_V60_PRIMARY) != _v60_sha256(selected_path):\n        raise RuntimeError(\"selected metadata submission copy drift\")\n    audit[\"status\"] = \"CANDIDATE_SELECTED\"\n    audit[\"selected_key\"] = selected_key\n    audit[\"selected_spec\"] = spec_name\n    audit[\"selected_alpha\"] = alpha\n    audit[\"selected_gamma\"] = gamma\n    audit[\"selected_targets\"] = [_V60_TARGETS[index] for index in indices]\n    audit[\"selected_feature_count\"] = feature_count\n    audit[\"selected_path\"] = str(selected_path)\n    audit[\"selected_sha256\"] = _v60_sha256(selected_path)\n    return audit\n\n\ndef _v60_execute():\n    started = _v60_time.monotonic()\n    _V60_OUT.mkdir(parents=True, exist_ok=True)\n    if not _V60_PRIMARY.is_file():\n        raise FileNotFoundError(\"incoming submission.csv is absent\")\n    _v60_shutil.copy2(_V60_PRIMARY, _V60_PARENT)\n    result = None\n    try:\n        result = _v60_main()\n    except Exception as error:\n        _v60_shutil.copy2(_V60_PARENT, _V60_PRIMARY)\n        result = {\n            \"status\": \"ERROR_PARENT_PRESERVED\",\n            \"error\": f\"{type(error).__name__}: {error}\",\n            \"traceback\": _v60_traceback.format_exc(),\n        }\n        print(f\"[DICOM V60] rejected safely: {result['error']}\", flush=True)\n        _v60_traceback.print_exc()\n    finally:\n        if result.get(\"status\") != \"CANDIDATE_SELECTED\":\n            _v60_shutil.copy2(_V60_PARENT, _V60_PRIMARY)\n        result[\"parent_sha256\"] = _v60_sha256(_V60_PARENT)\n        result[\"primary_sha256\"] = _v60_sha256(_V60_PRIMARY)\n        result[\"wall_seconds\"] = _v60_time.monotonic() - started\n        _V60_AUDIT.write_text(\n            _v60_json.dumps(result, indent=2, sort_keys=True) + \"\\n\"\n        )\n        print(_v60_json.dumps(result, indent=2, sort_keys=True), flush=True)\n    return result\n\n\nif __name__ == \"__main__\":\n    _v60_result = _v60_execute()\n"},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"# V61: train the one high-yield architectural fact disclosed by the public .903\n# notebook: a fold-matched 288px EfficientNet-B3 max-MIL family.  The current\n# submission is copied before any work and is restored unless one repeated\n# candidate clears a strict five-fold nested official-label gate.\nimport base64 as _v61_base64\nimport contextlib as _v61_contextlib\nimport gc as _v61_gc\nimport hashlib as _v61_hashlib\nimport json as _v61_json\nimport os as _v61_os\nimport re as _v61_re\nimport shutil as _v61_shutil\nimport subprocess as _v61_subprocess\nimport sys as _v61_sys\nimport time as _v61_time\nimport traceback as _v61_traceback\nimport zlib as _v61_zlib\nfrom pathlib import Path as _V61Path\n\nimport numpy as _v61_np\nimport pandas as _v61_pd\n\n\n_V61_STARTED = _v61_time.monotonic()\n_V61_UID = \"StudyInstanceUID\"\n_V61_TARGETS = [\n    \"ACL\", \"MCL\", \"Medial Meniscus\", \"Lateral Meniscus\",\n    \"Medial OA\", \"Lateral OA\", \"PF OA\", \"Effusion\",\n    \"Synovitis\", \"Baker's\", \"Contusion\", \"Fracture\",\n]\n_V61_COMP = _V61Path(\"/kaggle/input/competitions/rsna-knee-abnormality-detection\")\n_V61_DATA = _V61Path(\"/kaggle/input/rsna-knee-b3-v47-folds-0-3\")\n_V61_SOURCE = _V61Path(\"/kaggle/working/rsna_b3_v61_source\")\n_V61_TRAIN = _V61Path(\"/kaggle/working/rsna_b3_v61_warmstart288\")\n_V61_INFER = _V61Path(\"/kaggle/working/rsna_b3_v61_inference\")\n_V61_PRIMARY = _V61Path(\"/kaggle/working/submission.csv\")\n_V61_FALLBACK = _V61_TRAIN / \"submission_parent_before_v61.csv\"\n_V61_AUDIT = _V61Path(\"/kaggle/working/b3_warmstart288_v61_audit.json\")\n_V61_RAD_DIR = _V61Path(\"/kaggle/working/rsna_rad_v52\")\n_V61_DICOM_DIR = _V61Path(\"/kaggle/working/rsna_dicom_metadata_v60\")\n_V61_DICOM_AUDIT = _V61Path(\"/kaggle/working/dicom_metadata_v60_audit.json\")\n_V61_RAD_ALPHA = 0.20\n_V61_MIN_GAIN = 0.0015\n_V61_BETAS = (0.05, 0.10, 0.15, 0.20, 0.25, 0.30)\n_V61_DIRECT_ALPHAS = (0.05, 0.10, 0.15, 0.20)\n_V61_TRAIN_DEADLINE_SECONDS = 6 * 60 * 60\n_V61_SOURCES = {\"efficientnet_b3_public_repro_v1.py\":{\"b85\":\"c-rMXYm?hHlHcc7V7XPdq)pK?<3}aBD0@!g<Wg%pF57$iK~b?N5gIW=kqjU6$n5NIzkUE9KvJ4XZol1D)}}~wqtR&e3k~?se|pw6^|N(WJd?$hsBYU`Sv)`SJnyYcE|Uv6+muCHmw7Hz@n*ZtHkmBi_p+UR^IW|9@vUg<BrCGwLTt-geEjtOwfI{hWdsOLPPTQq7jeAp+OC#yEV6x7)~!g2qHL2kD~smjM8B&qs-$kD{=G|@U7oG=&tICd(9ip%-Rb8#DbjLpei`tt$Tnpv)1*z93Vhn+Nz=&2kkO>srVaZ~+_D1n(6es!0ehnl?QMnQRPVnd*tb~&f4)m9{1l3xDjZdkhvJj`wUfn0hT?Nq<<g9^==RkubS;WXzp7vi@B;p;(vy?VuRs3q=JTgH!!4V(4n-72E3pv%iGcrkufKikg~EHM{*q~y<l>zyvS!mY`08!a%6jllHT&tc7cv&z2;T_r!}sa|rnqZh4EX%%wkWT%HX{VzB$u-OOQXTR1*tF%zps-GPBDB8PCmSS{r=4~{obeKB5T_uN1$)Z8d-&puWx`hFF5)8kKaKv?^D~Qw?7t5n*amh9Z1s2f0<eGVKRP`#o}|_$>4-uiO(p6K+-n5m=k^39z-{8vft!fWJL>D&%gMBzU-5mxRG@xo4Codjcjy-S1;&0$@AdT`6+$6O7bine0qLPpSES5ns+bhRU>8EBRGF``o-y&^kJQ}o1IG=y}vGNV6nmSLN(r8uFFEkn_X92>ThSNSK>r0t8%k5aA)+XD<(fWaCto!+q_Kh!xwV)!UY_B<w0G`>|)o(soW&D{pT|cd;x5TxlRA|dBp6i8rh|#`W(a8((2(fQcTtZo(m8ilm@AlP9fWP{TziM{_geXH}I{N(Wcy2SuS~L)ZWi)|F842FaLRd8vLBzo!>8K&!QDPorUi6&+C7B24vzpJp(F2@=y5l#eMMe+5_>9-~avHH*Y?=y)%}l0I-Zg!0|NrE3}5jpFX_)_RXkm^Ax_FoTPFqin88=Sp6pBTB0_HL5;>2b=g%uh+7A2DA&KpO&f^W*T@kT?^fO*(>zaPTS4ta)`<7OHX-vyVnYTG?B<qIj7-12_x}68eG97)8wJrOX(S4>??4j6qo!N?p7;vj<<~+(IL(+hx*_PgLAzRlnkWRbMKv>B1JPOKYsM&c5G&#=J;n`%Rhf*a*(K*MU-=JlHi$q!13Ua6h<5TO%`Rlq`oVI3{$izQh_fP<H!_X$WDR@;&6DhbhANG|10H^lzd}*aJ0OOLvcGBSe(kavtmi`9fierl6Q$zsSs!IhgJxx68b)ch-O3uR2loh~lXv@~VIIPaD;UO4eIyyo`v2gt-qdwj`&;i0>AFX;&^;(Db8*)IJ7nss55c`>FjId^AafO^pue+Cf(?TxDQ^AXG5w*p01C7b2AoEHFY5BT@f<1OzLpM8Rx2261${*gtaf6*4-J5Ts>*MD;BH&SAXlKkH=&T&ceC&|rK~s7tf_UEff}m1H4qR3L5(b$uJL>IKA1w*PFa3=od8oOzBR49l9Nz*c5?vTL&^?J<+Qwj=k(^+F3D$2*}<qpQ*K+q0{EXw|2Dwpy%c4U-wJX~l8ai(J?O}18(Aa&+gtL$)U48=|In-w<fA5a4T=MX#$EBGdLL=wc@Ix7bD7550FtF4X}ALQECF);=mp`y<~(uGm>-TcrJ015^tW!fqCd+HCk+P})v_l%7!1=4P6I7HTZwt-0?}LDKH4XhKhSf5)g$~FrFB{PYUM68Bsbc0Yvg!Qf#w9G=`WdqKqJ#f1k+=nRt*qGA{dafuH*~mS8;x2)BxZtl473~K6Vm_uRvr^1;D5e$&LE(wK$7Tk4TWmt5C_W*mq4U*3{?nG!$nmHI(Ek$)HKD#*;N$h$&jhoq(O#`^in#EKY3#QFwc}uf(}6$>lXz6^JQh%NXJDdIvgn0I@t>9h3g8yq2|ScS#}I>r%L5ft3v1Ebicsd)?XMP6_&wf3LJ^HCl+tK_1G4xf=?5Y-!e*#q74lQLGyR5wjlQ60QC!aIp+fY;rKHFi`+^4u3vzdI@oEL!3uNs@5PDEJJAu%vfds`Z5?IU>ya}6FN{l6^f9Fj{)@IoB246UcH1D{ZO95tLOO2P)Kd9<(4(Eys0FvEwx~mIRt3Fnv`G#ZckezsQL1oMd}Qm8;I;A@)vM^VXc)wx%RdA#YhuKt|a>dO3>i#1rN#CdOWuI{EmU{iJX4M=ArmyWpzty^7(l%z@q$U7Yo09EDH3;2M8P!N;*Qz>e!wRS)mL!;MNWjtE~p3`KHY_phj_V#4WBc*DM~ec1jFD!;#{j;2ljs&wXu4`go&<9SrvvWGy#kof6#2OtV(PAwMCIHJ*rGu5W!;t%m~LDGo<_K|oRfIW<`tGFL-I+-o1E2mKsbr2d`xKrVsP;k>8QZG|Bve9dLyQ;T5mo!15s(iN>o5h1s877l6KU<j(F3BKv>m)6XUhGnbl0Vd)_RW@0hT_Lrs;jle^F=}iI>ar-W3kES30zWw2=|#bW^l0uK5w-=<SG?EmV*+0BMWIA{o#dE>NMnldSvwt+aR>(D20Wy;PX&b2c>ui{K2avZIDX&BR@Pv4X)y`ZeCe%FB#7&*-SzQeI%A>W*=WS2tXi;1<v!ckuAe3d&D}9OZQ5xQ4dyMEz)Q4z%803+7RQ&piU?`hQI-P$XfjyXnkCDr7Bun9&}aj)bSd`+8X7N0Be5jXvVJ3MWt-%&mboM7T(KP5Q$>`8XMlKm5WUns)H93rk%80{A_6dHJR=a)q|4hF-uozFO1hIeBg2h0_8B$@S1sE^w=TVwwXW){R6xDkeRg_E79V_@rK>i2f9WZ-qDDNq&{tZvK0yqa87jJhzyiiBZj_f7XBES+Q1mTOAIBk5fNGeT6~T(HG3HSAXyuBfYQ*c;kM+nF`XLnYFIfvH7QnTrD>CR~@1yG?XRpFuZ#F>?OkzewA5-$6@t2<I08^L>Gd{XbKg?*e0hJDc_G0;ck~g3w)L&>aR)NVM>cs+_{7u$n<7)x1n%K*N8<|!-0PT_l7LY6c1QzGnCFsZ=<*>OWwZ{OM*XW+jDI?DLUQl-EYTVtQJYpt$t<-=TmX=A@z}PE-g6Ln!o-1?5L~LMPWSE*VI!CkAj>^WZ4M(l99fz>k>9F?^P@1a)J|Bm~F)R^cC7v#XwOEdhXaHfW%Lm3afE+C52ZuKWuGE_Yi(>$#(x@1PVjSNeOykTpB&ZZI^t4{FB7?x;i8y!pXWWy69jKX~xh6-&Qv>ZeZgIG0&9GyA8yT7nZOgI#-8K(UKs<kt2A0U#*kV0Mht=vZCA_4ySKCjyiRJ8RdMaL!L2()QRK=`=LG}HtB@GeXS&sBEZ-GxuQrF3?&ol&Ru;lFQTm`ZNWn$T;UZ<fpW9#JV^nk)k=+;tGO4%&&^}&T|>6k&o$@S@|mjS?`WjRnSQ&9G51kQjP0mHF8xZ!XqPt|k9Bw-j9d~pt`r;>@Un+AM`tSpq9*<jw%bzHAyEy1mA72jp*W{FiNgCJeG>Q%P6jB$5N?dXA{5?A0YOP^m%grL2Z(wnvTYqc6D>pgbP?Ttv>t1Q^b25;&E_C))id6%W}Cy4HOHt<!_#-F;X0`h5>^fAePruh$L`x#Ww^gTY|zuA=y=Cu#;#I@nXShT}xhqt%e-MXPfn;Jk~ZkGJG9J6Z`$zOd!6|9DK4+hJ)BDt^H+kt^6VwVJpl%eKv8<a(x6`gc^2j&4%?LTIk22KT2vt%UW?1}q%B7d^Y@*Eb&X)wVn9;oe7I$bXxLoRO8kA!sP#?Hsx&%SL><B-#+1r@m&&h6=HIn}MRY#E2WOwFcy**25QGZYkJn4NW6u2^8PoLZ?6<9?_mEFFQno?GPAtkJwuk|;nQu-Jy8-N{1Wd<c>)L3Ef%`Ty6tr=7Qcy;`gA|F&d!NAJ*TalSsdSh%&v>YOr{>dk~kLHl%9f#4ARStu$uPs<7`xHo+~*Rd}O5KlYrw4!*ceuhIQ#dQWsYjVjNO8>wrwN@yu$*BdZcF1HdH!Y@<Xj$ND=%b}Z%7STVXlh$)O8fB340$pFV-?RUp8SO|vmI^C?a_GSE>cgxK#MA}jT@-3Pbras|JdV<k~CF@!zKZPUfn?;77L|B94zZvCYQWTP^Z$K?*nuBIMv(F_+zJctbiwGnfB&bSppTvSw;0E!?b6F1qky$&KPZe`+m)=Dw#Vzr3TKi0=gOa-W)054iC`*A%{bt_UKerhO4Y_qZNy-d}w;n?#MFOR)Q=9CXiF|^yer2(4o2bY9VOTk~_p-$1<P_2*M);pfjPy$qvT&2Rv27#%;5*z<u%p)z7kzHyAPqvB|c}bDJ6PA=Id60yNEFYwFbthqssuR54GU6-||ZB{{VbC{6_vXqeYcTW)p<CU76e?hvz(b1Mn?gDg<Nmi5CEG0muBv#rS|7@1y3mA@JfR<*<cFVKa0ogD1#<Z_Dxpsh&|>)*1<C#1{y><q0DJ)g~2)*{MF6;@KNlC~&)L}TfwT?S1sCjo7rBFjzljm&bNy0c?TrcAFIY>cT$_*-7c8b)4*0;AVmrpFd&=zi99T+6sZk>cbohLc;BWiHcrpKR(fPP&bM8@Dx9Q1#OkirbiiWBVmdCD(3JVmGv+1A3tyFe*H0E_0dGMYNYKn0QlKTbCQ`ld-d@g~*Jv2lEj^H_UkSUMrPj4hjZ%ki0z_i2oRTBXZQo-sTy~^hT&DwKaYXWw@O|>7IxPL}-}VzZ%#wF4fCY-uNIz1LVOH__Z6O3nETq>XJ?UKPvBIdb}bjykOwJ^H@>zxhxPlstN=PYp9X3d4VdXFihgoxRot(zDe*?3W*yA1gUx(5d{ja?@J3Lg#Ae^X(KHr$K#>RG;C><5Y_hDg}x`2q`GkSFD0y8)r-BlsSa}ZoYV()3Rbn?WVaimaslw~zycm4VF5{WKHK}P%B!@x(G^?N6Nkcrr&0Aj438%Q1BN0r&j^s}?FD7K`lR|D{%N9usonC?j<)O5_jFEF8%z}FnYGZw3ZI2x%ogAqHq&Jr2HfdZsTH@(!D%tB$B4KZ=wFJCkh(d9pMFeMV>s=`Ek?bt7)Mr_bl|G9qhoSe@3cS6dXf$AP&bE%cBZPf$ti5@i5@0H$$Zrf_;Da%Hd(`l4<!fty=|Q$%=rw8KqJ;P%x+Sgy-2&+!k)~~8mX>U(Pvd+Hd7|_X=`gSDHFE2Iv>8OZoxk(_ruq1wjaMM>&<TRWmFV&{G+g<W;|R%-?}!-n+OZE%`p>rep@Es|A+JpbHNVL%?3C8f23B#Kg0@ehfMwmo>8{fPOdqvqw)|9cO23nLOsHecwB;yvbmb^ksvQ%I-v(A&2x2bK`|1Au_(ex>Ul1}Sfnw5#aEaS&{+_8SI_CtO*lNn5%SrM!J&;w*V82{w*SNOg|QrP1g%h0aLtB=y6v&AR<2WtXN9<clJYG55Tc8`T>IXWDBbMA9L_DxGDvGu4U!@iTSG*~5i;T)dJxDEXr4AvmEC|jA)=0zf!R>}x8ARZg`Tv~P1;ZeYXc#^{FK9-d%T;27RM9`xTu~kj^Owal>y48M|FXO=X8W;ODP5i{==?pOZFoP2lxwoNDKpP;34`#-n*aP|McyDfBfm)n={Yl<7lL~X$AhxjDNY&D^0Za!pRMHNYMfI`)<FcJ$*HdK*^*vEovRUp}&CG`Qel_dRh=uD=9m+Ot+v4MJhLCN_jnoKsmeXB}`jwH5QaW|5DyA@?^hGL4CAxKc_#Jr^?yob>>tI)AzF#r_JD*WSI~u7z{`##sQcL=M|{T!Am4mp;(?_<>!}QM!F9Z53zvkvOqcc5ZcfVoR$Qe7%DMUI58A1an})G`FU7gs!|)=94nA5t<b00!IYtX@a?8Et0{Ex%yO*_IsC9B_&H;FYJr(eGqO@PEBoyE)p^7X{F_kty%Mt(T0oH^2zi;4j4u%C$M>J#e2iazeDm5ZgSo+JVZylVg+KpUyu*`ZnAz5)$<ddm&qbA05`tv;m&P2<oS|%q4!6$4o6m2>8scTpXt$B$=j&lT5$7*oMyGb(pySfmv8I8UqQ*+C?~_Xzr>yB?KCY*fpV7d|Jn#=?6pkomKH!IEO6gdj0`S$}mS!ajA7elg$Cy{zQqHppRZdHDzb;D?PlSd+T&5R{l3d$JdHdvc+tvSAn6J(tj$l+)9>QAGa+nlTMU%?Qr&;ZsI#ZR3*CnQOuz>1HDOVWWk2|r4Lof80=!ZQpqKL#5?W*4c28S4NGT<t#BE{4Ji<Ic7R339H7(#<?Mdf0qh0i1m5MjkW6GPc+g^M?w2w>$5(*f=%51`Y^c+O=|OGD|Qg?7K~BcANvS`6c0R<*QGd^#BaY`HeXG4Q<-Sc`my6UJ@TaVB|w8wv4Uom?aS5^Fipml`0%;>#qx0=t8)XqAYM?|%>%*%gqP0EldMBIz!051ykr_)lKR`c_<JP1X+ZVYw)tBWb{S&hQ8&dVO6n%K;=u2fHrQZj-GuhzOM;Fxn8&#Y$$-I<j|)SwH**+W7p*ljmogWc1I%Ai&)Os^*+#=czl_M{60iw1TmSPft(b|Ia}qkqE%Mo-}ugIo@5kg}wUR%<|v5r8g8V+UdluCw4iFGKgRi__XAS(txY=)W+1Ha*4J((A#Y;If)tC6>-yB1ZpdP>=HRSeRW85+^p|cdqsVLA*~)0yV~|_^hzbRm2C(nyP;u?dQqLM%Q8OzH`?AG+3s;nF^OoPJ;8z@Yo1wr3J(!5V8F3vqjTot*5h$z=?y8e!+Ck_%&43<rQ{E-x`WyE0g^EtO|q|YTK&o&0>V}g#TiE2EJCIwtw*oDKmF#3a7uHdFJ7YOcSeVd;E77i4z!r;39FQ#IV&2U53;?SCdwsDJz=sK|HDm!sVYqu$P5DyM!k6PC3;7v(fRqSljCbmfsIiQ*9xrNV@pL@<GT?a4}{vAi%HS;NT3_|%vv5zlRC^{yjvzWP~1=_&c>LK+@pACJi!zwr=ARO3knzlp(WaLD;su$xhdlk>jOP{3q<v+WOJeuPIMjW#Lb2M@viP|r+VnP+J5rBOd*D!@RCP)lm`+Qy^s9v$G7hgt6vlmJaoA{?jh^j78avx7C<-b<mR#}!QwA)#>2r~6pOR-QMD9=h<*^6J|<ja7l(2$+?Pk7Fy#uoi8i&w3RYsEfevwtfjA`7h1z0=83k1eE&w<xAd9iO0%ifq1T!>X`Ej3Y;U!lW-Y7-UGhnPS@azkbLxO;ohmtluNPq^siot#50MFZ09WYfL;EwU^w({+#lqh&P%2)BJCv(RP?;Cma5^Y`UQS^X659nG&DE+;(0aphHMPbWj0Pzkid?3NwlPE=$i9t0y7gw}4E<<?G{gV#}B6&Z;i%ESd16a7z8THiolWg$7m|n#V;EmMUHN6oZ)SyPKR(J_SB{8+JScvCGI5P#@b88B5xA)U*j{<cmWrdI9D9~Z%rM#Vsem4$&hW(bD!#Nx>e5Q?0#j-IoX=E{{_YwW%*r-px#T+JY=`lk<DEXa6zl4U0SqI#%2S`3-b*)atpB7@GzY*m9q&H!Wc6u5o^Jhk3u#RUmSRl8@lnGYGS$OZg@i0tXv@5XWVp>wG7AIa^vS|+pLSHGZ>hNM<LW2oK^|~uGd+5s_872#xH9kf?9N8AsW>;fyvCGnwq88M@?if)|>V_aN&%byvR>(=)7V%YR{piq}E26iceG?#U&=Cd(hd&;`&$l4-aNK%u&S?y*4k?&p=z!oug7F~4<SLS-7|gfvgoZaqP_DD2nMTM#%)|jKH!nnv_)aG20cJfgvVC>~2IV3<Fe>HG%)lKUKtp3R$<a-6mB~8Rfq_5f*C2U=`7UFf$+lufS~(9~D=;5Lrj@2k)sI=SWj;A&hsG<`Un9Le6<^fEEb7W74SKpx>Qv{H$cg2kd4QSj+X_ust)el3enA0o5=bG99BgxzwmZFj&`fBlZBEy>sPCAMffhdD$D=G<d->#{9;TP1d+e>>gtlBCH^f8xhi5LT0kdcZj&nX3Cle^sZO+(nX_b%T4Z<nMZDR7YLx(_v{P_fe)s&iaM?9?uN4urH%VoB1&6{ZqisL%2>YKZYFytYHqHFLDDR7N}IXWRbR`VEg&mYyH(~HmiJSY@ARB$3KHGxjY=|Kq7og<*(SUzWG!AQJgTtKvk(_B7)U~uLvVFypCqNLdQmZk7}?F#VRrpYe$WtLjaI{$zYhOfBuR2F&_X#3z(w?TyD;vxMk&;5UyaoF)H4p+ViBD_usH>k*m8zkPSWX_C1W^^rqw}xnYU2JDxuiGtnzlIjeWK0vwy1E%7Y`0-c_y8Su8cvw5K(8Fj`kc7gft$XIfr05c`8#;i^F`H%x!q+qW+=|w8M8E*jZpW!KqH*`5p$jr-Y3>zhT*f0X;tXVN&*!L-BV}PRZVa8fRt2Si(95#QRc+4dgbd}=7G(sVJ*Y4U|c1&F-7gGT%gKlZ%vI+pZh-AjCs9JiVgzRqb*f-0m4M&6s_O!MqD_Nb{JI0g>H@8xMt5`_tq+EvN7enFS4+BrZ?<-yRTzAb6gxhG2YO+Ky#tK_iq5{uG?;NxnKn7%&Lnubz}7tmL3qYm;185RgEinoi!++2Vlh4;?yn$JT}mtk;ezBG8GSv^sFD5g0P2%EBDMG(bVD=x6-7kyaLr=)-V6`y-k-Ubcv)gGw?IYn%LZu1u-e5k|}#(_TL#g!b}(jcZ}tD;n5Et{*Jp3`uLN%7OPhB2$XT+%FL)X!beu?&?$-9(r4Njy~YggFU|`3%Xv<H=x&3-*cqDdy71|GGrb?EAyipJ)5Kd)3RLn^V<6K2mRs)Yn{VIzBWrhWNt?Zerap|g+i)7F0CK%RX_hG2SJ4mPwS6KyYCZz9-_yXDG4q1Xq3;TcxurG0Lly2zzVj6iUHHy2vhH@Zp$2!w@<EvjCJ*Ee#+GbwCoHaZe8V5-0?p}5^d1Q=6h0!75*HzK;FfRek+^469MpeW2#sJ^MWdM?9f(IXoNIv07>wA{o9K0#?Ef*|8(LPq@65f~J~JJ6a$W7!IK`aTi@YAcwy(`KD#d!|LJZsqm;HLE_HlIyPH#V0SY}vf<eZP?@esp&x40xvZaGl1xNL$3&ijpGH8(*?7%0A1%~>5|HD?rmgOBgwaUx<Mz(|6kv}nSmHFXIHM_F0nn3(yvI!(oZbpIPbEcN!r%2E!Y{ze-XtD)L30T}sRVSk~Wf>L`GlNe2&@BmITO_VYqG4fHUv9K5O@Z%H`UHhc8Mc0;_q-m9&nAnUsdo{u5I=QCZQPNpHC(WIBS)aHV2Q>(U8(WCtiDE>c4lq1Zzxihb4#mET@$|~5Y>>5NM1T2+G9VG7@O`pnSI|T38+Hw5Mpe@45HQawVr5)VT4TI=L-FzuQLU^Fb)%`N@V<-zr7Wj9q|-HSO#>#RO*T7ZNY(j)b5@9K#DOc`NQZiJrAys!TI3ttrE^bXFd3u!+QvE(xBs+ycH7bk9`s{TQRDq__2||#W~jANy9^5@+q%O#T{`w+3Kfj;vIMtKl5U?Bw%4@nP0jz!(r+Rpy<7DAwL&ngHfkby)8Zs52;5#Bl!Xv)^jr@Whq_d8;;s}8;f7f}1|B5dS52bjVj0ZE;0Pjp<?0OkHMLe#j5YT5u^n{(hiAy_PRzZVD!PjHm++^L@j}ry%A=MyAoOv0Y3a4H+z#t>@G@cAoo?>`$eo+zcJc%SzlRVe9i7^MQSdP^H5UG%$-`3?o94>z#|FA?OJ{ocz>GSj5@TCv=&6!6dp|FgO$!T8<tz?k+Cn2xRjp?Qd`)r%Jn=|gVLjUJkF{SG_N$y44cumr-@qsScP^A;jbL;*X|AF-`UNPHi=w2G6EoY;L<z;HO)n*?VgCYJD@6yd)f2qKx-HN`@kQ13r3Mo&w>2uu;g9)Fcf{yk@;=cv!V3C$ecPhn;#}>1AKKI6oEx#;n;m+vWH6$ud4DO2{zpB`?I`GD(E)_Ms*3&<6ZAOszi1M@t}i;Q+513WK&LdDIwO>eI8Mt=9QQ(xm#Fj23#7PbEM4lF2mMRmd_Civh&^+^we^Cov9^A*1kOML?F$>2SUg!DCyGjc>&<3-nLFOhj~bRwW9ZP$TFO)408P4ehEcAK;V6in$5}s*kA@|2R*f@V&2*rxN@&b*``zQv27@w`Qb{K;^@Y^+1?4{&3;l(78j=Wer85_+T4xe{-Fdqs7ZyQOxhc$DNN80-p!Gt}r@D$<H=`+a-_fk9SF8S-4C)IsDoH+SE;>8noKPW>R{e=f1UYa|KBHWof^t-k=l$dJ{_(}i1eQ~mIa__{^N_eIj$E#)hpK~FeCZ@c`nidIzujT%TDK37fLdlHr&=A*>ieX~wtU@K@4g+;v~)i{-^j~|@ue~z>Kkr5Xf7SQhL_1BElDUH#&Wt=eZ|}L@z|ZZr5;DDEXDwteX(bcvp>u_bs|%nkKf;ARsgTDn>%y5X{65a?P2{r7U+A%&y@pHbE+pS!hb-@3!>{f0|Scx-iVKHq7+=H#@DTQ#*21w&c*k%9nT^po(k`0eb}&~ix++$0+T%BC9e{zsE3BPA~+5C?o0J=yWk-|d}$T04nIv8#BtTHQfu<inLyU5p|Djsp9!4S)Pa4z=@hSf=xP7TS<VM&*eW$X{09Q=Y*;Kezh601;GwHT%70>0pfQ>XFL)R*E*z?nu%|gZ$V!v!##<ILtfoUl8OL-sEsoJ=h+~xr-E*$xe*u&jlwb\",\"sha256\":\"2548480302b175aa15700888902835c91c377deec935a4edfe2a6a978e3ac232\"},\"efficientnet_b3_public_repro_v1_infer.py\":{\"b85\":\"c-q}r>u=jQ694YMf~&&;Qfn(|(iVFGR3N+C?iSZ8f=&1CgD)UtiFUZwYeg!FYxsY^c~B3_PTK9=rwfwEB8T(h@HaE`U%oqOOntJ@#fd8JM76QEWpO_7Jnx;pQ?q56r9u}=RjVRVVp-P0-YQWwi%cit?Q*FTtqS{5+1U^0qE=O1rcGjXS%@l5*6~V(z-lsC)@3fFTsF3;m6Sr~RasjR7e#4fBsY^uJ65l%xHhVNPs+lo2b<|dJHihb2pg-jcJy~s7VT%NbHy^N*xmwVEAR?lJ4r>8R~r!<QB>_v6&EQCz;BgKCX-YxMOMbC%)zWo`N-{pm;@IS0Y9cviHJrTg*Ylvyj<!`$*fGsM&DD*WudHO_epY>HzgedVyBnPQ5*5GER+Cu<9moYsJb0y(fCux+9>gBQ=q|b>$<G{rI(ObTZ^~IedjTMHserX9%q$NppXBYO3cP=3>&;Lj@rygwIa^d4RGYa&M8jiP(6SvocH*_Y-<IPP_(Pq)AzNuN}6VoYh%z~y}PHP(W#jV3(=u0e5h)bY6>(Hh}oNgU+VJSfIjmZdc%l?r4Yl!p?BR~oHYsw2b@CS?HgPBKpIRTEW%}zWqzg#-wB4%)9}?)q;^xOXbDEzAn3OPM2B%zsUr2a-nDJg&HKXG7>>?w?|<?xP=%>@_6+6upLOMT=HzkZ6St{TEC_-fGp^F`r`X2tYH+lV5~gq@vL-J~G(VI9>>bmaAhe}S%pJLL3LZ$+L-a1r3~T{AJdbtZkK9LRuLi%g=P<miS4|Fw{EBA$RGFmK6*?@EGA$D+gTcx$PE#4XCB8SC@fL$(<TBpOTK&;z@O$*BZq$>jtul57Mttjf7}Lu9Hym^hC^oCh(taz4<W?o?s?>!w`z#8eGSRC)!>ep;)!138>u=R&(WDR_v)i(%J9S*>K}^I=W~1-J)2}pWrtVa|Y1zO)&*bdYlRV-qEAMAX)yzy@t}#AhI)ssDYzaBSf*lUE&`WqwnGrpZmGL2t_fTyJ)kqKy5l2qlFR)M0`2`Pj*cF0II1sv6Nd`R>;7}O>mDYtxlg6PoGQNv-7B4_j0BI~G3X(~NNia;3VNZg!_w&_ni0bX`IfGMmrxP@YMe?ifp=b}ad0Z0(r(1X+ymeh(Nx~{%=69O#`UUO!i`U4JMTATl2`38e8pwjMkxVF)xQZ7#({|&Z2I37c{~nOBM?%trmpUml#RUH-Q<e2@&*QbiT~nD%ve>j<rx60XVyRc~0c#OP_Nn{p5FQZ@A)Yp1aNK(5_k*2VF<Klp0{nJaIwJQ?#l6z2TWe%lWSbT?TW+LEi}b+L@7Yrh@sI#`%w7b8^uU(%hJV6v`;i9yXU}>TbB`&#n|^KXd`7<Wc|Ke~iM0Uao({~5KGHxtC6n8xSVyA;JUZe|4v~;bAPG~38fruo9(WznBg_q8reCihb@`Zee2$XQeCIHF8@r{O<=4I*0(X+T1Myu%Z*^R(l>aIS#z*U)m^s@jNIgImVpS^C0{~05ap7|shA*dL1&4J@18rplk}%D0v5E{+C=88%2SC$m19k%-$hF!SHl4!GZ72kuvtdU{zIZ}0ii}}2x|Jf;CSKL5#k?+7-2D9XhQSUKB7$KW)(W-@^E*X7j^i6I=d^c>j%RvGgcDdgcQFYy#0}VAL468wkivGx#m0Zi)gEbtRXRj7*wL<qNP*f!y#L8KlU8-vRInkD*tRv+1rLTvS%HwYSex=F`jxc}>3y;BdnI{8+_~Hlf0b8u!zT9yQ&%c=26)%RbH8ZvE{oWfIVhgVt1K=sA9?RGP)1&>!e|T45DmpvUN;rgj8vg|#MK>BtS=E<#@>t#!YbC4cs4R^tl#IgkP|u_xH`<&@E;HfdM(RgHihT~dReXq2d*!WCE;E)W+YC8*As<U`*0@iD_vlWn+H!95B!RIvzxj<b}F1{X<$c{;S7D}u-bI783GCTsU^Z=#1u|ls7EbrJLbI*|Fwx5_nXSnX<MT!=*d(K(SzY(24QFGP5(k{v(lO#ODmnMrL%>md>3wn#}rf)A3>)euDbeg`SI=b0HV3$fr5`}a<lI9%^nFV?u0qTN#?$W9f0+8<NobXY>R@>dwf=PX?%Zg2b{-=RvP_T9T{?+3#Q%kvyp8bAfs&uvi3wR$})RoH6C_dH0ogTV3)dX3y-1dw~JB@1S9mDJmnu<)}0cz4vTOIZgl0zz^~Is*B-j-OQd=H(1#H@;^>M)zo>^qRaU&E2i||g2FH%OWAD(-m);hv*qv;9lXf2V)ObPQ1C!wYHeiz5(x{@3i>A<UwsPhv0UMAtHQ*d3pz1*8p)dmlZ$@C_7R*^fhr$Cx#SE%}moNVoKro$#XJ@Y`PkY$$J`_6k4IJ?=y9O75nm`T!nXG+cG6Ev*J8)*4g`esviW18Cq)f3TaFOO6K{#n4v6XdK4%6iiU5WXfQtW_WelZpBi`aD&%;|Bg%4^Mff-_EkfkRc73)fq895~tr!2Rj+*Pq{hx(=qUbBP8$Py2JPAvyB;qLix|s{JD#L@4LtS{JTPonP!Fx|`!?7@}dXsXD+yk{mY{`U*(-+)}kSM_peQJi7kTu8exC{#G<Bnk;j#e(;rO)%gb$xp-8CtX%0f$wJpL44dh2r;qa=ZshZJCyid^rA~cj&1caMf)KV3YEAP;qts{RpYC&{D=~ETkmbpKITC~%VMmV;1hh6w=<T7fAZ7u!>0f`KiWx5+lu+I{YP;HM4)~4eO@bzWDnGb{;20c-|09xy7nKJG<lw$?(y>eNjyd8IOgtAaCd1gnxxbD(YDxD!7>uWQIn4|rDyc>V8%5aoLU&ZUwzs<bI`3cZj<Ml)?&=^I2IV9`xhpdf%t@<fy;9?dp!qk^BevnCM=$3MQ}Lylq)Rhkeg69N^k}g+Kz&&H;`Q;fHKc-e@i6hI40!EV+i%4ABRDiF%Fjn0!|RDd)bqCJ1*kbXFYS7(+0JO&yWxaQ4$qc5;T_?|#zfnJWjpS;qwSu8I~U^-3Fgni&O5BKmRVzNxe1!=W7*_?8d|xsF}6K@e~33AjqxsocXfGv?X?)<IwZbf!x{On_n&@|?=C-l_~G)$zh4h*@UZp8w0qG9;b4|I!KCZK2J41(ArS1n59sazy`W45)K8J@=t&p~>~}>1ki{9w9@Hm2<=$R(w0oli>fy!qg_vVkPv2!dc_()>U_cQ9OX<nu+;;j$T3Y%VQsRDvcuV62AqN2^zQMf>Qt;Oug^kV*Z*HCo@82za^YQrl#rVw$#DkRhwU?BXm8Hb*QKa<veDaqnlm7q+e-&~\",\"sha256\":\"42a53bb80a7cece4d06a5e4ddc1f887e900a3adc4991e51ebfbbec58476438f5\"},\"efficientnet_b3_public_repro_v2_anatomy.py\":{\"b85\":\"c-pNzYm?hHlHc_!u*?_Irs$cmlgnwN)H;cia-HLK#m?n}PQ_3nG~y0LGJI%8an1hr>jwaWBs3nUb}Q=<L7>rSH2MvG@waDPQ$O2e#WPtPMRjWTW$}FGdEUz+Y0JZDp4=w26t8x>Y@5lVeJk7fchAMouiuEaPO>5^uEnmb#V_yQz7+pcNEyv$ANE-z4rSWqQlv7^HnL7ynV&>0yGAym-Aj>lY1YbA952NWufPB2&mt*OktZ9Oiz=&NP$9+bKFj57yHAR1oTe-QM7yu0<f)Slz)zZ{J5+5}76><K5#YA$@>CRME96~O%cfb(W>17nwtLa!*;b03Oxg}sRm-i+j(}0tHX_;n(Pd4BpypMc!2F0(QPzhf&;A5dn>N|rh^pj1e@Bpam8{!2p`B9?SQUUniZ#ih!nShP%0kGZ?5_8sEk#!B@=g|jg*?joBxF@?_XunUOP;qKQ55J0c(--7MH;~<BpGx_a?#$FgWW*1u4O}Hm!j<2s%yn|FSj>UnHBJ}EPz_u762ZAXbm(Rs-(@3c(+-*C(uANz)RMISKSqbtixGVm&Yu{`EfswS9(j~q3NnB&lt7ef1l6!&F{Yp*nl2Tn;n21fR|>oU0ohT9Pc`$dmM}GP?dEH6oy?&X!m9_-Bw>$N!>{Oyib}v@Q{AwKhO|$ZI<iSe>G*He;<-|uYcFDhvh*(HK&F#NfX%19C!!!UQOGiK<%)96#~V854Mtr0L>5$-TfU-LND4=MO()0KPMGFhT@;Zlt~_n_wo<gaI?Y!IQ9e+ilWj@72pXC@L!eAX0uf8M4XpN8gG(D#!Oy6Sj+_c4>1u-!oUZT!WoI<Ai}+ji{v1EPipokWE*du$5pohwvS;sbr~Noqw3@Z1S#9mDM5fy)(|K?3H}cbStG@d(E7G)e=NHqeO1?G?e9G6^ro_sMGEUXA!8()2IgIej|lT#O@mkg!d~4JNqhq`A_ffIxWzp<kH$8Y+``!5u1DA!BlUUtdATJf@^Pk+)?*e(006VMC7fswRQMbrX=SaO$1DD&E0A^dVz<PlhEH5eA1zoNlnL70&WIx(KK{c)fHl?YW`(O*!)SgsWKdW~@(vcSSc3mBh+4woD)gw??7NrmUjY;cAI(0z{N<-tAKt@T>=dcT*}FF{-@Y=f{51RZ^$)rQe$GCi=>5JdcG<NV-Z#x?GB`=F!eg0twDHtw@?r*}4m1Z)0%dZX74Q;{qw4N_Icr)FmPykB$2dowdq1nn5`=MlNVdRcNw?Jl#_d41e;5hi!$$oni{*#90}@k{_@I>DyAt4^m~%kFeZMaW58S5=P9F+JkeH}JK)%)z)&eS_g?~XyNRV%=5J`Yo<cmmHB1~2-F>4Xv7?9w=;0v(>I<|oQMRa*Z9lL_DgdN`AY=E2N4IGD?W+71eF9r9yu9FlcLcGnhDn_N&gZeVM&|E9?c$XZq92gq351!evEJ@fbh@n$_yz;cH(zZa%9N%a$qIi;M?dYV2SK0O^Ze$I*6ZWlyH~g>wX&C(g+wo&f2OAVS3Kw43^T1I57oG)A_%6Fc66Yw^;Dj@}*%lNh>GX)H`+;>{BXhZhO+kpTM3UFAD@aL@D>;ZWambnm#XDOBq%aeZ2C%0BY?)n`^YJ8fBLs{Z5C&Et&#F3uZ6@<5DHJMTP+b=#awyX~6dTajw`8#-Eim4m0HRX9txGepJZxl|lAbAmv0$cmU?9l)h_9;lL@NeeRy3_lVEt%kr1DPuDeE#Mh(Mf>`4J3=B)?5gV53REt4Xdim^erB)~vCxTXL5H(O?qb=oXBm{1j<&Ytaf+^Jz;0@wcVOW#O~Pn5dBdO~B;Q;uMZW4H_{6(7@%u*Ff=)*5Lcert#vGY~?+JJ=P}}c?a2E@ek~JOrsPJ4P@g=k6uxiHSG=00d!>q)`}?)$&HLr7Bmu%4a$R_<G$;%Jk_QXntdmsx-yS28@d+~X%fbv`VUxeyVnmtC^$>uJTN$rMUk@5U!sAImiK!^Lyh*;Uu!GqoPt2i+BgQ<<U25M%3CmVm`))mD=;MEx-45{1r$f2vGK8&Of6_?D|QjW09^4;ge=*{#Cd)S{I4YEWDL&;qdpGa(fhXcy9{OmkyUn8v@NTXAJ9HQbI7f@S?b8q^XBA(Y62XANXs9Syctb;KSiNmtTv_k76-;#t?>BAw1aVmmSO`s#n;cy)f4yP=^>)dMq31DLG93p-inFwtQgWyMqkSo?307pW7>@|pz>C4F(6`KMHy(06%AO2_PPgyVH<D|>=TSx0o~WbHIhUylw!iLHg3os8&FHaUg@4k@Uv3Ya{%#(Zdt6z=1*7#2}vg?LTewNE<$nj<jM0(F&CGi@Grw4h>$3L5S&5bVgPlxpqh5Is3LYIBA|29<!ub@e!rnl#btbPaRL8-E}kgq`_w{x2$O~Oa-iR)E8C)>R5Z)g<PC|T_jCkagUnCBy?~wYLU+fPMh?aAP!65z_hSy58^4zYy12*l-I6wb4;~=8)}S*T-PR9~bGt9WN5Hi^w7wAz5%953C!R<ZgtmSe0wmE2^6QW^Hz-tAz`Uy=1}Hw%Ovt_<J{XI|l$myej$;R5I&TE39=rhG*@e8HDnc#X_^d1S!BY~b$^Mj%CRi_wBMa^MRp5l}ZY3gUoXpf}W|T~UdmM%dvZ9ldm&S^p81U{TkEQD~5*W`cs)pwf7*XY+ZinQqXB`w6)82tYP-ti24C-gR2On9F*BmG#G(@T3rsP<_I6INhar!up$%03$O^2J7YQt6v+&Y2<ohs=+#7<+B-TdB90kF`I@O8@*q=t6c{EO)8FGFz|U497*08xAyd;**Sa~94yLSuK9%3vdW15VMGi?io&GT&+u3jA*|zg#Oj;{mdo#m-@x7cc%h0G_^xuCBhCJuDa0uqo*ANd-Cc&_Pwoe`+y#aHPD2a2_m=j#C4#SwT|dfx=HXBLy^lMmJE@KykMY9FP#*2jS3q0@u(ssX>~X!Qg6vq2@k<E3_$gYQ$pkk$>EKY^n1Tif<LM07;)BcP|xM%;3}G;y~(FX4m`ALy9aJ9`_j3oUk0|hM&UserEeH51IY8>%#85+i^)0zH@Ln+3}@_J`FkJz--I12IC4G+OY%EW`~3EVTywNn9p}#00F-!qMtdU^`PIPLmkU;c)|;<(D(8!`Mf4BvfX#Z&GPbU?8S88GWtPe=5UzoLm0|<A2vQP3i>0BqHQgKe7rGJPSd@o>;Q%qky57Y+=LFz7Gs*ESzb=x!7+_htb2JeeSMvm8?Xy85VrJ^<Mqgz8!ih(ZcpEZ!+FDa$5I_BG#GsJ6Z#<mT4|Mu*Dk$n)xn<enQCIp@VNXp9dBqdW$XGhQ>dfu0Yxl4YD5#I^~giS$1|!-AxeK&_Q3VoMSaAj+{jZ9sl0&iaXdAG6?h)%xrI1lq=N(k^bFOb#)W(kI0g5UIWXF%f*J$PiF(X@CrLc8hmLUra3dwq%xpvxwZKWQb=2&WO0F)~V!0I0-BkKQyzdU8PAJ;d0)KJ9s8W$9PUK;Ps4T#m0Zjwvr%PqOn$779TpZYQH3*x$m31SMT}v}bj8@H&qYIEnuTh6#rc3AIh<Bp%8|zwczu7KYbO%4p4sd#4+hm15r;&>RFTn0Ta<g%l;5D{ux=i<<HY1i8A#;aC@P9O?-*#Y5_%s*qA)G00)8JRoQSI*;WB(H<pR=~CNipE(m~S%j-f3%Et95gpGY@0LDeGzmg8|w@(r)()IG`3>or)QnP!0I)lQTw+jdjqJY~l3l8Ad?clb}T^KpX=R8q6=L=C07>p*MeJnapg~`WQ5>Z0Ou_HXA(PeU_%=T61j4UL<M(-4QtE`PVO|`id;UxDPdhp<hI=^rLS%;Gvn+q4so!Jg(~UUy||`4C)OW_N4Y-gci!1ENNUKAG)JDj`+;8;vXZZ4lr(2y1a-kE*}G8ikt(?bWva^74dPfjxpVXGo!k{G^*^{!8q>CVGw0v29B|8*C{KN0lZ5vx&)F{Mad|j;@fbn?K}?>cghWrV+;etTwzc0a`%{?aMR-xD?4O&U^rZ7XI7;Q_&NBCftwHRl4B<8SbN?6lwPA?Dh9KSkw;|~%KoHE43%eFUX1KXFpht$rTkNpA~RN<C@Yg-s$~gYpQ&lV>Ju*RP_-P|zP9CcrPs^BZ~JVUSB)P`1&w(d>9k9X*}&L$oH~19N{rY|6Jv1D|C^<_SU7|(dM$%y13%B2v7CC$MBfstsGg1)$2ljr4)l;5vOzoLqYeJvsxDF!Qkpj5ER~w}eojBk{(7H?(cl*H2-rS`v6|b{F4ohPO!l3+eqz5L-1Dh6!#Sux1Hk)i9N`&7n|4!Nd5#J4<RmdS4Ce+URXyCmP;d<f`d&`8$)mG|Do@S_S0_uSagSK~BbIN*{R35OwuP?yG95acMo(;#CTn~ZWQ2D|_@SZ>BMkJA-T?x4r9wvmgc8QV@Sh=|<pAE2p}N9843GmYlV!UZT%!{{_%>?(=)eK@19vwyZ&vtNRTWj4cw&Y=IAjYN;ZZ{zO=y45FUNl9V1Ix;Zj(CIIWCHFaRy+tDI2t%LX#uV<X650YGI}PZ?lxjsu-5ToCV%xC})%14<x;fOEk=MTUp^yT*HC*$Wx51*IiDP8VWbyXT;8@wIOvGttX#a-6#cBY|E5f&QY?2<6KtoVwf+Z;(+4hKUUFvj`o^8EI@((Yl^YeNde0Ef^7yQP7t~RHIBW}0UR8(Vt5M$_o)lS+9kt=6gnfos5X$!0@i*tkyVqaO3C_`nLA0qJEUX8efP1*B}_vsNQuGnz<SR@=fS5k7*h|6$ILgF#AkvAPS3+gylv9@3Q9=F_#zl`>+~ef$L2O0`B~aSO?bQYNF_S;<PkqRa)L1?wd9@g_(tx7Z)OTs94b`tZ}_o3C)o+mIqfh)$!?fRb^~LK<<vv*jr>%I8gxg9S{q`1J{=L`>15_lfpBl3C2!`JQ*jleFvcQA`11(>BL=umI1;Ap`lQ(VmJ8<ht_lFq`sw;mW~tS!^UruReZYG;bhdn&Hi!^kaN4o{YbxUYZB}NdOOOC^=uZbxQ|{VBa>sF{@y?gIwL%GILU-`N+-k#3v73LpX?I|L8BEVP+ol7%Mee-jXo_|hwlo7wz-WXHzcJ9V&~p7A5Y3p3H7JaHy?!75W0dWw8c{@Qh#I(whOz^`F#e;{!1BR56LTDmY&9Y6Q-b@%TIz#2nV(<3(Up;uiy2gNQsE3r3suy~wN)yE(r0SkQ)h5HI2B_o2!>@wQ@RF&1!>l7lbQlIRU^B!<eDt2UJotq3s~ki{orrVDt(|_&vVzIibL15Vk1?M3~MH3dkd<E3J%FcK%XeEN?BEn*viWqMGkKrwPl}01ikikptIGK0OjIw)f*g(Sgng4Q*Nf%Z4iXi<o}x+1J!C49%~%HMa3Xp4{G4Eo?E`U1VcnX`kMO1O$j!mf{lx2sQrZyT=9Sk@uADx4DU{$2W80lq{rx)@D(rWw*kFYD<fDcU&5LATdXhkHIXLMklzuEs{p=w!H}+HftPd2+MJo6bc>HCt?0l(MfUVfxf&0gm^WDds12Ln2tS$rLaBLX<eCqRIs!1Avr{h^CW&UXXv<%~m?pz(mS_l4JzW1<rD+auk|~PYzcH&`nqT~KP%NrY4XtpzJ`S|6i=H?84!nzf2YwDG5c&=BR3)}Qc@vC=Q8Zcg(#&B-8*br2_e%b-sfy<8uFO@H60v98VuT~K%9r`KRRzj*!*ru4PBTR!1jD~9Ls=W9{6ZCv@?{mOlV1+Y=F!_!f^J*`i}_<rX1}4k4(Aied|as>kJwYy1*uXzL{f<lQ?~bDH|92*-_P|_nf6fpOv&=4vrag4q~K@0Z^iPT>m617EY~vEcr}2xieC8kRx#-nbp!h}h5APfj#Ppab7`H~(7_`lX1z^<8xy(3ecx?Jl}jvl{4WqsJyRHB+6Rg!RF`g+*8cW5>AGc`Ko1or(}5O_P3qAgW0OkBma1Rh()Uu-S!z-aq|0qt-+*JP$E!z+YJlrO9?JSOk}yC;a179hI0Hj`D=usjF*7`hL_Oa;Od_rocr=ZyfJ?LQcDr1Ts|{?5pb>Uyfa^s)dnT>|ds*xV8Rf!{zPIht#3N6d-9~uC3uwtP$#Ud%Ki~>Est+(~oJU8f$S?-OP#(Oz<^A;TSH4V13OiM>wq>mjCtybB*f$)aeQK9<6Qf=#$^d4JZ#$tHa#2umFJ>yh8o)X9w83cYJUXW8FIfAx;!;;e>k%xt%chBUDx3D*_pg2f)9(#gqi^7i4-4$JTvt)_<`-CntZ<bxeW;?J>Lh(n&1x&S=RU(5N~b;|)WF)+O}3s5R~UCugiu$H@;X?96c{$r#n#?7e+mPYkU4i{$$&K-hJa181)4F(+{yu%7$jjrw^jNaM(;4sbQp<-hr%~;elveAlbRF_-p>jsXAewI*Oz@w3@fMvn>d^fxN;11F)EJN8yY#XOXIOaPsJ?K>?O5CFVp1k+tj04^)a_e1dcJVBX1PmbO-Ppm27#v9FS_-gRQkVmD}WGM4#36P?gbm3&&(Jow2A1%*~!KR$9Sy;<XYCE5L{(QK^W+-W3y;RF^TwX@+4L5<x(tMR#WvgBj;2{tkb>g`ekgD3JtFZsCN-TQ<XhtSTQh4vckELBDFtHpvHXDA`28VOX54d`D}s!sd8y1xKuO$>`dWYN38dMWE}kyfb>Aq6NB~tB0uSKb;Uw4bB;uUx-({a<Y^5$&u5l4Q8yaL6_;t=#pKI*W+c5eQ3&NSR^!}<x}<}m17YdfIkLd1@W!9#DX*F*a-WU`aX$s0^Sh?A3YN_^vrpt1UelxAB6%s#aP@p&DHUJeHP<gTW-OrQv%#k2YdOIGc|EbrTnBve6B<sWKTjxx@p`P2caTB5KqK_7~r%jA~J2j;yqL`5)^&UA&E%sMD)i;kO3VzO6E9wb_M;2eM7Dh)6t6I1ga=I-fyL3UqMO!siX_FtvkIF>qfPe)w#(vj#Y{pV)%|vvks=SlZI`e?D3i>lBw!^yHrByU+0YyN8LB6G4TIbojJ;`)xmEX<0OJNge81F<K`~b?uG48J9`VhX*JeS4j2R7U?v|bSI4k&Lv@8NA9r+gpJ5=*9~?KVzYNdZ`cnaB<^@aWc3Ils9mlQ(bD%*jV6G3DD@4|rE7T#i&ecHD+BW>+KYVaae3j4aUy;?p_!?gW?p}+WrbggJg3)C@qK#(Au{h_r%vc5@8TE)^7?#(-i-cj!|3fIFy}*kmWw8Z*!NPN&-tyHbyP0|`(;F}Hhv<IAsv`f)MStI(DHugMqQedR>!UATv<<s<Lf*mgjLREKE|=wQe24Gjudjcg%NWglU<^(bMyE@Gp><WDH!gMEQ@`;5d$sdW@eJ-7ESW?YB)6fiN8>8+1xDj&rS5Ss21Bvi=IHlCfJ3(P&|Un9|J>_CiG87?r*4+owR@>-TD&2^*_HE<sO%-GTF(l-G@#jgi8qvX9`QU#0o!`E3Zp|uZ}lAm7^I}V-8vdzYIqi72XEAo-$FP3Sd4NS$Mc(L70=vv=jU;E6?ia0j)OR(PDnTuq=Pt%c>{FNUlr&%9J=#qc({LmRMeopVKh>yC+MPnjo#y3SJVsFRoCA*a=5o=sBbyiakEdZzWfSj;*bAC+;INVgf$qP7yRDIsg;w0f@m-A(hMZ#0E_$3DlIV_><9>3Z_Py;uyKG^$A$Uc&q{ypXVtXzdaXRrK+M0js-5TqN9f1tj~zuX>+239)$iyHi0o!tX9RK?$7#8Z<6fMYa;@2r=jL=S`$i59ja#cM2Xm{q<}5`s7|_b2EF3TL>@IV)^BJGR8CR{A0OK`ANt(uqTAaW0=JSt;{yoSK(tCLIJ~Z(eOSZa(!)VfYa}0FZ5FQ+&d1T)6$fL*7xl^4peG~IcUlqNbV@;>Wfd~x6&{tn#klH#R17lE;D=72_;WWfZ%2Y9KPk98Rwa+_IMxzl;2^;{*2VEhZFTg7`$AMD>L;kLy;k5>YKnbg)#epD~kC#HZI{X0=oNSlTTk+znd0Ee!vj5B*D*)wFnXd=v%P`hgY|zHGJf`K9b^=%c!k}?Fz<k$=yx-ox2dbzK2mlj@!*+H3cltszc+C7d9!5Owen)IRc(qo^E?`+g4Rk$;7gUw>nJE=pX~_Vj6|DOliwW#+@YbFF*3}lT%M?x5^yil+SY50`{qw5-`Mm%6Vr^n%D$%53DqvHUwI#85`W5SqwMHXpz!WQD`ep{0_c_KeQpz@{OhX@nqsBq-xM$wz^DQNYu8oWl5U>weEva5-%!@-(WIGVSg9|dB*2$ltk3E&PLzkU#3Bq%^KXUCAmF<+a%16(J3YSzLfgWrOM;g>yAB#q6{rwb-Do^Yi2Br`h!Hn?Z9YYV)fu8)s2R}&}6sGs*Ba#)cc{n*;i=(43(bn``Ia?bK6!7%qSc=%QzE}1*Yk<Dt!rz%P-^6oT0%xoM?-P@aKDT#vSvvhng}Dw!*IXRoCkwqGx~(%XWU&mZUk|>DQt*qghL*eYc_=`b@@H=7RuLaC@l<&KS9sRxuWIDZ&x2yJEG$D+CK7d-m0{^?`E^I+Bm47u)M$PFVf@NruuZ?Rp;V*f9F9>yrtE_Cawu$bfWKPh*7k$19bVu?g&y~B1`_WK4m)KEn7S8W+0{iTKH6jPYcuz2&)x#dv;zMowM-{hUVeCPhi0-;0GFd+{D|C+m*f2X;N{exIJZgRZ2A?Ba8%WX(zI_gdBE8$!yDXa_Qf$)+<0+}hI1UN<lTWo^ZyTToK<Q\",\"sha256\":\"17a8a8c99afc02e6e75928cf7db2d014609d31d1280a1f05b07f15564c198696\"},\"efficientnet_b3_public_repro_v4_t4.py\":{\"b85\":\"c-pO6ZFAhXk^Vlv0%1NJX>(|4B)fK1nN%rR`OcL@tCD>AqD-+kBu0aXLvjgF(yYt>{hn@sZ!?s#TXia1A^~)x(eK^hAOG;7(e;P5Dn7{KNmP5YEsL|6=Xqbx=j*a5GMR;<DCe78RpM&1QK^!Jxt3=Bmos7NL=~!d5Sy|Vw|Cc<;%|kN(QNj0tF*wsq!0=yDv4i(5w)!9GHX(!%0iSIVYX6K%{o`9h<-dd!+feL842;lh({?aO)F+elPM!J@w5=TGLt#5W`wG&%{))m7)Z+kq@+fNV%-?L*1%){{agyQqlF1}S4GvBStd=QaxF5Y)4bHcyWV4)#Nd=huFLZA0#C(U|I#G26dJ@y90ZS5l4Z19;<OS3X!WZMXGBa<ifvh|UqMEa3uYA#FP~&BlB&uTpiC+9#K=0ymD$gAg_DAp!X)*BG#W>~oatSX=KxtO&Z`&kL1^_d=k4c}tmjpp6d)(v%B;zy*d>o3Yongzd{gEbo{A)&uk$i}oZ(r{O=Be^6Rp+&P=!{R6n{$|9&-6#S`@TsTQ)V=mC1Qp?kZ^%OJ?0<ASxm)n9VkIxf5}`X-regIClJm48X30mh0K99jhOzq}H;1-zIvSt9AQo|6m|$jLO^5f9tYnpLdDbw$C*fTkhI7z1NnKEHO!%CtBn5?Y25osR>CX@+6gZT?JIws$FwKn0dnNE8>eA|GA<gBzY+A<S&gZ0N!p{G`ngK<f5qBp$ar%0RO6NHiIf{M4XpN7C$Zg;9@55L;H#)@8^SSDhJPk=vkR<TqHZ`d$P5mLYjDe7F+#~*))JBPKv~oyM0vcy?`;Q4QVDLj+ADU?Pl}?6Vy_C#^`lvK9j{>)pc3>8*gAYtqG=kw#v1}{ukmK;r#BlA?iT5SL-5)A7REZD1C)Sa~}5?Jq9O@2KZRU5sk*gcFPH9C##H4n}vL&?u?88#9fzAbGQ?FkBGUj2seCq=-Z|+YA4(CrmT~lR!p7#;LLC*WJ^b6+i^=c(c#lC3j#%S)B2uHu>!Px@7Um|j^qmtZ|8jb$D*huOsQz0X0v}TfTP!=*<UX2t}yb?MMO`tua~#~ef9M&6gPjryuQNU|NZilo$hAv+)m;V^Rutyng3cA8};DU_5*ejMsQO?s{6rwH#VzO`WS0jE2(3hm++;$>S%ZdF9UZ7Hj$%>;q2L6^5V&<nKoq#$6%mO6ELsM&dA@z?EYgmf4lr}bQZTEDYhbY>yCz7>xHp3@X7I`-20>%7i_uCyjo8LAr0hanhFB#R8bWMkhKX&iRl4GnyCk6{6NTDOFNb(T9TjkgAR~lCd~l@P{ULZ8T{Jw|CPNDJZl^gX_$ow9L20AU<owysmvRA6eo^M)t2}WbSDoortow@<c097FP5E}2f&k0=kAQ~@jpq503uh9k|Q#QVpVKpP5zJx0nQ@LVNvj$E-+Fw#A`%Bf)J%o3lP0$(n1jr|B-qh3cxYKh1aFHU(I@^z=eV-8tY`{vn%Zn8AK0xx%RzxQI_t!AVi|DUk>Fs@QtS4_b>F5(N>N~g0E$g`P$Shd*H(Msa$@o8ySq?n4<vAG)&+H+LqLHvX7vs`%3yn6>T6FbGDd*dbboj>m#Nn276^lQ@mTL(<GO7d09zF`!qTU$m`MH2^{a=!ttd*UXjYm0MCiTb8>_S9u{yT_?vBsP$NyPQpQo$(uQ;(^1L7rOT>eJ{CxfMU;pR!=YL$CdShk|8DpsfBcN(N4+4B{r^j>@8p&8nFe6&Nxbf>|x0W>sAcsFb!jUu&@R239uWXIo9O4H>1`aK3lk_w~C0~ev!v6|+ay)@2P4dP*2XAhBgsY8mhey04UH`kAoIIi0^uR|kul=b?t)|d_Azt6ONf-^~7?Zmk(AeU;U#!@LwdjXTqa-(%h2KsF;?GNQ_6jd2<uusP>PkBh9G?3BSyF?Ik8oM<6Cp=Oe&Bb?6LvrYH|*SM(DTw!a>akJc)T~&B&ulSFo$zK`0X?X#Y9FPIialG`A2xeZ@mko^kv{ZBXX2|Kq7ybC%bi)2qZwe3*nFGI$_Je#Qn+YnBAw{?CHepA{?u0i{%Z3Cles2t2cLn@0tKwtop$=B+_pi1tDEHABZn~^zV87*@}Y+J##Q&R;_|#pB+L})qeRo!QK{SUWQ_;EXs;pE98TaZ+8mCJ)CqXe*7^yS=sfxmK>ke?5(0m;*>lP?|Y{{q%}pA=sTS7l->FOl^;-yhS;x%Jsz?cSfd4oEwbzDgZ#h>z>%8by+2$(KU|MJKZr#K)RAc)?k$oFhwS|!^r}sdDC5Y+vE0T7@ZX~93EN8butjR}tTHZ`cwwZsN!6Puj%k;PKP>ISv59d(YdJqcH+FZfY-gtbD(h1FJ^_Xd4%TL*4@<j0KOaP$vVuR58|EVKICoT8u7%yG-FwW+o`8EG0uSl;`|k9N_rm^pZxO6oW=tj^7FX{;EYDjHHg-Tyi^$H~fDA%8LUzev5Q>8VJAy=@%-Q`d3CEY~uUEJ6<?YpFaA1HgAqHP-Lev?cqbd;Z#Ny*eDDA-L8m#<b!U{?4d9llLIkTr?U!w%&+}`gq*3ZbnGS?AtYXp_pOnre6wHN<Z#!&kOflIz1wN91E_tdDN?o5}ob~&m+;<s&IesU<M(^i}9at(4_eg-z`dEJ>N+dDYl{T-Mg*BYp0l!;^HviT+yb@_~_ZJm0Xve#v4$Q@CY7Ixh=S_N(%=su7{gacI49}+b<McuX+KdZ}XB8z1hyCa~!9fe9t5F3S^#6w*+a8dA!sA9;DF&x@acBYIY;`jC5N5L6#wmx*C?a%jP68gioosxcJg5tC-W9`H7g02m@6zj6g{q}wG^y~^qU74w~0};VZItd-{+vz&F(T?>;&=1r>le<pFER91%<#Q`y5Ypj_6mpJx23Uo|bB}-_1_FYd0CNw%uY!Z5qv7(-f?CGJZ4R+H;1&m4xAG~;RpwTXFg_`mQ|#F0PvpVIXWNOWUq^3<Q;T=+&Q9SFPvLS-!yt%=DSi;VLc+-i>-a!5OO-qfw*w>aIcaheW8CiriVz#0oSfkQ8GH^i5o0~+W1MnV8R~jeSwxolxu(9{(aBN@ScP8RAI-26x?>Y)s}@MY3#S)eaz9U@P>XNKVZFBQa}MFgckI5@)9cF;6W=uzN;R#yj5u1hdV`%{nM@7K@~D;L58OipP4?O5nOQ-|if?;N!9jSXLuu+U=to<~hiw8d>~y%DNP*dbSUM&OtIcTXYei7M^JMHB{`6s=t)n%*sE>OSLpbqkRsA$)+aG2_OH;SC+nKymFby{<7e)ryCS%8)pGicA9`*w}jSuWJdd>#nYg8Pt=B;I%1GXAOp5u6RFyS%UcHb-XztRC5X0V}f46P5H{78)ko@R0pd!NZJ<-}&eZ#bJyc2&;S<uYf7ZuOr!MN=eCD}BF3TSK%X?}!_1)73wTe)xeZhSSAS_3%2%4%Mr;W^Wr|N6Z|1o}d3nS=~vrSbRDLJTw%T`CE44;_q?N;z66e3r<ju=U9=ocVY9ph#UHRuhS>1_5i{CMJO(8w`4V893GMTCLTUdFa>Jw_&Q+9P*LblisGKa`o=+ATzs<+-#uH)+Bfo_uJgZYs~1O>Rjp%1ognI&E0Ets6eklF?zfOOiEDz~@ITS}ebj!R&bd)F{E-^{mPiOO=37xj$ufVEe2)xUr#_Ard=03K7Ugq$3nQqfjNCPv0mqS}opQdj_V!^^qi{8PIM`J*s_?<4r;F~^@xHxuM2TImR<1>b0{zc}u*X-bHRjsu)#$Q*00o7);Tb=ZI;|C>T*h%$rg0o}1vp>pK`gy!tsv*Aei|eXp7rC0qm{}ZpOfOj-m~iwfRRtKn3%}bi<%lQ8&frAo~hc64iaN~#t6l2oVdgJ8*e`UM&y6@LcwZdZ#Qc2Ow}?Q6gX4hfM)BTQxqIxcoSp|Q(lAh^K>iI$EsAtZ-*s~YZt3FV7#&x`5R{Bye#rPU=sc$VhN^}wM5a`NOwaC7{0QwSoja&(X2>qE!q|n?gG3Q_2AOa9~f01-j|ETc{sQ(>C1)Xr+$1`94r@u=G;0l>DD3YN1x03QPz5SK3r(o8mHT)cwC+iki!~(MbRZIgnhj{?I)UIJheE2$?N40a(+Gmf|*Zp{;{8VfNY%aS}vlKW6G9?7xS%>S}C?SrPD0#?oEeWmM@j8QGakmj5oED@kw<0af0HGL;MMcUY_-9?EU5@*{OVgL}px+)X|_T`KR+S&Cpj)$U8)&IL<@D&cpGzOXQ9xS|1QwlRgAvH!wZNo?tZP8iO_aVUUIrjIQ8lS9=9Fx&Y)8SP+W@Aao@jts4b#x~~$T{u<G%(f&X`XoLF`#7FnI=pWDe$MaQ}`d3;uS?N?HMrbPQ0Wa<L_a2-0swD%yT|E>rw?QDloGQVL0}m2Nd+HWF4|Z>TP0D<-o{>8H(e4qy={^|(*6iU>g1snh;_`9e@GSPol+3vUB*?j#P{;X*eY&lRhYgxtg7i*mGin9F10|-RjLEyXy}G%)y&4v_o~0kN_M02uz7Le%>l$X(WcsD{eYlj`9^M64mX^}tq_ZBkh8hd443v)Tq_>K6Lw$t-7@wQ7NJc#p8#Or^>1O?auT5OTEFe}gnMcSRAqO}k>eh^J{U*qhzxby}zwQ3r>fcWYf4N4Rf9vk5FWRrGsAs<T_=!iYF|GFosdNxTTltbH<iEzBu&eI_{O)@fi^)4eeef~UtD6ik7G>Oj5f_t(0$Nt*fvWL`1hM;&AhzGt#kP4oT5Vg1Q)=Goa715`c|r7CD}(qcUk3MKG!<n?*0tX*x7JV~6xhBceKugNjjn6Fe-+-)k8*SS_hF&?pGI#77K`f6dQ%(tV+#C(i$Kn8M+-TB9efYPgM+&MaB2`@+sx&-ed6*6$ybe?t+pwZ7B-0q#V~gYxrsiV8jZeCI}v|c4ru?=$oIWM!^Y)JV%NYEtvOY_N3r^C2urQR?<;R8j)4{>Pcj?$sG}mlpD@#Lf#IUT=|RnM!10O6UKuAQnUlxE6LJb?$IX@Tn020m3(!F>)G1oneJG@;8BM8k9P>9EaZJTW9J>!VxDp9w{|6yHEy(\",\"sha256\":\"825b82656eea4ab3dd53c103fd04ce9ead803451486d29f61febdfc73e3751ec\"},\"efficientnet_b3_public_repro_v5_warmstart288.py\":{\"b85\":\"c-pO6QE%Kf5`N!b!P7ld+iDftaeBE$H$c)PMRC0}NYeJfXAs;KcUNbLR7J`5*6@G7nIS1rTG>f}GZI@Ohr{7;=9}SAfB)N)*3?hdxq2eiHLDKIUaP0GD2jenav>|d*(XfjsKr(n89RIayt-vKqAnMv5p~11x@OH@GSOyvBQy5l<GYvaUrI{$>M1LACW~}7`?Svuled+u^HQpYZT3R#q+wbW2O4cE(d437)Y9mpZE~&HMysY4n}+EvkyP!v$T#fzd}g^a_C{>>jHDt_4BOVaWH~&5xU7)kO6=r<_~@bl*{;uFX1Telbgr6&0sCB;8LmJp$qIe5fI+p&)oxLXMlxDF6OCVW-t3vlZ(%xU7p4Jm49Elvft9hU5K1O9p<p6W<QZ@<b_|OD^lU-2ELU4uO9fBe3bWj73T;5b^&wnq`p=W6Go>53*7^z-{0X~6Z8g}hqY!IZn8c}~DspI3da*6?idD6&Y7HV9Sd|sd{MiDJMJ2Kf+E`vzg~VMEq;}|Hq9|A`8_*7jghRGKDC??*>139oZOmq*Y|giB)7Fx6<fFkJgaUzym6^?aTfGCZjr8xJyM2+b{j2>04Jqt=;amS}wDQk22%$^=W)8;A4k{L#LKp*z_EVWG-@y7|*@f6hO9#e<^Ld*0@M=4nL*-O}?f(`Pz9#I~3gs6?!amABTimN|DXJ~21I)ry<(n!H4jSNJmCa_eOl}!3w8;4Nc|2dv82q3BY(*2tpt8IGh0Rm64X#8f<4A6|`6h==P_p6cr`#F<r_BRCyg5&+Lo_F5`4)D!A)aY&h?}1Wf551XWN)GMoo?RhR%LJMTG#P5@-7N)kjYA_3|4kv;F)4=VCp5iL!S3;8Wap<j%uR>zXFHmAmjt$As&zq@}6--Tudc5z-73_DYkOl0jrBbZ^&X|oHL=7bxS1%Amcl9QSelJPf@+#ajtp1_7AOU0B8KNH)N8XflMi3v~&QNwvfkS+jc8B$`Ru)I|OXfb(;%Z#U+fk@0}W))Tz9M#oNBv{;@2;<JwyJQM1|a=fI=4q}lIh348VO;~QxG{XE4t_=63zPcJ|G^ybsYg#GsOYunqkz%w+?J|WcosMR(H^QIRt^!FMz@!$r81pd9FnU&&7a&$KX20$*CFlr+#wk?*?6s2zth-bWUGjOY9H~gsCdz9IWvuEGOxJ^rPq&Zu>03@!Lq4FHEqycqWOVHr@0B9V~=jmSFW;w*31~eKvtHo8`<7zOKh1MmrQA5|8z4JG)`3CKT+Gu}ckcdXkQ9)H_+N)?&wNb)uWWL)shGXPgy%mL#1E<abrhzFSPE5QL4@z^i{fHD8SxRMR|BUe;g$sxln=3<6X4@9Uf>?H;XygbW6>5SeN_Vn}BUA(|V{tm4v%jsV7n|mfiqZAq&o=6ki7o(c1;79(xJI}&=$_PnhrjO=AM=bj7<ShS^9^=?Grx~I{&R=6b}Px6VRr)YZ5u4$?tZYs!4ONDmykVhlm8`y^`4!be@5Y$<!o>T0*N7O0HCql6SjkWyBkauC>jkXkktChR+M>h00E-N3UCxDl*Tv@0RliG`iyL?>ZEfjwfvU@t%ftBb_eK0bk1?eRl$-05n&_jE>kYT8?i3r65?U{8j#>^jj(PV1~Nctrh2TpYpomSbJVKarfMOo<aOBEd*figI~xyat)3krfmt#m=s*&yFP^x6<i?OYsf3|)zYtzS5906`mtv<D%1lxtWfnW@!Dsdz3F#tfm3|4C1(8yyRuueEO0X77ylFFmIe@qZ2SS~Pi9kRSFy)ZkO){o&uPtx`><}!h*fE^P^JD0D%CH7e>l?s)N`dJaxL}k@F%^ZKB4eSBI4~9qv5DI*7gGhMrmz9-ftQezVwgBfIY~e#iiW%H9%ceLx;DG_>0k&%azGzI=bmh+nbQ+~a`F%Mtrc#lS^F9;++`)Jz<7F@oMz)tN7fsX!1lcYFi5vDVG|qGi?J5XW={yUa&&!5Pvx|;^4jb=AeH4ey1tUN@#EcVpf!-Sl%=i@D-x*h0J0pA8vJgW2YbOzLKRti_#zecrt`RpD13goj!pxe?Az_OkTw8MYorvn@oC50FY3t?b~g9RqCc6u##5co0qiRT8gRSlQAcHPW|NH4g3hb$=oF46lE9w0JPPX3Pw#)jJQyQqUdt>BLtvLbP9-;^7Vm&Iy>ZZ4J1+-D1PF0wXPc`v0DnUHcc&%a3~XC<Wtq_YaI9|&6sKoN*%GI4!tu!FB)V`p&SioD5!%V_It2yX(lVz*Ex(eG7C?3s_VtmDx<3;aF%jyL3`*&McZrq*kuhNFW8pwC!IBys+a2tV5k#RB1-tj62J9sz`VsSyh1k)M8-RRZAurgewUU0sY$QO6xZLR-gx|^7L0zlhpz@bgefJu&(JNzx({RQu!7b{}bg3nMnThg`-X||WEfAv{pu;*z|CTFP-RYVj3gB|O_te>>;k#O7m%W=6wa1E1#3H~Yn<B$Waw85Ms03{xsubG530PSrzxwgbA8@4pMep*)`~p2OI7GX#>9+y}xa~6=Q3UG-Tugrg2|rS^b6rch2Z#gg1Ja92mwD8#Dg^7Vv4|CqmqOhph{Mh{5cgf2^qdNDu$m0^({GkPO@<NOC#?M;fjD5s;>m3YP`WNt{t6%8!Q+U9&C?Ee6a}aaWI}Bs*xFE$x%k?@H@Xo8DI)=JXD5Tj(8RcyFc^+xM-az+fM_plify9H-f^&SrL3KTYX`pHDN~rMo&mmn)$m5tkmL3Y?E6=-u%_6vu<u{RVyIvCojpe#?NMx51JJ2B%gfd2Jca#&0}^|&@{qg3WXjUbQ`465Jk1+f#={Axr?=bbW-p|2^Ua6U9Z;n3Uz0D#2s(}C=u=Y~qME1mt+s&!uxXrEB`m|(wz8dDu#x8U`w8o;#SJLfId@Es5nE$P$O0e*wL3RIHf&9mh!|OX%LbZZPu#mx6r`?Y#ZkMb>B&e%gas-6iIL@;9P>uhSv*PEDth2?r>>FV(Tc%Ov5;h7YB;l@Dml6;kB^}=@fj;p<rJ%N43bZa+Q{mm$-Rsf)}%-BTc1xwg|)DlwuBZ8v8WN7J*uf{<9T2d4ix>60k^xO{fGz;qosw-<hb9%g)u;6=Z6DRDTRuyZ1ed;qcD*gN4IEc5A>gjffFo!?Vn@qzmw%SQcaE3Ye-bDJD2Vb7QklY>`xeGmID_|*y)JpeNfTpQ;86++dMOPgVADj%3%C(xNQH=Mt-QA;|9Jq!0a$PU;dw+xuAG#>!IMsNBv!z{-psSd5p!EgoI-TL!Ut9!b!0~^LA{ZDY_iFG%DYF!8o4KJwZGl`yuR;A7RyI@?EL(EVfFrTEsFXIo*)UXtTjdz&1Bvs|vlEdI2>?%|z|eQL(9Yf+siLg+Y5T*BWh+`nhN~aX6DRz`hWZmRIoGrR&BX3z@tHP~iG1hz443VS=#G6W{&z^XnyP*Hz!>@{nZVxyt0Nhjlbb35%ELim@wZ2c`wQFSZfJzUA`H4!Mu8<W7=bERzA(%h9+B(!$X=TI%#T7z5VCH|9F-l>lzMrCO*hQF^yWS|9}1AO>N9H<P9T#G*U46?>Rdca?JX{CQe6QBWE(;#&6ZGlW`qY+4SE2L0h`Wb(u6;?jqjuyr)R<0s89JOd61U4?b&ZsHt%j_2>HX?EtG+`f;=UG1VnQ(Foe5m9M+IV#n3J)j4%QyMRAdrJp$>lKgF_b%fG^$nwuE;a&*dm8m$L+*}2Yge@xsUAG~jfNea*4=dou;?#FMkEjL81H;|>h{f-(?cIqqa~pN@^N&0c;GNg?Weu(9|Ok;Pkyc*9rU`YbcyFI*qiHRcaMAFuW>I-Q(vOLW7&eL!2SIp-B9DH(BD?4FY8^47vt}#2jYp@)H$(Sah~Z7=RJziZ7HBuzj|xq3xb2b(bluZtyNw?ZmD()dzC*J(8z14COV4zHlJ#@(6Rt%ajgMv+BK#k%eZihi?`8YafjmH18h-1w^v@iq7*>-)NmM0QG0>4xsb!7L#(+hdTagSk?sPDh|$U)oyAt17kJqQV}vUOpkqo7p~4pgU<~Fbjtl)kK8DO_J`3&<fXaPY&~J2bbs9Que}A+BC{ISkY5U#C>ifRkreLo9>DgonGidd7%JYygEWfRSIR~YQ)k)tms#=6K(jenqoyo<sz9%?yuFlevV*ys<TdFAZ+{#}&PiwNfbPkm{UAAAHq^Hj&BpxOEtEb7Z7~^nm_0VNn7=!usv!V20plB=?9ZR$nTi(GWAsMz?f1l{^F3;W9JELLb_0xUi_=7H2o5CE6M1!P@F1;~lp}S~TyK5-drxjfGw{_hG&AQEDW8JqMVDCmM)0#N2BAnpj1Wp+4@vMJ*+CQFOb`{~u$Tri?SU6<2x(=&%H2sB*+?QSqVIV11a9+`E1zTBQp(~@{<8F*h$DSGoHvqjur(d74TJApE8!>@I%?2_@^+29i3YBjmD8Xv(LKGJbMf~Xfhd1wEet7dbgzV4eOka^NMR;!l{aDYjl7^kmj~orvecT@8#wD?Ec%kx0IE>a!sR^+jY@4k~xn4N47+t&k8D2n@<NVVybApU`rr@s^!}7$0c=FE}4$Lm$>!`n8kX*sm;^cSTM;pt*hb9w#6C@^)`yA*f6&rr$70&ob9yd&zJSz(oN2wiFfv3CdeF0`Qvx&4SQkw5|o&Tj(BLD^g>|pBQ&0b``59IFqK)#k+441W}Pa%$|^^LNLVF#K68e7Sj@+I#mDSqe|&C{DYhto;}$L?Mg*p+5*2sW`Nw=Sn3^1uNVcNp(?tPivASo9wig<y?eEXH5Q1z@ab&3<_AWR>kMurZwoZj>ky9T3vTE5SEw)Y#u(9(5h>;?b9FE;o_(($B+zPtpvtiBZDBY%HN~H>Z-V!8b=I?1xn#_YVVbeL#Y(%fwjO0@ST3^}RSr*j+f5E!gS(WfX2jU}bQa&7yHyH2j!k2m-@HiHkL-qqE9L)f1h)6;5;>A0DSSXh;)&V>0~SAer*Q-oCn<<QeP*r*A<yXR8&9ILC7g=kE5mw3kQ!14l+$N&\",\"sha256\":\"4be9f0f3fa0ec6cb32e5f75a417a5114a0969b27dabc4682aa101a62561863f8\"},\"efficientnet_b3_public_repro_v6_yash288.py\":{\"b85\":\"c-oa&ZExE~68_F#v8Q}Es+AbqiG#$b3gi-}7od&<$L-<ZGqx1DvUsAn>@F!w4f5abnc1aCN=$k+5=q>bnb~=LX8GiAPit$Q&a>jFDmJ3rRZCqw3r3^S<u*<#arW|MxfQ8Obea_pBGE-<XnbMxT2xCV%6gt>iHKgEJQG>5P(~Gr5{0hRT<cX71h-hUIT!k|m@aglipSWjr?!esCE_9#xr$fugL0I0ZbfXAFsh2PLZy)qx690mO>DBbsDfk}7Z1uZ<7`da%D7sJO7lpX*(BFifk##pU2!kO`vNQ>*@t>PS4LQEs!32PGquWOpm;i+zIr7vkf@@{az!LkjgN<kpdkpA7+n&p7te*ok(Hf*qfto}&<6yo^Vm!)J%!G20-ODzCVa#?E2@{GGIf!}Ri+DJbX}yC$x58>eI-&20iZFbKcP*K=yHdxJB-8$IN59tkxG+o;tHav>|t5ih^!O@YzZkBbyXWBCG=8i)-5P9I~xSen0Y8;V^#CKjO{Yd=FOw~!a!6*1<mMxtuC71c4r;GG_K+#kFAB|4RM=hNj1UA%REk$TSw*r!Dh`zJi3W$SCWDJ_;pEw#`#2iQoq;aL3^2tdQG~3g34y7gi<kpe`Oj3L8=x)<~mO0<~$r<1Ok8T7Bl7-LN_p|l;h|ztCq5e*DB;i^fywAMV4f+aG@$We<qzjOPs=J5R~Mu?5(hBRPIK+bIhOFf~`ipqRcXvhClt8II~K;!{`THy(5!f8lz3P7{zs(Rd7>;MXE|wq&P8bjU&x1B)JfuiRPD)V?zStEF)u!Sgs1CBxL@Iv^!W7g36Yx6WEq20gb+eBN~-t4yU19JO2!c<Al$(KPVG`gdcRF;Gz5+k<#I?e)#a@H#H(@t(rq8?2&tK8B)Q)!Ph#ib9F4X8+U?}93g$UEpSjjZSUw5_t@>8_iFG`M{0}1ySdPPITFY{wJDliLGbH2Xte$n{Q7(%{_*<LB}RWekLU?sG!WdrzWM3$_R~as{Q32VOML(G{w=L~9_?rFyP3pK%m=qnR`Es`iwv2D9}pKGHLSXG9C{0y=lz;s9j}z6aJ29U4oSD|cswYxWF=j`lQ!3tb;2DD2X&Dxw1ILd%}FFe%64w?;8b8Mjnr%_xZ=Q`ugwOs@@J9G!d~>F;Xr<TeRKKYR$g7dz5H3ed;RA2`sUxAbHZ3E_ows;!7k&o^A{oc(*@^&v6%iSY-KLGR`wY%>a328f~n`bO4)Ecj+SbhW~ez8Oz%l^HB_?9wko$_YJJQj8~*~gr%GfgWU)4`b7aJF2NwS=73_z}Lj3gh&6O*uobBloxr{1kL>ZlERc6--CnsxaBW5XMW8xjps%@gmN?h_6BDdXBk}Bv2tc6n+X-<J!h;X!*W;79_hcsD_IytMym;S(%Bwd#%Xg2hOIET*s!??Hp>#}}ZUizbIofWkU)qxx`it}7B+2%x6EMjD?bkx}9&>O<mHaCnZ*gTwkVlJsb8wKiGMRhfgce<{`9RHy;pcusvXdQ2-SMProm@DgQZ=#R9^@?~=f*A5pav44thvikgy)N?IznX%=Tf+{SjveSMoN`z)NtHFUDql=QJ3zk;zGt_lCZqwB@KDU&#kp1EzG9E8>MUj9oe<sDfIZ<jI^=h>gEQzcd-t^$z<nKU?z@S1gxg)|Y<nTZ69IGxC#|6RrJzzx#$a<XMX^%5M?}kCKgd^Q?#uK<fX!NGY54MFOht{&pYH^fJb2CmbgO4)j7e!=iIhnt488)8kYdLHImm7lW|arZ_(6yhwtrCUR$^{|NtYAxxFpD#ELE~9HP9lR^^lrTYw}+wIi#YXM7f?bRwWQjJPdGm!gcjI18u>c7WGJxA4ThIi<<V3RT0&Q2=zagICpe1{?=LpbOy+4<%5BWL+5Kl@$YOBbW`o1`}?qEi@`uB=o_8a0BF7*0bslAJ#lzn5~{v{@?lCS(A5*m18{&B&)aan%E+%=r%+JcC#`Uo()1>Ox8gU>^^VtX1T%8AMl>=C+6Q{FIinoT;!T9?{-eM@ELNdY2q7w}XVht^lq(D8V*UX;GBj;ZIga2<20y|v!7VOCwgxuciW#t9#Cf`lI+^~TK(0cIx9{IvUwL7`^>fRyL0L|Pko4|;UoaXW510&82ki5zArVQP2sgvW9rr!M^=q?%@)-deAQ6N`8Tl9++WFngMPP{Fi7W6((SRYV+a%7_r(Bl`1??0LOz`yd(2*UD@4rRkeNj>SS5}~N;^`a(1VLf!>+S!z)RkT<3QVR^Qu|6SL=|>49<v3nu0LGA`RC2`)#d5vI2kv}aneB%aPE)?v3=^M`*Qexme6r=aGrjzOQ&&XIS6gGdlZ+1zQeE~=mV`|yuVPiy>I%c{8Q&-PvAQ2o$y;88c`&LNQ|>QUeD85AYj%P{CRhB?^e3x0i8gDI8L$(@0@1Isii*}ad9N?2V4*BKAP`YC$;7JaUzyzxgFj1jv#dGCgKk6HW9C0MU5QL18?|YM!Q=@cq$kZ-;<9X37zCw8G5>~I~I<`QxWn$_sG6EX&oG_a#i>(WAP)qLz@~qNWk5>%tY8-KRsLzeow{Oe$!q!_v6kHx$wlkKiqxqGd`v1v}=bw!MncqY|bLLIvkFP$<c5~EZbrLg&YiwkN0F8mOAICoDtLaA8s#iBrx0S{_1{=klXBi#%W+bf8V(Y@Fgy64*s1uJ3ohoI_|}nKtrhx_T|MWg-_5ep(^>VO)TmnarZ^a>U_iZUVtThAHbI=yxMok54oPhXCw5d(NQ^NpF0>MZs*v~mvUUh0KPh$lwceUgq;AA?L5cxA&$pg7`5HQjTwQM5C~{<vA^}XYc74@UFM!zcDhdBeyzP%;EOARx#kipdTkzRs@5NQCQOmVO~xc=Ql>hQa=b5StTSqV+KV4P@-(rVQR@LCmwXOK%Pe0##>K-lkLRday$zMxsA6Cu&$ijnxa%UONN;|0+-=(EBvG8E=$qW(kSM3q&m{gABy#ednS(<filvPH$HCXOV>dhS)M#CGDSUH_Ym25WApO60k~cMrvG3blj3P!;4OoahF+2MVDg|T)tg4mI|JcJg!%^WcqGQ>Onih*MNU+LZKemSM+@FM<)X*l3+1bmNlMcqga-Ys-FWT`<oM&k>jxw>In8#JJ^n}_`bSf+Kr?cnXg~@6Tf0oIzE>^P+PlVPVs*tMG$<oeF+KIa8Pn`}-<z{xKrswU<156;@9r5feIyn|#c6jO8c%m~_bN4i?$^Fe!SD9*ys#r-I6M~it<4t)Iot_U!yjR3L*U1XnY}dGav541MzB>|=Z#N}Xo9?Bo+`Q<^4;N)PyC+J%5;?mfuqXX-4>yh{nxL01h7Y%K+z-!j==Y{PvNi}~7OhO(EfcspsN6%{d;+({5y2L^V2tK<2GjaGUwFJ#g{^Ja@grgYG&E^`p0z)pwLhQVPkg{crB!vRy}L-%ByBpnbnNd&9M$)Y7_J{j52))B#>@#erjpj!(4V>?&x89Xt5n*CbmUSYA-!n5qSnWy9qTVN)Hbt~)T>U0bJU|sv*r$nppdelUh@$rap*Hu+b*12$3?bKmXOfrZXS=T+Kw)O=5H?PF6V7GX*>Bb>t1q%?zs~HqkR|X!lzy8p1Vtpek4YO!knbtNgqjm4I!Jx2AP#^a&(W*a2<AWjyrdvfv}l$&ukG=SFQYPIA$CzSuJ>gDp|}UVqTB~nt&EbAHfZYr+yDlfy*5K7BKx~LEP&#*?GMt9m0Y<>?7?`hF7wE$0qmh*pAmWt&S|XL6RiL@v1`0r8e0g=x<`Aq<n~yek|DU+k|wyhVm8CndB;WH0!T{k@E7K@}lJXV63|v`TJvPB9HHE?|bJ#1dxqB?_2MS-UawTdim0o(7jJ_8%FKc{@b&zVO*ff79X^m`#8QK<udBwF23p*o)T~>#qe)W&kOK{KZk<9iR_48S&qihBi+x*N^LuLnKTup^}4iSv)tu~0>yH|w>6S3-(1o`gERV1(S7Gv8j8ukMg@5{ukEsv$M(l|2#9^-@V>vdkU@Iq^&1a52t&Z`XzEy>M=$9-oKo&;y~$q<uC&5oxryikNcTQ_2p{(DrcS8wZRP%<R}H=*!NP+_v96Ju=Zx0x(CB{d?nS$L`sIGqHAN?d#~YP)vNqZhSM&*L(Dra3arL`DUms8J`@;v?9++$$abS}5cszV$qsigL3cd2{jk#hnISdIFeeYro7zzTIL-M;CDftdcO6tC(^cPUvij9N+0XTb^TL\",\"sha256\":\"d26fefc311008a4e6e0b85d2586898b94fb40e79d66bca961ce26c670fd2dcb5\"}}\n_V61_PARENT_SHA256 = {\"0\":\"71a4cafb8f13341c4ca8a50b20bcc1ca3ad63c88b721d632865e2305cbb282e3\",\"1\":\"088065c483017a7497a770c73530c8850467422738ddff3a6ca111a3f1749948\",\"2\":\"58482553af61a9afdfaccd7b7319a0af81046421054af0cf995c1da70dc36486\",\"3\":\"6b60f18a28c6fc80ab04939ae25be2358705013e4779706bbd9534c52fec4472\",\"4\":\"ccb18c38000fbdb152c2717be2f340c446978cad40f2db7667f15a1a49e31249\"}\n_V61_INPUT_SHA256 = {\n    str(_V61_COMP / \"train.csv\"):\n        \"8ca2203c0e9d61c080c7a314c7cdb51c1b03a1d9eb4770819f7f34af53ef4e33\",\n    str(_V61_COMP / \"train_series.csv\"):\n        \"573c1d80772bf41211c91b149c95677385a1c22d63f485c347f1b46c0177aef3\",\n    \"/kaggle/input/rsna-knee-llm-labels/report_labels_v2.csv\":\n        \"6f704a7bdb2f894cc49445b19ba7c4378c3f548d3449e00361e10044bee40920\",\n    \"/kaggle/input/rsna-knee-llm-report-labels/llm_labels_v2.csv\":\n        \"3c082a987939528a55202028f655bd6f9dc5c9e28eff5058bbae7ef088a9f144\",\n    \"/kaggle/input/rsna-knee-llm-report-labels-sol56/labels_llm_gpt56sol.csv\":\n        \"a79b05e609336811d51c97f9a48f00ea8004244a1a80fe886504a62bb39867ae\",\n}\n\n\ndef _v61_sha256(path):\n    digest = _v61_hashlib.sha256()\n    with open(path, \"rb\") as handle:\n        for chunk in iter(lambda: handle.read(8 << 20), b\"\"):\n            digest.update(chunk)\n    return digest.hexdigest()\n\n\ndef _v61_rank(values):\n    from scipy.stats import rankdata\n    values = _v61_np.asarray(values, _v61_np.float64)\n    ranked = _v61_np.empty_like(values)\n    for column in range(values.shape[1]):\n        ranked[:, column] = rankdata(values[:, column], method=\"average\")\n    return ranked / max(1, len(values))\n\n\ndef _v61_auc(truth, prediction):\n    from sklearn.metrics import roc_auc_score\n    values = []\n    for column in range(truth.shape[1]):\n        y = (_v61_np.asarray(truth)[:, column] >= 0.5).astype(_v61_np.uint8)\n        if _v61_np.unique(y).size == 2:\n            values.append(roc_auc_score(y, _v61_np.asarray(prediction)[:, column]))\n    if not values:\n        raise RuntimeError(\"macro AUC subset contains no nonconstant targets\")\n    return float(_v61_np.mean(values))\n\n\ndef _v61_validate_submission(frame, expected_uids):\n    if frame.columns.tolist() != [_V61_UID, *_V61_TARGETS]:\n        raise RuntimeError(f\"submission schema drift: {frame.columns.tolist()}\")\n    if frame[_V61_UID].astype(str).tolist() != list(map(str, expected_uids)):\n        raise RuntimeError(\"submission study order drift\")\n    if frame[_V61_UID].astype(str).duplicated().any():\n        raise RuntimeError(\"duplicate submission study\")\n    if not _v61_np.isfinite(frame[_V61_TARGETS].to_numpy(float)).all():\n        raise RuntimeError(\"submission contains non-finite values\")\n\n\ndef _v61_stop_workers(active):\n    for entry in list(active.values()):\n        if entry[\"process\"].poll() is None:\n            entry[\"process\"].terminate()\n    for entry in list(active.values()):\n        if entry[\"process\"].poll() is None:\n            try:\n                entry[\"process\"].wait(timeout=30)\n            except _v61_subprocess.TimeoutExpired:\n                entry[\"process\"].kill()\n                entry[\"process\"].wait(timeout=30)\n        if not entry[\"handle\"].closed:\n            entry[\"handle\"].close()\n    active.clear()\n\n\ndef _v61_tail(path, count=20):\n    try:\n        return \"\\n\".join(path.read_text(errors=\"replace\").splitlines()[-count:])\n    except Exception as error:\n        return f\"<unavailable {type(error).__name__}: {error}>\"\n\n\ndef _v61_materialize_sources():\n    _V61_SOURCE.mkdir(parents=True, exist_ok=True)\n    observed = {}\n    for name, contract in _V61_SOURCES.items():\n        payload = _v61_zlib.decompress(\n            _v61_base64.b85decode(contract[\"b85\"].encode(\"ascii\"))\n        )\n        digest = _v61_hashlib.sha256(payload).hexdigest()\n        if digest != contract[\"sha256\"]:\n            raise RuntimeError(f\"embedded source hash mismatch for {name}\")\n        path = _V61_SOURCE / name\n        path.write_bytes(payload)\n        if _v61_sha256(path) != digest:\n            raise RuntimeError(f\"written source hash mismatch for {name}\")\n        observed[name] = digest\n    return observed\n\n\ndef _v61_parent_contract():\n    # The immediately preceding V60 cell owns the exact V52/V58 reconstruction.\n    for name in (\"_v60_parent_oof\", \"_v60_rank\", \"_v60_auc\"):\n        if name not in globals() or not callable(globals()[name]):\n            raise RuntimeError(f\"required preceding DICOM-gate function absent: {name}\")\n    # V60 deliberately changes submission.csv after its gate.  Its parent OOF\n    # reconstruction pins the pre-metadata submission, so point that function\n    # at V60's immutable saved parent for the duration of this read-only call.\n    saved_parent = _V61_DICOM_DIR / \"submission_parent_before_metadata.csv\"\n    dicom_receipt = _v61_json.loads(_V61_DICOM_AUDIT.read_text())\n    if _v61_sha256(saved_parent) != str(dicom_receipt.get(\"parent_sha256\")):\n        raise RuntimeError(\"V60 saved-parent submission hash drift\")\n    original_v60_primary = globals().get(\"_V60_PRIMARY\")\n    globals()[\"_V60_PRIMARY\"] = saved_parent\n    try:\n        parent = _v60_parent_oof()\n    finally:\n        globals()[\"_V60_PRIMARY\"] = original_v60_primary\n    if parent.get(\"special_parent\"):\n        raise RuntimeError(\"specialist parent cannot be compared with this OOF topology\")\n    expert = parent[\"expert\"].copy().reset_index(drop=True)\n    truth = parent[\"truth\"].astype(_v61_np.float64)\n    folds = parent[\"outer_folds\"].astype(int)\n    baseline_pre_metadata = parent[\"baseline\"].astype(_v61_np.float64)\n\n    exact = _v61_np.load(_V61_DATA / \"exact_public_family_oof.npz\", allow_pickle=False)\n    exact_index = {str(uid): i for i, uid in enumerate(exact[\"ids\"].tolist())}\n    rows = _v61_np.asarray([exact_index[str(uid)] for uid in expert[_V61_UID]], int)\n    exact_rank = _v61_rank(exact[\"ours\"][rows].astype(_v61_np.float64))\n    rad_frame = _v61_pd.read_csv(\n        _V61_RAD_DIR / \"v52_oof.csv\", dtype={_V61_UID: str}\n    )\n    rad_aligned = expert[[_V61_UID]].merge(\n        rad_frame[[_V61_UID, *_V61_TARGETS]],\n        on=_V61_UID, how=\"left\", validate=\"one_to_one\",\n    )\n    rad_rank = _v61_rank(rad_aligned[_V61_TARGETS].to_numpy(float))\n\n    dicom = _v61_json.loads(_V61_DICOM_AUDIT.read_text())\n    dicom_status = str(dicom.get(\"status\"))\n    metadata_rank = None\n    metadata_indices = []\n    metadata_gamma = 0.0\n    baseline = baseline_pre_metadata.copy()\n    if dicom_status == \"CANDIDATE_SELECTED\":\n        alpha = float(dicom[\"selected_alpha\"])\n        metadata_key = f\"{dicom['selected_spec']}::alpha={alpha}\"\n        safe = _v61_re.sub(r\"[^a-zA-Z0-9]+\", \"_\", metadata_key).strip(\"_\")\n        frame = _v61_pd.read_csv(\n            _V61_DICOM_DIR / \"metadata_gold_oof.csv\", dtype={_V61_UID: str}\n        )\n        columns = [f\"{safe}::{target}\" for target in _V61_TARGETS]\n        aligned = expert[[_V61_UID]].merge(\n            frame[[_V61_UID, *columns]], on=_V61_UID,\n            how=\"left\", validate=\"one_to_one\",\n        )\n        metadata_rank = _v61_rank(aligned[columns].to_numpy(float))\n        metadata_indices = [\n            _V61_TARGETS.index(target) for target in dicom[\"selected_targets\"]\n        ]\n        metadata_gamma = float(dicom[\"selected_gamma\"])\n        baseline = _v61_rank(baseline_pre_metadata)\n        baseline[:, metadata_indices] = (\n            (1.0 - metadata_gamma) * baseline[:, metadata_indices]\n            + metadata_gamma * metadata_rank[:, metadata_indices]\n        )\n        expected = float(dicom[\"oof\"][\"selected_descriptive_macro_auc\"])\n    elif dicom_status in {\n        \"OOF_REJECTED_PARENT_PRESERVED\", \"ERROR_PARENT_PRESERVED\",\n        \"PARENT_PRESERVED\", \"PARENT_SPECIALIST_SELECTED_PRESERVED\",\n    }:\n        expected = float(parent[\"baseline_score\"])\n    else:\n        raise RuntimeError(f\"unexpected DICOM gate status: {dicom_status}\")\n    observed = _v61_auc(truth, baseline)\n    if abs(observed - expected) > 1e-10:\n        raise RuntimeError(f\"current parent OOF drift: {observed} != {expected}\")\n    return {\n        \"expert\": expert, \"truth\": truth, \"folds\": folds,\n        \"baseline\": baseline, \"baseline_score\": observed,\n        \"exact_rank\": exact_rank, \"rad_rank\": rad_rank,\n        \"dicom_status\": dicom_status, \"dicom_audit\": dicom,\n        \"metadata_rank\": metadata_rank,\n        \"metadata_indices\": metadata_indices,\n        \"metadata_gamma\": metadata_gamma,\n    }\n\n\ndef _v61_apply_metadata(values, contract):\n    indices = contract[\"metadata_indices\"]\n    if not indices:\n        return _v61_np.asarray(values, _v61_np.float64).copy()\n    result = _v61_rank(values)\n    gamma = contract[\"metadata_gamma\"]\n    metadata = contract[\"metadata_rank\"]\n    result[:, indices] = (\n        (1.0 - gamma) * result[:, indices]\n        + gamma * metadata[:, indices]\n    )\n    return result\n\n\ndef _v61_build_candidates(contract, warm):\n    warm_rank = _v61_rank(warm)\n    candidates = {\"parent\": contract[\"baseline\"]}\n    definitions = {\"parent\": {\"family\": \"parent\", \"weight\": 0.0}}\n    for beta in _V61_BETAS:\n        b3 = (1.0 - beta) * contract[\"exact_rank\"] + beta * warm_rank\n        pre_metadata = (\n            (1.0 - _V61_RAD_ALPHA) * _v61_rank(b3)\n            + _V61_RAD_ALPHA * contract[\"rad_rank\"]\n        )\n        key = f\"replace_b3_beta_{int(round(100 * beta)):02d}\"\n        candidates[key] = _v61_apply_metadata(pre_metadata, contract)\n        definitions[key] = {\"family\": \"replace_b3\", \"weight\": float(beta)}\n    parent_rank = _v61_rank(contract[\"baseline\"])\n    for alpha in _V61_DIRECT_ALPHAS:\n        key = f\"add_to_parent_alpha_{int(round(100 * alpha)):02d}\"\n        candidates[key] = (1.0 - alpha) * parent_rank + alpha * warm_rank\n        definitions[key] = {\"family\": \"add_to_parent\", \"weight\": float(alpha)}\n    return candidates, definitions\n\n\ndef _v61_select(contract, candidates, definitions):\n    truth, folds = contract[\"truth\"], contract[\"folds\"]\n    baseline_score = contract[\"baseline_score\"]\n    scores = {key: _v61_auc(truth, value) for key, value in candidates.items()}\n    nested = _v61_np.zeros_like(contract[\"baseline\"])\n    choices, train_scores, held_gains = [], [], []\n    keys = list(candidates)\n    for outer in range(5):\n        train_mask = folds != outer\n        held_mask = ~train_mask\n        scored = {}\n        for key in keys:\n            raw = _v61_auc(truth[train_mask], candidates[key][train_mask])\n            definition = definitions[key]\n            if definition[\"family\"] == \"replace_b3\":\n                penalty = 0.004 * abs(float(definition[\"weight\"]) - 0.10)\n            elif definition[\"family\"] == \"add_to_parent\":\n                penalty = 0.010 * float(definition[\"weight\"])\n            else:\n                penalty = 0.0\n            scored[key] = float(raw - penalty)\n        best = max(keys, key=lambda key: (scored[key], key == \"parent\", key))\n        choices.append(best)\n        train_scores.append(scored)\n        nested[held_mask] = candidates[best][held_mask]\n        held_gains.append(\n            _v61_auc(truth[held_mask], candidates[best][held_mask])\n            - _v61_auc(truth[held_mask], contract[\"baseline\"][held_mask])\n        )\n    nested_score = _v61_auc(truth, nested)\n    non_parent_folds = sum(choice != \"parent\" for choice in choices)\n    eligible = [\n        key for key in keys\n        if key != \"parent\" and choices.count(key) >= 2\n    ]\n    selected = (\n        max(\n            eligible,\n            key=lambda key: (\n                scores[key], choices.count(key),\n                -float(definitions[key][\"weight\"]), key,\n            ),\n        ) if eligible else \"parent\"\n    )\n    supported = bool(\n        selected != \"parent\"\n        and non_parent_folds >= 4\n        and sum(gain > 0 for gain in held_gains) >= 4\n        and nested_score >= baseline_score + _V61_MIN_GAIN\n        and scores[selected] >= baseline_score + _V61_MIN_GAIN\n    )\n    return {\n        \"candidate_macro_auc\": scores,\n        \"outer_choices\": choices,\n        \"outer_penalized_train_scores\": train_scores,\n        \"outer_held_gains\": held_gains,\n        \"non_parent_outer_folds\": int(non_parent_folds),\n        \"nested_macro_auc\": nested_score,\n        \"selected_key\": selected,\n        \"selected_descriptive_macro_auc\": scores[selected],\n        \"selected_definition\": definitions[selected],\n        \"selection_supported\": supported,\n    }\n\n\ndef _v61_train(audit):\n    global active\n    observed_inputs = {}\n    for text, expected in _V61_INPUT_SHA256.items():\n        path = _V61Path(text)\n        observed_inputs[text] = _v61_sha256(path)\n        if observed_inputs[text] != expected:\n            raise RuntimeError(f\"pinned input drift: {text}\")\n    for fold in range(5):\n        path = _V61_DATA / f\"fold{fold}\" / f\"fold{fold}_final.pt\"\n        digest = _v61_sha256(path)\n        observed_inputs[str(path)] = digest\n        if digest != _V61_PARENT_SHA256[str(fold)]:\n            raise RuntimeError(f\"fold {fold} parent checkpoint drift\")\n    audit[\"observed_inputs\"] = observed_inputs\n    audit[\"materialized_sources\"] = _v61_materialize_sources()\n\n    import torch as _v61_torch\n    if not _v61_torch.cuda.is_available():\n        raise RuntimeError(\"V61 requires CUDA\")\n    visible = int(_v61_torch.cuda.device_count())\n    workers = list(range(min(2, visible)))\n    if not workers:\n        raise RuntimeError(\"V61 found no visible GPU\")\n    audit[\"visible_gpus\"] = visible\n    audit[\"worker_gpus\"] = workers\n    module = _V61_SOURCE / \"efficientnet_b3_public_repro_v6_yash288.py\"\n    common = [\n        _v61_sys.executable, str(module),\n        \"--train-csv\", str(_V61_COMP / \"train.csv\"),\n        \"--series-csv\", str(_V61_COMP / \"train_series.csv\"),\n        \"--image-root\", str(_V61_COMP / \"train_series\"),\n        \"--pilkwang-labels\", \"/kaggle/input/rsna-knee-llm-labels/report_labels_v2.csv\",\n        \"--steven-labels\", \"/kaggle/input/rsna-knee-llm-report-labels/llm_labels_v2.csv\",\n        \"--lixin-labels\", \"/kaggle/input/rsna-knee-llm-report-labels-sol56/labels_llm_gpt56sol.csv\",\n    ]\n    pending = list(range(5))\n    active.clear()\n    results = {}\n    deadline = _v61_time.monotonic() + _V61_TRAIN_DEADLINE_SECONDS\n    while pending or active:\n        if _v61_time.monotonic() >= deadline:\n            _v61_stop_workers(active)\n            raise TimeoutError(\"V61 warm-start training exceeded its six-hour reserve\")\n        free = [gpu for gpu in workers if gpu not in active]\n        while pending and free:\n            fold, gpu = pending.pop(0), free.pop(0)\n            fold_dir = _V61_TRAIN / f\"fold{fold}\"\n            fold_dir.mkdir(parents=True, exist_ok=True)\n            log = _V61_TRAIN / f\"fold{fold}.log\"\n            handle = open(log, \"w\")\n            parent = _V61_DATA / f\"fold{fold}\" / f\"fold{fold}_final.pt\"\n            command = common + [\n                \"--output-dir\", str(fold_dir), \"--fold\", str(fold),\n                \"--parent-checkpoint\", str(parent),\n            ]\n            process = _v61_subprocess.Popen(\n                command,\n                env={\n                    **_v61_os.environ,\n                    \"CUDA_VISIBLE_DEVICES\": str(gpu),\n                    \"PYTHONUNBUFFERED\": \"1\",\n                },\n                stdout=handle, stderr=_v61_subprocess.STDOUT,\n            )\n            active[gpu] = {\n                \"fold\": fold, \"process\": process, \"handle\": handle,\n                \"log\": log, \"started\": _v61_time.monotonic(),\n            }\n            print(f\"[B3 V61] fold {fold} started on cuda:{gpu}\", flush=True)\n        completed = []\n        for gpu, entry in active.items():\n            code = entry[\"process\"].poll()\n            if code is None:\n                continue\n            entry[\"handle\"].close()\n            fold = entry[\"fold\"]\n            results[str(fold)] = {\n                \"exit_code\": int(code),\n                \"seconds\": _v61_time.monotonic() - entry[\"started\"],\n                \"log\": str(entry[\"log\"]),\n            }\n            print(\n                f\"[B3 V61] fold {fold} exit={code}\\n{_v61_tail(entry['log'])}\",\n                flush=True,\n            )\n            if code != 0:\n                _v61_stop_workers(active)\n                raise RuntimeError(f\"V61 fold {fold} failed with exit {code}\")\n            completed.append(gpu)\n        for gpu in completed:\n            active.pop(gpu)\n        if active and not completed:\n            _v61_time.sleep(10)\n    audit[\"fold_processes\"] = results\n\n\ndef _v61_collect_oof(contract, audit):\n    paths = [\n        _V61_TRAIN / f\"fold{fold}\" / f\"fold{fold}_expert_oof.csv\"\n        for fold in range(5)\n    ]\n    checkpoints = [\n        _V61_TRAIN / f\"fold{fold}\" / f\"fold{fold}_warmstart288.pt\"\n        for fold in range(5)\n    ]\n    if not all(path.is_file() for path in [*paths, *checkpoints]):\n        raise FileNotFoundError(\"one or more V61 fold artifacts are absent\")\n    frame = _v61_pd.concat(\n        [\n            _v61_pd.read_csv(path, dtype={_V61_UID: str}).assign(fold=fold)\n            for fold, path in enumerate(paths)\n        ],\n        ignore_index=True,\n    )\n    if len(frame) != 58 or frame[_V61_UID].duplicated().any():\n        raise RuntimeError(\"V61 OOF is not 58 unique expert studies\")\n    aligned = contract[\"expert\"][[_V61_UID]].merge(\n        frame, on=_V61_UID, how=\"left\", validate=\"one_to_one\"\n    )\n    if not _v61_np.array_equal(aligned[\"fold\"].to_numpy(int), contract[\"folds\"]):\n        raise RuntimeError(\"V61 own-fold assignments disagree with parent outer folds\")\n    for target in _V61_TARGETS:\n        if not _v61_np.array_equal(\n            aligned[f\"gold::{target}\"].to_numpy(int),\n            contract[\"truth\"][:, _V61_TARGETS.index(target)].astype(int),\n        ):\n            raise RuntimeError(f\"V61 official label drift for {target}\")\n    frame.to_csv(_V61_TRAIN / \"expert_oof_all.csv\", index=False)\n    warm = aligned[[f\"pred::{target}\" for target in _V61_TARGETS]].to_numpy(float)\n    if not _v61_np.isfinite(warm).all():\n        raise RuntimeError(\"V61 OOF contains non-finite values\")\n    audit[\"checkpoint_sha256\"] = {\n        str(path.relative_to(_V61_TRAIN)): _v61_sha256(path)\n        for path in checkpoints\n    }\n    audit[\"oof_sha256\"] = {\n        str(path.relative_to(_V61_TRAIN)): _v61_sha256(path)\n        for path in paths\n    }\n    return warm, checkpoints\n\n\ndef _v61_infer(checkpoints, audit):\n    command = [\n        _v61_sys.executable,\n        str(_V61_SOURCE / \"efficientnet_b3_public_repro_v1_infer.py\"),\n        \"--module\", str(_V61_SOURCE / \"efficientnet_b3_public_repro_v6_yash288.py\"),\n        \"--test-csv\", str(_V61_COMP / \"test.csv\"),\n        \"--series-csv\", str(_V61_COMP / \"test_series.csv\"),\n        \"--image-root\", str(_V61_COMP / \"test_series\"),\n        \"--checkpoints\", *map(str, checkpoints),\n        \"--output-dir\", str(_V61_INFER),\n        \"--budget-hours\", \"3.0\", \"--checkpoint-every\", \"10\",\n    ]\n    log = _V61_TRAIN / \"inference.log\"\n    with open(log, \"w\") as handle:\n        result = _v61_subprocess.run(\n            command,\n            env={\n                **_v61_os.environ,\n                \"CUDA_VISIBLE_DEVICES\": \"0\",\n                \"PYTHONUNBUFFERED\": \"1\",\n            },\n            stdout=handle, stderr=_v61_subprocess.STDOUT,\n            timeout=3 * 60 * 60 + 300,\n        )\n    print(f\"[B3 V61] inference exit={result.returncode}\\n{_v61_tail(log)}\", flush=True)\n    if result.returncode != 0:\n        raise RuntimeError(f\"V61 inference failed with exit {result.returncode}\")\n    meta = _v61_json.loads((_V61_INFER / \"infer_meta.json\").read_text())\n    if int(meta.get(\"eval_slices_final\", -1)) != 32:\n        raise RuntimeError(f\"V61 inference reduced slice count: {meta}\")\n    if int(meta.get(\"active_models_final\", -1)) != 5:\n        raise RuntimeError(f\"V61 inference reduced fold count: {meta}\")\n    if len(meta.get(\"errors\", {})) > max(1, int(_v61_np.ceil(0.01 * meta[\"studies\"]))):\n        raise RuntimeError(\"V61 test-study error cap exceeded\")\n    audit[\"inference\"] = meta\n    return _v61_pd.read_csv(\n        _V61_INFER / \"submission.csv\", dtype={_V61_UID: str}\n    )\n\n\ndef _v61_test_candidate(selection, warm_frame, contract):\n    test = _v61_pd.read_csv(_V61_COMP / \"test.csv\", dtype={_V61_UID: str})\n    uids = test[_V61_UID].astype(str).tolist()\n    fallback = _v61_pd.read_csv(_V61_FALLBACK, dtype={_V61_UID: str})\n    _v61_validate_submission(fallback, uids)\n    _v61_validate_submission(warm_frame, uids)\n    warm_rank = _v61_rank(warm_frame[_V61_TARGETS].to_numpy(float))\n    definition = selection[\"selected_definition\"]\n    if definition[\"family\"] == \"replace_b3\":\n        exact = _v61_pd.read_csv(\n            \"/kaggle/working/submission_public_0899.csv\", dtype={_V61_UID: str}\n        )\n        rad = _v61_pd.read_csv(\n            _V61_RAD_DIR / \"submission_rad_only.csv\", dtype={_V61_UID: str}\n        )\n        _v61_validate_submission(exact, uids)\n        _v61_validate_submission(rad, uids)\n        beta = float(definition[\"weight\"])\n        b3 = (\n            (1.0 - beta) * _v61_rank(exact[_V61_TARGETS].to_numpy(float))\n            + beta * warm_rank\n        )\n        values = (\n            (1.0 - _V61_RAD_ALPHA) * _v61_rank(b3)\n            + _V61_RAD_ALPHA * _v61_rank(rad[_V61_TARGETS].to_numpy(float))\n        )\n        if contract[\"metadata_indices\"]:\n            metadata = _v61_pd.read_csv(\n                _V61_DICOM_DIR / \"submission_metadata_only.csv\",\n                dtype={_V61_UID: str},\n            )\n            _v61_validate_submission(metadata, uids)\n            indices = contract[\"metadata_indices\"]\n            gamma = contract[\"metadata_gamma\"]\n            values = _v61_rank(values)\n            metadata_rank = _v61_rank(metadata[_V61_TARGETS].to_numpy(float))\n            values[:, indices] = (\n                (1.0 - gamma) * values[:, indices]\n                + gamma * metadata_rank[:, indices]\n            )\n    elif definition[\"family\"] == \"add_to_parent\":\n        alpha = float(definition[\"weight\"])\n        values = (\n            (1.0 - alpha) * _v61_rank(fallback[_V61_TARGETS].to_numpy(float))\n            + alpha * warm_rank\n        )\n    else:\n        raise RuntimeError(f\"unsupported selected family: {definition}\")\n    selected = fallback.copy()\n    selected[_V61_TARGETS] = values\n    _v61_validate_submission(selected, uids)\n    path = _V61_TRAIN / \"submission_v61_selected.csv\"\n    selected.to_csv(path, index=False)\n    return path\n\n\nactive = {}\n_v61_audit = {\n    \"status\": \"PARENT_PRESERVED\",\n    \"evidence_boundary\": (\n        \"All 58-study AUC values are local official-label diagnostics, not a \"\n        \"competition score. Only a completed Kaggle competition row with a \"\n        \"nonempty public score establishes leaderboard performance.\"\n    ),\n    \"model_family\": \"efficientnet_b3_yash_max_warmstart288_v6\",\n    \"image_size\": 288,\n    \"direct_rectangular_resize\": True,\n    \"uint8_truncation_roundtrip\": True,\n    \"instance_number_sort\": True,\n    \"max_mil_only\": True,\n    \"train_slices_per_plane\": 6,\n    \"validation_and_inference_slices_per_plane\": 32,\n    \"own_fold_warmstart\": True,\n    \"min_descriptive_and_nested_gain\": _V61_MIN_GAIN,\n}\n\ntry:\n    if not _V61_PRIMARY.is_file():\n        raise FileNotFoundError(\"incoming submission.csv is absent\")\n    _V61_TRAIN.mkdir(parents=True, exist_ok=True)\n    _v61_shutil.copy2(_V61_PRIMARY, _V61_FALLBACK)\n    _v61_audit[\"parent_sha256\"] = _v61_sha256(_V61_FALLBACK)\n    contract = _v61_parent_contract()\n    _v61_audit[\"parent_oof_macro_auc\"] = contract[\"baseline_score\"]\n    _v61_audit[\"dicom_status\"] = contract[\"dicom_status\"]\n    _v61_train(_v61_audit)\n    warm, checkpoints = _v61_collect_oof(contract, _v61_audit)\n    candidates, definitions = _v61_build_candidates(contract, warm)\n    selection = _v61_select(contract, candidates, definitions)\n    _v61_audit[\"warmstart_oof_macro_auc\"] = _v61_auc(contract[\"truth\"], warm)\n    _v61_audit[\"oof\"] = selection\n    (_V61_TRAIN / \"v61_blend_policy.json\").write_text(\n        _v61_json.dumps(selection, indent=2, sort_keys=True) + \"\\n\"\n    )\n    if selection[\"selection_supported\"]:\n        warm_test = _v61_infer(checkpoints, _v61_audit)\n        selected_path = _v61_test_candidate(selection, warm_test, contract)\n        _v61_shutil.copy2(selected_path, _V61_PRIMARY)\n        if _v61_sha256(_V61_PRIMARY) != _v61_sha256(selected_path):\n            raise RuntimeError(\"V61 selected submission copy drift\")\n        _v61_audit[\"status\"] = \"CANDIDATE_SELECTED\"\n        _v61_audit[\"selected_path\"] = str(selected_path)\n        _v61_audit[\"selected_sha256\"] = _v61_sha256(selected_path)\n    else:\n        _v61_shutil.copy2(_V61_FALLBACK, _V61_PRIMARY)\n        _v61_audit[\"status\"] = \"OOF_REJECTED_PARENT_PRESERVED\"\nexcept Exception as _v61_error:\n    _v61_stop_workers(active)\n    if _V61_FALLBACK.is_file():\n        _v61_shutil.copy2(_V61_FALLBACK, _V61_PRIMARY)\n    _v61_audit.update({\n        \"status\": \"ERROR_PARENT_PRESERVED\",\n        \"error\": f\"{type(_v61_error).__name__}: {_v61_error}\",\n        \"traceback\": _v61_traceback.format_exc(),\n    })\n    print(_v61_audit[\"traceback\"], flush=True)\nfinally:\n    if _v61_audit.get(\"status\") != \"CANDIDATE_SELECTED\" and _V61_FALLBACK.is_file():\n        _v61_shutil.copy2(_V61_FALLBACK, _V61_PRIMARY)\n    _v61_audit[\"primary_sha256\"] = (\n        _v61_sha256(_V61_PRIMARY) if _V61_PRIMARY.is_file() else None\n    )\n    _v61_audit[\"elapsed_seconds\"] = _v61_time.monotonic() - _V61_STARTED\n    _V61_AUDIT.write_text(\n        _v61_json.dumps(_v61_audit, indent=2, sort_keys=True) + \"\\n\"\n    )\n    print(_v61_json.dumps(_v61_audit, indent=2, sort_keys=True), flush=True)\n"},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"# ==== own-model blend (appended) ====\nimport os, sys, subprocess\nimport pandas as pd\n\nTARGETS = [\"ACL\", \"MCL\", \"Medial Meniscus\", \"Lateral Meniscus\", \"Medial OA\",\n           \"Lateral OA\", \"PF OA\", \"Effusion\", \"Synovitis\", \"Baker's\",\n           \"Contusion\", \"Fracture\"]\nW_OWN = 0.15\n\ndef _find_code():\n    for c in (\"/kaggle/input/rsna-ens-code\",\n              \"/kaggle/input/datasets/analyticaobscura/rsna-ens-code\"):\n        if os.path.isfile(os.path.join(c, \"ours_srv_infer.py\")):\n            return c\n    for root, dirs, files in os.walk(\"/kaggle/input\"):\n        dirs[:] = [d for d in dirs if d not in (\"train_series\", \"test_series\")]\n        if \"ours_srv_infer.py\" in files:\n            return root\n    return None\n\ndef _rank01(s):\n    r = s.rank(method=\"average\")\n    return (r - 1) / max(len(r) - 1, 1)\n\nprint(\"[blend] cell running\")\nif os.path.exists(\"submission.csv\"):\n    base = pd.read_csv(\"submission.csv\")\n    base.to_csv(\"out_base.csv\", index=False)\n    CODE = _find_code()\n    print(\"[blend] code dir:\", CODE)\n    owns = []\n    if CODE:\n        for name, script, w, tmo in [(\"srv\", \"ours_srv_infer.py\", 0.75, 5400),\n                                     (\"v2ft\", \"ours_v2_infer.py\", 0.25, 2400)]:\n            try:\n                r = subprocess.run([sys.executable, os.path.join(CODE, script)],\n                                   cwd=\"/kaggle/working\", capture_output=True,\n                                   text=True, timeout=tmo)\n                print(f\"[own:{name}] rc={r.returncode}\")\n                print(r.stdout[-500:])\n                if r.returncode == 0 and os.path.exists(\"submission.csv\"):\n                    df = pd.read_csv(\"submission.csv\")\n                    os.replace(\"submission.csv\", f\"out_own_{name}.csv\")\n                    if set(TARGETS).issubset(df.columns) and \\\n                            float(df[TARGETS].std().sum()) > 1e-6:\n                        owns.append((df.set_index(\"StudyInstanceUID\")[TARGETS], w))\n            except Exception as e:\n                print(f\"[own:{name}] failed: {type(e).__name__}: {e}\")\n    b = base.set_index(\"StudyInstanceUID\")[TARGETS]\n    idx = b.index\n    if owns:\n        wsum = sum(w for _d, w in owns)\n        out = pd.DataFrame(index=idx, columns=TARGETS, dtype=float)\n        for t in TARGETS:\n            oc = sum((w / wsum) * _rank01(d[t].reindex(idx).fillna(0.5))\n                     for d, w in owns)\n            out[t] = ((1 - W_OWN) * _rank01(b[t]) + W_OWN * oc).values\n        out = out.clip(0.005, 0.995)\n        sub = base[[\"StudyInstanceUID\"]].join(out, on=\"StudyInstanceUID\").fillna(0.5)\n        sub.to_csv(\"submission.csv\", index=False)\n        print(f\"[blend] DONE: base + {len(owns)} own arm(s) at {W_OWN}\")\n    else:\n        base.to_csv(\"submission.csv\", index=False)\n        print(\"[blend] own arms unavailable; base stands\")\nelse:\n    print(\"[blend] no base submission found!\")"},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":"# ==== official-metric self-check (honest version) ====\nimport os, sys, subprocess\nCODE = None\nfor c in (\"/kaggle/input/rsna-ens-code\",\n          \"/kaggle/input/datasets/analyticaobscura/rsna-ens-code\"):\n    if os.path.isfile(os.path.join(c, \"gold_eval.py\")):\n        CODE = c\n        break\nif CODE is None:\n    for root, dirs, files in os.walk(\"/kaggle/input\"):\n        dirs[:] = [d for d in dirs if d not in (\"train_series\", \"test_series\")]\n        if \"gold_eval.py\" in files:\n            CODE = root\n            break\nif CODE:\n    r = subprocess.run([sys.executable, os.path.join(CODE, \"gold_eval.py\")],\n                       capture_output=True, text=True, timeout=3000)\n    print(r.stdout[-4500:])\n    if r.returncode != 0:\n        print(r.stderr[-1500:])\nelse:\n    print(\"gold_eval.py not found\")"}]}