{
  "id": 611652,
  "title": "What's the possibility of a shakeup?",
  "url": "/competitions/rsna-intracranial-aneurysm-detection/discussion/611652",
  "author_name": "AC",
  "post_date": "2025-10-13T11:49:20.148000",
  "votes": 2,
  "comment_count": 1,
  "views": 0,
  "content": "<p>When I changed my pipeline from its original  settings to a clinically and computationally correct, standardized approach, my model's validation AUC immediately dropped by 0.04, which I believe is indicative of the public models being optimized to a particular setting and not generalizing well to the unknowns. Do you think there's a possibility of a shakeup? </p>",
  "messages": [
    {
      "id": 3301442,
      "postDate": "2025-10-13T11:49:20.147Z",
      "content": "<p>When I changed my pipeline from its original  settings to a clinically and computationally correct, standardized approach, my model's validation AUC immediately dropped by 0.04, which I believe is indicative of the public models being optimized to a particular setting and not generalizing well to the unknowns. Do you think there's a possibility of a shakeup? </p>",
      "rawMarkdown": "When I changed my pipeline from its original  settings to a clinically and computationally correct, standardized approach, my model's validation AUC immediately dropped by 0.04, which I believe is indicative of the public models being optimized to a particular setting and not generalizing well to the unknowns. Do you think there's a possibility of a shakeup? \n",
      "votes": 2
    },
    {
      "id": 3301713,
      "postDate": "2025-10-14T03:00:18.557Z",
      "content": "<p>It's because the public models are mostly exploiting class and dataset imbalance not actually finding aneurysm. If you just throw images of multiple modalities at a model it will first learn which modalities or image artifacts have higher aneurysm rates before anything else, not actually finding the aneurysms within the images. </p>\n<p>For example the aneurysm rate on CTA is around 50/50 whereas on MRI T1 and T2 its around 1/4. The CNN's will see this and predict higher aneurysm prob on CTA images and lower on MRI T1 and T2 without looking at vasculature.</p>\n<p>But I think the dataset imbalances carries over to their test set so things won't really be shaken up. </p>",
      "rawMarkdown": "It's because the public models are mostly exploiting class and dataset imbalance not actually finding aneurysm. If you just throw images of multiple modalities at a model it will first learn which modalities or image artifacts have higher aneurysm rates before anything else, not actually finding the aneurysms within the images. \n\nFor example the aneurysm rate on CTA is around 50/50 whereas on MRI T1 and T2 its around 1/4. The CNN's will see this and predict higher aneurysm prob on CTA images and lower on MRI T1 and T2 without looking at vasculature.\n\nBut I think the dataset imbalances carries over to their test set so things won't really be shaken up. "
    }
  ],
  "comments": [
    {
      "id": 3301713,
      "author_name": "Jackson Fenner",
      "author_url": "",
      "post_date": "2025-10-14T03:00:18.557000",
      "content": "<p>It's because the public models are mostly exploiting class and dataset imbalance not actually finding aneurysm. If you just throw images of multiple modalities at a model it will first learn which modalities or image artifacts have higher aneurysm rates before anything else, not actually finding the aneurysms within the images. </p>\n<p>For example the aneurysm rate on CTA is around 50/50 whereas on MRI T1 and T2 its around 1/4. The CNN's will see this and predict higher aneurysm prob on CTA images and lower on MRI T1 and T2 without looking at vasculature.</p>\n<p>But I think the dataset imbalances carries over to their test set so things won't really be shaken up. </p>",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "3301442": "When I changed my pipeline from its original  settings to a clinically and computationally correct, standardized approach, my model's validation AUC immediately dropped by 0.04, which I believe is indicative of the public models being optimized to a particular setting and not generalizing well to the unknowns. Do you think there's a possibility of a shakeup? \n",
    "3301713": "It's because the public models are mostly exploiting class and dataset imbalance not actually finding aneurysm. If you just throw images of multiple modalities at a model it will first learn which modalities or image artifacts have higher aneurysm rates before anything else, not actually finding the aneurysms within the images. \n\nFor example the aneurysm rate on CTA is around 50/50 whereas on MRI T1 and T2 its around 1/4. The CNN's will see this and predict higher aneurysm prob on CTA images and lower on MRI T1 and T2 without looking at vasculature.\n\nBut I think the dataset imbalances carries over to their test set so things won't really be shaken up. "
  }
}