{
  "id": 437036,
  "title": "SAM-Med2D {Segment Anything Model}",
  "url": "/competitions/rsna-2023-abdominal-trauma-detection/discussion/437036",
  "author_name": "Nischay Dhankhar",
  "post_date": "2023-09-05T06:57:35.227000",
  "votes": 21,
  "comment_count": 7,
  "views": 0,
  "content": "<p>Sharing a recent MICCAI 2023 publication &amp; it's Github repo which could potentially be useful in this competition. </p>\n<h2>A new segment anything model is released which is pretrained on medical related 4.6M images and 19.7M masks.</h2>\n<ul>\n<li>Covers more than 30+ human organs</li>\n<li>Finetuned original SAM on these images</li>\n<li>SAM-Med2D outperforms SAM in medical image segmentation with over 5% performance gap</li>\n<li>Pretrained Checkpoint weights are available. </li>\n</ul>\n<h3><em>Training code is still not released but should be easy to finetune the model by extracting encoder blocks or using the model to extract masks of oragns.</em></h3>\n<p>Github Link: <a href=\"https://github.com/OpenGVLab/SAM-Med2D\" target=\"_blank\">https://github.com/OpenGVLab/SAM-Med2D</a><br>\nPaper Link: <a href=\"https://arxiv.org/pdf/2308.16184.pdf\" target=\"_blank\">https://arxiv.org/pdf/2308.16184.pdf</a></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4712534%2F4a0f1b08676a07ba2900a5287549cf8e%2FScreenshot%202023-09-05%20at%2012.03.36%20PM.png?generation=1693895639308885&amp;alt=media\" alt=\"\"></p>",
  "messages": [
    {
      "id": 2424282,
      "postDate": "2023-09-05T06:57:35.227Z",
      "content": "<p>Sharing a recent MICCAI 2023 publication &amp; it's Github repo which could potentially be useful in this competition. </p>\n<h2>A new segment anything model is released which is pretrained on medical related 4.6M images and 19.7M masks.</h2>\n<ul>\n<li>Covers more than 30+ human organs</li>\n<li>Finetuned original SAM on these images</li>\n<li>SAM-Med2D outperforms SAM in medical image segmentation with over 5% performance gap</li>\n<li>Pretrained Checkpoint weights are available. </li>\n</ul>\n<h3><em>Training code is still not released but should be easy to finetune the model by extracting encoder blocks or using the model to extract masks of oragns.</em></h3>\n<p>Github Link: <a href=\"https://github.com/OpenGVLab/SAM-Med2D\" target=\"_blank\">https://github.com/OpenGVLab/SAM-Med2D</a><br>\nPaper Link: <a href=\"https://arxiv.org/pdf/2308.16184.pdf\" target=\"_blank\">https://arxiv.org/pdf/2308.16184.pdf</a></p>\n<p><img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4712534%2F4a0f1b08676a07ba2900a5287549cf8e%2FScreenshot%202023-09-05%20at%2012.03.36%20PM.png?generation=1693895639308885&amp;alt=media\" alt=\"\"></p>",
      "rawMarkdown": "Sharing a recent MICCAI 2023 publication & it's Github repo which could potentially be useful in this competition. \n\n## A new segment anything model is released which is pretrained on medical related 4.6M images and 19.7M masks. \n\n- Covers more than 30+ human organs\n- Finetuned original SAM on these images\n- SAM-Med2D outperforms SAM in medical image segmentation with over 5% performance gap\n- Pretrained Checkpoint weights are available. \n### *Training code is still not released but should be easy to finetune the model by extracting encoder blocks or using the model to extract masks of oragns.*\n\n\nGithub Link: https://github.com/OpenGVLab/SAM-Med2D\nPaper Link: https://arxiv.org/pdf/2308.16184.pdf\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4712534%2F4a0f1b08676a07ba2900a5287549cf8e%2FScreenshot%202023-09-05%20at%2012.03.36%20PM.png?generation=1693895639308885&alt=media)",
      "votes": 20
    },
    {
      "id": 2424462,
      "postDate": "2023-09-05T09:44:29.400Z",
      "content": "<p>This model requires manually provided points/boxes, isn't it?</p>",
      "rawMarkdown": "This model requires manually provided points/boxes, isn't it?",
      "votes": 1,
      "replies": [
        {
          "id": 2424464,
          "postDate": "2023-09-05T09:46:12.157Z",
          "content": "<p>For training: Yes, for inference: No</p>",
          "rawMarkdown": "For training: Yes, for inference: No",
          "replies": [
            {
              "id": 2424514,
              "postDate": "2023-09-05T10:37:19.097Z",
              "content": "<p>I have seen gradio demo and it requires manually points like original SAM model</p>",
              "rawMarkdown": "I have seen gradio demo and it requires manually points like original SAM model",
              "votes": 2
            },
            {
              "id": 2426788,
              "postDate": "2023-09-06T20:20:32.257Z",
              "content": "<blockquote>\n  <p>For training: Yes, for inference: No</p>\n</blockquote>\n<p>Are you sure?</p>",
              "rawMarkdown": "> For training: Yes, for inference: No\n\n\nAre you sure?\n\n",
              "votes": 2
            }
          ]
        },
        {
          "id": 2426768,
          "postDate": "2023-09-06T19:50:24.310Z",
          "content": "<p>Yes, SAM works with either one point or bounding box prompts. It's very useful when you have weak labels like that but it doesn't provide segmentation maps of each organ out of the box.</p>",
          "rawMarkdown": "Yes, SAM works with either one point or bounding box prompts. It's very useful when you have weak labels like that but it doesn't provide segmentation maps of each organ out of the box."
        }
      ]
    },
    {
      "id": 2426096,
      "postDate": "2023-09-06T12:01:46.340Z",
      "content": "<p>Is there a straightforward way of using SAM to actually label/annotate the segmented images?</p>",
      "rawMarkdown": "Is there a straightforward way of using SAM to actually label/annotate the segmented images?",
      "replies": [
        {
          "id": 2428798,
          "postDate": "2023-09-08T06:59:42.440Z",
          "content": "<p>You train a good enough model with high specificity (true negative rate) so it won't output that much false positives. You use that model's outputs as a prompt to SAM and refine masks. </p>",
          "rawMarkdown": "You train a good enough model with high specificity (true negative rate) so it won't output that much false positives. You use that model's outputs as a prompt to SAM and refine masks. ",
          "votes": 1
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 2424462,
      "author_name": "JanGlinko2",
      "author_url": "",
      "post_date": "2023-09-05T09:44:29.400000",
      "content": "<p>This model requires manually provided points/boxes, isn't it?</p>",
      "votes": 1,
      "replies": [
        {
          "id": 2424464,
          "author_name": "ThomCat",
          "author_url": "",
          "post_date": "2023-09-05T09:46:12.157000",
          "content": "<p>For training: Yes, for inference: No</p>",
          "votes": 0,
          "replies": [
            {
              "id": 2424514,
              "author_name": "Aleksandr Lavrikov",
              "author_url": "",
              "post_date": "2023-09-05T10:37:19.097000",
              "content": "<p>I have seen gradio demo and it requires manually points like original SAM model</p>",
              "votes": 2,
              "replies": []
            },
            {
              "id": 2426788,
              "author_name": "Gyula Maloveczky4",
              "author_url": "",
              "post_date": "2023-09-06T20:20:32.257000",
              "content": "<blockquote>\n  <p>For training: Yes, for inference: No</p>\n</blockquote>\n<p>Are you sure?</p>",
              "votes": 2,
              "replies": []
            }
          ]
        },
        {
          "id": 2426768,
          "author_name": "Gunes Evitan",
          "author_url": "",
          "post_date": "2023-09-06T19:50:24.310000",
          "content": "<p>Yes, SAM works with either one point or bounding box prompts. It's very useful when you have weak labels like that but it doesn't provide segmentation maps of each organ out of the box.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 2426096,
      "author_name": "Gyula Maloveczky4",
      "author_url": "",
      "post_date": "2023-09-06T12:01:46.340000",
      "content": "<p>Is there a straightforward way of using SAM to actually label/annotate the segmented images?</p>",
      "votes": 0,
      "replies": [
        {
          "id": 2428798,
          "author_name": "Gunes Evitan",
          "author_url": "",
          "post_date": "2023-09-08T06:59:42.440000",
          "content": "<p>You train a good enough model with high specificity (true negative rate) so it won't output that much false positives. You use that model's outputs as a prompt to SAM and refine masks. </p>",
          "votes": 1,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2424282": "Sharing a recent MICCAI 2023 publication & it's Github repo which could potentially be useful in this competition. \n\n## A new segment anything model is released which is pretrained on medical related 4.6M images and 19.7M masks. \n\n- Covers more than 30+ human organs\n- Finetuned original SAM on these images\n- SAM-Med2D outperforms SAM in medical image segmentation with over 5% performance gap\n- Pretrained Checkpoint weights are available. \n### *Training code is still not released but should be easy to finetune the model by extracting encoder blocks or using the model to extract masks of oragns.*\n\n\nGithub Link: https://github.com/OpenGVLab/SAM-Med2D\nPaper Link: https://arxiv.org/pdf/2308.16184.pdf\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F4712534%2F4a0f1b08676a07ba2900a5287549cf8e%2FScreenshot%202023-09-05%20at%2012.03.36%20PM.png?generation=1693895639308885&alt=media)",
    "2424462": "This model requires manually provided points/boxes, isn't it?",
    "2426096": "Is there a straightforward way of using SAM to actually label/annotate the segmented images?"
  }
}