{ "id": 372673, "title": "GMIC transfer learning", "url": "/competitions/rsna-breast-cancer-detection/discussion/372673", "author_name": "@kaggleqrdl", "post_date": "2022-12-17T10:07:20.084000", "votes": 3, "comment_count": 17, "views": 0, "content": "

https://github.com/nyukat/GMIC

\n

I gave this a spin, seems to work fairly well on a quick test. I got about a .2 prediction for cancer, .03 otherwise, and that was with some fairly unintelligent resizing. Probably do better with more intelligent cropping and a deeper read of what the model needs to work properly.

\n

https://www.kaggle.com/code/kaggleqrdl/gmic-test

", "messages": [ { "id": 2067914, "postDate": "2022-12-17T10:07:20.083Z", "content": "

https://github.com/nyukat/GMIC

\n

I gave this a spin, seems to work fairly well on a quick test. I got about a .2 prediction for cancer, .03 otherwise, and that was with some fairly unintelligent resizing. Probably do better with more intelligent cropping and a deeper read of what the model needs to work properly.

\n

https://www.kaggle.com/code/kaggleqrdl/gmic-test

", "rawMarkdown": "https://github.com/nyukat/GMIC\n\nI gave this a spin, seems to work fairly well on a quick test. I got about a .2 prediction for cancer, .03 otherwise, and that was with some fairly unintelligent resizing. Probably do better with more intelligent cropping and a deeper read of what the model needs to work properly.\n\nhttps://www.kaggle.com/code/kaggleqrdl/gmic-test\n", "votes": 3 }, { "id": 2068058, "postDate": "2022-12-17T13:11:56.540Z", "content": "

seems that many past winning solution are using smiliar 2 stages approaches:
\nhttps://blogs.nvidia.com/blog/2018/02/01/making-mammography-more-meaningful/

\n

\"Using 10,000 standard mammograms, Therapixel trained one set of algorithms to find all of the cancer-specific anomalies in the images. A second neural network, acting on a coarser scale, was then trained based on the first network. This second network was able to return the best results when it came to calculating the likelihood of a patient developing breast cancer\"

\n

https://www.lri.fr/~gcharpia/deeppractice/2020/tpx_for_mva_dl_course_2020.pdf

\n

\"Selection-222\"
\n\"Selection-221\"

", "rawMarkdown": "seems that many past winning solution are using smiliar 2 stages approaches:\nhttps://blogs.nvidia.com/blog/2018/02/01/making-mammography-more-meaningful/\n\n\"Using 10,000 standard mammograms, Therapixel trained one set of algorithms to find all of the cancer-specific anomalies in the images. A second neural network, acting on a coarser scale, was then trained based on the first network. This second network was able to return the best results when it came to calculating the likelihood of a patient developing breast cancer\"\n\nhttps://www.lri.fr/~gcharpia/deeppractice/2020/tpx_for_mva_dl_course_2020.pdf\n\n\"Selection-222\"\n\"Selection-221\"", "votes": 1, "replies": [ { "id": 2068059, "postDate": "2022-12-17T13:14:06.047Z", "content": "

Another thing I'm seeing with GMIC is the notion of benign/malignant lesions. We just have binary cancer for this comp.

", "rawMarkdown": "Another thing I'm seeing with GMIC is the notion of benign/malignant lesions. We just have binary cancer for this comp. " }, { "id": 2068069, "postDate": "2022-12-17T13:27:02Z", "content": "

actually their solutions may more sense.

\n
    \n
  1. pretain a \"coarse scale segmentation model, e.g. at 1/32 scale\" using open dataset like DDSM, inBreast with lesion/abnormality annotation
    \n2.extend the pretrain model with image classification head and finetune on kaggle dataset
    2. \n
", "rawMarkdown": "actually their solutions may more sense.\n1. pretain a \"coarse scale segmentation model, e.g. at 1/32 scale\" using open dataset like DDSM, inBreast with lesion/abnormality annotation\n2.extend the pretrain model with image classification head and finetune on kaggle dataset", "replies": [ { "id": 2068080, "postDate": "2022-12-17T13:33:47.327Z", "content": "

hmm … gpt3chat knows everything …
\n\"Selection-224\"
\n\"Selection-223\"

", "rawMarkdown": "hmm ... gpt3chat knows everything ...\n\"Selection-224\"\n\"Selection-223\"", "votes": 1 }, { "id": 2068543, "postDate": "2022-12-18T05:26:11.730Z", "content": "

What about using yolo trained on ddsm to extract ROI patches? Probably faster than GMIC. Maybe more accurate as well.

", "rawMarkdown": "What about using yolo trained on ddsm to extract ROI patches? Probably faster than GMIC. Maybe more accurate as well.", "votes": 1 } ] } ] }, { "id": 2099990, "postDate": "2023-01-14T20:05:37.503Z", "content": "

Have you had any luck finetuning GMIC? It seems to be well-trained but even after reading the scrubbing paper and other literature that finetunes it, I am missing something. I can't get better results than about .75 AUC for malignant

", "rawMarkdown": "Have you had any luck finetuning GMIC? It seems to be well-trained but even after reading the scrubbing paper and other literature that finetunes it, I am missing something. I can't get better results than about .75 AUC for malignant ", "replies": [ { "id": 2099994, "postDate": "2023-01-14T20:10:20.690Z", "content": "

It is also interesting to note that their sampling strategy during training seems superior, they upsample both benign and malignant samples and match their cases with normal findings. However, we only have labels for cancerous (malignant) exams. But using the provided BIRADS (only for site_id == 1) we can upsample the benign findings as well but I saw no noticeable improve in training. Maybe we don't have as much data as they did.

", "rawMarkdown": "It is also interesting to note that their sampling strategy during training seems superior, they upsample both benign and malignant samples and match their cases with normal findings. However, we only have labels for cancerous (malignant) exams. But using the provided BIRADS (only for site_id == 1) we can upsample the benign findings as well but I saw no noticeable improve in training. Maybe we don't have as much data as they did.", "replies": [ { "id": 2100469, "postDate": "2023-01-15T07:44:03.033Z", "content": "

Had an issue on my end and AUC .84+, with only using one site. might be worth looking at. very fast arch but data requirements are 292 GB for the training dataset.

", "rawMarkdown": "Had an issue on my end and AUC .84+, with only using one site. might be worth looking at. very fast arch but data requirements are 292 GB for the training dataset. " } ] } ] }, { "id": 2067936, "postDate": "2022-12-17T10:46:51.533Z", "content": "

I tried more on the following patient ids from site 2. No cherry picking here, just grabbed first 5 patients with cancer.

\n

10432 - 0.2 on L-CC, 0.11 on L-MLO, 0.03 on R-CC, 0.025 in R-MLO (cancer in L)
\n106 - 0.11 on L-MLO, 0.017 in L-CC, 0.013 in R-CC, 0.018 in R-MLO (cancer in L, signal in MLO but not CC)
\n10635 - 0.025 on L-MLO, 0.023 in L-CC, 0.014 in R-CC, 0.036 R-MLO (cancer in L, looks like it missed it)
\n10638 - 0.022 on L-MLO, 0.03 in L-CC, 0.028 in R-CC, 0.039 in R-MLO (cancer in L, missed as well)
\n10940 - 0.023 on L-MLO, 0.045 in L-CC, 0.36 in R-CC, 0.123 in R-MLO (cancer in R, caught that nicely)

\n

no false positives. Let's see what the pf score is with thresh around .11..

", "rawMarkdown": "I tried more on the following patient ids from site 2. No cherry picking here, just grabbed first 5 patients with cancer.\n\n10432 - 0.2 on L-CC, 0.11 on L-MLO, 0.03 on R-CC, 0.025 in R-MLO (cancer in L)\n106 - 0.11 on L-MLO, 0.017 in L-CC, 0.013 in R-CC, 0.018 in R-MLO (cancer in L, signal in MLO but not CC)\n10635 - 0.025 on L-MLO, 0.023 in L-CC, 0.014 in R-CC, 0.036 R-MLO (cancer in L, looks like it missed it)\n10638 - 0.022 on L-MLO, 0.03 in L-CC, 0.028 in R-CC, 0.039 in R-MLO (cancer in L, missed as well)\n10940 - 0.023 on L-MLO, 0.045 in L-CC, 0.36 in R-CC, 0.123 in R-MLO (cancer in R, caught that nicely)\n\nno false positives. Let's see what the pf score is with thresh around .11..", "replies": [ { "id": 2067947, "postDate": "2022-12-17T10:55:28.883Z", "content": "

pfbeta([1,1,1,1,1,0,0,0,0,0], [1,1,1,0,0,0,0,0,0,0],1)

\n

0.7499999999999999

\n

Let me know if I'm out of whack there. Doing a bunch of TNs now, as that's where it gets you. So far the thresh holds, but have to do a lot clearly.

", "rawMarkdown": "pfbeta([1,1,1,1,1,0,0,0,0,0], [1,1,1,0,0,0,0,0,0,0],1)\n\n0.7499999999999999\n\nLet me know if I'm out of whack there. Doing a bunch of TNs now, as that's where it gets you. So far the thresh holds, but have to do a lot clearly. ", "replies": [ { "id": 2068120, "postDate": "2022-12-17T14:32:09.110Z", "content": "

Interesting ..dicomsdl gives different results!

", "rawMarkdown": "Interesting ..dicomsdl gives different results!" } ] } ] }, { "id": 2067926, "postDate": "2022-12-17T10:29:49.863Z", "content": "

\"Probably do better with more intelligent cropping and a deeper read of what the model needs to work properly.\"

\n

start by using this as a pretrain models and fine-tune on kaggle dataset. check if the license allows

", "rawMarkdown": "\"Probably do better with more intelligent cropping and a deeper read of what the model needs to work properly.\"\n\nstart by using this as a pretrain models and fine-tune on kaggle dataset. check if the license allows", "replies": [ { "id": 2067943, "postDate": "2022-12-17T10:51:43.193Z", "content": "

Well, there's meta transfer learning, right. Read the paper, reproduce from scratch, etc. Good just to see the model and ideas prove out with such a basic test

", "rawMarkdown": "Well, there's meta transfer learning, right. Read the paper, reproduce from scratch, etc. Good just to see the model and ideas prove out with such a basic test" } ] }, { "id": 2067925, "postDate": "2022-12-17T10:29:00.783Z", "content": "

actually, you can write e-mail to the author and invite him for a team-up. maybe you can learn some tricks from him

", "rawMarkdown": "actually, you can write e-mail to the author and invite him for a team-up. maybe you can learn some tricks from him", "replies": [ { "id": 2067938, "postDate": "2022-12-17T10:48:03.893Z", "content": "

Not going to compete in this one, just encourage folks to do well. Breast cancer.. boy it'd be nice for kaggle to shine here. Even if we're not dealing with SOTA data, having models that work well on basic stuff would be great as not everyone has access to SOTA machine / staff.

", "rawMarkdown": "Not going to compete in this one, just encourage folks to do well. Breast cancer.. boy it'd be nice for kaggle to shine here. Even if we're not dealing with SOTA data, having models that work well on basic stuff would be great as not everyone has access to SOTA machine / staff." } ] }, { "id": 2067923, "postDate": "2022-12-17T10:21:35.913Z", "content": "

more here https://github.com/nyukat/mammography_metarepository

", "rawMarkdown": "more here https://github.com/nyukat/mammography_metarepository", "replies": [ { "id": 2068496, "postDate": "2022-12-18T03:04:41.310Z", "content": "

Sort of ironic how they have this crazy copy left license and then create this arch / framework to try to get around it. lol

", "rawMarkdown": "Sort of ironic how they have this crazy copy left license and then create this arch / framework to try to get around it. lol", "votes": 1 } ] } ], "comments": [ { "id": 2068058, "author_name": "hengck23", "author_url": "", "post_date": "2022-12-17T13:11:56.540000", "content": "

seems that many past winning solution are using smiliar 2 stages approaches:
\nhttps://blogs.nvidia.com/blog/2018/02/01/making-mammography-more-meaningful/

\n

\"Using 10,000 standard mammograms, Therapixel trained one set of algorithms to find all of the cancer-specific anomalies in the images. A second neural network, acting on a coarser scale, was then trained based on the first network. This second network was able to return the best results when it came to calculating the likelihood of a patient developing breast cancer\"

\n

https://www.lri.fr/~gcharpia/deeppractice/2020/tpx_for_mva_dl_course_2020.pdf

\n

\"Selection-222\"
\n\"Selection-221\"

", "votes": 1, "replies": [ { "id": 2068059, "author_name": "@kaggleqrdl", "author_url": "", "post_date": "2022-12-17T13:14:06.047000", "content": "

Another thing I'm seeing with GMIC is the notion of benign/malignant lesions. We just have binary cancer for this comp.

", "votes": 0, "replies": [] }, { "id": 2068069, "author_name": "hengck23", "author_url": "", "post_date": "2022-12-17T13:27:02", "content": "

actually their solutions may more sense.

\n
    \n
  1. pretain a \"coarse scale segmentation model, e.g. at 1/32 scale\" using open dataset like DDSM, inBreast with lesion/abnormality annotation
    \n2.extend the pretrain model with image classification head and finetune on kaggle dataset
    2. \n
", "votes": 0, "replies": [ { "id": 2068080, "author_name": "hengck23", "author_url": "", "post_date": "2022-12-17T13:33:47.327000", "content": "

hmm … gpt3chat knows everything …
\n\"Selection-224\"
\n\"Selection-223\"

", "votes": 1, "replies": [] }, { "id": 2068543, "author_name": "@kaggleqrdl", "author_url": "", "post_date": "2022-12-18T05:26:11.730000", "content": "

What about using yolo trained on ddsm to extract ROI patches? Probably faster than GMIC. Maybe more accurate as well.

", "votes": 1, "replies": [] } ] } ] }, { "id": 2099990, "author_name": "outwrest", "author_url": "", "post_date": "2023-01-14T20:05:37.503000", "content": "

Have you had any luck finetuning GMIC? It seems to be well-trained but even after reading the scrubbing paper and other literature that finetunes it, I am missing something. I can't get better results than about .75 AUC for malignant

", "votes": 0, "replies": [ { "id": 2099994, "author_name": "outwrest", "author_url": "", "post_date": "2023-01-14T20:10:20.690000", "content": "

It is also interesting to note that their sampling strategy during training seems superior, they upsample both benign and malignant samples and match their cases with normal findings. However, we only have labels for cancerous (malignant) exams. But using the provided BIRADS (only for site_id == 1) we can upsample the benign findings as well but I saw no noticeable improve in training. Maybe we don't have as much data as they did.

", "votes": 0, "replies": [ { "id": 2100469, "author_name": "outwrest", "author_url": "", "post_date": "2023-01-15T07:44:03.033000", "content": "

Had an issue on my end and AUC .84+, with only using one site. might be worth looking at. very fast arch but data requirements are 292 GB for the training dataset.

", "votes": 0, "replies": [] } ] } ] }, { "id": 2067936, "author_name": "@kaggleqrdl", "author_url": "", "post_date": "2022-12-17T10:46:51.533000", "content": "

I tried more on the following patient ids from site 2. No cherry picking here, just grabbed first 5 patients with cancer.

\n

10432 - 0.2 on L-CC, 0.11 on L-MLO, 0.03 on R-CC, 0.025 in R-MLO (cancer in L)
\n106 - 0.11 on L-MLO, 0.017 in L-CC, 0.013 in R-CC, 0.018 in R-MLO (cancer in L, signal in MLO but not CC)
\n10635 - 0.025 on L-MLO, 0.023 in L-CC, 0.014 in R-CC, 0.036 R-MLO (cancer in L, looks like it missed it)
\n10638 - 0.022 on L-MLO, 0.03 in L-CC, 0.028 in R-CC, 0.039 in R-MLO (cancer in L, missed as well)
\n10940 - 0.023 on L-MLO, 0.045 in L-CC, 0.36 in R-CC, 0.123 in R-MLO (cancer in R, caught that nicely)

\n

no false positives. Let's see what the pf score is with thresh around .11..

", "votes": 0, "replies": [ { "id": 2067947, "author_name": "@kaggleqrdl", "author_url": "", "post_date": "2022-12-17T10:55:28.883000", "content": "

pfbeta([1,1,1,1,1,0,0,0,0,0], [1,1,1,0,0,0,0,0,0,0],1)

\n

0.7499999999999999

\n

Let me know if I'm out of whack there. Doing a bunch of TNs now, as that's where it gets you. So far the thresh holds, but have to do a lot clearly.

", "votes": 0, "replies": [ { "id": 2068120, "author_name": "@kaggleqrdl", "author_url": "", "post_date": "2022-12-17T14:32:09.110000", "content": "

Interesting ..dicomsdl gives different results!

", "votes": 0, "replies": [] } ] } ] }, { "id": 2067926, "author_name": "hengck23", "author_url": "", "post_date": "2022-12-17T10:29:49.863000", "content": "

\"Probably do better with more intelligent cropping and a deeper read of what the model needs to work properly.\"

\n

start by using this as a pretrain models and fine-tune on kaggle dataset. check if the license allows

", "votes": 0, "replies": [ { "id": 2067943, "author_name": "@kaggleqrdl", "author_url": "", "post_date": "2022-12-17T10:51:43.193000", "content": "

Well, there's meta transfer learning, right. Read the paper, reproduce from scratch, etc. Good just to see the model and ideas prove out with such a basic test

", "votes": 0, "replies": [] } ] }, { "id": 2067925, "author_name": "hengck23", "author_url": "", "post_date": "2022-12-17T10:29:00.783000", "content": "

actually, you can write e-mail to the author and invite him for a team-up. maybe you can learn some tricks from him

", "votes": 0, "replies": [ { "id": 2067938, "author_name": "@kaggleqrdl", "author_url": "", "post_date": "2022-12-17T10:48:03.893000", "content": "

Not going to compete in this one, just encourage folks to do well. Breast cancer.. boy it'd be nice for kaggle to shine here. Even if we're not dealing with SOTA data, having models that work well on basic stuff would be great as not everyone has access to SOTA machine / staff.

", "votes": 0, "replies": [] } ] }, { "id": 2067923, "author_name": "hengck23", "author_url": "", "post_date": "2022-12-17T10:21:35.913000", "content": "

more here https://github.com/nyukat/mammography_metarepository

", "votes": 0, "replies": [ { "id": 2068496, "author_name": "@kaggleqrdl", "author_url": "", "post_date": "2022-12-18T03:04:41.310000", "content": "

Sort of ironic how they have this crazy copy left license and then create this arch / framework to try to get around it. lol

", "votes": 1, "replies": [] } ] } ], "raw_markdown_by_id": { "2067914": "https://github.com/nyukat/GMIC\n\nI gave this a spin, seems to work fairly well on a quick test. I got about a .2 prediction for cancer, .03 otherwise, and that was with some fairly unintelligent resizing. Probably do better with more intelligent cropping and a deeper read of what the model needs to work properly.\n\nhttps://www.kaggle.com/code/kaggleqrdl/gmic-test\n", "2068058": "seems that many past winning solution are using smiliar 2 stages approaches:\nhttps://blogs.nvidia.com/blog/2018/02/01/making-mammography-more-meaningful/\n\n\"Using 10,000 standard mammograms, Therapixel trained one set of algorithms to find all of the cancer-specific anomalies in the images. A second neural network, acting on a coarser scale, was then trained based on the first network. This second network was able to return the best results when it came to calculating the likelihood of a patient developing breast cancer\"\n\nhttps://www.lri.fr/~gcharpia/deeppractice/2020/tpx_for_mva_dl_course_2020.pdf\n\n\"Selection-222\"\n\"Selection-221\"", "2099990": "Have you had any luck finetuning GMIC? It seems to be well-trained but even after reading the scrubbing paper and other literature that finetunes it, I am missing something. I can't get better results than about .75 AUC for malignant ", "2067936": "I tried more on the following patient ids from site 2. No cherry picking here, just grabbed first 5 patients with cancer.\n\n10432 - 0.2 on L-CC, 0.11 on L-MLO, 0.03 on R-CC, 0.025 in R-MLO (cancer in L)\n106 - 0.11 on L-MLO, 0.017 in L-CC, 0.013 in R-CC, 0.018 in R-MLO (cancer in L, signal in MLO but not CC)\n10635 - 0.025 on L-MLO, 0.023 in L-CC, 0.014 in R-CC, 0.036 R-MLO (cancer in L, looks like it missed it)\n10638 - 0.022 on L-MLO, 0.03 in L-CC, 0.028 in R-CC, 0.039 in R-MLO (cancer in L, missed as well)\n10940 - 0.023 on L-MLO, 0.045 in L-CC, 0.36 in R-CC, 0.123 in R-MLO (cancer in R, caught that nicely)\n\nno false positives. Let's see what the pf score is with thresh around .11..", "2067926": "\"Probably do better with more intelligent cropping and a deeper read of what the model needs to work properly.\"\n\nstart by using this as a pretrain models and fine-tune on kaggle dataset. check if the license allows", "2067925": "actually, you can write e-mail to the author and invite him for a team-up. maybe you can learn some tricks from him", "2067923": "more here https://github.com/nyukat/mammography_metarepository" } }