\n
![\"https://i.ibb.co/vXx9MX3/Selection-401.png\"](\"https://i.ibb.co/vXx9MX3/Selection-401.png\")
\n
![\"https://i.ibb.co/L00DzmW/Selection-400.png\"](\"https://i.ibb.co/L00DzmW/Selection-400.png\")
results on validation:
\n
\n
Question : How is very small lesions detected? i can find many similar regions that are false positives. How do you look at the whole image to decide if it is cancerous or not? Or do you really can decide just based on local context region?
\n(basically i am trying to decide the context for good detection but it seems impossible. It is even difficult on training set)
", "messages": [ { "id": 2081540, "postDate": "2022-12-31T10:11:24.500Z", "content": "results on validation:
\n
\n
Question : How is very small lesions detected? i can find many similar regions that are false positives. How do you look at the whole image to decide if it is cancerous or not? Or do you really can decide just based on local context region?
\n(basically i am trying to decide the context for good detection but it seems impossible. It is even difficult on training set)
", "rawMarkdown": "results on validation:\n\n\n\nQuestion : How is very small lesions detected? i can find many similar regions that are false positives. How do you look at the whole image to decide if it is cancerous or not? Or do you really can decide just based on local context region?\n\n(basically i am trying to decide the context for good detection but it seems impossible. It is even difficult on training set)", "votes": 29 }, { "id": 2081791, "postDate": "2022-12-31T16:18:52.997Z", "content": "I am not a radiologist, but I have worked on medical imaging software for 30 years and am very familiar with mammography. I don't think it's a reasonable assumption that a radiologist would be able to diagnose cancer from a tiny density like the ones in these images.
\nNormally, CAD will pick up the small densities and direct the radiologists attention to the area. Then, the radiologist will \"window\" the image manually to try and find the \"sweet spot\" where the density in question is separated from the rest of the tissue as much as possible in order to determine the location, shape, symmetry etc. Most of the time, a follow on ultrasound (and a biopsy) is needed to verify what the dense tissue actually is.
\nSince these are screening mammograms (not follow ups), we don't expect to see many cases of large tumors. It seems the goal of this competition is to do something that humans cannot do .. which is to diagnose cancer from imaging alone.
", "rawMarkdown": "I am not a radiologist, but I have worked on medical imaging software for 30 years and am very familiar with mammography. I don't think it's a reasonable assumption that a radiologist would be able to diagnose cancer from a tiny density like the ones in these images.\n\nNormally, CAD will pick up the small densities and direct the radiologists attention to the area. Then, the radiologist will \"window\" the image manually to try and find the \"sweet spot\" where the density in question is separated from the rest of the tissue as much as possible in order to determine the location, shape, symmetry etc. Most of the time, a follow on ultrasound (and a biopsy) is needed to verify what the dense tissue actually is.\n\nSince these are screening mammograms (not follow ups), we don't expect to see many cases of large tumors. It seems the goal of this competition is to do something that humans cannot do .. which is to diagnose cancer from imaging alone.", "votes": 20, "replies": [ { "id": 2081918, "postDate": "2022-12-31T20:15:56.937Z", "content": "thx for this answer @davidbroberts, so much good info here 🙏 I think I learned more from your reply here on how all of this works, how the work gets done vs many of the much longer posts, really cool 🙂
\nthank you!
", "rawMarkdown": "thx for this answer @davidbroberts, so much good info here 🙏 I think I learned more from your reply here on how all of this works, how the work gets done vs many of the much longer posts, really cool 🙂\n\nthank you!", "votes": 1 }, { "id": 2082005, "postDate": "2023-01-01T01:18:40.297Z", "content": "@davidbroberts
\nthanks a lot for the reply.
so i can assume:
\n[1] the annotation process is likely to follows what you have described. even in open dataset like vindr, the process will be CAD+verification, i.e. there may be miss detection by CAD.
[2] hence one can give out of trying to perfect a segmentation model for mask annotation. instead one should devise a mask model to include all positive mask, but reduce the num of negative mask. it is ok that for one cancer image to have at least one +ve mask and several neg mask.
\nor even in aux loss, we cannot assume to have perfect annotation for mask.
\nnow i understand why they use weak models like resnet22 in heatmap prediction (and not a strong one like renset50)
[3] Now i understand whay the two-stage approach:
\nthe loss function is important to ensure the model is high sensitivity or specificity in the two stages.
\nactually i also have another observation at external data:
\nalthough the sore for mask detection is poor (e.g. for small region), the image level prediction (e.g. BIRADS) is generally good.
\ni use a joint loss end-to-end model (heatmap prediction + image level prediction)
check this !!!
\nhttps://www.youtube.com/watch?v=FJl_yf5Au68&t=138s
\nHow to read a screening mammogram
hence, it is not important to consider use of simple model like resnet22,34 efficientnet b2 or complicated large ones like efficientnet b7.
\na good workflow is more important. e.g. models for different scales, different parts, models for single image, models for multiple images of same laterality, multiple images of different laterality, models for ROI level, models for image levels ….
\nan ensemble of different type of models seems to be the key.
\n![\"Selection-421\"](\"https://i.ibb.co/1MbP1Jf/Selection-421.png\")
\n![\"Selection-420\"](\"https://i.ibb.co/pWPCZLD/Selection-420.png\")
\n![\"Selection-419\"](\"https://i.ibb.co/4fN13Dk/Selection-419.png\")
\n\n\n\n\n",
"votes": 4
},
{
"id": 2082044,
"postDate": "2023-01-01T03:28:17.150Z",
"content": "![\"https://i.ibb.co/Z2bW7MY/Selection-425.png\"](\"https://i.ibb.co/Z2bW7MY/Selection-425.png\")
\n![\"https://i.ibb.co/LRy2GjL/Selection-432.png\"](\"https://i.ibb.co/LRy2GjL/Selection-432.png\")
That's pretty interesting. Looks like laterality alignment (left ontop of right) is a critical pre-processing activity.
", "rawMarkdown": "That's pretty interesting. Looks like laterality alignment (left ontop of right) is a critical pre-processing activity." }, { "id": 2090891, "postDate": "2023-01-07T18:36:26.867Z", "content": "\n\na good workflow is more important. e.g. models for different scales, different parts, models for single image, models for multiple images of same laterality, multiple images of different laterality, models for ROI level, models for image levels ….
\n
and just how would such multiplicity of models work in a notebook in under 9 hours on 43K high resolution images?
", "rawMarkdown": ">a good workflow is more important. e.g. models for different scales, different parts, models for single image, models for multiple images of same laterality, multiple images of different laterality, models for ROI level, models for image levels ….\n\nand just how would such multiplicity of models work in a notebook in under 9 hours on 43K high resolution images?" }, { "id": 2091004, "postDate": "2023-01-07T22:29:54.830Z", "content": "Yep, that's the issue. TPU / tensorflow might be the way to go as it performs generally better than pytorch, but harder to use.
", "rawMarkdown": "Yep, that's the issue. TPU / tensorflow might be the way to go as it performs generally better than pytorch, but harder to use.", "votes": 1 }, { "id": 2091010, "postDate": "2023-01-07T22:45:57.960Z", "content": "I agree…
\nmy point is, we need to be extremely stingy in our approach… simple and effective trickery is worth more than here than complex models…
\nwork in progress…
Check this out - https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/376630
", "rawMarkdown": "@houssamassila \n\nCheck this out - https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/376630", "votes": 1 }, { "id": 2091031, "postDate": "2023-01-08T00:03:33.193Z", "content": "yes I did check that earlier today. thanks.
\neven further than just enlarging the usable ram, trickery regarding the dataset is also useful, like using smaller files when possible/reducing the dataset (even if it seems counter-intuitive at first). To me, this seems more the way to go than spending time trying ever more extravaggant models that won't run in 9 hours.
Hmm, someone is saying we can't use TPU for this comp. Which is weird, given that someone created a TPU notebook. But it might be useful for training.
\nedit: reading up on it, I think it's because the TPUs require internet access which is not allowed in this comp
", "rawMarkdown": "Hmm, someone is saying we can't use TPU for this comp. Which is weird, given that someone created a TPU notebook. But it might be useful for training.\n\nedit: reading up on it, I think it's because the TPUs require internet access which is not allowed in this comp", "votes": 1 }, { "id": 2091353, "postDate": "2023-01-08T10:19:10.467Z", "content": "it's as you said, TPU will be useful only to train models that can later be loaded into the submission notebook.
", "rawMarkdown": "it's as you said, TPU will be useful only to train models that can later be loaded into the submission notebook.\n" }, { "id": 2094507, "postDate": "2023-01-10T19:39:39.123Z", "content": "@houssamassila
\n\"just how would such multiplicity of models work in a notebook in under 9 hours on 43K high resolution images?\"
\"early rejection\" is a common trick in computer vision. 10x speed up is not problem
\nhttps://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/377008#2094477
", "rawMarkdown": "@houssamassila \n\"just how would such multiplicity of models work in a notebook in under 9 hours on 43K high resolution images?\"\n\n\n\"early rejection\" is a common trick in computer vision. 10x speed up is not problem\n\nhttps://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/377008#2094477" } ] } ] }, { "id": 2125004, "postDate": "2023-02-01T11:12:57.127Z", "content": "I see you are looking for a radiologist to team up @hengck23. I worked with @sandorkonya before and he is the best one I've ever worked with. Tagged both of you for potential collab.
", "rawMarkdown": "I see you are looking for a radiologist to team up @hengck23. I worked with @sandorkonya before and he is the best one I've ever worked with. Tagged both of you for potential collab.", "votes": 2, "replies": [ { "id": 2125090, "postDate": "2023-02-01T12:14:31.927Z", "content": "The idea is simple.
\nWith my top models, i will generate CAM heatmap.
\nThen the radiology correct it. (Delete very sure wrong heat map for activated case, mark possible one for missed cases)
i do not need very accurate annotation. one can simply just annotate a point.
\nI do not need all annotation, just at least one per pos image.
only 2% of train data are pos. this is very few images. annotation can done within hours (because i do not need good annotation).
\nBasically it is manual active learning. we will go through 2~3 rounds.
\nLet's see if this will fight imbalance data and gives better generalization, rather than relying on unstable LB/CV
\nhttps://ojs.aaai.org/index.php/AAAI/article/download/16153/15960
\nfigure.2 is example of point annotation
@hengck23, you probably know this, but the vindr-mammo dataset already has the bounding box annotations that you are looking for. It would be interesting to see how much juice you can squeeze out of those annotations - gives you a quick way to assess if it's worth getting the RSNA dataset annotated by a radiologist…
\nMy intuition is that it would help a lot. If you can show a model the ROI that causes a mammogram to be tagged as positive, the model is likely to learn the important features better. I explored this a bit, but haven't found a good way to incorporate the bounding box data. One possible method is to divide the images into tiles, label each tile and then train on the tiles and their labels. The hard part, at least for me is aggregating the tile-level predictions into a single prediction for the whole image. A global maxpooling operation doesn't seem to work well.
", "rawMarkdown": "@hengck23, you probably know this, but the [vindr-mammo](https://www.physionet.org/content/vindr-mammo/1.0.0/) dataset already has the bounding box annotations that you are looking for. It would be interesting to see how much juice you can squeeze out of those annotations - gives you a quick way to assess if it's worth getting the RSNA dataset annotated by a radiologist...\n\nMy intuition is that it would help a lot. If you can show a model the ROI that causes a mammogram to be tagged as positive, the model is likely to learn the important features better. I explored this a bit, but haven't found a good way to incorporate the bounding box data. One possible method is to divide the images into tiles, label each tile and then train on the tiles and their labels. The hard part, at least for me is aggregating the tile-level predictions into a single prediction for the whole image. A global maxpooling operation doesn't seem to work well." } ] }, { "id": 2125687, "postDate": "2023-02-01T19:56:41.807Z", "content": "@gunesevitan - thank you for the generous words i barely deserve it though.
\nI read through some of the discussions of the contest.
@hengck23 are you using the dataset from https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/377790 already? Those images have a region annotation and you can do a point annotation from that pretty easily. The mammography technique is quite standardised (in contrary to other projectionsradiographic material). I see that there are labels on the above set which could misguide the classification… so i would mask out the mamma on all of the images (of the official and this one above) and just add black bg to all as a preprocessing step. That area does not contain any valuable information regarding the tumor anyway. The fine-grained structure of the organ on the image is very similar!
\nI hope you are already working on full resolution (patches, sliding windows ) because you have only chance to detect the suspect microcalcifications on (near?) full res.
Dense (fibrous and glandular) breast tissue looks \"white\" on a mammogram. Breast masses and cancers can also look \"white\", so the classification in this category is the hardest task. These breasts have usually an uncertain BIRAD score! This is why you probably see some patients having more than 4 images, especially those with fibrous tissue. The extra images are focused, zoomed images or repeated images due to false compression.
\nThe problem with annotating mammography is that the annotation itself doesn't take long (one or 4 points doesn't matter) … but fiding the lesion! =) Even annotating 1.1k images (2% positive of 55k) would take a substantial time.
\nWe could try some to see how it affects the generalisation… i have a reporting monitor @home for good contrast.
Thank you very much for explanation. Is any chace you show us the example of dense (fibrous and glandular) breast tissue, breast masses and cancers?
\na. It is hard for me do see these issues on images.
\nb. second - I made prototype for synthetic breast cancer generator (but I am not able to confirm if it works - it was made by me for fun - to see if it is possible to generate such changes on images. I am aware that it is far from \"good\" issue we should classify)
![\"image_1\"](\"http://zapodaj.net/images/6608d09e85fec.jpg\")
![\"image_2\"](\"http://zapodaj.net/images/a896deacf2e13.jpg\")
![\"image_3\"](\"http://zapodaj.net/images/9131c16081453.jpg\")
![\"image_4\"](\"http://zapodaj.net/images/002e5863d493a.jpg\")
Hi there,
\nfor densities take a look at this pic, you will see what the differences are:
\nhttps://www.cancer.gov/sites/g/files/xnrzdm211/files/styles/cgov_article/public/cgov_image/media_image/2022-04/BIRADS_Updated%20Image.png?itok=sCglGBKJ
i really like synthetic data, and i am pretty sure that it would help alot. but. For human reporters, a mass must be seen on at least two different mammographic projections to be suspect (this does not apply for ml models though).
\nThe first 3 images show a rather indistinct \"blurry\" lesion where none of the circumference is well defined - this is usually consiered as suspicious. this is only one of the characteristics though a mammographeur is looking for. Suspicious masses can have other characteristics, like they can be spiculated ( like the last image of yours, however the density there is a bit high ). Other features might be seen:
skin retraction
nipple retraction
skin thickening
trabecular thickening
axillary adenopathy
architectural distortion <---**
calcifications <---***
\n** and *** are important ones.
\nHowever there are benign and suspicious calcifications… diffuse ones (like speckles) are usually benign characteristic, clustered, ergional and linearly arranged ones are suspicious.
\nsee this vid here: https://youtu.be/jWogsnqwb6I?t=70
Thank you very much @sandorkonya for your help and explanation. I see that synthetic data is posible to create as a separate long term project :) Certainly it requires test and trial where radiologist will play the main role in this process. BTW: this video from youtube is great! 👍👍👍
\n@hengck23 - look here - full res images and this paper: https://arxiv.org/abs/2301.13817
", "rawMarkdown": "Thank you very much @sandorkonya for your help and explanation. I see that synthetic data is posible to create as a separate long term project :) Certainly it requires test and trial where radiologist will play the main role in this process. BTW: this video from youtube is great! 👍👍👍\n\n@hengck23 - look here - full res images and this paper: https://arxiv.org/abs/2301.13817" }, { "id": 2132021, "postDate": "2023-02-06T14:45:50.130Z", "content": "chatGPT is trained via human feedback (like RL critic).
\nbut it uses something like ranking.
\nI am thinking that this can also be used in image recognition.
\n(this may be something new)
we need papers that apply human in the loop for large scale data. then it would be automatic online training when people work with Ai assistant.
", "rawMarkdown": "chatGPT is trained via human feedback (like RL critic).\nbut it uses something like ranking.\nI am thinking that this can also be used in image recognition.\n(this may be something new)\n\nwe need papers that apply human in the loop for large scale data. then it would be automatic online training when people work with Ai assistant." }, { "id": 2132033, "postDate": "2023-02-06T14:55:28.740Z", "content": "Do you have a concrete proposal for this in practice?
", "rawMarkdown": "Do you have a concrete proposal for this in practice?" }, { "id": 2132043, "postDate": "2023-02-06T15:02:19.677Z", "content": "not yet. but medical AI is mainly CAD (assistant diagnostic tool). so it can get human feedback easily. e.g. for breast mammography, CAD gives some candidates and radiologists verify them or detect new lesion.
\nthis is already human feedback.
\nhow to use this feedback to train like chatGPT is really interesting
", "rawMarkdown": "not yet. but medical AI is mainly CAD (assistant diagnostic tool). so it can get human feedback easily. e.g. for breast mammography, CAD gives some candidates and radiologists verify them or detect new lesion.\n\nthis is already human feedback.\n\nhow to use this feedback to train like chatGPT is really interesting" } ] } ] }, { "id": 2083939, "postDate": "2023-01-03T01:43:21.670Z", "content": "one thing to note that radiologist make decision by comparing mammography of different years apart, e.g. if there new emerging calcification, has the mass changes, etc.
", "rawMarkdown": "one thing to note that radiologist make decision by comparing mammography of different years apart, e.g. if there new emerging calcification, has the mass changes, etc. \n", "votes": 2 }, { "id": 2081701, "postDate": "2022-12-31T14:28:48.233Z", "content": "i would recommend reading https://radiologyassistant.nl/breast/bi-rads/bi-rads-for-mammography-and-ultrasound-2013
\nthe part on distribution of calcifications might be of assistance
\"Doctors or radiologists\" - I love the distinction, @vaillant ! 😏
", "rawMarkdown": "\"Doctors or radiologists\" - I love the distinction, @vaillant ! 😏" }, { "id": 2130469, "postDate": "2023-02-05T13:24:30.720Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 2081791, "author_name": "David Roberts", "author_url": "", "post_date": "2022-12-31T16:18:52.997000", "content": "I am not a radiologist, but I have worked on medical imaging software for 30 years and am very familiar with mammography. I don't think it's a reasonable assumption that a radiologist would be able to diagnose cancer from a tiny density like the ones in these images.
\nNormally, CAD will pick up the small densities and direct the radiologists attention to the area. Then, the radiologist will \"window\" the image manually to try and find the \"sweet spot\" where the density in question is separated from the rest of the tissue as much as possible in order to determine the location, shape, symmetry etc. Most of the time, a follow on ultrasound (and a biopsy) is needed to verify what the dense tissue actually is.
\nSince these are screening mammograms (not follow ups), we don't expect to see many cases of large tumors. It seems the goal of this competition is to do something that humans cannot do .. which is to diagnose cancer from imaging alone.
", "votes": 20, "replies": [ { "id": 2081918, "author_name": "Radek Osmulski", "author_url": "", "post_date": "2022-12-31T20:15:56.937000", "content": "thx for this answer @davidbroberts, so much good info here 🙏 I think I learned more from your reply here on how all of this works, how the work gets done vs many of the much longer posts, really cool 🙂
\nthank you!
", "votes": 1, "replies": [] }, { "id": 2082005, "author_name": "hengck23", "author_url": "", "post_date": "2023-01-01T01:18:40.297000", "content": "@davidbroberts
\nthanks a lot for the reply.
so i can assume:
\n[1] the annotation process is likely to follows what you have described. even in open dataset like vindr, the process will be CAD+verification, i.e. there may be miss detection by CAD.
[2] hence one can give out of trying to perfect a segmentation model for mask annotation. instead one should devise a mask model to include all positive mask, but reduce the num of negative mask. it is ok that for one cancer image to have at least one +ve mask and several neg mask.
\nor even in aux loss, we cannot assume to have perfect annotation for mask.
\nnow i understand why they use weak models like resnet22 in heatmap prediction (and not a strong one like renset50)
[3] Now i understand whay the two-stage approach:
\nthe loss function is important to ensure the model is high sensitivity or specificity in the two stages.
\nactually i also have another observation at external data:
\nalthough the sore for mask detection is poor (e.g. for small region), the image level prediction (e.g. BIRADS) is generally good.
\ni use a joint loss end-to-end model (heatmap prediction + image level prediction)
check this !!!
\nhttps://www.youtube.com/watch?v=FJl_yf5Au68&t=138s
\nHow to read a screening mammogram
hence, it is not important to consider use of simple model like resnet22,34 efficientnet b2 or complicated large ones like efficientnet b7.
\na good workflow is more important. e.g. models for different scales, different parts, models for single image, models for multiple images of same laterality, multiple images of different laterality, models for ROI level, models for image levels ….
\nan ensemble of different type of models seems to be the key.
\n
\n
\n
\n
That's pretty interesting. Looks like laterality alignment (left ontop of right) is a critical pre-processing activity.
", "votes": 0, "replies": [] }, { "id": 2090891, "author_name": "TheStrugglingEngineer", "author_url": "", "post_date": "2023-01-07T18:36:26.867000", "content": "\n\na good workflow is more important. e.g. models for different scales, different parts, models for single image, models for multiple images of same laterality, multiple images of different laterality, models for ROI level, models for image levels ….
\n
and just how would such multiplicity of models work in a notebook in under 9 hours on 43K high resolution images?
", "votes": 0, "replies": [] }, { "id": 2091004, "author_name": "@kaggleqrdl", "author_url": "", "post_date": "2023-01-07T22:29:54.830000", "content": "Yep, that's the issue. TPU / tensorflow might be the way to go as it performs generally better than pytorch, but harder to use.
", "votes": 1, "replies": [] }, { "id": 2091010, "author_name": "TheStrugglingEngineer", "author_url": "", "post_date": "2023-01-07T22:45:57.960000", "content": "I agree…
\nmy point is, we need to be extremely stingy in our approach… simple and effective trickery is worth more than here than complex models…
\nwork in progress…
Check this out - https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/376630
", "votes": 1, "replies": [] }, { "id": 2091031, "author_name": "TheStrugglingEngineer", "author_url": "", "post_date": "2023-01-08T00:03:33.193000", "content": "yes I did check that earlier today. thanks.
\neven further than just enlarging the usable ram, trickery regarding the dataset is also useful, like using smaller files when possible/reducing the dataset (even if it seems counter-intuitive at first). To me, this seems more the way to go than spending time trying ever more extravaggant models that won't run in 9 hours.
Hmm, someone is saying we can't use TPU for this comp. Which is weird, given that someone created a TPU notebook. But it might be useful for training.
\nedit: reading up on it, I think it's because the TPUs require internet access which is not allowed in this comp
", "votes": 1, "replies": [] }, { "id": 2091353, "author_name": "TheStrugglingEngineer", "author_url": "", "post_date": "2023-01-08T10:19:10.467000", "content": "it's as you said, TPU will be useful only to train models that can later be loaded into the submission notebook.
", "votes": 0, "replies": [] }, { "id": 2094507, "author_name": "hengck23", "author_url": "", "post_date": "2023-01-10T19:39:39.123000", "content": "@houssamassila
\n\"just how would such multiplicity of models work in a notebook in under 9 hours on 43K high resolution images?\"
\"early rejection\" is a common trick in computer vision. 10x speed up is not problem
\nhttps://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/377008#2094477
", "votes": 0, "replies": [] } ] } ] }, { "id": 2125004, "author_name": "Gunes Evitan", "author_url": "", "post_date": "2023-02-01T11:12:57.127000", "content": "I see you are looking for a radiologist to team up @hengck23. I worked with @sandorkonya before and he is the best one I've ever worked with. Tagged both of you for potential collab.
", "votes": 2, "replies": [ { "id": 2125090, "author_name": "hengck23", "author_url": "", "post_date": "2023-02-01T12:14:31.927000", "content": "The idea is simple.
\nWith my top models, i will generate CAM heatmap.
\nThen the radiology correct it. (Delete very sure wrong heat map for activated case, mark possible one for missed cases)
i do not need very accurate annotation. one can simply just annotate a point.
\nI do not need all annotation, just at least one per pos image.
only 2% of train data are pos. this is very few images. annotation can done within hours (because i do not need good annotation).
\nBasically it is manual active learning. we will go through 2~3 rounds.
\nLet's see if this will fight imbalance data and gives better generalization, rather than relying on unstable LB/CV
\nhttps://ojs.aaai.org/index.php/AAAI/article/download/16153/15960
\nfigure.2 is example of point annotation
@hengck23, you probably know this, but the vindr-mammo dataset already has the bounding box annotations that you are looking for. It would be interesting to see how much juice you can squeeze out of those annotations - gives you a quick way to assess if it's worth getting the RSNA dataset annotated by a radiologist…
\nMy intuition is that it would help a lot. If you can show a model the ROI that causes a mammogram to be tagged as positive, the model is likely to learn the important features better. I explored this a bit, but haven't found a good way to incorporate the bounding box data. One possible method is to divide the images into tiles, label each tile and then train on the tiles and their labels. The hard part, at least for me is aggregating the tile-level predictions into a single prediction for the whole image. A global maxpooling operation doesn't seem to work well.
", "votes": 0, "replies": [] } ] }, { "id": 2125687, "author_name": "dr. Konya", "author_url": "", "post_date": "2023-02-01T19:56:41.807000", "content": "@gunesevitan - thank you for the generous words i barely deserve it though.
\nI read through some of the discussions of the contest.
@hengck23 are you using the dataset from https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/377790 already? Those images have a region annotation and you can do a point annotation from that pretty easily. The mammography technique is quite standardised (in contrary to other projectionsradiographic material). I see that there are labels on the above set which could misguide the classification… so i would mask out the mamma on all of the images (of the official and this one above) and just add black bg to all as a preprocessing step. That area does not contain any valuable information regarding the tumor anyway. The fine-grained structure of the organ on the image is very similar!
\nI hope you are already working on full resolution (patches, sliding windows ) because you have only chance to detect the suspect microcalcifications on (near?) full res.
Dense (fibrous and glandular) breast tissue looks \"white\" on a mammogram. Breast masses and cancers can also look \"white\", so the classification in this category is the hardest task. These breasts have usually an uncertain BIRAD score! This is why you probably see some patients having more than 4 images, especially those with fibrous tissue. The extra images are focused, zoomed images or repeated images due to false compression.
\nThe problem with annotating mammography is that the annotation itself doesn't take long (one or 4 points doesn't matter) … but fiding the lesion! =) Even annotating 1.1k images (2% positive of 55k) would take a substantial time.
\nWe could try some to see how it affects the generalisation… i have a reporting monitor @home for good contrast.
Thank you very much for explanation. Is any chace you show us the example of dense (fibrous and glandular) breast tissue, breast masses and cancers?
\na. It is hard for me do see these issues on images.
\nb. second - I made prototype for synthetic breast cancer generator (but I am not able to confirm if it works - it was made by me for fun - to see if it is possible to generate such changes on images. I am aware that it is far from \"good\" issue we should classify)
Hi there,
\nfor densities take a look at this pic, you will see what the differences are:
\nhttps://www.cancer.gov/sites/g/files/xnrzdm211/files/styles/cgov_article/public/cgov_image/media_image/2022-04/BIRADS_Updated%20Image.png?itok=sCglGBKJ
i really like synthetic data, and i am pretty sure that it would help alot. but. For human reporters, a mass must be seen on at least two different mammographic projections to be suspect (this does not apply for ml models though).
\nThe first 3 images show a rather indistinct \"blurry\" lesion where none of the circumference is well defined - this is usually consiered as suspicious. this is only one of the characteristics though a mammographeur is looking for. Suspicious masses can have other characteristics, like they can be spiculated ( like the last image of yours, however the density there is a bit high ). Other features might be seen:
skin retraction
nipple retraction
skin thickening
trabecular thickening
axillary adenopathy
architectural distortion <---**
calcifications <---***
\n** and *** are important ones.
\nHowever there are benign and suspicious calcifications… diffuse ones (like speckles) are usually benign characteristic, clustered, ergional and linearly arranged ones are suspicious.
\nsee this vid here: https://youtu.be/jWogsnqwb6I?t=70
Thank you very much @sandorkonya for your help and explanation. I see that synthetic data is posible to create as a separate long term project :) Certainly it requires test and trial where radiologist will play the main role in this process. BTW: this video from youtube is great! 👍👍👍
\n@hengck23 - look here - full res images and this paper: https://arxiv.org/abs/2301.13817
", "votes": 0, "replies": [] }, { "id": 2132021, "author_name": "hengck23", "author_url": "", "post_date": "2023-02-06T14:45:50.130000", "content": "chatGPT is trained via human feedback (like RL critic).
\nbut it uses something like ranking.
\nI am thinking that this can also be used in image recognition.
\n(this may be something new)
we need papers that apply human in the loop for large scale data. then it would be automatic online training when people work with Ai assistant.
", "votes": 0, "replies": [] }, { "id": 2132033, "author_name": "dr. Konya", "author_url": "", "post_date": "2023-02-06T14:55:28.740000", "content": "Do you have a concrete proposal for this in practice?
", "votes": 0, "replies": [] }, { "id": 2132043, "author_name": "hengck23", "author_url": "", "post_date": "2023-02-06T15:02:19.677000", "content": "not yet. but medical AI is mainly CAD (assistant diagnostic tool). so it can get human feedback easily. e.g. for breast mammography, CAD gives some candidates and radiologists verify them or detect new lesion.
\nthis is already human feedback.
\nhow to use this feedback to train like chatGPT is really interesting
", "votes": 0, "replies": [] } ] } ] }, { "id": 2083939, "author_name": "hengck23", "author_url": "", "post_date": "2023-01-03T01:43:21.670000", "content": "one thing to note that radiologist make decision by comparing mammography of different years apart, e.g. if there new emerging calcification, has the mass changes, etc.
", "votes": 2, "replies": [] }, { "id": 2081701, "author_name": "Redicee", "author_url": "", "post_date": "2022-12-31T14:28:48.233000", "content": "i would recommend reading https://radiologyassistant.nl/breast/bi-rads/bi-rads-for-mammography-and-ultrasound-2013
\nthe part on distribution of calcifications might be of assistance
\"Doctors or radiologists\" - I love the distinction, @vaillant ! 😏
", "votes": 0, "replies": [] }, { "id": 2130469, "author_name": "", "author_url": "", "post_date": "2023-02-05T13:24:30.720000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2081540": "results on validation:\n\n\n\nQuestion : How is very small lesions detected? i can find many similar regions that are false positives. How do you look at the whole image to decide if it is cancerous or not? Or do you really can decide just based on local context region?\n\n(basically i am trying to decide the context for good detection but it seems impossible. It is even difficult on training set)", "2081791": "I am not a radiologist, but I have worked on medical imaging software for 30 years and am very familiar with mammography. I don't think it's a reasonable assumption that a radiologist would be able to diagnose cancer from a tiny density like the ones in these images.\n\nNormally, CAD will pick up the small densities and direct the radiologists attention to the area. Then, the radiologist will \"window\" the image manually to try and find the \"sweet spot\" where the density in question is separated from the rest of the tissue as much as possible in order to determine the location, shape, symmetry etc. Most of the time, a follow on ultrasound (and a biopsy) is needed to verify what the dense tissue actually is.\n\nSince these are screening mammograms (not follow ups), we don't expect to see many cases of large tumors. It seems the goal of this competition is to do something that humans cannot do .. which is to diagnose cancer from imaging alone.", "2125004": "I see you are looking for a radiologist to team up @hengck23. I worked with @sandorkonya before and he is the best one I've ever worked with. Tagged both of you for potential collab.", "2083939": "one thing to note that radiologist make decision by comparing mammography of different years apart, e.g. if there new emerging calcification, has the mass changes, etc. \n", "2081701": "i would recommend reading https://radiologyassistant.nl/breast/bi-rads/bi-rads-for-mammography-and-ultrasound-2013\nthe part on distribution of calcifications might be of assistance", "2081781": "\"Doctors or radiologists\" - I love the distinction, @vaillant ! 😏", "2130469": "" } }