![\"\"](\"https://i.ibb.co/SszygDS/circle.jpg\")
Hi all,
\nLate joiner so asking away…
\nI have noticed that some images have markings on them, like circles.
\nfor example train_images/12195/100147863.dcm
\nI assume these are more common on cancer=1 and the model will probably associate markings with 1
\nI also assume the test dataset does not contain these.
Anyone looked into this? Couldn't find discussion but perhaps I didn't use the right keywords
", "messages": [ { "id": 2117980, "postDate": "2023-01-27T17:23:11.030Z", "content": "I believe those are mole markers (source)
", "rawMarkdown": "I believe those are mole markers ([source](https://blog.pdchealthcare.com/articles/mammography/quick-reference-guide-mammography-skin-markers/))", "votes": 3 }, { "id": 2117628, "postDate": "2023-01-27T11:54:42.310Z", "content": "\"The circles you see on mammogram images are typically used to indicate the location of any abnormalities or suspicious areas that require further examination. They are usually placed there by radiologists or other medical professionals to help guide further testing and treatment. These circles are used to mark areas that require attention and are not an indication of a specific diagnosis.\" chatGPT
", "rawMarkdown": "\"The circles you see on mammogram images are typically used to indicate the location of any abnormalities or suspicious areas that require further examination. They are usually placed there by radiologists or other medical professionals to help guide further testing and treatment. These circles are used to mark areas that require attention and are not an indication of a specific diagnosis.\" chatGPT", "votes": 1, "replies": [ { "id": 2117661, "postDate": "2023-01-27T12:26:54.627Z", "content": "Hehehe, he is great this chatbot. However, this does not answer my question. The markings are probably more common on cancer=1,so the model will learn that. This will create a biased model that will not generalize well to unmarked images.
", "rawMarkdown": "Hehehe, he is great this chatbot. However, this does not answer my question. The markings are probably more common on cancer=1,so the model will learn that. This will create a biased model that will not generalize well to unmarked images. ", "replies": [ { "id": 2117673, "postDate": "2023-01-27T12:47:14.390Z", "content": "This is why I made - synthetic circle generator 😂
\nThis is custom augumentation function (frequency -> proba = 0.2) which I use for 'non cancer' images.
Was thinking that before I went to sleep. If you can't fight them… Join them.
", "rawMarkdown": "Was thinking that before I went to sleep. If you can't fight them... Join them. ", "votes": 3 }, { "id": 2118540, "postDate": "2023-01-28T05:05:19.617Z", "content": "@remekkinas
\nThank you for sharing your nice idea.
Is this just adding synthetic circle?
\nDid this augmentation work well?
\nThanks.
Does it help on CV? It should not if it's mole marker.
", "rawMarkdown": "Does it help on CV? It should not if it's mole marker." }, { "id": 2123991, "postDate": "2023-01-31T19:26:05.040Z", "content": "Ibstill didn't get to trying it, fighting getting the dicom conversion fast enough
", "rawMarkdown": "Ibstill didn't get to trying it, fighting getting the dicom conversion fast enough " }, { "id": 2124028, "postDate": "2023-01-31T19:54:31.087Z", "content": "I made only one test (unfortunately) - it did not improve score. I have to spend more time experimenting with this. This was only fast check. So I am sorry - it wasn't too detailed to confirm for sure whether it works or not.
", "rawMarkdown": "I made only one test (unfortunately) - it did not improve score. I have to spend more time experimenting with this. This was only fast check. So I am sorry - it wasn't too detailed to confirm for sure whether it works or not. ", "votes": 1 } ] } ] }, { "id": 2117544, "postDate": "2023-01-27T10:48:59.767Z", "content": "Hi all,
\nLate joiner so asking away…
\nI have noticed that some images have markings on them, like circles.
\nfor example train_images/12195/100147863.dcm
\nI assume these are more common on cancer=1 and the model will probably associate markings with 1
\nI also assume the test dataset does not contain these.
Anyone looked into this? Couldn't find discussion but perhaps I didn't use the right keywords
", "rawMarkdown": "Hi all,\nLate joiner so asking away... \nI have noticed that some images have markings on them, like circles.\nfor example train_images/12195/100147863.dcm\nI assume these are more common on cancer=1 and the model will probably associate markings with 1\nI also assume the test dataset does not contain these.\n\nAnyone looked into this? Couldn't find discussion but perhaps I didn't use the right keywords\n", "votes": 1 }, { "id": 2117866, "postDate": "2023-01-27T15:46:11.673Z", "content": "What made you to think that there are no circles in test set?
", "rawMarkdown": "What made you to think that there are no circles in test set?", "replies": [ { "id": 2118180, "postDate": "2023-01-27T20:27:23.063Z", "content": "Nothing, but it's unknown. Especially with the big final dataset.
", "rawMarkdown": "Nothing, but it's unknown. Especially with the big final dataset. ", "replies": [ { "id": 2118270, "postDate": "2023-01-27T22:12:31.787Z", "content": "There is no meaning in curating data without circle for test, so I think you don't have to be wary about it.
", "rawMarkdown": "There is no meaning in curating data without circle for test, so I think you don't have to be wary about it." } ] } ] } ], "comments": [ { "id": 2117980, "author_name": "AbdouE", "author_url": "", "post_date": "2023-01-27T17:23:11.030000", "content": "I believe those are mole markers (source)
", "votes": 3, "replies": [] }, { "id": 2117628, "author_name": "Remek Kinas", "author_url": "", "post_date": "2023-01-27T11:54:42.310000", "content": "\"The circles you see on mammogram images are typically used to indicate the location of any abnormalities or suspicious areas that require further examination. They are usually placed there by radiologists or other medical professionals to help guide further testing and treatment. These circles are used to mark areas that require attention and are not an indication of a specific diagnosis.\" chatGPT
", "votes": 1, "replies": [ { "id": 2117661, "author_name": "Moshel", "author_url": "", "post_date": "2023-01-27T12:26:54.627000", "content": "Hehehe, he is great this chatbot. However, this does not answer my question. The markings are probably more common on cancer=1,so the model will learn that. This will create a biased model that will not generalize well to unmarked images.
", "votes": 0, "replies": [ { "id": 2117673, "author_name": "Remek Kinas", "author_url": "", "post_date": "2023-01-27T12:47:14.390000", "content": "This is why I made - synthetic circle generator 😂
\nThis is custom augumentation function (frequency -> proba = 0.2) which I use for 'non cancer' images.
Was thinking that before I went to sleep. If you can't fight them… Join them.
", "votes": 3, "replies": [] }, { "id": 2118540, "author_name": "taruto", "author_url": "", "post_date": "2023-01-28T05:05:19.617000", "content": "@remekkinas
\nThank you for sharing your nice idea.
Is this just adding synthetic circle?
\nDid this augmentation work well?
\nThanks.
Does it help on CV? It should not if it's mole marker.
", "votes": 0, "replies": [] }, { "id": 2123991, "author_name": "Moshel", "author_url": "", "post_date": "2023-01-31T19:26:05.040000", "content": "Ibstill didn't get to trying it, fighting getting the dicom conversion fast enough
", "votes": 0, "replies": [] }, { "id": 2124028, "author_name": "Remek Kinas", "author_url": "", "post_date": "2023-01-31T19:54:31.087000", "content": "I made only one test (unfortunately) - it did not improve score. I have to spend more time experimenting with this. This was only fast check. So I am sorry - it wasn't too detailed to confirm for sure whether it works or not.
", "votes": 1, "replies": [] } ] } ] }, { "id": 2117866, "author_name": "luddite^", "author_url": "", "post_date": "2023-01-27T15:46:11.673000", "content": "What made you to think that there are no circles in test set?
", "votes": 0, "replies": [ { "id": 2118180, "author_name": "Moshel", "author_url": "", "post_date": "2023-01-27T20:27:23.063000", "content": "Nothing, but it's unknown. Especially with the big final dataset.
", "votes": 0, "replies": [ { "id": 2118270, "author_name": "luddite^", "author_url": "", "post_date": "2023-01-27T22:12:31.787000", "content": "There is no meaning in curating data without circle for test, so I think you don't have to be wary about it.
", "votes": 0, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2117980": "I believe those are mole markers ([source](https://blog.pdchealthcare.com/articles/mammography/quick-reference-guide-mammography-skin-markers/))", "2117628": "\"The circles you see on mammogram images are typically used to indicate the location of any abnormalities or suspicious areas that require further examination. They are usually placed there by radiologists or other medical professionals to help guide further testing and treatment. These circles are used to mark areas that require attention and are not an indication of a specific diagnosis.\" chatGPT", "2117544": "Hi all,\nLate joiner so asking away... \nI have noticed that some images have markings on them, like circles.\nfor example train_images/12195/100147863.dcm\nI assume these are more common on cancer=1 and the model will probably associate markings with 1\nI also assume the test dataset does not contain these.\n\nAnyone looked into this? Couldn't find discussion but perhaps I didn't use the right keywords\n", "2117866": "What made you to think that there are no circles in test set?" } }