{ "id": 336063, "title": "Duplicate areas within images", "url": "/competitions/mayo-clinic-strip-ai/discussion/336063", "author_name": "Nikita Kuzmenkov", "post_date": "2022-07-09T07:28:49.330000", "votes": 16, "comment_count": 7, "views": 0, "content": "

While looking at raw images I've recently discovered that a lot of them contain duplicate areas (within the same image), e.g.:

\n

\"drawing\"

\n

Actually, most of them look like the same original image was copied & pasted several times (some may be also rotated). As have zero medical background, I wonder if this can be due to the features related to the process of data collection?

\n

Update

\n

For more such images, please refer to the great Stroke (AIS) FastDup notebook by @mpwolke.

", "messages": [ { "id": 1849076, "postDate": "2022-07-09T07:28:49.330Z", "content": "

While looking at raw images I've recently discovered that a lot of them contain duplicate areas (within the same image), e.g.:

\n

\"drawing\"

\n

Actually, most of them look like the same original image was copied & pasted several times (some may be also rotated). As have zero medical background, I wonder if this can be due to the features related to the process of data collection?

\n

Update

\n

For more such images, please refer to the great Stroke (AIS) FastDup notebook by @mpwolke.

", "rawMarkdown": "While looking at raw images I've recently discovered that a lot of them contain duplicate areas (within the same image), e.g.:\n\n\"drawing\"\n\nActually, most of them look like the same original image was copied & pasted several times (some may be also rotated). As have zero medical background, I wonder if this can be due to the features related to the process of data collection?\n\n\n## Update\n\nFor more such images, please refer to the great **[Stroke (AIS) FastDup][1]** notebook by @mpwolke.\n\n[1]: https://www.kaggle.com/code/mpwolke/stroke-ais-fastdup", "votes": 16 }, { "id": 1858381, "postDate": "2022-07-16T21:34:31.187Z", "content": "

Maybe organizers can describe what is it?

", "rawMarkdown": "Maybe organizers can describe what is it?", "votes": 1, "replies": [ { "id": 1860158, "postDate": "2022-07-18T06:58:43.363Z", "content": "

Good point

\n

@barbaroserdal, @ashleychow, can we please get some clarification on this topic?

", "rawMarkdown": "Good point\n\n@barbaroserdal, @ashleychow, can we please get some clarification on this topic?" } ] }, { "id": 1862711, "postDate": "2022-07-19T23:15:55.523Z", "content": "

I have noted, from a very examples, that the repeated images have slight differences in coloring and some small color patches are different. Outer shapes do look pretty much the same though.

", "rawMarkdown": "I have noted, from a very examples, that the repeated images have slight differences in coloring and some small color patches are different. Outer shapes do look pretty much the same though.", "votes": 2, "replies": [ { "id": 1862712, "postDate": "2022-07-19T23:17:19.830Z", "content": "

Maybe different scans of the same sample, put in a single slide for extra information. Just guessing.

", "rawMarkdown": "Maybe different scans of the same sample, put in a single slide for extra information. Just guessing.", "votes": 2 }, { "id": 1877558, "postDate": "2022-07-30T19:14:15.927Z", "content": "

@icemantd I would guess you're right. Perhaps it would make sense to treat both images as two separate images of the same object? It could be something similar to image augmentation, where the same image is altered, but still has the same target label.

", "rawMarkdown": "@icemantd I would guess you're right. Perhaps it would make sense to treat both images as two separate images of the same object? It could be something similar to image augmentation, where the same image is altered, but still has the same target label.", "votes": 2 }, { "id": 1878987, "postDate": "2022-07-31T19:32:21.803Z", "content": "

yes, there seems to be no evidence that this repetition is redundant…. maybe one of the repetition has a better signal, and we don't know which one… this problem is really complicated at multiple levels

", "rawMarkdown": "yes, there seems to be no evidence that this repetition is redundant.... maybe one of the repetition has a better signal, and we don't know which one... this problem is really complicated at multiple levels", "votes": 2 } ] }, { "id": 1849933, "postDate": "2022-07-10T00:13:53.540Z", "content": "

Thank you for sharing!
\nThis is an important discovery it might mess up our models completely..

", "rawMarkdown": "Thank you for sharing! \nThis is an important discovery it might mess up our models completely.. \n" } ], "comments": [ { "id": 1858381, "author_name": "anthony", "author_url": "", "post_date": "2022-07-16T21:34:31.187000", "content": "

Maybe organizers can describe what is it?

", "votes": 1, "replies": [ { "id": 1860158, "author_name": "Nikita Kuzmenkov", "author_url": "", "post_date": "2022-07-18T06:58:43.363000", "content": "

Good point

\n

@barbaroserdal, @ashleychow, can we please get some clarification on this topic?

", "votes": 0, "replies": [] } ] }, { "id": 1862711, "author_name": "tdiceman", "author_url": "", "post_date": "2022-07-19T23:15:55.523000", "content": "

I have noted, from a very examples, that the repeated images have slight differences in coloring and some small color patches are different. Outer shapes do look pretty much the same though.

", "votes": 2, "replies": [ { "id": 1862712, "author_name": "tdiceman", "author_url": "", "post_date": "2022-07-19T23:17:19.830000", "content": "

Maybe different scans of the same sample, put in a single slide for extra information. Just guessing.

", "votes": 2, "replies": [] }, { "id": 1877558, "author_name": "yqz", "author_url": "", "post_date": "2022-07-30T19:14:15.927000", "content": "

@icemantd I would guess you're right. Perhaps it would make sense to treat both images as two separate images of the same object? It could be something similar to image augmentation, where the same image is altered, but still has the same target label.

", "votes": 2, "replies": [] }, { "id": 1878987, "author_name": "tdiceman", "author_url": "", "post_date": "2022-07-31T19:32:21.803000", "content": "

yes, there seems to be no evidence that this repetition is redundant…. maybe one of the repetition has a better signal, and we don't know which one… this problem is really complicated at multiple levels

", "votes": 2, "replies": [] } ] }, { "id": 1849933, "author_name": "The Devastator", "author_url": "", "post_date": "2022-07-10T00:13:53.540000", "content": "

Thank you for sharing!
\nThis is an important discovery it might mess up our models completely..

", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1849076": "While looking at raw images I've recently discovered that a lot of them contain duplicate areas (within the same image), e.g.:\n\n\"drawing\"\n\nActually, most of them look like the same original image was copied & pasted several times (some may be also rotated). As have zero medical background, I wonder if this can be due to the features related to the process of data collection?\n\n\n## Update\n\nFor more such images, please refer to the great **[Stroke (AIS) FastDup][1]** notebook by @mpwolke.\n\n[1]: https://www.kaggle.com/code/mpwolke/stroke-ais-fastdup", "1858381": "Maybe organizers can describe what is it?", "1862711": "I have noted, from a very examples, that the repeated images have slight differences in coloring and some small color patches are different. Outer shapes do look pretty much the same though.", "1849933": "Thank you for sharing! \nThis is an important discovery it might mess up our models completely.. \n" } }