hi. I'm an engineer participating in this competition and I'm wondering how everyone is dealing with these huge images.
\n I'm currently trying to resize the images to be smaller somehow, but I'm not sure if using resized images will have a good effect on the model.
How is everyone dealing with these large image sizes? In other posts, it seems like they're using a tailed variant of the image, but what does that mean?
\nI don't understand how to input these tailed images into the model, is there anyone who can explain it to me?
\nThanks.
", "messages": [ { "id": 2492164, "postDate": "2023-10-22T09:59:48.280Z", "content": "hi. I'm an engineer participating in this competition and I'm wondering how everyone is dealing with these huge images.
\n I'm currently trying to resize the images to be smaller somehow, but I'm not sure if using resized images will have a good effect on the model.
How is everyone dealing with these large image sizes? In other posts, it seems like they're using a tailed variant of the image, but what does that mean?
\nI don't understand how to input these tailed images into the model, is there anyone who can explain it to me?
\nThanks.
", "rawMarkdown": "hi. I'm an engineer participating in this competition and I'm wondering how everyone is dealing with these huge images.\n I'm currently trying to resize the images to be smaller somehow, but I'm not sure if using resized images will have a good effect on the model. \n\nHow is everyone dealing with these large image sizes? In other posts, it seems like they're using a tailed variant of the image, but what does that mean?\n\nI don't understand how to input these tailed images into the model, is there anyone who can explain it to me? \n\nThanks.\n", "votes": 3 }, { "id": 2493093, "postDate": "2023-10-23T06:44:41.090Z", "content": "Hello, I believe that simply performing a reshape operation on medical images is not advisable (even though I'm a beginner, haha). This is especially true for extremely large images like whole-slide images (WSI), as compressing the image to a relatively small size can likely result in the loss of critical features.
\nBased on my current research on WSIs, most work involves segmenting them into smaller patches, and these patches may be input to a final classifier, possibly with features extracted from them using other pre-trained models.
\nAdditionally, in this context, the foreground region of WSI doesn't constitute a significant portion of the overall image, making patches a potentially better choice.
I have made some simple filtering of patches which does not have any information (mostly black or edge cases) in https://www.kaggle.com/code/jirkaborovec/cancer-subtype-decompose-large-image-tiles so you can adjust it to your need or use already prepared dataset with tiles 512*512px with scale factor 0.25 -> https://www.kaggle.com/datasets/jirkaborovec/tiles-of-cancer-2048px-scale-0-25
", "rawMarkdown": "I have made some simple filtering of patches which does not have any information (mostly black or edge cases) in https://www.kaggle.com/code/jirkaborovec/cancer-subtype-decompose-large-image-tiles so you can adjust it to your need or use already prepared dataset with tiles 512*512px with scale factor 0.25 -> https://www.kaggle.com/datasets/jirkaborovec/tiles-of-cancer-2048px-scale-0-25" } ] }, { "id": 2492303, "postDate": "2023-10-22T11:24:19.053Z", "content": "Hi, regarding reading such large-sized images, you can use the pyvips library. Here's a great notebook describing it: https://www.kaggle.com/code/gunesevitan/libvips-pyvips-installation-and-getting-started. For training, I'm currently trying to slice the WSI into multiple patches, but this also requires some methods to address subsequent issues.
", "rawMarkdown": "Hi, regarding reading such large-sized images, you can use the pyvips library. Here's a great notebook describing it: https://www.kaggle.com/code/gunesevitan/libvips-pyvips-installation-and-getting-started. For training, I'm currently trying to slice the WSI into multiple patches, but this also requires some methods to address subsequent issues.", "votes": 2, "replies": [ { "id": 2492328, "postDate": "2023-10-22T11:50:33.790Z", "content": "So if we take CNN as an example of a model, does it take a single image as input, or a single patch as input?
", "rawMarkdown": "So if we take CNN as an example of a model, does it take a single image as input, or a single patch as input?", "votes": 1, "replies": [ { "id": 2492456, "postDate": "2023-10-22T14:32:47.983Z", "content": "patch is preferable under memory constraints here.
", "rawMarkdown": "patch is preferable under memory constraints here.", "votes": 1 } ] } ] } ], "comments": [ { "id": 2493093, "author_name": "li_ne", "author_url": "", "post_date": "2023-10-23T06:44:41.090000", "content": "Hello, I believe that simply performing a reshape operation on medical images is not advisable (even though I'm a beginner, haha). This is especially true for extremely large images like whole-slide images (WSI), as compressing the image to a relatively small size can likely result in the loss of critical features.
\nBased on my current research on WSIs, most work involves segmenting them into smaller patches, and these patches may be input to a final classifier, possibly with features extracted from them using other pre-trained models.
\nAdditionally, in this context, the foreground region of WSI doesn't constitute a significant portion of the overall image, making patches a potentially better choice.
I have made some simple filtering of patches which does not have any information (mostly black or edge cases) in https://www.kaggle.com/code/jirkaborovec/cancer-subtype-decompose-large-image-tiles so you can adjust it to your need or use already prepared dataset with tiles 512*512px with scale factor 0.25 -> https://www.kaggle.com/datasets/jirkaborovec/tiles-of-cancer-2048px-scale-0-25
", "votes": 0, "replies": [] } ] }, { "id": 2492303, "author_name": "Huang Jin Feng", "author_url": "", "post_date": "2023-10-22T11:24:19.053000", "content": "Hi, regarding reading such large-sized images, you can use the pyvips library. Here's a great notebook describing it: https://www.kaggle.com/code/gunesevitan/libvips-pyvips-installation-and-getting-started. For training, I'm currently trying to slice the WSI into multiple patches, but this also requires some methods to address subsequent issues.
", "votes": 2, "replies": [ { "id": 2492328, "author_name": "JaewooChoi", "author_url": "", "post_date": "2023-10-22T11:50:33.790000", "content": "So if we take CNN as an example of a model, does it take a single image as input, or a single patch as input?
", "votes": 1, "replies": [ { "id": 2492456, "author_name": "Alexandra", "author_url": "", "post_date": "2023-10-22T14:32:47.983000", "content": "patch is preferable under memory constraints here.
", "votes": 1, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2492164": "hi. I'm an engineer participating in this competition and I'm wondering how everyone is dealing with these huge images.\n I'm currently trying to resize the images to be smaller somehow, but I'm not sure if using resized images will have a good effect on the model. \n\nHow is everyone dealing with these large image sizes? In other posts, it seems like they're using a tailed variant of the image, but what does that mean?\n\nI don't understand how to input these tailed images into the model, is there anyone who can explain it to me? \n\nThanks.\n", "2493093": "Hello, I believe that simply performing a reshape operation on medical images is not advisable (even though I'm a beginner, haha). This is especially true for extremely large images like whole-slide images (WSI), as compressing the image to a relatively small size can likely result in the loss of critical features. \nBased on my current research on WSIs, **most work involves segmenting them into smaller patches**, and these patches may be input to a final classifier, possibly with features extracted from them using other pre-trained models. \nAdditionally, in this context, the foreground region of WSI doesn't constitute a significant portion of the overall image, making patches a potentially better choice.", "2492303": "Hi, regarding reading such large-sized images, you can use the pyvips library. Here's a great notebook describing it: https://www.kaggle.com/code/gunesevitan/libvips-pyvips-installation-and-getting-started. For training, I'm currently trying to slice the WSI into multiple patches, but this also requires some methods to address subsequent issues." } }