{ "id": 450775, "title": "Why people train the model on thumbnail images instead or train_images ?", "url": "/competitions/UBC-OCEAN/discussion/450775", "author_name": "Sonu Jha", "post_date": "2023-10-25T15:19:11.608000", "votes": 5, "comment_count": 5, "views": 0, "content": "

I see so many notebook using thumbnail image to train the model, is it just to test the pipeline? Because we won't get thumbnail for test images right?

", "messages": [ { "id": 2498874, "postDate": "2023-10-25T15:19:11.607Z", "content": "

I see so many notebook using thumbnail image to train the model, is it just to test the pipeline? Because we won't get thumbnail for test images right?

", "rawMarkdown": "I see so many notebook using thumbnail image to train the model, is it just to test the pipeline? Because we won't get thumbnail for test images right? ", "votes": 5 }, { "id": 2501129, "postDate": "2023-10-27T09:01:23.617Z", "content": "

@sonujha090 I think the main problem is the png format which is very unusual for digital pathology. Please refer to this discussion. Having tiff or svs format would make use of high res image much more feasible. Upvote the post of @erntkn 🥇 Hope we will have feedback from @sohier soon :)

", "rawMarkdown": "@sonujha090 I think the main problem is the png format which is very unusual for digital pathology. Please refer to [this discussion](https://www.kaggle.com/competitions/UBC-OCEAN/discussion/446688#2501120). Having tiff or svs format would make use of high res image much more feasible. Upvote the post of @erntkn 🥇 Hope we will have feedback from @sohier soon :)", "votes": 1 }, { "id": 2498915, "postDate": "2023-10-25T15:49:54.417Z", "content": "

We do get thumbnail images for test too, and can also confirm that unlike train where we didn't get thumbnails for tma, we are getting thumbnails for every instance.
\nThe main images are very heavy so people are using the tiles/some kind of preprocessing before using them raw. And training on thumbnails is a quicker way to test a pipeline

", "rawMarkdown": "We do get thumbnail images for test too, and can also confirm that unlike train where we didn't get thumbnails for tma, we are getting thumbnails for every instance. \nThe main images are very heavy so people are using the tiles/some kind of preprocessing before using them raw. And training on thumbnails is a quicker way to test a pipeline ", "votes": 1, "replies": [ { "id": 2498967, "postDate": "2023-10-25T16:21:47.310Z", "content": "

But for real submission we need to use original train/test image, because using thumbnail means we are losing the information right?

", "rawMarkdown": "But for real submission we need to use original train/test image, because using thumbnail means we are losing the information right?", "replies": [ { "id": 2498977, "postDate": "2023-10-25T16:27:37.083Z", "content": "

Correct. The same preprocessing pipeline needs to be applied on test images.

", "rawMarkdown": "Correct. The same preprocessing pipeline needs to be applied on test images." } ] } ] }, { "id": 2499717, "postDate": "2023-10-26T07:38:06.107Z", "content": "

Also with a thumbnail, you have the whole image, so you can assume that the cancer features are present in each presented image
\nsee: https://www.kaggle.com/code/jirkaborovec/cancer-subtype-baseline-with-lightning-timm
\nCompared to working with just tiles, you may use a tissue section, which is healthy and misses any cancer features
\nsee: https://www.kaggle.com/code/jirkaborovec/cancer-subtype-tiles-w-lightning-timm-models
\nBut over both approaches seem to be learning something useful and converging. Also, with tiles, you can say that samples with eventually normal tissue bring some random noise, which prevents the model from overfitting…

", "rawMarkdown": "Also with a thumbnail, you have the whole image, so you can assume that the cancer features are present in each presented image\nsee: https://www.kaggle.com/code/jirkaborovec/cancer-subtype-baseline-with-lightning-timm\nCompared to working with just tiles, you may use a tissue section, which is healthy and misses any cancer features\nsee: https://www.kaggle.com/code/jirkaborovec/cancer-subtype-tiles-w-lightning-timm-models\nBut over both approaches seem to be learning something useful and converging. Also, with tiles, you can say that samples with eventually normal tissue bring some random noise, which prevents the model from overfitting...", "votes": 2 } ], "comments": [ { "id": 2501129, "author_name": "simjeg", "author_url": "", "post_date": "2023-10-27T09:01:23.617000", "content": "

@sonujha090 I think the main problem is the png format which is very unusual for digital pathology. Please refer to this discussion. Having tiff or svs format would make use of high res image much more feasible. Upvote the post of @erntkn 🥇 Hope we will have feedback from @sohier soon :)

", "votes": 1, "replies": [] }, { "id": 2498915, "author_name": "pjmathematician", "author_url": "", "post_date": "2023-10-25T15:49:54.417000", "content": "

We do get thumbnail images for test too, and can also confirm that unlike train where we didn't get thumbnails for tma, we are getting thumbnails for every instance.
\nThe main images are very heavy so people are using the tiles/some kind of preprocessing before using them raw. And training on thumbnails is a quicker way to test a pipeline

", "votes": 1, "replies": [ { "id": 2498967, "author_name": "Sonu Jha", "author_url": "", "post_date": "2023-10-25T16:21:47.310000", "content": "

But for real submission we need to use original train/test image, because using thumbnail means we are losing the information right?

", "votes": 0, "replies": [ { "id": 2498977, "author_name": "pjmathematician", "author_url": "", "post_date": "2023-10-25T16:27:37.083000", "content": "

Correct. The same preprocessing pipeline needs to be applied on test images.

", "votes": 0, "replies": [] } ] } ] }, { "id": 2499717, "author_name": "Jirka", "author_url": "", "post_date": "2023-10-26T07:38:06.107000", "content": "

Also with a thumbnail, you have the whole image, so you can assume that the cancer features are present in each presented image
\nsee: https://www.kaggle.com/code/jirkaborovec/cancer-subtype-baseline-with-lightning-timm
\nCompared to working with just tiles, you may use a tissue section, which is healthy and misses any cancer features
\nsee: https://www.kaggle.com/code/jirkaborovec/cancer-subtype-tiles-w-lightning-timm-models
\nBut over both approaches seem to be learning something useful and converging. Also, with tiles, you can say that samples with eventually normal tissue bring some random noise, which prevents the model from overfitting…

", "votes": 2, "replies": [] } ], "raw_markdown_by_id": { "2498874": "I see so many notebook using thumbnail image to train the model, is it just to test the pipeline? Because we won't get thumbnail for test images right? ", "2501129": "@sonujha090 I think the main problem is the png format which is very unusual for digital pathology. Please refer to [this discussion](https://www.kaggle.com/competitions/UBC-OCEAN/discussion/446688#2501120). Having tiff or svs format would make use of high res image much more feasible. Upvote the post of @erntkn 🥇 Hope we will have feedback from @sohier soon :)", "2498915": "We do get thumbnail images for test too, and can also confirm that unlike train where we didn't get thumbnails for tma, we are getting thumbnails for every instance. \nThe main images are very heavy so people are using the tiles/some kind of preprocessing before using them raw. And training on thumbnails is a quicker way to test a pipeline ", "2499717": "Also with a thumbnail, you have the whole image, so you can assume that the cancer features are present in each presented image\nsee: https://www.kaggle.com/code/jirkaborovec/cancer-subtype-baseline-with-lightning-timm\nCompared to working with just tiles, you may use a tissue section, which is healthy and misses any cancer features\nsee: https://www.kaggle.com/code/jirkaborovec/cancer-subtype-tiles-w-lightning-timm-models\nBut over both approaches seem to be learning something useful and converging. Also, with tiles, you can say that samples with eventually normal tissue bring some random noise, which prevents the model from overfitting..." } }