Hi Everyone,
\nI'm having some difficulty with getting the image generation part of my submission script to run quickly enough. I trained an efficient_net classifier (keras-cv with jax backend) on individual tiles and I'm using pyvips to cut the tiles into a regular grid ( <= 32 tiles per image) for inference, then predicting by using a simple sum of the logits. My NN whips through these at about 5-6 seconds per image (less on the smaller batches), but my total processing time varies widely. Naturally, TMAs don't take that long (~3-4 seconds total processing time), but larger images take ~30-100 seconds, depending on the size.
\nI'm just trying to get a baseline in, but my last submission timed out. I was hopeful that the majority of the test set was TMAs, but it seems there aren't enough to compensate for the time. With an average 30 seconds of processing time X 2000 images, you end up with about 17 hours of total processing time give or take a bit.
\nWhat sort of timings are others here seeing and are you willing to share some tips and tricks on how you were able to achieve it? I was hoping to use this competition to learn how to build a model that generalizes well and can perform outlier detection, but I've spent quite a lot of time fighting with submissions and trying to get the most out of pyvips…Any advice would be greatly appreciated.
", "messages": [ { "id": 2574327, "postDate": "2023-12-25T20:21:12.793Z", "content": "something like:
\n concurrent.futures \n\n (): \n do tiling stuff...\n tiles\n\n concurrent.futures.ThreadPoolExecutor(max_workers=) executor:\n results = ((executor.(process_tile) )\nthis is a dummy example, you need to adapt for your method and process!
", "rawMarkdown": "something like:\n\n\n```python\nimport concurrent.futures \n\ndef process_tile(): \n do tiling stuff...\n return tiles\n \nwith concurrent.futures.ThreadPoolExecutor(max_workers=4) as executor:\n results = list((executor.map(process_tile) )\n```\n\nthis is a dummy example, you need to adapt for your method and process!", "votes": 3, "replies": [ { "id": 2574998, "postDate": "2023-12-26T12:52:35.337Z", "content": "Thanks @aliabbasi
", "rawMarkdown": "Thanks @aliabbasi " } ] }, { "id": 2573127, "postDate": "2023-12-24T18:27:43.127Z", "content": "almost had same issue,
\nnow I'm tiling images with pyvips on 4 core of CPUs, each core cropping a tile, and it runs at lease 25% faster.
\nyou can use python naive concurrent module
Hey @aliabbasi , thanks for the tip! Your pyvips offline notebook gave me a great starting point with this. However, I don't see anything in that notebook that shows how explicitly use a core to crop a tile. I'm also not sure what you mean by \"you can use python naive concurrent module\". Do you have an example of such a module so I can go learn how to use it?
", "rawMarkdown": "Hey @aliabbasi , thanks for the tip! Your pyvips offline notebook gave me a great starting point with this. However, I don't see anything in that notebook that shows how explicitly use a core to crop a tile. I'm also not sure what you mean by \"you can use python naive concurrent module\". Do you have an example of such a module so I can go learn how to use it?", "replies": [ { "id": 2574852, "postDate": "2023-12-26T09:37:47.297Z", "content": "Hi @aliabbasi While converting the whole images into tiles I did not get any issue related to the time . Are you converting images from train_images folder or from thumbnail folder?
", "rawMarkdown": "Hi @aliabbasi While converting the whole images into tiles I did not get any issue related to the time . Are you converting images from train_images folder or from thumbnail folder?", "isDeleted": true } ] } ] }, { "id": 2573025, "postDate": "2023-12-24T16:35:14.287Z", "content": "Hi Everyone,
\nI'm having some difficulty with getting the image generation part of my submission script to run quickly enough. I trained an efficient_net classifier (keras-cv with jax backend) on individual tiles and I'm using pyvips to cut the tiles into a regular grid ( <= 32 tiles per image) for inference, then predicting by using a simple sum of the logits. My NN whips through these at about 5-6 seconds per image (less on the smaller batches), but my total processing time varies widely. Naturally, TMAs don't take that long (~3-4 seconds total processing time), but larger images take ~30-100 seconds, depending on the size.
\nI'm just trying to get a baseline in, but my last submission timed out. I was hopeful that the majority of the test set was TMAs, but it seems there aren't enough to compensate for the time. With an average 30 seconds of processing time X 2000 images, you end up with about 17 hours of total processing time give or take a bit.
\nWhat sort of timings are others here seeing and are you willing to share some tips and tricks on how you were able to achieve it? I was hoping to use this competition to learn how to build a model that generalizes well and can perform outlier detection, but I've spent quite a lot of time fighting with submissions and trying to get the most out of pyvips…Any advice would be greatly appreciated.
", "rawMarkdown": "Hi Everyone,\n\nI'm having some difficulty with getting the image generation part of my submission script to run quickly enough. I trained an efficient_net classifier (keras-cv with jax backend) on individual tiles and I'm using pyvips to cut the tiles into a regular grid ( <= 32 tiles per image) for inference, then predicting by using a simple sum of the logits. My NN whips through these at about 5-6 seconds per image (less on the smaller batches), but my total processing time varies widely. Naturally, TMAs don't take that long (~3-4 seconds total processing time), but larger images take ~30-100 seconds, depending on the size.\n\nI'm just trying to get a baseline in, but my last submission timed out. I was hopeful that the majority of the test set was TMAs, but it seems there aren't enough to compensate for the time. With an average 30 seconds of processing time X 2000 images, you end up with about 17 hours of total processing time give or take a bit.\n\nWhat sort of timings are others here seeing and are you willing to share some tips and tricks on how you were able to achieve it? I was hoping to use this competition to learn how to build a model that generalizes well and can perform outlier detection, but I've spent quite a lot of time fighting with submissions and trying to get the most out of pyvips...Any advice would be greatly appreciated.", "votes": 4 } ], "comments": [ { "id": 2574327, "author_name": "Ali", "author_url": "", "post_date": "2023-12-25T20:21:12.793000", "content": "something like:
\n concurrent.futures \n\n (): \n do tiling stuff...\n tiles\n\n concurrent.futures.ThreadPoolExecutor(max_workers=) executor:\n results = ((executor.(process_tile) )\nthis is a dummy example, you need to adapt for your method and process!
", "votes": 3, "replies": [ { "id": 2574998, "author_name": "chemdatafarmer", "author_url": "", "post_date": "2023-12-26T12:52:35.337000", "content": "Thanks @aliabbasi
", "votes": 0, "replies": [] } ] }, { "id": 2573127, "author_name": "Ali", "author_url": "", "post_date": "2023-12-24T18:27:43.127000", "content": "almost had same issue,
\nnow I'm tiling images with pyvips on 4 core of CPUs, each core cropping a tile, and it runs at lease 25% faster.
\nyou can use python naive concurrent module
Hey @aliabbasi , thanks for the tip! Your pyvips offline notebook gave me a great starting point with this. However, I don't see anything in that notebook that shows how explicitly use a core to crop a tile. I'm also not sure what you mean by \"you can use python naive concurrent module\". Do you have an example of such a module so I can go learn how to use it?
", "votes": 0, "replies": [ { "id": 2574852, "author_name": "", "author_url": "", "post_date": "2023-12-26T09:37:47.297000", "content": "Hi @aliabbasi While converting the whole images into tiles I did not get any issue related to the time . Are you converting images from train_images folder or from thumbnail folder?
", "votes": 0, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2574327": "something like:\n\n\n```python\nimport concurrent.futures \n\ndef process_tile(): \n do tiling stuff...\n return tiles\n \nwith concurrent.futures.ThreadPoolExecutor(max_workers=4) as executor:\n results = list((executor.map(process_tile) )\n```\n\nthis is a dummy example, you need to adapt for your method and process!", "2573127": "almost had same issue, \nnow I'm tiling images with pyvips on 4 core of CPUs, each core cropping a tile, and it runs at lease 25% faster. \nyou can use python naive concurrent module ", "2573025": "Hi Everyone,\n\nI'm having some difficulty with getting the image generation part of my submission script to run quickly enough. I trained an efficient_net classifier (keras-cv with jax backend) on individual tiles and I'm using pyvips to cut the tiles into a regular grid ( <= 32 tiles per image) for inference, then predicting by using a simple sum of the logits. My NN whips through these at about 5-6 seconds per image (less on the smaller batches), but my total processing time varies widely. Naturally, TMAs don't take that long (~3-4 seconds total processing time), but larger images take ~30-100 seconds, depending on the size.\n\nI'm just trying to get a baseline in, but my last submission timed out. I was hopeful that the majority of the test set was TMAs, but it seems there aren't enough to compensate for the time. With an average 30 seconds of processing time X 2000 images, you end up with about 17 hours of total processing time give or take a bit.\n\nWhat sort of timings are others here seeing and are you willing to share some tips and tricks on how you were able to achieve it? I was hoping to use this competition to learn how to build a model that generalizes well and can perform outlier detection, but I've spent quite a lot of time fighting with submissions and trying to get the most out of pyvips...Any advice would be greatly appreciated." } }