In the training section, I wanted to use OpenCV to read images (I wanted Numpy format), but I found it to be very slow, taking 30 seconds to read a relatively large image. Does anyone know how to read faster?
", "messages": [ { "id": 2508111, "postDate": "2023-11-01T13:37:21.340Z", "content": "In the training section, I wanted to use OpenCV to read images (I wanted Numpy format), but I found it to be very slow, taking 30 seconds to read a relatively large image. Does anyone know how to read faster?
", "rawMarkdown": "In the training section, I wanted to use OpenCV to read images (I wanted Numpy format), but I found it to be very slow, taking 30 seconds to read a relatively large image. Does anyone know how to read faster?", "votes": 1 }, { "id": 2508523, "postDate": "2023-11-01T19:15:13.170Z", "content": "What is your expectation, loading the whole image to memory?
\nYou may check pyVips: https://www.kaggle.com/code/jirkaborovec/pyvips-load-thumbnail-crop-large-image
Yes, I want to load the entire original image into memory and hope it is in numpy format, but opencv is too slow. Thank you for your suggestion. I'll try this method and get back to you later!
", "rawMarkdown": "Yes, I want to load the entire original image into memory and hope it is in numpy format, but opencv is too slow. Thank you for your suggestion. I'll try this method and get back to you later!" }, { "id": 2508751, "postDate": "2023-11-02T01:24:32.880Z", "content": "It seems that this method is also very slow. It reads images very quickly, but the format of the read images is not numpy, so I need to manually convert to numpy format, which takes too much time.
", "rawMarkdown": "It seems that this method is also very slow. It reads images very quickly, but the format of the read images is not numpy, so I need to manually convert to numpy format, which takes too much time.", "replies": [ { "id": 2508845, "postDate": "2023-11-02T03:16:26.410Z", "content": "Note that what you try to achieve is loading several GB image with PNG decoding on a machine with limited resources and IO speed
", "rawMarkdown": "Note that what you try to achieve is loading several GB image with PNG decoding on a machine with limited resources and IO speed ", "votes": 2 }, { "id": 2508848, "postDate": "2023-11-02T03:22:55.733Z", "content": "yes, I'll try another method, Thank you!
", "rawMarkdown": "yes, I'll try another method, Thank you!" } ] } ] } ], "comments": [ { "id": 2508523, "author_name": "Jirka", "author_url": "", "post_date": "2023-11-01T19:15:13.170000", "content": "What is your expectation, loading the whole image to memory?
\nYou may check pyVips: https://www.kaggle.com/code/jirkaborovec/pyvips-load-thumbnail-crop-large-image
Yes, I want to load the entire original image into memory and hope it is in numpy format, but opencv is too slow. Thank you for your suggestion. I'll try this method and get back to you later!
", "votes": 0, "replies": [] }, { "id": 2508751, "author_name": "WangXuC", "author_url": "", "post_date": "2023-11-02T01:24:32.880000", "content": "It seems that this method is also very slow. It reads images very quickly, but the format of the read images is not numpy, so I need to manually convert to numpy format, which takes too much time.
", "votes": 0, "replies": [ { "id": 2508845, "author_name": "Jirka", "author_url": "", "post_date": "2023-11-02T03:16:26.410000", "content": "Note that what you try to achieve is loading several GB image with PNG decoding on a machine with limited resources and IO speed
", "votes": 2, "replies": [] }, { "id": 2508848, "author_name": "WangXuC", "author_url": "", "post_date": "2023-11-02T03:22:55.733000", "content": "yes, I'll try another method, Thank you!
", "votes": 0, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2508111": "In the training section, I wanted to use OpenCV to read images (I wanted Numpy format), but I found it to be very slow, taking 30 seconds to read a relatively large image. Does anyone know how to read faster?", "2508523": "What is your expectation, loading the whole image to memory?\nYou may check pyVips: https://www.kaggle.com/code/jirkaborovec/pyvips-load-thumbnail-crop-large-image" } }