![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F2853990%2F33d1973498eee2f4911b5c3945f0c02f%2Finference_time.jpg?generation=1676868937855006&alt=media\")
my notebook takes about 8+ hours scoring but it fails and then throws an exception although it runs successfully and it takes about 5-7 min ,what that could be
\nI am using pydicom library and gdcm,pylibjpeg dependencies I store the test processed images in \"/kaggle/tmp/output/\"
\nis that a memory problem?
\nit's my first featured competition so I am a little bit confused
Yes, scoring is very long and it also takes me around 8 hours on GPU P100. This is normal.
\nI also got the \"Notebook Threw Exception\" error at the beginning which was due I guess to some scans of the hidden set. In my preprocessing, I applied windowing on all the training examples which was fine, but maybe it did not work on some examples of the hidden set. Anyway, I fixed the issue by using an exeption:
\ntry:\n img = windowing(img, scan)\nexcept:\n pass\nI don't know if this will help you. Otherwise, look at all the steps of the processing from start to finish and identify the places where the problem can potentially come from. Then detect this place with an exception and try to solve it. It can be annoying but it worked for me!
\nAfter that it is still possible to encounter errors such as Notebook Timeout or Submission Scoring Error, but for other reasons. Good luck!
", "rawMarkdown": "@bassantashraf\n\nYes, scoring is very long and it also takes me around 8 hours on GPU P100. This is normal.\n\nI also got the \"Notebook Threw Exception\" error at the beginning which was due I guess to some scans of the hidden set. In my preprocessing, I applied windowing on all the training examples which was fine, but maybe it did not work on some examples of the hidden set. Anyway, I fixed the issue by using an exeption:\n\n```\ntry:\n img = windowing(img, scan)\nexcept:\n pass\n```\n\nI don't know if this will help you. Otherwise, look at all the steps of the processing from start to finish and identify the places where the problem can potentially come from. Then detect this place with an exception and try to solve it. It can be annoying but it worked for me!\n\nAfter that it is still possible to encounter errors such as Notebook Timeout or Submission Scoring Error, but for other reasons. Good luck!", "votes": 1, "replies": [ { "id": 2134995, "postDate": "2023-02-08T11:42:03.457Z", "content": "thank you for your reply, I will definitely try debugging the preprocessing steps
", "rawMarkdown": "thank you for your reply, I will definitely try debugging the preprocessing steps" }, { "id": 2135092, "postDate": "2023-02-08T12:54:16.547Z", "content": "Oh, Thank you for your explanation. Actually, I used your preprocessor method and encountered this issue during inference. I will try your suggestion to avoid mistakes.
\nDo you save the image to the disk before predicting, or do you directly predict after processing and then proceed to the next batch of processing?
", "rawMarkdown": "Oh, Thank you for your explanation. Actually, I used your preprocessor method and encountered this issue during inference. I will try your suggestion to avoid mistakes. \n\nDo you save the image to the disk before predicting, or do you directly predict after processing and then proceed to the next batch of processing?" }, { "id": 2151394, "postDate": "2023-02-20T04:57:16.407Z", "content": "
The inference time against four images is too high. Automatically, it will exceed the nine hours limit. Plz, assist.
", "rawMarkdown": "\n\nThe inference time against four images is too high. Automatically, it will exceed the nine hours limit. Plz, assist." } ] }, { "id": 2134379, "postDate": "2023-02-08T00:15:23.757Z", "content": "my notebook takes about 8+ hours scoring but it fails and then throws an exception although it runs successfully and it takes about 5-7 min ,what that could be
\nI am using pydicom library and gdcm,pylibjpeg dependencies I store the test processed images in \"/kaggle/tmp/output/\"
\nis that a memory problem?
\nit's my first featured competition so I am a little bit confused
Yes, scoring is very long and it also takes me around 8 hours on GPU P100. This is normal.
\nI also got the \"Notebook Threw Exception\" error at the beginning which was due I guess to some scans of the hidden set. In my preprocessing, I applied windowing on all the training examples which was fine, but maybe it did not work on some examples of the hidden set. Anyway, I fixed the issue by using an exeption:
\ntry:\n img = windowing(img, scan)\nexcept:\n pass\nI don't know if this will help you. Otherwise, look at all the steps of the processing from start to finish and identify the places where the problem can potentially come from. Then detect this place with an exception and try to solve it. It can be annoying but it worked for me!
\nAfter that it is still possible to encounter errors such as Notebook Timeout or Submission Scoring Error, but for other reasons. Good luck!
", "votes": 1, "replies": [ { "id": 2134995, "author_name": "Bassant", "author_url": "", "post_date": "2023-02-08T11:42:03.457000", "content": "thank you for your reply, I will definitely try debugging the preprocessing steps
", "votes": 0, "replies": [] }, { "id": 2135092, "author_name": "JimmyLiao", "author_url": "", "post_date": "2023-02-08T12:54:16.547000", "content": "Oh, Thank you for your explanation. Actually, I used your preprocessor method and encountered this issue during inference. I will try your suggestion to avoid mistakes.
\nDo you save the image to the disk before predicting, or do you directly predict after processing and then proceed to the next batch of processing?
", "votes": 0, "replies": [] }, { "id": 2151394, "author_name": "Muhammad", "author_url": "", "post_date": "2023-02-20T04:57:16.407000", "content": "
The inference time against four images is too high. Automatically, it will exceed the nine hours limit. Plz, assist.
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "2134868": "@bassantashraf\n\nYes, scoring is very long and it also takes me around 8 hours on GPU P100. This is normal.\n\nI also got the \"Notebook Threw Exception\" error at the beginning which was due I guess to some scans of the hidden set. In my preprocessing, I applied windowing on all the training examples which was fine, but maybe it did not work on some examples of the hidden set. Anyway, I fixed the issue by using an exeption:\n\n```\ntry:\n img = windowing(img, scan)\nexcept:\n pass\n```\n\nI don't know if this will help you. Otherwise, look at all the steps of the processing from start to finish and identify the places where the problem can potentially come from. Then detect this place with an exception and try to solve it. It can be annoying but it worked for me!\n\nAfter that it is still possible to encounter errors such as Notebook Timeout or Submission Scoring Error, but for other reasons. Good luck!", "2134379": "my notebook takes about 8+ hours scoring but it fails and then throws an exception although it runs successfully and it takes about 5-7 min ,what that could be\nI am using pydicom library and gdcm,pylibjpeg dependencies I store the test processed images in \"/kaggle/tmp/output/\"\nis that a memory problem?\nit's my first featured competition so I am a little bit confused" } }