{ "id": 379041, "title": "My submissions keep return error (Notebook Threw Exception)", "url": "/competitions/rsna-breast-cancer-detection/discussion/379041", "author_name": "OLAF2357", "post_date": "2023-01-17T21:00:24.060000", "votes": 3, "comment_count": 3, "views": 0, "content": "

\"\"

\n

I changed my code many time but I don't know why..

\n

This is my submissions notebook url.
\nhttps://www.kaggle.com/olaf2357/submission-code-rsna

\n

'
\n'
\nWould it be because of the GPU memory?
\n(To prevent this I set the batch_size = 2 & used torch.cuda.memory_allocated())
\n(But Im not sure about memory..)
\n'
\n'

\n

Or is it because I make png files at the output(directory)file?
\nI don't know if is it not allowed to make anything at the output directory except the submission.csv file.
\n(So I deleted the files that a made by using !rm)
\n'
\nI tried but I failed so I need help… TT
\nThanks

\n

This is my submissions notebook url.
\nhttps://www.kaggle.com/olaf2357/submission-code-rsna

", "messages": [ { "id": 2104712, "postDate": "2023-01-18T01:39:29.473Z", "content": "

I had the same problem, until I put a check for 0 in the image preprocessing function. In your case you should make sure that in your process() function img.max() - img.min() is not 0:

\n

`def process(f, size=512, save_folder=None, dicom_process = True, extension=\"png\"):

\n
patient = f.split('/')[-2]\nimage_name = f.split('/')[-1][:-4]\n\ndicom = pydicom.dcmread(f)\nimg = dicom.pixel_array\n\nif img.max() - img.min() != 0:\n    img = (img - img.min()) / (img.max() - img.min())\n\nif dicom.PhotometricInterpretation == \"MONOCHROME1\":  \n    img = 1 - img\n\nimage = (img * 255).astype(np.uint8)\n\n# img = cv2.resize(image, (size1, size2))\nimg = image\nfile_name = f'{save_folder}' + f\"{patient}_{image_name}.{extension}\"\n\ncv2.imwrite(file_name, img)`\n
", "rawMarkdown": "I had the same problem, until I put a check for 0 in the image preprocessing function. In your case you should make sure that in your process() function img.max() - img.min() is not 0:\n\n`def process(f, size=512, save_folder=None, dicom_process = True, extension=\"png\"):\n \n patient = f.split('/')[-2]\n image_name = f.split('/')[-1][:-4]\n\n dicom = pydicom.dcmread(f)\n img = dicom.pixel_array\n\n if img.max() - img.min() != 0:\n img = (img - img.min()) / (img.max() - img.min())\n\n if dicom.PhotometricInterpretation == \"MONOCHROME1\": \n img = 1 - img\n\n image = (img * 255).astype(np.uint8)\n \n # img = cv2.resize(image, (size1, size2))\n img = image\n file_name = f'{save_folder}' + f\"{patient}_{image_name}.{extension}\"\n\n cv2.imwrite(file_name, img)`", "votes": 3 }, { "id": 2104495, "postDate": "2023-01-17T21:00:24.060Z", "content": "

\"\"

\n

I changed my code many time but I don't know why..

\n

This is my submissions notebook url.
\nhttps://www.kaggle.com/olaf2357/submission-code-rsna

\n

'
\n'
\nWould it be because of the GPU memory?
\n(To prevent this I set the batch_size = 2 & used torch.cuda.memory_allocated())
\n(But Im not sure about memory..)
\n'
\n'

\n

Or is it because I make png files at the output(directory)file?
\nI don't know if is it not allowed to make anything at the output directory except the submission.csv file.
\n(So I deleted the files that a made by using !rm)
\n'
\nI tried but I failed so I need help… TT
\nThanks

\n

This is my submissions notebook url.
\nhttps://www.kaggle.com/olaf2357/submission-code-rsna

", "rawMarkdown": "![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F13099637%2Fbc9af8c70073e0d640fa86d22caf48fd%2F20230118_055405_1.png?generation=1673988857917479&alt=media)\n\n\n\nI changed my code many time but I don't know why..\n\nThis is my submissions notebook url.\nhttps://www.kaggle.com/olaf2357/submission-code-rsna\n\n\n'\n'\nWould it be because of the GPU memory? \n(To prevent this I set the batch_size = 2 & used torch.cuda.memory_allocated())\n(But Im not sure about memory..)\n'\n'\n\nOr is it because I make png files at the output(directory)file?\nI don't know if is it not allowed to make anything at the output directory except the submission.csv file.\n(So I deleted the files that a made by using !rm)\n'\nI tried but I failed so I need help... TT\nThanks\n\n\n\n\nThis is my submissions notebook url.\nhttps://www.kaggle.com/olaf2357/submission-code-rsna\n", "votes": 3 }, { "id": 2104717, "postDate": "2023-01-18T01:44:12.970Z", "content": "

you can debug by:

\n
def make_all_zero_submission():\n       set prediction to zero\n       save submission.csv\n\nmake sure this code is correct, i.e. can produce a LB score\n
\n

then, here is how you can debug your code

\n

check if bug is in section 1

\n
... code section 1 ....\nmake_all_zero_submission()\n
\n

`

\n

check if bug is in section 2

\n
... code section 1 ....\n... code section 2 ....\nmake_all_zero_submission()\n
\n

`

\n

check if bug is in section 3

\n
... code section 1 ....\n... code section 2 ....\n... code section 3 ....\nmake_all_zero_submission()\n
\n

`

", "rawMarkdown": "you can debug by:\n\n````\ndef make_all_zero_submission():\n set prediction to zero\n save submission.csv\n \nmake sure this code is correct, i.e. can produce a LB score\n\n````\n\nthen, here is how you can debug your code\n\n\ncheck if bug is in section 1\n```\n... code section 1 ....\nmake_all_zero_submission()\n\n````\n\n\ncheck if bug is in section 2\n```\n... code section 1 ....\n... code section 2 ....\nmake_all_zero_submission()\n\n````\n\n\n\ncheck if bug is in section 3\n```\n... code section 1 ....\n... code section 2 ....\n... code section 3 ....\nmake_all_zero_submission()\n\n````", "votes": 4 }, { "id": 2104826, "postDate": "2023-01-18T04:45:41.053Z", "content": "

I have the same problem. But I'm using tensorflow.
\nEverything runs fine in normal environment.
\nUpon submission, however, my notebook fails within 3-4 minutes.
\nI've tested the notebook inferring on 2500 samples from the training set with 0 issues, so it cannot be a memory problem.
\nI've narrowed down the problematic code to the following lines (unfortunately I have run out of submissions for today to debug further):

\n
BATCH_SIZE = \nbatched_test_ds = processed_test_ds.batch(BATCH_SIZE, num_parallel_calls=).prefetch()\npredictions = model.predict(batched_test_ds)\n
\n

The notebook runs perfectly fine in the normal (non-submission) environment.

\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n

>

\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n

Edit: I figured out what is wrong with my notebook.
\nI am using the tensorflow-io dicom processing functions and it seems to bug out on certain WindowWidth and WindowCenter values. I still have no clue as to why this happens since it works perfectly fine for all the training data.

", "rawMarkdown": "I have the same problem. But I'm using tensorflow.\nEverything runs fine in normal environment.\nUpon submission, however, my notebook fails within 3-4 minutes.\nI've tested the notebook inferring on 2500 samples from the training set with 0 issues, so it cannot be a memory problem.\nI've narrowed down the problematic code to the following lines (unfortunately I have run out of submissions for today to debug further):\n\n```python\nBATCH_SIZE = 64\nbatched_test_ds = processed_test_ds.batch(BATCH_SIZE, num_parallel_calls=4).prefetch(2)\npredictions = model.predict(batched_test_ds)\n```\n\nThe notebook runs perfectly fine in the normal (non-submission) environment.\n\n>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>\n\nEdit: I figured out what is wrong with my notebook.\nI am using the tensorflow-io dicom processing functions and it seems to bug out on certain WindowWidth and WindowCenter values. I still have no clue as to why this happens since it works perfectly fine for all the training data.", "votes": 1 } ], "comments": [ { "id": 2104712, "author_name": "Roland Luethy", "author_url": "", "post_date": "2023-01-18T01:39:29.473000", "content": "

I had the same problem, until I put a check for 0 in the image preprocessing function. In your case you should make sure that in your process() function img.max() - img.min() is not 0:

\n

`def process(f, size=512, save_folder=None, dicom_process = True, extension=\"png\"):

\n
patient = f.split('/')[-2]\nimage_name = f.split('/')[-1][:-4]\n\ndicom = pydicom.dcmread(f)\nimg = dicom.pixel_array\n\nif img.max() - img.min() != 0:\n    img = (img - img.min()) / (img.max() - img.min())\n\nif dicom.PhotometricInterpretation == \"MONOCHROME1\":  \n    img = 1 - img\n\nimage = (img * 255).astype(np.uint8)\n\n# img = cv2.resize(image, (size1, size2))\nimg = image\nfile_name = f'{save_folder}' + f\"{patient}_{image_name}.{extension}\"\n\ncv2.imwrite(file_name, img)`\n
", "votes": 3, "replies": [] }, { "id": 2104717, "author_name": "hengck23", "author_url": "", "post_date": "2023-01-18T01:44:12.970000", "content": "

you can debug by:

\n
def make_all_zero_submission():\n       set prediction to zero\n       save submission.csv\n\nmake sure this code is correct, i.e. can produce a LB score\n
\n

then, here is how you can debug your code

\n

check if bug is in section 1

\n
... code section 1 ....\nmake_all_zero_submission()\n
\n

`

\n

check if bug is in section 2

\n
... code section 1 ....\n... code section 2 ....\nmake_all_zero_submission()\n
\n

`

\n

check if bug is in section 3

\n
... code section 1 ....\n... code section 2 ....\n... code section 3 ....\nmake_all_zero_submission()\n
\n

`

", "votes": 4, "replies": [] }, { "id": 2104826, "author_name": "Bill Qi", "author_url": "", "post_date": "2023-01-18T04:45:41.053000", "content": "

I have the same problem. But I'm using tensorflow.
\nEverything runs fine in normal environment.
\nUpon submission, however, my notebook fails within 3-4 minutes.
\nI've tested the notebook inferring on 2500 samples from the training set with 0 issues, so it cannot be a memory problem.
\nI've narrowed down the problematic code to the following lines (unfortunately I have run out of submissions for today to debug further):

\n
BATCH_SIZE = \nbatched_test_ds = processed_test_ds.batch(BATCH_SIZE, num_parallel_calls=).prefetch()\npredictions = model.predict(batched_test_ds)\n
\n

The notebook runs perfectly fine in the normal (non-submission) environment.

\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n

>

\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n
\n

Edit: I figured out what is wrong with my notebook.
\nI am using the tensorflow-io dicom processing functions and it seems to bug out on certain WindowWidth and WindowCenter values. I still have no clue as to why this happens since it works perfectly fine for all the training data.

", "votes": 1, "replies": [] } ], "raw_markdown_by_id": { "2104712": "I had the same problem, until I put a check for 0 in the image preprocessing function. In your case you should make sure that in your process() function img.max() - img.min() is not 0:\n\n`def process(f, size=512, save_folder=None, dicom_process = True, extension=\"png\"):\n \n patient = f.split('/')[-2]\n image_name = f.split('/')[-1][:-4]\n\n dicom = pydicom.dcmread(f)\n img = dicom.pixel_array\n\n if img.max() - img.min() != 0:\n img = (img - img.min()) / (img.max() - img.min())\n\n if dicom.PhotometricInterpretation == \"MONOCHROME1\": \n img = 1 - img\n\n image = (img * 255).astype(np.uint8)\n \n # img = cv2.resize(image, (size1, size2))\n img = image\n file_name = f'{save_folder}' + f\"{patient}_{image_name}.{extension}\"\n\n cv2.imwrite(file_name, img)`", "2104495": "![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F13099637%2Fbc9af8c70073e0d640fa86d22caf48fd%2F20230118_055405_1.png?generation=1673988857917479&alt=media)\n\n\n\nI changed my code many time but I don't know why..\n\nThis is my submissions notebook url.\nhttps://www.kaggle.com/olaf2357/submission-code-rsna\n\n\n'\n'\nWould it be because of the GPU memory? \n(To prevent this I set the batch_size = 2 & used torch.cuda.memory_allocated())\n(But Im not sure about memory..)\n'\n'\n\nOr is it because I make png files at the output(directory)file?\nI don't know if is it not allowed to make anything at the output directory except the submission.csv file.\n(So I deleted the files that a made by using !rm)\n'\nI tried but I failed so I need help... TT\nThanks\n\n\n\n\nThis is my submissions notebook url.\nhttps://www.kaggle.com/olaf2357/submission-code-rsna\n", "2104717": "you can debug by:\n\n````\ndef make_all_zero_submission():\n set prediction to zero\n save submission.csv\n \nmake sure this code is correct, i.e. can produce a LB score\n\n````\n\nthen, here is how you can debug your code\n\n\ncheck if bug is in section 1\n```\n... code section 1 ....\nmake_all_zero_submission()\n\n````\n\n\ncheck if bug is in section 2\n```\n... code section 1 ....\n... code section 2 ....\nmake_all_zero_submission()\n\n````\n\n\n\ncheck if bug is in section 3\n```\n... code section 1 ....\n... code section 2 ....\n... code section 3 ....\nmake_all_zero_submission()\n\n````", "2104826": "I have the same problem. But I'm using tensorflow.\nEverything runs fine in normal environment.\nUpon submission, however, my notebook fails within 3-4 minutes.\nI've tested the notebook inferring on 2500 samples from the training set with 0 issues, so it cannot be a memory problem.\nI've narrowed down the problematic code to the following lines (unfortunately I have run out of submissions for today to debug further):\n\n```python\nBATCH_SIZE = 64\nbatched_test_ds = processed_test_ds.batch(BATCH_SIZE, num_parallel_calls=4).prefetch(2)\npredictions = model.predict(batched_test_ds)\n```\n\nThe notebook runs perfectly fine in the normal (non-submission) environment.\n\n>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>\n\nEdit: I figured out what is wrong with my notebook.\nI am using the tensorflow-io dicom processing functions and it seems to bug out on certain WindowWidth and WindowCenter values. I still have no clue as to why this happens since it works perfectly fine for all the training data." } }