My notebook took 1 hour to commit. The full test set is about 4 times the public test set. It should take about 4 hours to submit but it keeps going in timeout, meaning it takes more than 9 hours. Why? Am I missing something?
\nUpdate: I was using multithreading, the workers silently failed because of a pydicom error while reading .dcm files, and the main thread was stuck waiting the end of the workers. I don't know exactly how that was possible but doing all the computations on the main thread made the notebook finish and threw an exception. Then, installing gdcm resolved the pydicom reading exception. Only the private test set contains images not readable without gdcm
", "messages": [ { "id": 1025324, "postDate": "2020-09-24T13:32:24.113Z", "content": "@neuronffh I head the same issue. I found out the problem is the memory limitation. In my pipeline I had a loop which loaded all the dicom metadata into a Pandas DataFrame. This loop consumed a lot of memory which caused the notebook to use the disk swap space which really slowed the script and it timed - out. After improving the loop and saving only the necessary data, the notebook finished after the expected time.
", "rawMarkdown": "@neuronffh I head the same issue. I found out the problem is the memory limitation. In my pipeline I had a loop which loaded all the dicom metadata into a Pandas DataFrame. This loop consumed a lot of memory which caused the notebook to use the disk swap space which really slowed the script and it timed - out. After improving the loop and saving only the necessary data, the notebook finished after the expected time. ", "votes": 1, "replies": [ { "id": 1025692, "postDate": "2020-09-24T17:56:22.650Z", "content": "Thank you a lot for your suggestion!
\nI'm now looking at my code to see how it manages memory and whether ram is a problem or not
My notebook took 1 hour to commit. The full test set is about 4 times the public test set. It should take about 4 hours to submit but it keeps going in timeout, meaning it takes more than 9 hours. Why? Am I missing something?
\nUpdate: I was using multithreading, the workers silently failed because of a pydicom error while reading .dcm files, and the main thread was stuck waiting the end of the workers. I don't know exactly how that was possible but doing all the computations on the main thread made the notebook finish and threw an exception. Then, installing gdcm resolved the pydicom reading exception. Only the private test set contains images not readable without gdcm
", "rawMarkdown": "My notebook took 1 hour to commit. The full test set is about 4 times the public test set. It should take about 4 hours to submit but it keeps going in timeout, meaning it takes more than 9 hours. Why? Am I missing something?\n\nUpdate: I was using multithreading, the workers silently failed because of a pydicom error while reading .dcm files, and the main thread was stuck waiting the end of the workers. I don't know exactly how that was possible but doing all the computations on the main thread made the notebook finish and threw an exception. Then, installing gdcm resolved the pydicom reading exception. Only the private test set contains images not readable without gdcm\n", "votes": 2 }, { "id": 1025250, "postDate": "2020-09-24T12:52:07.997Z", "content": "I have exactly the same problem and I genuinely don't know how to solve it…
\nMade some changes this morning to streamline it as much as possible, submitted again, this time (after a still ridiculous amount of time spent processing it) I get a \"submission scoring error\" - \"no additional details provided for this error\".
\nAnybody know something about that?
Look here \"submission scoring error\" means your submission file is not in the correct format:
\nYou might want to compare your submission file to sample_submission.csv
", "rawMarkdown": "Look [here](https://www.kaggle.com/code-competition-debugging) \"submission scoring error\" means your submission file is not in the correct format:\n* You may be missing the header\n* Maybe you don't have all the required id's\n\nYou might want to compare your submission file to sample_submission.csv" }, { "id": 1025424, "postDate": "2020-09-24T14:41:07.110Z", "content": "Yeah I think I'll just resort to loading the sample submission file and replacing the values, instead of writing my own submission file… thanks!
", "rawMarkdown": "Yeah I think I'll just resort to loading the sample submission file and replacing the values, instead of writing my own submission file... thanks!" } ] } ], "comments": [ { "id": 1025324, "author_name": "yuval reina", "author_url": "", "post_date": "2020-09-24T13:32:24.113000", "content": "@neuronffh I head the same issue. I found out the problem is the memory limitation. In my pipeline I had a loop which loaded all the dicom metadata into a Pandas DataFrame. This loop consumed a lot of memory which caused the notebook to use the disk swap space which really slowed the script and it timed - out. After improving the loop and saving only the necessary data, the notebook finished after the expected time.
", "votes": 1, "replies": [ { "id": 1025692, "author_name": "Marco Stefani", "author_url": "", "post_date": "2020-09-24T17:56:22.650000", "content": "Thank you a lot for your suggestion!
\nI'm now looking at my code to see how it manages memory and whether ram is a problem or not
I have exactly the same problem and I genuinely don't know how to solve it…
\nMade some changes this morning to streamline it as much as possible, submitted again, this time (after a still ridiculous amount of time spent processing it) I get a \"submission scoring error\" - \"no additional details provided for this error\".
\nAnybody know something about that?
Look here \"submission scoring error\" means your submission file is not in the correct format:
\nYou might want to compare your submission file to sample_submission.csv
", "votes": 0, "replies": [] }, { "id": 1025424, "author_name": "Alex Bader", "author_url": "", "post_date": "2020-09-24T14:41:07.110000", "content": "Yeah I think I'll just resort to loading the sample submission file and replacing the values, instead of writing my own submission file… thanks!
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "1025324": "@neuronffh I head the same issue. I found out the problem is the memory limitation. In my pipeline I had a loop which loaded all the dicom metadata into a Pandas DataFrame. This loop consumed a lot of memory which caused the notebook to use the disk swap space which really slowed the script and it timed - out. After improving the loop and saving only the necessary data, the notebook finished after the expected time. ", "1024902": "My notebook took 1 hour to commit. The full test set is about 4 times the public test set. It should take about 4 hours to submit but it keeps going in timeout, meaning it takes more than 9 hours. Why? Am I missing something?\n\nUpdate: I was using multithreading, the workers silently failed because of a pydicom error while reading .dcm files, and the main thread was stuck waiting the end of the workers. I don't know exactly how that was possible but doing all the computations on the main thread made the notebook finish and threw an exception. Then, installing gdcm resolved the pydicom reading exception. Only the private test set contains images not readable without gdcm\n", "1025250": "I have exactly the same problem and I genuinely don't know how to solve it...\nMade some changes this morning to streamline it as much as possible, submitted again, this time (after a still ridiculous amount of time spent processing it) I get a \"submission scoring error\" - \"no additional details provided for this error\".\nAnybody know something about that?" } }