Hi all,
\n i am facing an issue with submitting my model/work to this competition. i successfully made an inference kernel and commit also went well and created a submission.csv file. it hardly took me 1-2hr for the commit to complete. but, when i submit it to the competition, it runs for more than 9hrs exhausting the allowed time. i am not sure why am i facing this issue .
i compared the size of the submission file created by my inference kernel with the mean submission csv and both are having the same shape proving that within the commit period my kernal was able to predict for 650 Exam that is required for the public LB.
\nany help is appreciated.
", "messages": [ { "id": 1032925, "postDate": "2020-09-30T14:14:17.883Z", "content": "i found a temporary solution. so, it turns out that when we submit our solution, it is run across both public and private dataset(i guess this is for checking whether the work will raise any issue on final test ). this is causing some RAM issue which i have not figured it out yet.
\nAs a temporary solution, i modified my solution to run only on public dataset for now and not on private dataset. i did this by simply checked the no of study files in the test folder. if it is greater than 650 study then i just return the sample submission instead of running my model for now.
\nNow i know how well my model perform on the public test set for now.
Are you using GPU or CPU? For me, it took 3-3.5 hrs to submit the inference result using GPU
Btw you can have a look at a similar issue here
", "rawMarkdown": "Btw you can have a look at a similar issue [here](https://www.kaggle.com/c/rsna-str-pulmonary-embolism-detection/discussion/186355)" }, { "id": 1032803, "postDate": "2020-09-30T13:04:22.593Z", "content": "i am using GPU . and it works well when i commit it . and for committing it took only 1-2hr. i guess it has to do with how my work is behaving with private dataset
", "rawMarkdown": "i am using GPU . and it works well when i commit it . and for committing it took only 1-2hr. i guess it has to do with how my work is behaving with private dataset" }, { "id": 1032870, "postDate": "2020-09-30T13:47:27.273Z", "content": "I updated my post explaining why it didn't work for me, maybe you have a similar problem. In my case the private test set contained .dcm not readable with pydicom without gdcm
", "rawMarkdown": "I updated my [post](https://www.kaggle.com/c/rsna-str-pulmonary-embolism-detection/discussion/186355) explaining why it didn't work for me, maybe you have a similar problem. In my case the private test set contained .dcm not readable with pydicom without gdcm " } ] }, { "id": 1032569, "postDate": "2020-09-30T09:47:46.917Z", "content": "Hi all,
\n i am facing an issue with submitting my model/work to this competition. i successfully made an inference kernel and commit also went well and created a submission.csv file. it hardly took me 1-2hr for the commit to complete. but, when i submit it to the competition, it runs for more than 9hrs exhausting the allowed time. i am not sure why am i facing this issue .
i compared the size of the submission file created by my inference kernel with the mean submission csv and both are having the same shape proving that within the commit period my kernal was able to predict for 650 Exam that is required for the public LB.
\nany help is appreciated.
", "rawMarkdown": "Hi all, \n i am facing an issue with submitting my model/work to this competition. i successfully made an inference kernel and commit also went well and created a submission.csv file. it hardly took me 1-2hr for the commit to complete. but, when i submit it to the competition, it runs for more than 9hrs exhausting the allowed time. i am not sure why am i facing this issue . \n\ni compared the size of the submission file created by my inference kernel with the mean submission csv and both are having the same shape proving that within the commit period my kernal was able to predict for 650 Exam that is required for the public LB.\n\nany help is appreciated. " } ], "comments": [ { "id": 1032925, "author_name": "yuvaramsingh", "author_url": "", "post_date": "2020-09-30T14:14:17.883000", "content": "i found a temporary solution. so, it turns out that when we submit our solution, it is run across both public and private dataset(i guess this is for checking whether the work will raise any issue on final test ). this is causing some RAM issue which i have not figured it out yet.
\nAs a temporary solution, i modified my solution to run only on public dataset for now and not on private dataset. i did this by simply checked the no of study files in the test folder. if it is greater than 650 study then i just return the sample submission instead of running my model for now.
\nNow i know how well my model perform on the public test set for now.
Are you using GPU or CPU? For me, it took 3-3.5 hrs to submit the inference result using GPU
Btw you can have a look at a similar issue here
", "votes": 0, "replies": [] }, { "id": 1032803, "author_name": "yuvaramsingh", "author_url": "", "post_date": "2020-09-30T13:04:22.593000", "content": "i am using GPU . and it works well when i commit it . and for committing it took only 1-2hr. i guess it has to do with how my work is behaving with private dataset
", "votes": 0, "replies": [] }, { "id": 1032870, "author_name": "Marco Stefani", "author_url": "", "post_date": "2020-09-30T13:47:27.273000", "content": "I updated my post explaining why it didn't work for me, maybe you have a similar problem. In my case the private test set contained .dcm not readable with pydicom without gdcm
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "1032925": "i found a temporary solution. so, it turns out that when we submit our solution, it is run across both public and private dataset(i guess this is for checking whether the work will raise any issue on final test ). this is causing some RAM issue which i have not figured it out yet. \nAs a temporary solution, i modified my solution to run only on public dataset for now and not on private dataset. i did this by simply checked the no of study files in the test folder. if it is greater than 650 study then i just return the sample submission instead of running my model for now.\nNow i know how well my model perform on the public test set for now. ", "1032745": "Are you using `GPU` or `CPU`? For me, it took 3-3.5 hrs to submit the inference result using `GPU`", "1032569": "Hi all, \n i am facing an issue with submitting my model/work to this competition. i successfully made an inference kernel and commit also went well and created a submission.csv file. it hardly took me 1-2hr for the commit to complete. but, when i submit it to the competition, it runs for more than 9hrs exhausting the allowed time. i am not sure why am i facing this issue . \n\ni compared the size of the submission file created by my inference kernel with the mean submission csv and both are having the same shape proving that within the commit period my kernal was able to predict for 650 Exam that is required for the public LB.\n\nany help is appreciated. " } }