Hello, guys!
\nLet me ask a question about the run time at submission.
Currently, when I submit my notebook, it takes about an hour for the score to be calculated (for preprocessing and inference). Does this situation result in a run time error when making the final submission?
\n(I was told that Public LB is calculated using 7% of the Private test data. If I simply calculate it, the upper limit of 9 hours at the time of submission would mean that only 63% of the total private data preprocessing and inference would be completed…?)
\nPlease let me know how long it is taking you guys!
\nthere is anything set in your setup, please let me know!
", "messages": [ { "id": 1957694, "postDate": "2022-09-27T06:01:13.703Z", "content": "When you submitted, it will run on all the data. But the LB only showed the score for that 7% data. Anyway, you have already got a PB score. Don't worry.
", "rawMarkdown": "When you submitted, it will run on all the data. But the LB only showed the score for that 7% data. Anyway, you have already got a PB score. Don't worry.", "votes": 5, "replies": [ { "id": 1957844, "postDate": "2022-09-27T07:17:07.257Z", "content": "Are you sure about this? I mean it makes sense cause my prediction time for 4 images was 200 seconds and it takes 4+ hours during the submission. Some people think that when we submit, it runs on 20 images or something but the numbers don't add up.
", "rawMarkdown": "Are you sure about this? I mean it makes sense cause my prediction time for 4 images was 200 seconds and it takes 4+ hours during the submission. Some people think that when we submit, it runs on 20 images or something but the numbers don't add up. ", "votes": 2 }, { "id": 1957877, "postDate": "2022-09-27T07:31:12.740Z", "content": "I confirm he is correct. Once you got the score, it means your kernel is running fine on the whole test set. Just that the score displayed is only from 7% of them (they will reveal the complete score a.k.a. private score at the end)
", "rawMarkdown": "I confirm he is correct. Once you got the score, it means your kernel is running fine on the whole test set. Just that the score displayed is only from 7% of them (they will reveal the complete score a.k.a. private score at the end) ", "votes": 4 }, { "id": 1971494, "postDate": "2022-10-04T16:38:38.803Z", "content": "I have had this question for a long time. Here are my two observations that hint that @forcewithme and @nyleve are right:
\n1) I made a fake submission where I read images upto 2Gb, resize them, delete them and input a 0.5 prediction without any inference. This notebook ran for over an hour, which means that it definitely reading more than 20 images, 280 makes sense given the sizes of images from the training data.
\n2) I saw somewhere hosts mentioning that kaggle is monitoring private LB data. This is only possible if inference happens on all hidden data.
\nSo if notebook runs within 9 hours, you should be fine. Otherwise each notebook should finish within 40min (7% of 9hours), which I don't think is even achievable.
That also explains how private lb scores of all previous submissions are instantly displayed when competitions end.
", "rawMarkdown": "That also explains how private lb scores of all previous submissions are instantly displayed when competitions end.", "votes": 1 } ] }, { "id": 1957434, "postDate": "2022-09-27T01:23:08.930Z", "content": "Hello, guys!
\nLet me ask a question about the run time at submission.
Currently, when I submit my notebook, it takes about an hour for the score to be calculated (for preprocessing and inference). Does this situation result in a run time error when making the final submission?
\n(I was told that Public LB is calculated using 7% of the Private test data. If I simply calculate it, the upper limit of 9 hours at the time of submission would mean that only 63% of the total private data preprocessing and inference would be completed…?)
\nPlease let me know how long it is taking you guys!
\nthere is anything set in your setup, please let me know!
", "rawMarkdown": "Hello, guys!\nLet me ask a question about the run time at submission.\n\nCurrently, when I submit my notebook, it takes about an hour for the score to be calculated (for preprocessing and inference). Does this situation result in a run time error when making the final submission?\n\n(I was told that Public LB is calculated using 7% of the Private test data. If I simply calculate it, the upper limit of 9 hours at the time of submission would mean that only 63% of the total private data preprocessing and inference would be completed...?)\n\nPlease let me know how long it is taking you guys!\n\nthere is anything set in your setup, please let me know!", "votes": 4 } ], "comments": [ { "id": 1957694, "author_name": "ForcewithMe", "author_url": "", "post_date": "2022-09-27T06:01:13.703000", "content": "When you submitted, it will run on all the data. But the LB only showed the score for that 7% data. Anyway, you have already got a PB score. Don't worry.
", "votes": 5, "replies": [ { "id": 1957844, "author_name": "Gunes Evitan", "author_url": "", "post_date": "2022-09-27T07:17:07.257000", "content": "Are you sure about this? I mean it makes sense cause my prediction time for 4 images was 200 seconds and it takes 4+ hours during the submission. Some people think that when we submit, it runs on 20 images or something but the numbers don't add up.
", "votes": 2, "replies": [] }, { "id": 1957877, "author_name": "yukiya", "author_url": "", "post_date": "2022-09-27T07:31:12.740000", "content": "I confirm he is correct. Once you got the score, it means your kernel is running fine on the whole test set. Just that the score displayed is only from 7% of them (they will reveal the complete score a.k.a. private score at the end)
", "votes": 4, "replies": [] }, { "id": 1971494, "author_name": "tdiceman", "author_url": "", "post_date": "2022-10-04T16:38:38.803000", "content": "I have had this question for a long time. Here are my two observations that hint that @forcewithme and @nyleve are right:
\n1) I made a fake submission where I read images upto 2Gb, resize them, delete them and input a 0.5 prediction without any inference. This notebook ran for over an hour, which means that it definitely reading more than 20 images, 280 makes sense given the sizes of images from the training data.
\n2) I saw somewhere hosts mentioning that kaggle is monitoring private LB data. This is only possible if inference happens on all hidden data.
\nSo if notebook runs within 9 hours, you should be fine. Otherwise each notebook should finish within 40min (7% of 9hours), which I don't think is even achievable.
That also explains how private lb scores of all previous submissions are instantly displayed when competitions end.
", "votes": 1, "replies": [] } ] } ], "raw_markdown_by_id": { "1957694": "When you submitted, it will run on all the data. But the LB only showed the score for that 7% data. Anyway, you have already got a PB score. Don't worry.", "1957434": "Hello, guys!\nLet me ask a question about the run time at submission.\n\nCurrently, when I submit my notebook, it takes about an hour for the score to be calculated (for preprocessing and inference). Does this situation result in a run time error when making the final submission?\n\n(I was told that Public LB is calculated using 7% of the Private test data. If I simply calculate it, the upper limit of 9 hours at the time of submission would mean that only 63% of the total private data preprocessing and inference would be completed...?)\n\nPlease let me know how long it is taking you guys!\n\nthere is anything set in your setup, please let me know!" } }