It would be really nice if the Kaggle notebook submission recorded the scoring time and not just the notebook time. There have been many competitions (and there will be more) where the scoring can take hours and it will be good for diagnosis to have that time recorded for debugging and planning your research.
", "messages": [ { "id": 2496311, "postDate": "2023-10-24T00:40:52.733Z", "content": "It would be really nice if the Kaggle notebook submission recorded the scoring time and not just the notebook time. There have been many competitions (and there will be more) where the scoring can take hours and it will be good for diagnosis to have that time recorded for debugging and planning your research.
", "rawMarkdown": "It would be really nice if the Kaggle notebook submission recorded the scoring time and not just the notebook time. There have been many competitions (and there will be more) where the scoring can take hours and it will be good for diagnosis to have that time recorded for debugging and planning your research.", "votes": 9 }, { "id": 2497912, "postDate": "2023-10-25T02:01:45.393Z", "content": "My frustration comes with contests like this where I get stuck for some time because of some error while the hidden data gets processed (usually out of CPU memory or out of GPU memory). This is not supposed to be a resource management contest.
\nAnd certainly, there are things that could be done to improve things for competitors:
I agree. Yes, people can probe LB with that but they are already doing it by counting seconds anyway.
", "rawMarkdown": "I agree. Yes, people can probe LB with that but they are already doing it by counting seconds anyway.", "votes": 3 }, { "id": 2499294, "postDate": "2023-10-25T23:55:28.250Z", "content": "Yes, as a college student it’s really impossible for me to count how long my notebook run and returned error.But I had to do it because it’s really important for me.Also I think it is not really good to waste a great number of time in debugging and planning!
", "rawMarkdown": "Yes, as a college student it’s really impossible for me to count how long my notebook run and returned error.But I had to do it because it’s really important for me.Also I think it is not really good to waste a great number of time in debugging and planning!", "votes": 2 } ], "comments": [ { "id": 2497912, "author_name": "Seung Jae Lee", "author_url": "", "post_date": "2023-10-25T02:01:45.393000", "content": "My frustration comes with contests like this where I get stuck for some time because of some error while the hidden data gets processed (usually out of CPU memory or out of GPU memory). This is not supposed to be a resource management contest.
\nAnd certainly, there are things that could be done to improve things for competitors:
I agree. Yes, people can probe LB with that but they are already doing it by counting seconds anyway.
", "votes": 3, "replies": [] }, { "id": 2499294, "author_name": "LuoZiqian", "author_url": "", "post_date": "2023-10-25T23:55:28.250000", "content": "Yes, as a college student it’s really impossible for me to count how long my notebook run and returned error.But I had to do it because it’s really important for me.Also I think it is not really good to waste a great number of time in debugging and planning!
", "votes": 2, "replies": [] } ], "raw_markdown_by_id": { "2496311": "It would be really nice if the Kaggle notebook submission recorded the scoring time and not just the notebook time. There have been many competitions (and there will be more) where the scoring can take hours and it will be good for diagnosis to have that time recorded for debugging and planning your research.", "2497912": "My frustration comes with contests like this where I get stuck for some time because of some error while the hidden data gets processed (usually out of CPU memory or out of GPU memory). This is not supposed to be a resource management contest.\nAnd certainly, there are things that could be done to improve things for competitors:\n- More submissions per day.\n- Record time of scoring process especially when it fails.\n- Tell us the maximum size of data per prediction item. In the case of image processing contests like this one, what is the maximum image size in the hidden data?\n- Use a system with more RAM and GPU with more memory for scoring.", "2496497": "I agree. Yes, people can probe LB with that but they are already doing it by counting seconds anyway.", "2499294": "Yes, as a college student it’s really impossible for me to count how long my notebook run and returned error.But I had to do it because it’s really important for me.Also I think it is not really good to waste a great number of time in debugging and planning!" } }