Yes, another one of these posts. Sorry. I am not totally clear on the requirements even after reading the other 4 or so..
\n\nFrom reading the two-stage FAQ, I interpret there as being two options for uploading our models:
\n\na) Upload the weights of the models that we have trained along with the code to perform image processing and use the models to perform inference and then generate a submission.\nb) Upload the code used to train models, along with code to use them to perform inference and generate submission.
\n\nSince Stage1 ground truth data will be released at the beginning of Stage2, option b is optimal as it allows the opportunity to retain some models using Stage1 data.
\n\nHowever, does the option b submission requires us to have beautifully structured, plug-and-play code like Appiass'? Or can it just have basic instructions for how to tie the different scripts and models together? \nWill it be expected that competition staff can just unzip the model, read a readme and reproduce the result, or will the winners have the opportunity to have a dialog and help to troubleshoot bugs? \nI ask this because having flawless code that can run and train multiple models and then ensemble them together turns this into more of a coding competition as opposed to a DataScience competition. I am a Mechatronics Engineer, not a Software Engineer - as hard as I try I don't think I can reliably produce code / documentation of this quality in the very short time period that I will have to do it.
\n\nI guess what I am asking is: can I just zip my git repo, spend an hour or two writing a README in as much detail as I can and submit that? Then for some crazy reason I do happen to win, have the opportunity to troubleshoot.. Or do I need to spend a whole week and hundreds of GPU hours writing a consolidated workflow and then testing it to ensure that it produces the expected results?
\n\nMy models is already an ensemble of 5, and probably another 5 before the competition ends. It is a non-trivial task to reproduce this. If I want to create a flawless workflow that can reproduce this then I basically need to cancel all my experiments, finish progressing in the competition now and get started on it.
\n\nAlso, do I need to document the hardware and exact environment that was used to generate the results. I am not sure how much of an impact variations in python package versions will have on producing deterministic results, however certainly hardware specific things (like number of GPU, VRAM, etc) is going to have an impact (it split batches 32GB ram between 2x V100 or 3x K80)
\n\nIs this code only used to reproduce the results for the prize winners? I presume so, since it is a huge amount of resources to train all medal winners models.
\n\nFinally, if we do decide that we will want to retrain on Stage1 data - do we need to decide that before we submit our model?
\n\nSorry for the wall of text..\n-C
", "messages": [ { "id": 655488, "postDate": "2019-10-23T05:13:52.850Z", "content": "Yes, another one of these posts. Sorry. I am not totally clear on the requirements even after reading the other 4 or so..
\n\nFrom reading the two-stage FAQ, I interpret there as being two options for uploading our models:
\n\na) Upload the weights of the models that we have trained along with the code to perform image processing and use the models to perform inference and then generate a submission.\nb) Upload the code used to train models, along with code to use them to perform inference and generate submission.
\n\nSince Stage1 ground truth data will be released at the beginning of Stage2, option b is optimal as it allows the opportunity to retain some models using Stage1 data.
\n\nHowever, does the option b submission requires us to have beautifully structured, plug-and-play code like Appiass'? Or can it just have basic instructions for how to tie the different scripts and models together? \nWill it be expected that competition staff can just unzip the model, read a readme and reproduce the result, or will the winners have the opportunity to have a dialog and help to troubleshoot bugs? \nI ask this because having flawless code that can run and train multiple models and then ensemble them together turns this into more of a coding competition as opposed to a DataScience competition. I am a Mechatronics Engineer, not a Software Engineer - as hard as I try I don't think I can reliably produce code / documentation of this quality in the very short time period that I will have to do it.
\n\nI guess what I am asking is: can I just zip my git repo, spend an hour or two writing a README in as much detail as I can and submit that? Then for some crazy reason I do happen to win, have the opportunity to troubleshoot.. Or do I need to spend a whole week and hundreds of GPU hours writing a consolidated workflow and then testing it to ensure that it produces the expected results?
\n\nMy models is already an ensemble of 5, and probably another 5 before the competition ends. It is a non-trivial task to reproduce this. If I want to create a flawless workflow that can reproduce this then I basically need to cancel all my experiments, finish progressing in the competition now and get started on it.
\n\nAlso, do I need to document the hardware and exact environment that was used to generate the results. I am not sure how much of an impact variations in python package versions will have on producing deterministic results, however certainly hardware specific things (like number of GPU, VRAM, etc) is going to have an impact (it split batches 32GB ram between 2x V100 or 3x K80)
\n\nIs this code only used to reproduce the results for the prize winners? I presume so, since it is a huge amount of resources to train all medal winners models.
\n\nFinally, if we do decide that we will want to retrain on Stage1 data - do we need to decide that before we submit our model?
\n\nSorry for the wall of text..\n-C
", "rawMarkdown": "Yes, another one of these posts. Sorry. I am not totally clear on the requirements even after reading the other 4 or so..\n\n**From reading the [two-stage FAQ](https://www.kaggle.com/two-stage-frequently-asked-questions), I interpret there as being two options for uploading our models:**\n\na) Upload the weights of the models that we have trained along with the code to perform image processing and use the models to perform inference and then generate a submission.\nb) Upload the code used to train models, along with code to use them to perform inference and generate submission.\n\n__________\n\nSince Stage1 ground truth data will be released at the beginning of [Stage2](https://www.kaggle.com/c/rsna-intracranial-hemorrhage-detection/discussion/112145#latest-652165), option b is optimal as it allows the opportunity to retain some models using Stage1 data. \n\nHowever, does the option b submission requires us to have beautifully structured, plug-and-play code like Appiass'? Or can it just have basic instructions for how to tie the different scripts and models together? \nWill it be expected that competition staff can just unzip the model, read a readme and reproduce the result, or will the winners have the opportunity to have a dialog and help to troubleshoot bugs? \nI ask this because having flawless code that can run and train multiple models and then ensemble them together turns this into more of a coding competition as opposed to a DataScience competition. I am a Mechatronics Engineer, not a Software Engineer - as hard as I try I don't think I can reliably produce code / documentation of this quality in the very short time period that I will have to do it.\n\n\n**I guess what I am asking is:** can I just zip my git repo, spend an hour or two writing a README in as much detail as I can and submit that? Then for some crazy reason I do happen to win, have the opportunity to troubleshoot.. Or do I need to spend a whole week and hundreds of GPU hours writing a consolidated workflow and then testing it to ensure that it produces the expected results?\n\nMy models is already an ensemble of 5, and probably another 5 before the competition ends. It is a non-trivial task to reproduce this. If I want to create a flawless workflow that can reproduce this then I basically need to cancel all my experiments, finish progressing in the competition now and get started on it.\n\n__________\n\nAlso, do I need to document the hardware and exact environment that was used to generate the results. I am not sure how much of an impact variations in python package versions will have on producing deterministic results, however certainly hardware specific things (like number of GPU, VRAM, etc) is going to have an impact (it split batches 32GB ram between 2x V100 or 3x K80)\n\nIs this code only used to reproduce the results for the prize winners? I presume so, since it is a huge amount of resources to train all medal winners models. \n\nFinally, if we do decide that we will want to retrain on Stage1 data - do we need to decide that before we submit our model? \n\n\nSorry for the wall of text..\n-C", "votes": 4 } ], "comments": [], "raw_markdown_by_id": { "655488": "Yes, another one of these posts. Sorry. I am not totally clear on the requirements even after reading the other 4 or so..\n\n**From reading the [two-stage FAQ](https://www.kaggle.com/two-stage-frequently-asked-questions), I interpret there as being two options for uploading our models:**\n\na) Upload the weights of the models that we have trained along with the code to perform image processing and use the models to perform inference and then generate a submission.\nb) Upload the code used to train models, along with code to use them to perform inference and generate submission.\n\n__________\n\nSince Stage1 ground truth data will be released at the beginning of [Stage2](https://www.kaggle.com/c/rsna-intracranial-hemorrhage-detection/discussion/112145#latest-652165), option b is optimal as it allows the opportunity to retain some models using Stage1 data. \n\nHowever, does the option b submission requires us to have beautifully structured, plug-and-play code like Appiass'? Or can it just have basic instructions for how to tie the different scripts and models together? \nWill it be expected that competition staff can just unzip the model, read a readme and reproduce the result, or will the winners have the opportunity to have a dialog and help to troubleshoot bugs? \nI ask this because having flawless code that can run and train multiple models and then ensemble them together turns this into more of a coding competition as opposed to a DataScience competition. I am a Mechatronics Engineer, not a Software Engineer - as hard as I try I don't think I can reliably produce code / documentation of this quality in the very short time period that I will have to do it.\n\n\n**I guess what I am asking is:** can I just zip my git repo, spend an hour or two writing a README in as much detail as I can and submit that? Then for some crazy reason I do happen to win, have the opportunity to troubleshoot.. Or do I need to spend a whole week and hundreds of GPU hours writing a consolidated workflow and then testing it to ensure that it produces the expected results?\n\nMy models is already an ensemble of 5, and probably another 5 before the competition ends. It is a non-trivial task to reproduce this. If I want to create a flawless workflow that can reproduce this then I basically need to cancel all my experiments, finish progressing in the competition now and get started on it.\n\n__________\n\nAlso, do I need to document the hardware and exact environment that was used to generate the results. I am not sure how much of an impact variations in python package versions will have on producing deterministic results, however certainly hardware specific things (like number of GPU, VRAM, etc) is going to have an impact (it split batches 32GB ram between 2x V100 or 3x K80)\n\nIs this code only used to reproduce the results for the prize winners? I presume so, since it is a huge amount of resources to train all medal winners models. \n\nFinally, if we do decide that we will want to retrain on Stage1 data - do we need to decide that before we submit our model? \n\n\nSorry for the wall of text..\n-C" } }