Let's say I have a model trained on stage 1 train set.
\n\nApparently, I am allowed to retrain it using stage 1 test set. As far as I understand, I should not change the number of epochs I trained for during the 1st stage. Am I correct ?
\n\nAlso, can I just use the saved models, load them, and fine-tune them with the test set. If I can do that, how many epochs am I allowed to add ?
\n\nMany thanks for answering those questions (I dare not say clarifications, because I am sure this is already clear to a lot of people, I am just confused by all the posts, threads and informations and don't know where to look for this particular point).
", "messages": [ { "id": 667666, "postDate": "2019-11-07T14:33:32.413Z", "content": "Let's say I have a model trained on stage 1 train set.
\n\nApparently, I am allowed to retrain it using stage 1 test set. As far as I understand, I should not change the number of epochs I trained for during the 1st stage. Am I correct ?
\n\nAlso, can I just use the saved models, load them, and fine-tune them with the test set. If I can do that, how many epochs am I allowed to add ?
\n\nMany thanks for answering those questions (I dare not say clarifications, because I am sure this is already clear to a lot of people, I am just confused by all the posts, threads and informations and don't know where to look for this particular point).
", "rawMarkdown": "Let's say I have a model trained on stage 1 train set.\n\nApparently, I am allowed to retrain it using stage 1 test set. As far as I understand, I should not change the number of epochs I trained for during the 1st stage. Am I correct ? \n\nAlso, can I just use the saved models, load them, and **fine-tune** them with the test set. If I can do that, how many epochs am I allowed to add ? \n\nMany thanks for answering those questions (I dare not say clarifications, because I am sure this is already clear to a lot of people, I am just confused by all the posts, threads and informations and don't know where to look for this particular point).", "votes": 3 }, { "id": 667715, "postDate": "2019-11-07T15:41:32.990Z", "content": "Now let's say I've produced three models in my uploaded zip file... can I do an ensemble of these three models even if I didn't put the code to do the ensembling (we did that in excel) in said uploaded zip file ? I mean, ensembling is just some multiplication/division, and stage2 philosophy is to avoid LB probing (which we can't do here to fine-tune the ensemble since we are with 1% of the dataset). Can I do that or not ? @juliaelliott sorry for the @mentionning but I'm fairly new to this and I want to make sure I follow the rules properly...
", "rawMarkdown": "Now let's say I've produced three models in my uploaded zip file... can I do an ensemble of these three models even if I didn't put the code to do the ensembling (we did that in excel) in said uploaded zip file ? I mean, ensembling is just some multiplication/division, and stage2 philosophy is to avoid LB probing (which we can't do here to fine-tune the ensemble since we are with 1% of the dataset). Can I do that or not ? @juliaelliott sorry for the @mentionning but I'm fairly new to this and I want to make sure I follow the rules properly...", "votes": 1 }, { "id": 667841, "postDate": "2019-11-07T18:25:44.810Z", "content": "Hi! Thanks for your question. You can't change the number of epochs unless your code is already doing that for you. Any alterations to your code apart from what's required to read in any new files / deal with new file structures, are potential violations of the rules.
", "rawMarkdown": "Hi! Thanks for your question. You can't change the number of epochs unless your code is _already_ doing that for you. Any alterations to your code apart from what's required to read in any new files / deal with new file structures, are potential violations of the rules.", "votes": 2, "replies": [ { "id": 668290, "postDate": "2019-11-08T08:16:50.123Z", "content": "Hi @philculliton ! Thank you for your answer. I'll do changes as minimal as possible then. Now let's say I don't change anything to my models (except the path to input files), but then produce an ensemble out of these three models for which there is no code (We do that on excel because each member of the team has his own preds). Is that considered a violation of the rules as well ? Thank you for your time !
\n\nedit: it would be nice for the person who downvoted this to comment if they understand how doing an average of three predictions would be a violation of the 2-stage spirit, which is just to avoid LB probing and get results as close to reality as possible. Since we can't do any probing with 1% of the data, I would not change the weights of the ensemble depending on how well it fares on the current LB. Our strategy was always to give weights to our models depending on how well they fared on stage1LB. Thank you for your time.
", "rawMarkdown": "Hi @philculliton ! Thank you for your answer. I'll do changes as minimal as possible then. Now let's say I don't change anything to my models (except the path to input files), but then produce an ensemble out of these three models for which there is no code (We do that on excel because each member of the team has his own preds). Is that considered a violation of the rules as well ? Thank you for your time !\n\nedit: it would be nice for the person who downvoted this to comment if they understand how doing an average of three predictions would be a violation of the 2-stage spirit, which is just to avoid LB probing and get results as close to reality as possible. Since we can't do any probing with 1% of the data, I would not change the weights of the ensemble depending on how well it fares on the current LB. Our strategy was always to give weights to our models depending on how well they fared on stage1LB. Thank you for your time.", "votes": 2 }, { "id": 668353, "postDate": "2019-11-08T10:06:29.743Z", "content": "If you provided the code for the 3 models AND a README file stating \"one of my 2 final submissions will be an average of the predictions of these 3\" then I assume it's fine.
\n\nIf not, no idea really, but my intuition is that it's a violation.
", "rawMarkdown": "If you provided the code for the 3 models **AND** a README file stating \"one of my 2 final submissions will be an average of the predictions of these 3\" then I assume it's fine.\n\nIf not, no idea really, but my intuition is that it's a violation.", "votes": 1 }, { "id": 668556, "postDate": "2019-11-08T14:55:03.900Z", "content": "Hi @cparrarojas , thank you for your answer. Yeah, my intuition is the same as yours, I am asking for confirmation because I would be really sad to not do it out of extra-consciousness that other participants may not have and then realize that would have won me a medal...
", "rawMarkdown": "Hi @cparrarojas , thank you for your answer. Yeah, my intuition is the same as yours, I am asking for confirmation because I would be really sad to not do it out of extra-consciousness that other participants may not have and then realize that would have won me a medal... ", "votes": 1 }, { "id": 668587, "postDate": "2019-11-08T15:37:43.490Z", "content": "@bdubreu, if you did not include your ensemble script in your zipped upload in stage-1 you may face removal from the LB. I uploaded all my models plus my ensemble script in stage-1 because of the same type of caution. So my ensemble can be based of stage-1 models or retrained versions of those depending my retraining CV scores.
", "rawMarkdown": "@bdubreu, if you did not include your ensemble script in your zipped upload in stage-1 you may face removal from the LB. I uploaded all my models plus my ensemble script in stage-1 because of the same type of caution. So my ensemble can be based of stage-1 models or retrained versions of those depending my retraining CV scores." } ] }, { "id": 667686, "postDate": "2019-11-07T14:55:27.440Z", "rawMarkdown": "", "votes": -1, "isDeleted": true, "replies": [ { "id": 667690, "postDate": "2019-11-07T15:01:12.900Z", "content": "thanks for your thorough answer !
", "rawMarkdown": "thanks for your thorough answer ! " } ] } ], "comments": [ { "id": 667715, "author_name": "Benjamin Dubreu", "author_url": "", "post_date": "2019-11-07T15:41:32.990000", "content": "Now let's say I've produced three models in my uploaded zip file... can I do an ensemble of these three models even if I didn't put the code to do the ensembling (we did that in excel) in said uploaded zip file ? I mean, ensembling is just some multiplication/division, and stage2 philosophy is to avoid LB probing (which we can't do here to fine-tune the ensemble since we are with 1% of the dataset). Can I do that or not ? @juliaelliott sorry for the @mentionning but I'm fairly new to this and I want to make sure I follow the rules properly...
", "votes": 1, "replies": [] }, { "id": 667841, "author_name": "Phil Culliton", "author_url": "", "post_date": "2019-11-07T18:25:44.810000", "content": "Hi! Thanks for your question. You can't change the number of epochs unless your code is already doing that for you. Any alterations to your code apart from what's required to read in any new files / deal with new file structures, are potential violations of the rules.
", "votes": 2, "replies": [ { "id": 668290, "author_name": "Benjamin Dubreu", "author_url": "", "post_date": "2019-11-08T08:16:50.123000", "content": "Hi @philculliton ! Thank you for your answer. I'll do changes as minimal as possible then. Now let's say I don't change anything to my models (except the path to input files), but then produce an ensemble out of these three models for which there is no code (We do that on excel because each member of the team has his own preds). Is that considered a violation of the rules as well ? Thank you for your time !
\n\nedit: it would be nice for the person who downvoted this to comment if they understand how doing an average of three predictions would be a violation of the 2-stage spirit, which is just to avoid LB probing and get results as close to reality as possible. Since we can't do any probing with 1% of the data, I would not change the weights of the ensemble depending on how well it fares on the current LB. Our strategy was always to give weights to our models depending on how well they fared on stage1LB. Thank you for your time.
", "votes": 2, "replies": [] }, { "id": 668353, "author_name": "César Parra Rojas", "author_url": "", "post_date": "2019-11-08T10:06:29.743000", "content": "If you provided the code for the 3 models AND a README file stating \"one of my 2 final submissions will be an average of the predictions of these 3\" then I assume it's fine.
\n\nIf not, no idea really, but my intuition is that it's a violation.
", "votes": 1, "replies": [] }, { "id": 668556, "author_name": "Benjamin Dubreu", "author_url": "", "post_date": "2019-11-08T14:55:03.900000", "content": "Hi @cparrarojas , thank you for your answer. Yeah, my intuition is the same as yours, I am asking for confirmation because I would be really sad to not do it out of extra-consciousness that other participants may not have and then realize that would have won me a medal...
", "votes": 1, "replies": [] }, { "id": 668587, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2019-11-08T15:37:43.490000", "content": "@bdubreu, if you did not include your ensemble script in your zipped upload in stage-1 you may face removal from the LB. I uploaded all my models plus my ensemble script in stage-1 because of the same type of caution. So my ensemble can be based of stage-1 models or retrained versions of those depending my retraining CV scores.
", "votes": 0, "replies": [] } ] }, { "id": 667686, "author_name": "", "author_url": "", "post_date": "2019-11-07T14:55:27.440000", "content": "", "votes": -1, "replies": [ { "id": 667690, "author_name": "Benjamin Dubreu", "author_url": "", "post_date": "2019-11-07T15:01:12.900000", "content": "thanks for your thorough answer !
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "667666": "Let's say I have a model trained on stage 1 train set.\n\nApparently, I am allowed to retrain it using stage 1 test set. As far as I understand, I should not change the number of epochs I trained for during the 1st stage. Am I correct ? \n\nAlso, can I just use the saved models, load them, and **fine-tune** them with the test set. If I can do that, how many epochs am I allowed to add ? \n\nMany thanks for answering those questions (I dare not say clarifications, because I am sure this is already clear to a lot of people, I am just confused by all the posts, threads and informations and don't know where to look for this particular point).", "667715": "Now let's say I've produced three models in my uploaded zip file... can I do an ensemble of these three models even if I didn't put the code to do the ensembling (we did that in excel) in said uploaded zip file ? I mean, ensembling is just some multiplication/division, and stage2 philosophy is to avoid LB probing (which we can't do here to fine-tune the ensemble since we are with 1% of the dataset). Can I do that or not ? @juliaelliott sorry for the @mentionning but I'm fairly new to this and I want to make sure I follow the rules properly...", "667841": "Hi! Thanks for your question. You can't change the number of epochs unless your code is _already_ doing that for you. Any alterations to your code apart from what's required to read in any new files / deal with new file structures, are potential violations of the rules.", "667686": "" } }