It is interesting how averaging model folds results leads to at most the same result as the best fold result on public leaderboard. \nFrom what I have seen in discussions it is not just my experience, but a common thing to most attempts. \nHappened to me on both regression and classification approaches.\nI am even wandering if it make sense to waste time for training 5 folds. The only thing that makes me do that is hoping that it will lead to a more robust model from the private leaderboard point of view.\nMy next thing will be to combine the results from classification with the regression ones. I am curious if this approach will be improved by multiple folds or again will not.\nWhat were your experiences with multiple folds on both classification and regression ?
", "messages": [ { "id": 878409, "postDate": "2020-06-08T13:59:25.493Z", "content": "It is interesting how averaging model folds results leads to at most the same result as the best fold result on public leaderboard. \nFrom what I have seen in discussions it is not just my experience, but a common thing to most attempts. \nHappened to me on both regression and classification approaches.\nI am even wandering if it make sense to waste time for training 5 folds. The only thing that makes me do that is hoping that it will lead to a more robust model from the private leaderboard point of view.\nMy next thing will be to combine the results from classification with the regression ones. I am curious if this approach will be improved by multiple folds or again will not.\nWhat were your experiences with multiple folds on both classification and regression ?
", "rawMarkdown": "It is interesting how averaging model folds results leads to at most the same result as the best fold result on public leaderboard. \nFrom what I have seen in discussions it is not just my experience, but a common thing to most attempts. \nHappened to me on both regression and classification approaches.\nI am even wandering if it make sense to waste time for training 5 folds. The only thing that makes me do that is hoping that it will lead to a more robust model from the private leaderboard point of view.\nMy next thing will be to combine the results from classification with the regression ones. I am curious if this approach will be improved by multiple folds or again will not.\nWhat were your experiences with multiple folds on both classification and regression ? ", "votes": 6 }, { "id": 881819, "postDate": "2020-06-11T11:43:41.033Z", "content": "My ensemble model get better cv than single model, around 0.005. But lb score didn't change though.
", "rawMarkdown": "My ensemble model get better cv than single model, around 0.005. But lb score didn't change though.", "votes": 1, "replies": [ { "id": 882043, "postDate": "2020-06-11T14:49:20.503Z", "content": "How many folds did you use ?
", "rawMarkdown": "How many folds did you use ?" }, { "id": 882538, "postDate": "2020-06-11T22:35:07.227Z", "content": "5 folds\nBut I trained them with slightly changes.
", "rawMarkdown": "5 folds\nBut I trained them with slightly changes." } ] }, { "id": 878502, "postDate": "2020-06-08T15:41:29.640Z", "content": "My 5-fold models are also more or less the same than single fold, a bit better (beyond 2 decimal places) based on sorting by LB score.
\n\nI do single fold experiments and will train 5-fold models at the end.
", "rawMarkdown": "My 5-fold models are also more or less the same than single fold, a bit better (beyond 2 decimal places) based on sorting by LB score. \n\nI do single fold experiments and will train 5-fold models at the end. ", "votes": 2, "replies": [ { "id": 878504, "postDate": "2020-06-08T15:43:18.947Z", "content": "Regression ?
", "rawMarkdown": "Regression ?" }, { "id": 878530, "postDate": "2020-06-08T16:08:41.107Z", "content": "This has been true for both regression and classification.
", "rawMarkdown": "This has been true for both regression and classification." }, { "id": 878536, "postDate": "2020-06-08T16:11:09.847Z", "content": "Good to know. Good luck !
", "rawMarkdown": "Good to know. Good luck !" } ] } ], "comments": [ { "id": 881819, "author_name": "Tsai29", "author_url": "", "post_date": "2020-06-11T11:43:41.033000", "content": "My ensemble model get better cv than single model, around 0.005. But lb score didn't change though.
", "votes": 1, "replies": [ { "id": 882043, "author_name": "Vlad Vaduva", "author_url": "", "post_date": "2020-06-11T14:49:20.503000", "content": "How many folds did you use ?
", "votes": 0, "replies": [] }, { "id": 882538, "author_name": "Tsai29", "author_url": "", "post_date": "2020-06-11T22:35:07.227000", "content": "5 folds\nBut I trained them with slightly changes.
", "votes": 0, "replies": [] } ] }, { "id": 878502, "author_name": "Ian Pan", "author_url": "", "post_date": "2020-06-08T15:41:29.640000", "content": "My 5-fold models are also more or less the same than single fold, a bit better (beyond 2 decimal places) based on sorting by LB score.
\n\nI do single fold experiments and will train 5-fold models at the end.
", "votes": 2, "replies": [ { "id": 878504, "author_name": "Vlad Vaduva", "author_url": "", "post_date": "2020-06-08T15:43:18.947000", "content": "Regression ?
", "votes": 0, "replies": [] }, { "id": 878530, "author_name": "Ian Pan", "author_url": "", "post_date": "2020-06-08T16:08:41.107000", "content": "This has been true for both regression and classification.
", "votes": 0, "replies": [] }, { "id": 878536, "author_name": "Vlad Vaduva", "author_url": "", "post_date": "2020-06-08T16:11:09.847000", "content": "Good to know. Good luck !
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "878409": "It is interesting how averaging model folds results leads to at most the same result as the best fold result on public leaderboard. \nFrom what I have seen in discussions it is not just my experience, but a common thing to most attempts. \nHappened to me on both regression and classification approaches.\nI am even wandering if it make sense to waste time for training 5 folds. The only thing that makes me do that is hoping that it will lead to a more robust model from the private leaderboard point of view.\nMy next thing will be to combine the results from classification with the regression ones. I am curious if this approach will be improved by multiple folds or again will not.\nWhat were your experiences with multiple folds on both classification and regression ? ", "881819": "My ensemble model get better cv than single model, around 0.005. But lb score didn't change though.", "878502": "My 5-fold models are also more or less the same than single fold, a bit better (beyond 2 decimal places) based on sorting by LB score. \n\nI do single fold experiments and will train 5-fold models at the end. " } }