I find this competition really disappointing since everyone who ended up with a private score of 0.915 used Qishen's public kernel. While people who actually worked hard and was honest have gone down below them.
\n\nI understand Qishen was only trying to give us new insight, but people copying them and going up in the private LB with a few submissions is demotivating.
\n\nI really hope the kaggle team adds some form of rule where people can't select someone else's public kernel for the final private LB submission and disable downloads for public kernels so that the trained weights cannot be downloaded.
", "messages": [ { "id": 940521, "postDate": "2020-07-23T02:10:43.617Z", "content": "I find this competition really disappointing since everyone who ended up with a private score of 0.915 used Qishen's public kernel. While people who actually worked hard and was honest have gone down below them.
\n\nI understand Qishen was only trying to give us new insight, but people copying them and going up in the private LB with a few submissions is demotivating.
\n\nI really hope the kaggle team adds some form of rule where people can't select someone else's public kernel for the final private LB submission and disable downloads for public kernels so that the trained weights cannot be downloaded.
", "rawMarkdown": "I find this competition really disappointing since everyone who ended up with a private score of 0.915 used Qishen's public kernel. While people who actually worked hard and was honest have gone down below them.\n\nI understand Qishen was only trying to give us new insight, but people copying them and going up in the private LB with a few submissions is demotivating.\n\nI really hope the kaggle team adds some form of rule where people can't select someone else's public kernel for the final private LB submission and disable downloads for public kernels so that the trained weights cannot be downloaded.", "votes": 2 }, { "id": 947405, "postDate": "2020-07-27T08:47:34.110Z", "content": "You are right, and unfortunately, it is not only in this competition.\nBut what matters is the capability of working on a whole project even if you might have worse results rather than stacking existing submissions without any model implemented (you don't learn anything this way)\nTo me: LB is for medals, but ranking is not a good proxy on one's maths comprehension or coding skills.
", "rawMarkdown": "You are right, and unfortunately, it is not only in this competition.\nBut what matters is the capability of working on a whole project even if you might have worse results rather than stacking existing submissions without any model implemented (you don't learn anything this way)\nTo me: LB is for medals, but ranking is not a good proxy on one's maths comprehension or coding skills.", "votes": 1 }, { "id": 951103, "postDate": "2020-07-29T22:02:02.170Z", "content": "Probably a good practice, which I try to follow, would be sharing only models trained at kaggle within the kernel time limit. It would be easier for people working independently to beat such a benchmark than a model trained for many hours externally.
", "rawMarkdown": "Probably a good practice, which I try to follow, would be sharing only models trained at kaggle within the kernel time limit. It would be easier for people working independently to beat such a benchmark than a model trained for many hours externally. ", "votes": 2 }, { "id": 942207, "postDate": "2020-07-23T16:31:24.663Z", "content": "That is mostly because Kaggle tries to be two things at the same time: A competition platform and a learning platform. Technically they could shut down kernels (during competition) and go back to a purely competitive experience but that is very anti-beginners and for their business not necessarily viable. On the other hand they still want a competition as it is what motivates innovation.
\n\nI think overall the mix is not too bad (could probably be better) since everybody is given the same information and if a public kernel is better than your solution then at least you hopefully learned from it and can now work on improving upon that. The worst used to be when people posted high scoring kernel the day before the end and kaggle now warns against such behavior (I personally think they should restrict it completely near the end).
", "rawMarkdown": "That is mostly because Kaggle tries to be two things at the same time: A competition platform and a learning platform. Technically they could shut down kernels (during competition) and go back to a purely competitive experience but that is very anti-beginners and for their business not necessarily viable. On the other hand they still want a competition as it is what motivates innovation.\n\nI think overall the mix is not too bad (could probably be better) since everybody is given the same information and if a public kernel is better than your solution then at least you hopefully learned from it and can now work on improving upon that. The worst used to be when people posted high scoring kernel the day before the end and kaggle now warns against such behavior (I personally think they should restrict it completely near the end).", "votes": 2 }, { "id": 941750, "postDate": "2020-07-23T11:47:44.870Z", "content": "I think the problem with such a rule would be that it would be very confusing for beginners. And easily overcomed anyways. Just copy/paste the code, change a line or two (or even the seed, for that matter) and then you have a brand new model that you can submit. Too much hassle for kaggle team, just to ensure \"fairness\" in a world where there is no such thing.
\n\nAlso, if you think of competition in terms of the rank you get, I understand why you'd be upset. If you think of them as a way of learning, then you learned much more than the people who just submitted a public kernel. Hence you won ;)
", "rawMarkdown": "I think the problem with such a rule would be that it would be very confusing for beginners. And easily overcomed anyways. Just copy/paste the code, change a line or two (or even the seed, for that matter) and then you have a brand new model that you can submit. Too much hassle for kaggle team, just to ensure \"fairness\" in a world where there is no such thing. \n\nAlso, if you think of competition in terms of the rank you get, I understand why you'd be upset. If you think of them as a way of learning, then you learned much more than the people who just submitted a public kernel. Hence you won ;)", "votes": 2, "replies": [ { "id": 941788, "postDate": "2020-07-23T12:15:24.750Z", "content": "@bdubreu I agree that these competitions are mostly a learning experience and to learn from what others share. However, it can be very demotivating when people just copy someone else's work and end up with a better result. Anyway what's done is done, hope the next competition is better.
", "rawMarkdown": "@bdubreu I agree that these competitions are mostly a learning experience and to learn from what others share. However, it can be very demotivating when people just copy someone else's work and end up with a better result. Anyway what's done is done, hope the next competition is better.", "votes": 2 } ] } ], "comments": [ { "id": 947405, "author_name": "BryanB", "author_url": "", "post_date": "2020-07-27T08:47:34.110000", "content": "You are right, and unfortunately, it is not only in this competition.\nBut what matters is the capability of working on a whole project even if you might have worse results rather than stacking existing submissions without any model implemented (you don't learn anything this way)\nTo me: LB is for medals, but ranking is not a good proxy on one's maths comprehension or coding skills.
", "votes": 1, "replies": [] }, { "id": 951103, "author_name": "Iafoss", "author_url": "", "post_date": "2020-07-29T22:02:02.170000", "content": "Probably a good practice, which I try to follow, would be sharing only models trained at kaggle within the kernel time limit. It would be easier for people working independently to beat such a benchmark than a model trained for many hours externally.
", "votes": 2, "replies": [] }, { "id": 942207, "author_name": "Arnaud Roussel", "author_url": "", "post_date": "2020-07-23T16:31:24.663000", "content": "That is mostly because Kaggle tries to be two things at the same time: A competition platform and a learning platform. Technically they could shut down kernels (during competition) and go back to a purely competitive experience but that is very anti-beginners and for their business not necessarily viable. On the other hand they still want a competition as it is what motivates innovation.
\n\nI think overall the mix is not too bad (could probably be better) since everybody is given the same information and if a public kernel is better than your solution then at least you hopefully learned from it and can now work on improving upon that. The worst used to be when people posted high scoring kernel the day before the end and kaggle now warns against such behavior (I personally think they should restrict it completely near the end).
", "votes": 2, "replies": [] }, { "id": 941750, "author_name": "Benjamin Dubreu", "author_url": "", "post_date": "2020-07-23T11:47:44.870000", "content": "I think the problem with such a rule would be that it would be very confusing for beginners. And easily overcomed anyways. Just copy/paste the code, change a line or two (or even the seed, for that matter) and then you have a brand new model that you can submit. Too much hassle for kaggle team, just to ensure \"fairness\" in a world where there is no such thing.
\n\nAlso, if you think of competition in terms of the rank you get, I understand why you'd be upset. If you think of them as a way of learning, then you learned much more than the people who just submitted a public kernel. Hence you won ;)
", "votes": 2, "replies": [ { "id": 941788, "author_name": "Yovin Yahathugoda", "author_url": "", "post_date": "2020-07-23T12:15:24.750000", "content": "@bdubreu I agree that these competitions are mostly a learning experience and to learn from what others share. However, it can be very demotivating when people just copy someone else's work and end up with a better result. Anyway what's done is done, hope the next competition is better.
", "votes": 2, "replies": [] } ] } ], "raw_markdown_by_id": { "940521": "I find this competition really disappointing since everyone who ended up with a private score of 0.915 used Qishen's public kernel. While people who actually worked hard and was honest have gone down below them.\n\nI understand Qishen was only trying to give us new insight, but people copying them and going up in the private LB with a few submissions is demotivating.\n\nI really hope the kaggle team adds some form of rule where people can't select someone else's public kernel for the final private LB submission and disable downloads for public kernels so that the trained weights cannot be downloaded.", "947405": "You are right, and unfortunately, it is not only in this competition.\nBut what matters is the capability of working on a whole project even if you might have worse results rather than stacking existing submissions without any model implemented (you don't learn anything this way)\nTo me: LB is for medals, but ranking is not a good proxy on one's maths comprehension or coding skills.", "951103": "Probably a good practice, which I try to follow, would be sharing only models trained at kaggle within the kernel time limit. It would be easier for people working independently to beat such a benchmark than a model trained for many hours externally. ", "942207": "That is mostly because Kaggle tries to be two things at the same time: A competition platform and a learning platform. Technically they could shut down kernels (during competition) and go back to a purely competitive experience but that is very anti-beginners and for their business not necessarily viable. On the other hand they still want a competition as it is what motivates innovation.\n\nI think overall the mix is not too bad (could probably be better) since everybody is given the same information and if a public kernel is better than your solution then at least you hopefully learned from it and can now work on improving upon that. The worst used to be when people posted high scoring kernel the day before the end and kaggle now warns against such behavior (I personally think they should restrict it completely near the end).", "941750": "I think the problem with such a rule would be that it would be very confusing for beginners. And easily overcomed anyways. Just copy/paste the code, change a line or two (or even the seed, for that matter) and then you have a brand new model that you can submit. Too much hassle for kaggle team, just to ensure \"fairness\" in a world where there is no such thing. \n\nAlso, if you think of competition in terms of the rank you get, I understand why you'd be upset. If you think of them as a way of learning, then you learned much more than the people who just submitted a public kernel. Hence you won ;)" } }