{ "id": 465363, "title": "Why only private data?", "url": "/competitions/UBC-OCEAN/discussion/465363", "author_name": "yang_zhou", "post_date": "2024-01-04T01:02:34.536000", "votes": -7, "comment_count": 10, "views": 0, "content": "

I think both public and private data are important, Is the final score based on the average score of public and private data?

", "messages": [ { "id": 2586100, "postDate": "2024-01-04T01:10:52.980Z", "content": "

The public test data is equivalent to the verification set, and the private test data is the real test set, because the test set strictly only allows one test.

", "rawMarkdown": "The public test data is equivalent to the verification set, and the private test data is the real test set, because the test set strictly only allows one test.", "votes": 3 }, { "id": 2586097, "postDate": "2024-01-04T01:07:59.207Z", "content": "

To prevent meaningless overfitting.

", "rawMarkdown": "To prevent meaningless overfitting.", "votes": 4, "replies": [ { "id": 2586103, "postDate": "2024-01-04T01:15:36.653Z", "content": "

I got 0.65 on public and 0.61 on private, the 4th team got 0.58 and 0.61, In comparison, I feel that my model is more robust.

", "rawMarkdown": "I got 0.65 on public and 0.61 on private, the 4th team got 0.58 and 0.61, In comparison, I feel that my model is more robust.", "votes": -7, "replies": [ { "id": 2586115, "postDate": "2024-01-04T01:28:09.523Z", "content": "
    \n
  1. 生产环境里，在一个看得到的线下数据集里分数刷得再高没有任何意义。只有在部署上线后，在真实世界的分数上测过才知道模型的真实效果。
    2. \n
  2. 在很多比赛里，可能只是简单地调模型融合的权重或者某些阈值，就可以带来大幅的提升。如果Public也算分，那毫无疑问会有非常多人去过拟合public，这样会造成很多比赛的混乱并且失去原本的意义。
    3. \n
  3. 这个比赛Public只占22%，Public的分数本来就不具备很强的说服力。
    4. \n
", "rawMarkdown": "1. 生产环境里，在一个看得到的线下数据集里分数刷得再高没有任何意义。只有在部署上线后，在真实世界的分数上测过才知道模型的真实效果。\n2. 在很多比赛里，可能只是简单地调模型融合的权重或者某些阈值，就可以带来大幅的提升。如果Public也算分，那毫无疑问会有非常多人去过拟合public，这样会造成很多比赛的混乱并且失去原本的意义。\n3. 这个比赛Public只占22%，Public的分数本来就不具备很强的说服力。", "votes": 1 }, { "id": 2586116, "postDate": "2024-01-04T01:30:56.833Z", "content": "

没有绝对公平的规则，kaggle是个平台，需要承办无数的比赛，它只能选择大多数时候公平，并且更符合真实世界的规则

", "rawMarkdown": "没有绝对公平的规则，kaggle是个平台，需要承办无数的比赛，它只能选择大多数时候公平，并且更符合真实世界的规则", "votes": -1 }, { "id": 2586138, "postDate": "2024-01-04T02:22:02.253Z", "content": "

Totally agree

", "rawMarkdown": "Totally agree", "votes": 1 }, { "id": 2586194, "postDate": "2024-01-04T03:38:44.593Z", "content": "

但是，我们不可能说在公开集上调到一个值后就再也不调了，而期待着在私有集上结果反超。任何努力都应该有意义。

", "rawMarkdown": "但是，我们不可能说在公开集上调到一个值后就再也不调了，而期待着在私有集上结果反超。任何努力都应该有意义。", "votes": -3 }, { "id": 2586204, "postDate": "2024-01-04T03:46:34.520Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2586224, "postDate": "2024-01-04T04:02:28.760Z", "content": "

不是说不调，而是在设计模型/方案的过程中不能盯着一个分数在那里调。如果public算分，那在很多比赛里可能很多人在Public里可以刷到很高，但是private可能非常差。

", "rawMarkdown": "不是说不调，而是在设计模型/方案的过程中不能盯着一个分数在那里调。如果public算分，那在很多比赛里可能很多人在Public里可以刷到很高，但是private可能非常差。", "votes": -1 }, { "id": 2586225, "postDate": "2024-01-04T04:03:50.720Z", "content": "

你可以去翻翻之前Kaggle比赛的记录，很多比赛里Public 0.9, private random的都有。这种public 再高有什么意义。kaggle要办所有领域的比赛，肯定需要设计一个在大多数情况下公平的规则。

", "rawMarkdown": "你可以去翻翻之前Kaggle比赛的记录，很多比赛里Public 0.9, private random的都有。这种public 再高有什么意义。kaggle要办所有领域的比赛，肯定需要设计一个在大多数情况下公平的规则。" } ] } ] }, { "id": 2586089, "postDate": "2024-01-04T01:02:34.537Z", "content": "

I think both public and private data are important, Is the final score based on the average score of public and private data?

", "rawMarkdown": "I think both public and private data are important, Is the final score based on the average score of public and private data?", "votes": -7 } ], "comments": [ { "id": 2586100, "author_name": "m1dsolo", "author_url": "", "post_date": "2024-01-04T01:10:52.980000", "content": "

The public test data is equivalent to the verification set, and the private test data is the real test set, because the test set strictly only allows one test.

", "votes": 3, "replies": [] }, { "id": 2586097, "author_name": "ForcewithMe", "author_url": "", "post_date": "2024-01-04T01:07:59.207000", "content": "

To prevent meaningless overfitting.

", "votes": 4, "replies": [ { "id": 2586103, "author_name": "yang_zhou", "author_url": "", "post_date": "2024-01-04T01:15:36.653000", "content": "

I got 0.65 on public and 0.61 on private, the 4th team got 0.58 and 0.61, In comparison, I feel that my model is more robust.

", "votes": -7, "replies": [ { "id": 2586115, "author_name": "ForcewithMe", "author_url": "", "post_date": "2024-01-04T01:28:09.523000", "content": "
    \n
  1. 生产环境里，在一个看得到的线下数据集里分数刷得再高没有任何意义。只有在部署上线后，在真实世界的分数上测过才知道模型的真实效果。
    2. \n
  2. 在很多比赛里，可能只是简单地调模型融合的权重或者某些阈值，就可以带来大幅的提升。如果Public也算分，那毫无疑问会有非常多人去过拟合public，这样会造成很多比赛的混乱并且失去原本的意义。
    3. \n
  3. 这个比赛Public只占22%，Public的分数本来就不具备很强的说服力。
    4. \n
", "votes": 1, "replies": [] }, { "id": 2586116, "author_name": "ForcewithMe", "author_url": "", "post_date": "2024-01-04T01:30:56.833000", "content": "

没有绝对公平的规则，kaggle是个平台，需要承办无数的比赛，它只能选择大多数时候公平，并且更符合真实世界的规则

", "votes": -1, "replies": [] }, { "id": 2586138, "author_name": "Seeing Times", "author_url": "", "post_date": "2024-01-04T02:22:02.253000", "content": "

Totally agree

", "votes": 1, "replies": [] }, { "id": 2586194, "author_name": "yang_zhou", "author_url": "", "post_date": "2024-01-04T03:38:44.593000", "content": "

但是，我们不可能说在公开集上调到一个值后就再也不调了，而期待着在私有集上结果反超。任何努力都应该有意义。

", "votes": -3, "replies": [] }, { "id": 2586204, "author_name": "", "author_url": "", "post_date": "2024-01-04T03:46:34.520000", "content": "", "votes": 0, "replies": [] }, { "id": 2586224, "author_name": "ForcewithMe", "author_url": "", "post_date": "2024-01-04T04:02:28.760000", "content": "

不是说不调，而是在设计模型/方案的过程中不能盯着一个分数在那里调。如果public算分，那在很多比赛里可能很多人在Public里可以刷到很高，但是private可能非常差。

", "votes": -1, "replies": [] }, { "id": 2586225, "author_name": "ForcewithMe", "author_url": "", "post_date": "2024-01-04T04:03:50.720000", "content": "

你可以去翻翻之前Kaggle比赛的记录，很多比赛里Public 0.9, private random的都有。这种public 再高有什么意义。kaggle要办所有领域的比赛，肯定需要设计一个在大多数情况下公平的规则。

", "votes": 0, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2586100": "The public test data is equivalent to the verification set, and the private test data is the real test set, because the test set strictly only allows one test.", "2586097": "To prevent meaningless overfitting.", "2586089": "I think both public and private data are important, Is the final score based on the average score of public and private data?" } }