Imagine the following scenario. Team 'leaky leaky' has a leaky 0.8 solution and a non-leak 0.78 solution. The team ensembles them for 0.79 solution and submit. But he needs a 0.79 non-leak solution to show to the host. So, he waits a few days, reads the top solutions on the forum, and incorporates several new ideas into the non-leak 0.78 solution. Now, he has a 0.79 non-leak solution to submit to the host.
\nSo, please only publish after the host gets all the top solutions and gives us the green light for publishing.
Imagine the following scenario. Team 'leaky leaky' has a leaky 0.8 solution and a non-leak 0.78 solution. The team ensembles them for 0.79 solution and submit. But he needs a 0.79 non-leak solution to show to the host. So, he waits a few days, reads the top solutions on the forum, and incorporates several new ideas into the non-leak 0.78 solution. Now, he has a 0.79 non-leak solution to submit to the host.
\nSo, please only publish after the host gets all the top solutions and gives us the green light for publishing.
Thank you for suggestion!
\n@jerrylin96
\nIs this a same as official view? If yes, could you pin it so that it is visible at the top of discussion?
\nWe do not intend to choose a leaky solution for final submission and would like to follow community policy.
Pinned. Thank you for following guidelines for final submission.
", "rawMarkdown": "Pinned. Thank you for following guidelines for final submission.", "votes": 6, "replies": [ { "id": 2922786, "postDate": "2024-07-15T13:46:00.190Z", "content": "\nSorry for all the questions. I'm very confused.
\nWhat do you mean by \"solution\"?
\nHow can I send the \"solution\" only to the host?
\nIn other words, what should people in the gold zone do and what should they not do before the deadline?
@jerrylin96 This topic is currently no pinned. Could you pin again, or could you announce this officially? Newcomer will confuse if no one shared their solution.
", "rawMarkdown": "@jerrylin96 This topic is currently no pinned. Could you pin again, or could you announce this officially? Newcomer will confuse if no one shared their solution." }, { "id": 2926649, "postDate": "2024-07-17T21:42:45.853Z", "content": "It is written in the new pinned post 'We request teams to not publish solutions until we have finished verifying solutions.'
", "rawMarkdown": "It is written in the [new pinned post](https://www.kaggle.com/competitions/leap-atmospheric-physics-ai-climsim/discussion/520378) 'We request teams to not publish solutions until we have finished verifying solutions.'", "votes": 1 }, { "id": 2926656, "postDate": "2024-07-17T21:52:17.070Z", "content": "According to the title of new topic, I think most of people consider this topic is only applied to prize winners. And It doesn’t describe any reason why we shouldn’t share the solution. So I still think guidance to non-prize winner is neccessary.
", "rawMarkdown": "According to the title of new topic, I think most of people consider this topic is only applied to prize winners. And It doesn’t describe any reason why we shouldn’t share the solution. So I still think guidance to non-prize winner is neccessary.", "votes": 1 }, { "id": 2926669, "postDate": "2024-07-17T22:02:47.407Z", "content": "At this point, IMHO the safe thing to do is for any team, regardless of their ranking status, to refrain from revealing anything about their solution until further notice. It's just like the XOR problem: stuff that appears useless may in fact be completely useful in a different perspective. So, a low-ranking team's solution might help a malicious team's solution. Who knows?
\nI'd just pretend the word \"top\" in the title of this post doesn't exist.
", "rawMarkdown": "At this point, IMHO the safe thing to do is for any team, regardless of their ranking status, to refrain from revealing anything about their solution until further notice. It's just like the XOR problem: stuff that appears useless may in fact be completely useful in a different perspective. So, a low-ranking team's solution might help a malicious team's solution. Who knows?\n\nI'd just pretend the word \"top\" in the title of this post doesn't exist." } ] } ] }, { "id": 2921531, "postDate": "2024-07-14T13:06:28.087Z", "content": "Good suggestion!!
\nHope you can get top1 in the end! You deserve it.
", "rawMarkdown": "Good suggestion!! \n\nHope you can get top1 in the end! You deserve it. ", "votes": 4, "replies": [ { "id": 2921553, "postDate": "2024-07-14T13:26:11.927Z", "content": "Thank you. I did my best. Even if I don't get 1st, it's ok. What I learned in this competition no one can take, and it was a LOT. This is the most important thing.
", "rawMarkdown": "Thank you. I did my best. Even if I don't get 1st, it's ok. What I learned in this competition no one can take, and it was a LOT. This is the most important thing.", "votes": 3, "replies": [ { "id": 2921838, "postDate": "2024-07-14T17:21:13.173Z", "content": "\n\nWhat I learned in this competition no one can take
\n
But then when they acquire the source code for vetting, wouldn't that reveal a lot (even if not 100%) of the \"no one can take\" part?
", "rawMarkdown": ">What I learned in this competition no one can take\n\nBut then when they acquire the source code for vetting, wouldn't that reveal a lot (even if not 100%) of the \"no one can take\" part?" } ] } ] }, { "id": 2921540, "postDate": "2024-07-14T13:17:02.963Z", "content": "greySnow, I agree with you for the intention. But, does your comment imply that we're allowed to submit after the deadline (as far as competing for ranking/medal/money goes)?
", "rawMarkdown": "greySnow, I agree with you for the intention. But, does your comment imply that we're allowed to submit *after* the deadline (as far as competing for ranking/medal/money goes)?", "votes": 1, "replies": [ { "id": 2921551, "postDate": "2024-07-14T13:23:59.683Z", "content": "Submitting solutions for scoring can be done only before the deadline.
\nHowever, submitting solutions to the host for validation can take a few more days (up to a week or two by Kaggle rules, I don't remember exactly).
\nSo, you see, there is a discrepancy that needs to be taken care of.
\n\nsubmitting solutions to the host for validation can take a few more days (up to a week or two by Kaggle rules
\n
If a team truly doesn't use any leaks at all, why would they have the chance to \"submit to the host @jerrylin96 for validation\" after the deadline? Why can't their pre-deadline submission be used for validation for ranking/medal/prize?
", "rawMarkdown": ">submitting solutions to the host for validation can take a few more days (up to a week or two by Kaggle rules\n\nIf a team truly doesn't use any leaks at all, why would they have the chance to \"submit to the host @jerrylin96 for validation\" after the deadline? Why can't their pre-deadline submission be used for validation for ranking/medal/prize?" }, { "id": 2923628, "postDate": "2024-07-16T03:13:46.487Z", "content": "dude, they might train the model in other server then uploaded the model and only did the submission part in kaggle because kaggle only give us accelerator with limited time so that the host could not see the training code from them. They need to submit the training code if they got price XD
", "rawMarkdown": "dude, they might train the model in other server then uploaded the model and only did the submission part in kaggle because kaggle only give us accelerator with limited time so that the host could not see the training code from them. They need to submit the training code if they got price XD", "votes": 1 }, { "id": 2923657, "postDate": "2024-07-16T03:54:38.387Z", "content": "I'm saying, even the \"submitting the training code\" part should be done prior to the deadline. Or at the very least, not too long after the deadline.
\nOtherwise, it'll be pretty easy for a competitor to gain an unfair advantage. Leak is not the only concern here.
", "rawMarkdown": "I'm saying, even the \"submitting the training code\" part should be done prior to the deadline. Or at the very least, not too long after the deadline.\n\nOtherwise, it'll be pretty easy for a competitor to gain an unfair advantage. Leak is not the only concern here." }, { "id": 2923745, "postDate": "2024-07-16T06:04:16.290Z", "content": "actually, it is hard to sort out all codes for your final model. someone may have like hundreds of model structure and strategy. it will take hours and hours to make them in order then submit.
\nthis is a modelling competition and all things to do is to make the model better. they only want to check whether your code is similar to other players. the model score will not change after the deadline. the only way that you cheat to a top scores is copying from other players or ask someone to make a model for you. if you copied from other players, you cannot give a training code which is not similar to others. if you asked someone to make a model for you, why dont he join this competition and win the price? that is the point i think.
", "rawMarkdown": "actually, it is hard to sort out all codes for your final model. someone may have like hundreds of model structure and strategy. it will take hours and hours to make them in order then submit. \n\nthis is a modelling competition and all things to do is to make the model better. they only want to check whether your code is similar to other players. the model score will not change after the deadline. the only way that you cheat to a top scores is copying from other players or ask someone to make a model for you. if you copied from other players, you cannot give a training code which is not similar to others. if you asked someone to make a model for you, why dont he join this competition and win the price? that is the point i think.", "votes": 2 }, { "id": 2923810, "postDate": "2024-07-16T06:37:10.700Z", "content": "and in this competition, i think the host only want to check whether they use leak. :/
", "rawMarkdown": "and in this competition, i think the host only want to check whether they use leak. :/" }, { "id": 2924176, "postDate": "2024-07-16T11:33:39.763Z", "content": "\n\nthe only way that you cheat to a top scores is copying from other players or ask someone to make a model for you
\n
Another way to cheat for someone who simply needs more time to make their model to prize range is for them to use the time while @jerrylin96 / Kaggle staff is vetting solutions to do that.
\nThere is only one way to prevent this: Require every team to send in their source code prior to the deadline AND require that the predictions spit out by their source code is bit-to-bit identical (or at least very very very very similar) to their pre-deadline test set prediction submission.
\nOtherwise, the amount of unfair leverage a malicious competitor could have is, in principle, unbounded. This is worse than the leak issue because the advantage a leak can provide is bounded.
", "rawMarkdown": ">the only way that you cheat to a top scores is copying from other players or ask someone to make a model for you\n\nAnother way to cheat for someone who simply needs more time to make their model to prize range is for them to use the time while @jerrylin96 / Kaggle staff is vetting solutions to do that.\n\nThere is only one way to prevent this: Require every team to send in their source code prior to the deadline AND require that the predictions spit out by their source code is bit-to-bit identical (or at least very very very very similar) to their pre-deadline test set prediction submission.\n\nOtherwise, the amount of unfair leverage a malicious competitor could have is, in principle, unbounded. This is worse than the leak issue because the advantage a leak can provide is bounded." } ] } ] }, { "id": 2926643, "postDate": "2024-07-17T21:31:19.407Z", "content": "Absolutely agree. Great suggestion! @shlomoron
", "rawMarkdown": "Absolutely agree. Great suggestion! @shlomoron " }, { "id": 2925688, "postDate": "2024-07-17T10:44:01.460Z", "content": "Great suggestion!!
\nDo check out my work
good！very good！
", "rawMarkdown": "good！very good！" } ], "comments": [ { "id": 2922155, "author_name": "kurupical", "author_url": "", "post_date": "2024-07-15T00:05:20.940000", "content": "Thank you for suggestion!
\n@jerrylin96
\nIs this a same as official view? If yes, could you pin it so that it is visible at the top of discussion?
\nWe do not intend to choose a leaky solution for final submission and would like to follow community policy.
Pinned. Thank you for following guidelines for final submission.
", "votes": 6, "replies": [ { "id": 2922786, "author_name": "ohkawa3", "author_url": "", "post_date": "2024-07-15T13:46:00.190000", "content": "\nSorry for all the questions. I'm very confused.
\nWhat do you mean by \"solution\"?
\nHow can I send the \"solution\" only to the host?
\nIn other words, what should people in the gold zone do and what should they not do before the deadline?
@jerrylin96 This topic is currently no pinned. Could you pin again, or could you announce this officially? Newcomer will confuse if no one shared their solution.
", "votes": 0, "replies": [] }, { "id": 2926649, "author_name": "greySnow", "author_url": "", "post_date": "2024-07-17T21:42:45.853000", "content": "It is written in the new pinned post 'We request teams to not publish solutions until we have finished verifying solutions.'
", "votes": 1, "replies": [] }, { "id": 2926656, "author_name": "Bilzard", "author_url": "", "post_date": "2024-07-17T21:52:17.070000", "content": "According to the title of new topic, I think most of people consider this topic is only applied to prize winners. And It doesn’t describe any reason why we shouldn’t share the solution. So I still think guidance to non-prize winner is neccessary.
", "votes": 1, "replies": [] }, { "id": 2926669, "author_name": "Truth Seeker", "author_url": "", "post_date": "2024-07-17T22:02:47.407000", "content": "At this point, IMHO the safe thing to do is for any team, regardless of their ranking status, to refrain from revealing anything about their solution until further notice. It's just like the XOR problem: stuff that appears useless may in fact be completely useful in a different perspective. So, a low-ranking team's solution might help a malicious team's solution. Who knows?
\nI'd just pretend the word \"top\" in the title of this post doesn't exist.
", "votes": 0, "replies": [] } ] } ] }, { "id": 2921531, "author_name": "ADAM.", "author_url": "", "post_date": "2024-07-14T13:06:28.087000", "content": "Good suggestion!!
\nHope you can get top1 in the end! You deserve it.
", "votes": 4, "replies": [ { "id": 2921553, "author_name": "greySnow", "author_url": "", "post_date": "2024-07-14T13:26:11.927000", "content": "Thank you. I did my best. Even if I don't get 1st, it's ok. What I learned in this competition no one can take, and it was a LOT. This is the most important thing.
", "votes": 3, "replies": [ { "id": 2921838, "author_name": "Truth Seeker", "author_url": "", "post_date": "2024-07-14T17:21:13.173000", "content": "\n\nWhat I learned in this competition no one can take
\n
But then when they acquire the source code for vetting, wouldn't that reveal a lot (even if not 100%) of the \"no one can take\" part?
", "votes": 0, "replies": [] } ] } ] }, { "id": 2921540, "author_name": "Truth Seeker", "author_url": "", "post_date": "2024-07-14T13:17:02.963000", "content": "greySnow, I agree with you for the intention. But, does your comment imply that we're allowed to submit after the deadline (as far as competing for ranking/medal/money goes)?
", "votes": 1, "replies": [ { "id": 2921551, "author_name": "greySnow", "author_url": "", "post_date": "2024-07-14T13:23:59.683000", "content": "Submitting solutions for scoring can be done only before the deadline.
\nHowever, submitting solutions to the host for validation can take a few more days (up to a week or two by Kaggle rules, I don't remember exactly).
\nSo, you see, there is a discrepancy that needs to be taken care of.
\n\nsubmitting solutions to the host for validation can take a few more days (up to a week or two by Kaggle rules
\n
If a team truly doesn't use any leaks at all, why would they have the chance to \"submit to the host @jerrylin96 for validation\" after the deadline? Why can't their pre-deadline submission be used for validation for ranking/medal/prize?
", "votes": 0, "replies": [] }, { "id": 2923628, "author_name": "YueeeeeZ", "author_url": "", "post_date": "2024-07-16T03:13:46.487000", "content": "dude, they might train the model in other server then uploaded the model and only did the submission part in kaggle because kaggle only give us accelerator with limited time so that the host could not see the training code from them. They need to submit the training code if they got price XD
", "votes": 1, "replies": [] }, { "id": 2923657, "author_name": "Truth Seeker", "author_url": "", "post_date": "2024-07-16T03:54:38.387000", "content": "I'm saying, even the \"submitting the training code\" part should be done prior to the deadline. Or at the very least, not too long after the deadline.
\nOtherwise, it'll be pretty easy for a competitor to gain an unfair advantage. Leak is not the only concern here.
", "votes": 0, "replies": [] }, { "id": 2923745, "author_name": "YueeeeeZ", "author_url": "", "post_date": "2024-07-16T06:04:16.290000", "content": "actually, it is hard to sort out all codes for your final model. someone may have like hundreds of model structure and strategy. it will take hours and hours to make them in order then submit.
\nthis is a modelling competition and all things to do is to make the model better. they only want to check whether your code is similar to other players. the model score will not change after the deadline. the only way that you cheat to a top scores is copying from other players or ask someone to make a model for you. if you copied from other players, you cannot give a training code which is not similar to others. if you asked someone to make a model for you, why dont he join this competition and win the price? that is the point i think.
", "votes": 2, "replies": [] }, { "id": 2923810, "author_name": "YueeeeeZ", "author_url": "", "post_date": "2024-07-16T06:37:10.700000", "content": "and in this competition, i think the host only want to check whether they use leak. :/
", "votes": 0, "replies": [] }, { "id": 2924176, "author_name": "Truth Seeker", "author_url": "", "post_date": "2024-07-16T11:33:39.763000", "content": "\n\nthe only way that you cheat to a top scores is copying from other players or ask someone to make a model for you
\n
Another way to cheat for someone who simply needs more time to make their model to prize range is for them to use the time while @jerrylin96 / Kaggle staff is vetting solutions to do that.
\nThere is only one way to prevent this: Require every team to send in their source code prior to the deadline AND require that the predictions spit out by their source code is bit-to-bit identical (or at least very very very very similar) to their pre-deadline test set prediction submission.
\nOtherwise, the amount of unfair leverage a malicious competitor could have is, in principle, unbounded. This is worse than the leak issue because the advantage a leak can provide is bounded.
", "votes": 0, "replies": [] } ] } ] }, { "id": 2926643, "author_name": "Din Ezra", "author_url": "", "post_date": "2024-07-17T21:31:19.407000", "content": "Absolutely agree. Great suggestion! @shlomoron
", "votes": 0, "replies": [] }, { "id": 2925688, "author_name": "Ahmed Arham", "author_url": "", "post_date": "2024-07-17T10:44:01.460000", "content": "Great suggestion!!
\nDo check out my work
good！very good！
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2921482": "Imagine the following scenario. Team 'leaky leaky' has a leaky 0.8 solution and a non-leak 0.78 solution. The team ensembles them for 0.79 solution and submit. But he needs a 0.79 non-leak solution to show to the host. So, he waits a few days, reads the top solutions on the forum, and incorporates several new ideas into the non-leak 0.78 solution. Now, he has a 0.79 non-leak solution to submit to the host.\nSo, please only publish after the host gets all the top solutions and gives us the green light for publishing.", "2922155": "Thank you for suggestion! \n\n@jerrylin96 \nIs this a same as official view? If yes, could you pin it so that it is visible at the top of discussion?\nWe do not intend to choose a leaky solution for final submission and would like to follow community policy.", "2921531": "Good suggestion!! \n\nHope you can get top1 in the end! You deserve it. ", "2921540": "greySnow, I agree with you for the intention. But, does your comment imply that we're allowed to submit *after* the deadline (as far as competing for ranking/medal/money goes)?", "2926643": "Absolutely agree. Great suggestion! @shlomoron ", "2925688": "Great suggestion!! \nDo check out my work\n\n", "2923564": "good！very good！" } }