Why I always get the exact same score when I submit different submission.csv ?
", "messages": [ { "id": 1035082, "postDate": "2020-10-02T12:37:30.017Z", "content": "Why I always get the exact same score when I submit different submission.csv ?
", "rawMarkdown": "Why I always get the exact same score when I submit different submission.csv ?", "votes": 1 }, { "id": 1036617, "postDate": "2020-10-04T04:37:18.507Z", "content": "i am also facing the same issue. in my case, it has to do with my pipeline. mine is taking longer than 9hrs to run on private test data while submitting . so i did changes to run only on public test data and return default sample submission csv for private data. unfortunately, this did not work and i keep getting 0.693.
", "rawMarkdown": "i am also facing the same issue. in my case, it has to do with my pipeline. mine is taking longer than 9hrs to run on private test data while submitting . so i did changes to run only on public test data and return default sample submission csv for private data. unfortunately, this did not work and i keep getting 0.693. " }, { "id": 1035593, "postDate": "2020-10-02T20:50:42.263Z", "content": "I had this happen when I had a prebuilt public test submission and tried to combine it with inference on the private test set. Your public Leaderboard score is just on the public test set.
\nI don't know if that applies to your submission method.
\nAnother possibility is if you are updating the private sample_submission.csv file, is that you are not merging to the keys correctly, and a pd.merge command with \"how=left\" is not merging anything in.
\nAlso, with the size of the test dataset, it is always possible you are not changing enough entries to make a difference. I tried something that \"fixed\" about 500 image level entries. Did not make any difference. I suspect the change was just too small.
\nAlso, if I understand the metric correctly, some things like RV/LV ratio are only scored for positive studies. So changes to those entries for negative studies will not make any difference. (I admit to not totally understanding the metric and if this is exactly correct).
", "rawMarkdown": "I had this happen when I had a prebuilt public test submission and tried to combine it with inference on the private test set. Your public Leaderboard score is just on the public test set.\n\nI don't know if that applies to your submission method.\n\nAnother possibility is if you are updating the private sample_submission.csv file, is that you are not merging to the keys correctly, and a pd.merge command with \"how=left\" is not merging anything in.\n\nAlso, with the size of the test dataset, it is always possible you are not changing enough entries to make a difference. I tried something that \"fixed\" about 500 image level entries. Did not make any difference. I suspect the change was just too small.\n\nAlso, if I understand the metric correctly, some things like RV/LV ratio are only scored for positive studies. So changes to those entries for negative studies will not make any difference. (I admit to not totally understanding the metric and if this is exactly correct).", "replies": [ { "id": 1036658, "postDate": "2020-10-04T06:11:45.590Z", "content": "Thanks for your answer! But why pd.merge doesn't work?
", "rawMarkdown": "Thanks for your answer! But why pd.merge doesn't work?", "replies": [ { "id": 1037168, "postDate": "2020-10-04T16:47:29.840Z", "content": "Just a possibility. Make sure your merge keys are correct. Run your commands interactively and make sure you are getting the results you expect. Also, after a merge, duplicate column names become things like \"label_x\", \"label_y\". This can be confusing if you are looking for the original \"label\" column.
", "rawMarkdown": "Just a possibility. Make sure your merge keys are correct. Run your commands interactively and make sure you are getting the results you expect. Also, after a merge, duplicate column names become things like \"label_x\", \"label_y\". This can be confusing if you are looking for the original \"label\" column." } ] } ] } ], "comments": [ { "id": 1036617, "author_name": "yuvaramsingh", "author_url": "", "post_date": "2020-10-04T04:37:18.507000", "content": "i am also facing the same issue. in my case, it has to do with my pipeline. mine is taking longer than 9hrs to run on private test data while submitting . so i did changes to run only on public test data and return default sample submission csv for private data. unfortunately, this did not work and i keep getting 0.693.
", "votes": 0, "replies": [] }, { "id": 1035593, "author_name": "quadcore/Richard Epstein", "author_url": "", "post_date": "2020-10-02T20:50:42.263000", "content": "I had this happen when I had a prebuilt public test submission and tried to combine it with inference on the private test set. Your public Leaderboard score is just on the public test set.
\nI don't know if that applies to your submission method.
\nAnother possibility is if you are updating the private sample_submission.csv file, is that you are not merging to the keys correctly, and a pd.merge command with \"how=left\" is not merging anything in.
\nAlso, with the size of the test dataset, it is always possible you are not changing enough entries to make a difference. I tried something that \"fixed\" about 500 image level entries. Did not make any difference. I suspect the change was just too small.
\nAlso, if I understand the metric correctly, some things like RV/LV ratio are only scored for positive studies. So changes to those entries for negative studies will not make any difference. (I admit to not totally understanding the metric and if this is exactly correct).
", "votes": 0, "replies": [ { "id": 1036658, "author_name": "MaXiuyu", "author_url": "", "post_date": "2020-10-04T06:11:45.590000", "content": "Thanks for your answer! But why pd.merge doesn't work?
", "votes": 0, "replies": [ { "id": 1037168, "author_name": "quadcore/Richard Epstein", "author_url": "", "post_date": "2020-10-04T16:47:29.840000", "content": "Just a possibility. Make sure your merge keys are correct. Run your commands interactively and make sure you are getting the results you expect. Also, after a merge, duplicate column names become things like \"label_x\", \"label_y\". This can be confusing if you are looking for the original \"label\" column.
", "votes": 0, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "1035082": "Why I always get the exact same score when I submit different submission.csv ?", "1036617": "i am also facing the same issue. in my case, it has to do with my pipeline. mine is taking longer than 9hrs to run on private test data while submitting . so i did changes to run only on public test data and return default sample submission csv for private data. unfortunately, this did not work and i keep getting 0.693. ", "1035593": "I had this happen when I had a prebuilt public test submission and tried to combine it with inference on the private test set. Your public Leaderboard score is just on the public test set.\n\nI don't know if that applies to your submission method.\n\nAnother possibility is if you are updating the private sample_submission.csv file, is that you are not merging to the keys correctly, and a pd.merge command with \"how=left\" is not merging anything in.\n\nAlso, with the size of the test dataset, it is always possible you are not changing enough entries to make a difference. I tried something that \"fixed\" about 500 image level entries. Did not make any difference. I suspect the change was just too small.\n\nAlso, if I understand the metric correctly, some things like RV/LV ratio are only scored for positive studies. So changes to those entries for negative studies will not make any difference. (I admit to not totally understanding the metric and if this is exactly correct)." } }