sub.Label = (sub.Label > 0.5).astype(float)credits to @anginscribe for showing that it is possible.
", "messages": [ { "id": 666938, "postDate": "2019-11-06T16:31:28.480Z", "content": "sub.Label = (sub.Label > 0.5).astype(float)credits to @anginscribe for showing that it is possible.
", "rawMarkdown": "1. Grab your best submission and run `sub.Label = (sub.Label > 0.5).astype(float)`\n2. ??\n3. Profit!\n\ncredits to @anginscribe for showing that it is possible.", "votes": 21 }, { "id": 666978, "postDate": "2019-11-06T17:23:23.533Z", "content": "You found the cure for cranial hemorrhagy, congratz !
", "rawMarkdown": "You found the cure for cranial hemorrhagy, congratz !", "votes": 9, "replies": [ { "id": 667125, "postDate": "2019-11-06T21:08:34.707Z", "content": "I don't want my doctor using that code :)
", "rawMarkdown": "I don't want my doctor using that code :)", "votes": 14 } ] }, { "id": 925491, "postDate": "2020-07-12T05:59:43.213Z", "content": "When we will use 100% of our brain!!!
", "rawMarkdown": "When we will use 100% of our brain!!!", "votes": 6 }, { "id": 667560, "postDate": "2019-11-07T11:29:46.603Z", "content": "For those who got 13.837 after these tricks, try lowering your thresholds. e.g. sub.Label = (sub.Label > 0.3).astype(float)
\n\nNow, please share prize if you win :D
", "rawMarkdown": "For those who got 13.837 after these tricks, try lowering your thresholds. e.g. sub.Label = (sub.Label > 0.3).astype(float)\n\nNow, please share prize if you win :D", "votes": 5, "replies": [ { "id": 667567, "postDate": "2019-11-07T11:40:10.077Z", "content": "I noticed the same.
\n\nDo you have an explanation for such bizarre behavior of the LB?
", "rawMarkdown": "I noticed the same.\n\nDo you have an explanation for such bizarre behavior of the LB?" }, { "id": 667572, "postDate": "2019-11-07T11:57:58.847Z", "content": "My theory is... as someone else pointed out, there's over representation of positive cases in the public LB (~40%), that's much higher than what our data is trained on ~16%, and higher than the private LB data. I think most models trained on the training set have learned to be \"cautious\" when giving a probability for positive cases - it'll rather win small than risk losing big. Hence, the thresholds to get the correct answers may be smaller than plain 0.5.
\n\nNot sure what this \"cautious behavior\" over actual accuracy means in terms of doing well in actual LB. I guess we'll see in a few days.
", "rawMarkdown": "My theory is... as someone else pointed out, there's over representation of positive cases in the public LB (~40%), that's much higher than what our data is trained on ~16%, and higher than the private LB data. I think most models trained on the training set have learned to be \"cautious\" when giving a probability for positive cases - it'll rather win small than risk losing big. Hence, the thresholds to get the correct answers may be smaller than plain 0.5.\n\nNot sure what this \"cautious behavior\" over actual accuracy means in terms of doing well in actual LB. I guess we'll see in a few days." }, { "id": 667609, "postDate": "2019-11-07T13:00:46.273Z", "content": "Humm, interesting. So you got 0.000 score after setting the threshold to 0.3?\nI just tested with threshold 0.4, 0.3. Same score: 13.837?\nMaybe there are something wrong with my model and predictions?
", "rawMarkdown": "Humm, interesting. So you got 0.000 score after setting the threshold to 0.3?\nI just tested with threshold 0.4, 0.3. Same score: 13.837?\nMaybe there are something wrong with my model and predictions?" }, { "id": 667650, "postDate": "2019-11-07T14:09:27.880Z", "content": "Try 0.1 or 0.2?
", "rawMarkdown": "Try 0.1 or 0.2?" }, { "id": 667704, "postDate": "2019-11-07T15:26:18.747Z", "content": "Had to use threshold of 0.1 to get 0.000 score. Used up all submissions for today. \nAny indication of the prediction performance from the threshold value?
", "rawMarkdown": "Had to use threshold of 0.1 to get 0.000 score. Used up all submissions for today. \nAny indication of the prediction performance from the threshold value?" }, { "id": 668667, "postDate": "2019-11-08T17:30:21.340Z", "content": "@yeeseng is private set is hidden in stage-2-test ,just that scoring is done on very fraction ?
", "rawMarkdown": "@yeeseng is private set is hidden in stage-2-test ,just that scoring is done on very fraction ?" }, { "id": 668703, "postDate": "2019-11-08T18:46:13.310Z", "content": "That is correct. The current public LB is only on <1% of the cases. It is completely useless and not at all predictive of the private LB which is scored on 100% of the cases. The stage 2 private LB is the one used to determine the final LB rankings.
\n\nIt has been fun and amusing to manipulate thresholds for the 0.000 public LB score but in all seriousness, those submissions are guaranteed to get horrible scores on the real thing. I am not selecting it for final scoring.
", "rawMarkdown": "That is correct. The current public LB is only on <1% of the cases. It is completely useless and not at all predictive of the private LB which is scored on 100% of the cases. The stage 2 private LB is the one used to determine the final LB rankings.\n\nIt has been fun and amusing to manipulate thresholds for the 0.000 public LB score but in all seriousness, those submissions are guaranteed to get horrible scores on the real thing. I am not selecting it for final scoring. " } ] }, { "id": 667073, "postDate": "2019-11-06T19:33:54.147Z", "content": "This like betting your life savings on a game of black jack, win once, and then walk. Brilliant!
", "rawMarkdown": "This like betting your life savings on a game of black jack, win once, and then walk. Brilliant!", "votes": 3 }, { "id": 680832, "postDate": "2019-11-25T10:05:55.490Z", "content": "nice one
", "rawMarkdown": "nice one", "votes": 4 }, { "id": 667451, "postDate": "2019-11-07T08:41:54.613Z", "content": "Zero on LB not total zero. I submit ur code and jump on 5 position, ignore 2 positions with zero.
", "rawMarkdown": "Zero on LB not total zero. I submit ur code and jump on 5 position, ignore 2 positions with zero.", "votes": 1 }, { "id": 667069, "postDate": "2019-11-06T19:26:48.147Z", "content": "funny, it didn't work for me - got 13.837.\nmaybe I did something wrong...\n:)\njust tried to see if it's really that crazy π
", "rawMarkdown": "funny, it didn't work for me - got 13.837.\nmaybe I did something wrong...\n:)\njust tried to see if it's really that crazy π ", "votes": 1, "replies": [ { "id": 667123, "postDate": "2019-11-06T21:04:58.743Z", "content": "the score of miss-classification is 15*log(10) = 34.538..., which means that you have about 40% of wrong classifications. Your initial predictions are very close to random, you need a decent model for the trick to work.
Don't know why I got exactly the same score: 13.837.
", "rawMarkdown": "Don't know why I got exactly the same score: 13.837. ", "votes": 1 }, { "id": 667276, "postDate": "2019-11-07T03:53:54.740Z", "content": "> you have about 40% of wrong classifications
\n\nHere 40% is just 40% of the public testset, which is <1% of the whole testset, thus don't worry. I bet it only contains n images where n < 10
\n\n--update--\niThe number should be like 35, see my post: https://www.kaggle.com/c/rsna-intracranial-hemorrhage-detection/discussion/116100\n--update 2nd times--\nI forgot to include 13.837, the numbr should be like: 455, 665, 910 ...
", "rawMarkdown": "> you have about 40% of wrong classifications\n\nHere 40% is just 40% of the public testset, which is <1% of the whole testset, thus don't worry. I bet it only contains n images where n < 10\n\n--update--\niThe number should be like 35, see my post: https://www.kaggle.com/c/rsna-intracranial-hemorrhage-detection/discussion/116100\n--update 2nd times--\nI forgot to include 13.837, the numbr should be like: 455, 665, 910 ...", "votes": 2 } ] }, { "id": 685586, "postDate": "2019-12-02T00:29:42.703Z", "content": "AI Doctors
", "rawMarkdown": "AI Doctors" }, { "id": 685359, "postDate": "2019-12-01T13:53:26.923Z", "content": "This is niceππ
", "rawMarkdown": "This is niceππ " }, { "id": 684565, "postDate": "2019-11-30T02:44:54.877Z", "content": "congratz~!
", "rawMarkdown": "congratz~!" }, { "id": 678706, "postDate": "2019-11-21T18:33:40.277Z", "content": "nice
", "rawMarkdown": "nice" }, { "id": 670597, "postDate": "2019-11-11T16:26:50.340Z", "content": "sub.Label = (sub.Label > 0.1).astype(float)
", "rawMarkdown": "sub.Label = (sub.Label > 0.1).astype(float)" }, { "id": 670015, "postDate": "2019-11-10T22:24:49.340Z", "content": "This is nuts π
", "rawMarkdown": "This is nuts π " }, { "id": 668044, "postDate": "2019-11-07T22:50:12.920Z", "content": "Haha, thanks. LB probing at its best here.
", "rawMarkdown": "Haha, thanks. LB probing at its best here. " }, { "id": 667277, "postDate": "2019-11-07T03:56:14.557Z", "content": "Where should I add these code ? Please introduce this method more detailed , thxπ
", "rawMarkdown": "Where should I add these code ? Please introduce this method more detailed , thxπ " }, { "id": 666975, "postDate": "2019-11-06T17:22:30.593Z", "content": "And then what? This is crazy:-)
", "rawMarkdown": "And then what? This is crazy:-)" }, { "id": 669383, "postDate": "2019-11-10T00:56:40.907Z", "rawMarkdown": "", "isDeleted": true }, { "id": 667636, "postDate": "2019-11-07T13:34:38.523Z", "rawMarkdown": "", "isDeleted": true }, { "id": 666971, "postDate": "2019-11-06T17:17:10.457Z", "rawMarkdown": "", "votes": 1, "isDeleted": true, "replies": [ { "id": 666981, "postDate": "2019-11-06T17:24:53.980Z", "content": "If someone decides to select his final submissions with this post-processing nobody will ask this question, because models outside of medals range are not checked.
", "rawMarkdown": "If someone decides to select his final submissions with this post-processing nobody will ask this question, because models outside of medals range are not checked. ", "votes": 6 }, { "id": 667003, "postDate": "2019-11-06T17:51:12.433Z", "content": "@portelaraian This technique will score terribly on the full private LB
", "rawMarkdown": "@portelaraian This technique will score terribly on the full private LB", "votes": 3 }, { "id": 667009, "postDate": "2019-11-06T18:01:09.417Z", "content": "But it's fun xd. I saw the .000 before finding out it's on <1% subset. Literally mindblown.
", "rawMarkdown": "But it's fun xd. I saw the .000 before finding out it's on <1% subset. Literally mindblown." }, { "id": 667284, "postDate": "2019-11-07T04:20:25.597Z", "rawMarkdown": "" } ] } ], "comments": [ { "id": 666978, "author_name": "Etienne R", "author_url": "", "post_date": "2019-11-06T17:23:23.533000", "content": "You found the cure for cranial hemorrhagy, congratz !
", "votes": 9, "replies": [ { "id": 667125, "author_name": "nosound", "author_url": "", "post_date": "2019-11-06T21:08:34.707000", "content": "I don't want my doctor using that code :)
", "votes": 14, "replies": [] } ] }, { "id": 925491, "author_name": "Abhishek", "author_url": "", "post_date": "2020-07-12T05:59:43.213000", "content": "When we will use 100% of our brain!!!
", "votes": 6, "replies": [] }, { "id": 667560, "author_name": "Yee Ng", "author_url": "", "post_date": "2019-11-07T11:29:46.603000", "content": "For those who got 13.837 after these tricks, try lowering your thresholds. e.g. sub.Label = (sub.Label > 0.3).astype(float)
\n\nNow, please share prize if you win :D
", "votes": 5, "replies": [ { "id": 667567, "author_name": "nosound", "author_url": "", "post_date": "2019-11-07T11:40:10.077000", "content": "I noticed the same.
\n\nDo you have an explanation for such bizarre behavior of the LB?
", "votes": 0, "replies": [] }, { "id": 667572, "author_name": "Yee Ng", "author_url": "", "post_date": "2019-11-07T11:57:58.847000", "content": "My theory is... as someone else pointed out, there's over representation of positive cases in the public LB (~40%), that's much higher than what our data is trained on ~16%, and higher than the private LB data. I think most models trained on the training set have learned to be \"cautious\" when giving a probability for positive cases - it'll rather win small than risk losing big. Hence, the thresholds to get the correct answers may be smaller than plain 0.5.
\n\nNot sure what this \"cautious behavior\" over actual accuracy means in terms of doing well in actual LB. I guess we'll see in a few days.
", "votes": 0, "replies": [] }, { "id": 667609, "author_name": "catlet", "author_url": "", "post_date": "2019-11-07T13:00:46.273000", "content": "Humm, interesting. So you got 0.000 score after setting the threshold to 0.3?\nI just tested with threshold 0.4, 0.3. Same score: 13.837?\nMaybe there are something wrong with my model and predictions?
", "votes": 0, "replies": [] }, { "id": 667650, "author_name": "Yee Ng", "author_url": "", "post_date": "2019-11-07T14:09:27.880000", "content": "Try 0.1 or 0.2?
", "votes": 0, "replies": [] }, { "id": 667704, "author_name": "catlet", "author_url": "", "post_date": "2019-11-07T15:26:18.747000", "content": "Had to use threshold of 0.1 to get 0.000 score. Used up all submissions for today. \nAny indication of the prediction performance from the threshold value?
", "votes": 0, "replies": [] }, { "id": 668667, "author_name": "Jaideep", "author_url": "", "post_date": "2019-11-08T17:30:21.340000", "content": "@yeeseng is private set is hidden in stage-2-test ,just that scoring is done on very fraction ?
", "votes": 0, "replies": [] }, { "id": 668703, "author_name": "Yee Ng", "author_url": "", "post_date": "2019-11-08T18:46:13.310000", "content": "That is correct. The current public LB is only on <1% of the cases. It is completely useless and not at all predictive of the private LB which is scored on 100% of the cases. The stage 2 private LB is the one used to determine the final LB rankings.
\n\nIt has been fun and amusing to manipulate thresholds for the 0.000 public LB score but in all seriousness, those submissions are guaranteed to get horrible scores on the real thing. I am not selecting it for final scoring.
", "votes": 0, "replies": [] } ] }, { "id": 667073, "author_name": "Yee Ng", "author_url": "", "post_date": "2019-11-06T19:33:54.147000", "content": "This like betting your life savings on a game of black jack, win once, and then walk. Brilliant!
", "votes": 3, "replies": [] }, { "id": 680832, "author_name": "Abhishek", "author_url": "", "post_date": "2019-11-25T10:05:55.490000", "content": "nice one
", "votes": 4, "replies": [] }, { "id": 667451, "author_name": "Leigh", "author_url": "", "post_date": "2019-11-07T08:41:54.613000", "content": "Zero on LB not total zero. I submit ur code and jump on 5 position, ignore 2 positions with zero.
", "votes": 1, "replies": [] }, { "id": 667069, "author_name": "Alexey Kotlik", "author_url": "", "post_date": "2019-11-06T19:26:48.147000", "content": "funny, it didn't work for me - got 13.837.\nmaybe I did something wrong...\n:)\njust tried to see if it's really that crazy π
", "votes": 1, "replies": [ { "id": 667123, "author_name": "nosound", "author_url": "", "post_date": "2019-11-06T21:04:58.743000", "content": "the score of miss-classification is 15*log(10) = 34.538..., which means that you have about 40% of wrong classifications. Your initial predictions are very close to random, you need a decent model for the trick to work.
Don't know why I got exactly the same score: 13.837.
", "votes": 1, "replies": [] }, { "id": 667276, "author_name": "Yiheng Wang", "author_url": "", "post_date": "2019-11-07T03:53:54.740000", "content": "> you have about 40% of wrong classifications
\n\nHere 40% is just 40% of the public testset, which is <1% of the whole testset, thus don't worry. I bet it only contains n images where n < 10
\n\n--update--\niThe number should be like 35, see my post: https://www.kaggle.com/c/rsna-intracranial-hemorrhage-detection/discussion/116100\n--update 2nd times--\nI forgot to include 13.837, the numbr should be like: 455, 665, 910 ...
", "votes": 2, "replies": [] } ] }, { "id": 685586, "author_name": "Joel Then", "author_url": "", "post_date": "2019-12-02T00:29:42.703000", "content": "AI Doctors
", "votes": 0, "replies": [] }, { "id": 685359, "author_name": "Abhinav Kulshreshth", "author_url": "", "post_date": "2019-12-01T13:53:26.923000", "content": "This is niceππ
", "votes": 0, "replies": [] }, { "id": 684565, "author_name": "DongshiLi", "author_url": "", "post_date": "2019-11-30T02:44:54.877000", "content": "congratz~!
", "votes": 0, "replies": [] }, { "id": 678706, "author_name": "ralfzu", "author_url": "", "post_date": "2019-11-21T18:33:40.277000", "content": "nice
", "votes": 0, "replies": [] }, { "id": 670597, "author_name": "Mathis", "author_url": "", "post_date": "2019-11-11T16:26:50.340000", "content": "sub.Label = (sub.Label > 0.1).astype(float)
", "votes": 0, "replies": [] }, { "id": 670015, "author_name": "Serban N", "author_url": "", "post_date": "2019-11-10T22:24:49.340000", "content": "This is nuts π
", "votes": 0, "replies": [] }, { "id": 668044, "author_name": "David Tang", "author_url": "", "post_date": "2019-11-07T22:50:12.920000", "content": "Haha, thanks. LB probing at its best here.
", "votes": 0, "replies": [] }, { "id": 667277, "author_name": "Wang", "author_url": "", "post_date": "2019-11-07T03:56:14.557000", "content": "Where should I add these code ? Please introduce this method more detailed , thxπ
", "votes": 0, "replies": [] }, { "id": 666975, "author_name": "YaGana Sheriff-Hussaini", "author_url": "", "post_date": "2019-11-06T17:22:30.593000", "content": "And then what? This is crazy:-)
", "votes": 0, "replies": [] }, { "id": 669383, "author_name": "", "author_url": "", "post_date": "2019-11-10T00:56:40.907000", "content": "", "votes": 0, "replies": [] }, { "id": 667636, "author_name": "", "author_url": "", "post_date": "2019-11-07T13:34:38.523000", "content": "", "votes": 0, "replies": [] }, { "id": 666971, "author_name": "", "author_url": "", "post_date": "2019-11-06T17:17:10.457000", "content": "", "votes": 1, "replies": [ { "id": 666981, "author_name": "nosound", "author_url": "", "post_date": "2019-11-06T17:24:53.980000", "content": "If someone decides to select his final submissions with this post-processing nobody will ask this question, because models outside of medals range are not checked.
", "votes": 6, "replies": [] }, { "id": 667003, "author_name": "yuval reina", "author_url": "", "post_date": "2019-11-06T17:51:12.433000", "content": "@portelaraian This technique will score terribly on the full private LB
", "votes": 3, "replies": [] }, { "id": 667009, "author_name": "Yifeng (Ethan) Zou", "author_url": "", "post_date": "2019-11-06T18:01:09.417000", "content": "But it's fun xd. I saw the .000 before finding out it's on <1% subset. Literally mindblown.
", "votes": 0, "replies": [] }, { "id": 667284, "author_name": "Kerem Turgutlu", "author_url": "", "post_date": "2019-11-07T04:20:25.597000", "content": "", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "666938": "1. Grab your best submission and run `sub.Label = (sub.Label > 0.5).astype(float)`\n2. ??\n3. Profit!\n\ncredits to @anginscribe for showing that it is possible.", "666978": "You found the cure for cranial hemorrhagy, congratz !", "925491": "When we will use 100% of our brain!!!", "667560": "For those who got 13.837 after these tricks, try lowering your thresholds. e.g. sub.Label = (sub.Label > 0.3).astype(float)\n\nNow, please share prize if you win :D", "667073": "This like betting your life savings on a game of black jack, win once, and then walk. Brilliant!", "680832": "nice one", "667451": "Zero on LB not total zero. I submit ur code and jump on 5 position, ignore 2 positions with zero.", "667069": "funny, it didn't work for me - got 13.837.\nmaybe I did something wrong...\n:)\njust tried to see if it's really that crazy π ", "685586": "AI Doctors", "685359": "This is niceππ ", "684565": "congratz~!", "678706": "nice", "670597": "sub.Label = (sub.Label > 0.1).astype(float)", "670015": "This is nuts π ", "668044": "Haha, thanks. LB probing at its best here. ", "667277": "Where should I add these code ? Please introduce this method more detailed , thxπ ", "666975": "And then what? This is crazy:-)", "669383": "", "667636": "", "666971": "" } }