LB probe is completed! I revealed the following three facts(almost no doubt).
\n\nany is x2 of otherssklearn.metrics.log_loss() with sample_weight=([1, 1, 1, 1, 1, 2] * 78545). LB score is round off, not round().If you need, please see my kernel for details.
\nhttps://www.kaggle.com/kambarakun/lb-probe-weights-n-of-positives-scoring
From now on, I will concentrate on making models!!
", "messages": [ { "id": 635681, "postDate": "2019-09-28T02:55:45.300Z", "content": "LB probe is completed! I revealed the following three facts(almost no doubt).
\n\nany is x2 of otherssklearn.metrics.log_loss() with sample_weight=([1, 1, 1, 1, 1, 2] * 78545). LB score is round off, not round().If you need, please see my kernel for details.
\nhttps://www.kaggle.com/kambarakun/lb-probe-weights-n-of-positives-scoring
From now on, I will concentrate on making models!!
", "rawMarkdown": "**LB probe is completed!** I revealed the following three facts(almost no doubt). \n\n* weights of `any` is x2 of others\n* n_positives of each class\n* Metrics is same as `sklearn.metrics.log_loss()` with `sample_weight=([1, 1, 1, 1, 1, 2] * 78545)`. LB score is round off, not round().\n\nIf you need, please see my kernel for details. \nhttps://www.kaggle.com/kambarakun/lb-probe-weights-n-of-positives-scoring\n\n**From now on, I will concentrate on making models!!**", "votes": 49 }, { "id": 635686, "postDate": "2019-09-28T03:05:38.387Z", "content": "Hi @kambarakun - correct on the weight. The metric is written in C# and is not in any way related to scikit-learn's excellent implementation, apart from doing the same math. I'm glad it lines up so nicely!
", "rawMarkdown": "Hi @kambarakun - correct on the weight. The metric is written in C# and is not in any way related to scikit-learn's excellent implementation, apart from doing the same math. I'm glad it lines up so nicely!", "votes": 4, "replies": [ { "id": 635702, "postDate": "2019-09-28T04:16:38.893Z", "content": "Hi @philculliton, thank you very much for your reply.
\nFirst of all, I'm really very very sorry if my posts prevent this competition.
And there are two requests.
\nThe first is that it doesn't matter even after the competition ended, so why did you not clarify the weight at the start of the competition and what is the significance of the weight (I'm MD and I think it is somehow related to clinical usage, or I think you expect high generalization models even for easy tasks).
\nSecond, please don't make big change (ex. change weights, delete metadata, etc.) in stage 2.
It would be a big surprise to re-weight the loss on stage 2 :D
", "rawMarkdown": "It would be a big surprise to re-weight the loss on stage 2 :D", "votes": 2 }, { "id": 636870, "postDate": "2019-09-30T09:37:42.497Z", "content": "I sincerely hope that this will not happen.
", "rawMarkdown": "I sincerely hope that this will not happen." }, { "id": 643521, "postDate": "2019-10-07T15:30:26.007Z", "content": "Hi, sorry I missed this, @kambarakun!
\n\nWe will not be making any changes to the weights or metadata in stage 2.
", "rawMarkdown": "Hi, sorry I missed this, @kambarakun!\n\nWe will not be making any changes to the weights or metadata in stage 2.", "votes": 1 }, { "id": 644030, "postDate": "2019-10-08T08:31:01.700Z", "content": "@philculliton I am really happy to receive a reply.
\nI hope that it will be a good competition.
so it's like score = any*0.28 + disease1*0.14 + disease2*0.14 + disease3*0.14 + disease4*0.14 + disease5*0.14 @kambarakun ?
It is almost correct. Exactly 1/7 and 2/7.
\nThe LB score is currently rounded down. If it is the same as other competitions, the number of digits may increase after the deadline.
Hi @kambarakun. Could u explain me how [1, 1, 1, 1, 1, 2] turned into [1/7, 1/7, 1/7, 1/7, 1/7, 2/7]? Aren't these 2 supposed to be the same thing (weight 2 times bigger)?
", "rawMarkdown": "Hi @kambarakun. Could u explain me how [1, 1, 1, 1, 1, 2] turned into [1/7, 1/7, 1/7, 1/7, 1/7, 2/7]? Aren't these 2 supposed to be the same thing (weight 2 times bigger)?" }, { "id": 644037, "postDate": "2019-10-08T08:35:25.697Z", "content": "They are exactly the same.
\nI use the weights divided by 7 to adjust the sum of the logloss calculated for each label.
could you explain more on\"weights of any is x2 of others\"\nI can NOT follow you.\nThanks
", "rawMarkdown": "could you explain more on\"weights of any is x2 of others\"\nI can NOT follow you.\nThanks", "replies": [ { "id": 644043, "postDate": "2019-10-08T08:42:46.050Z", "content": "This means that the impact on the score is twice that of other labels.
\n\nFrom this, I think that any prediction method is worth trial and error.
\nFor example, is it a direct model prediction, probability calculation, or some post-processing?
Perhaps some people think that the influence of any is minor.
", "rawMarkdown": "This means that the impact on the score is twice that of other labels.\n\nFrom this, I think that any prediction method is worth trial and error. \nFor example, is it a direct model prediction, probability calculation, or some post-processing?\n\nPerhaps some people think that the influence of any is minor." } ] }, { "id": 641855, "postDate": "2019-10-05T08:39:01.913Z", "rawMarkdown": "", "isDeleted": true, "replies": [ { "id": 644045, "postDate": "2019-10-08T08:44:04.330Z", "content": "I think they are more important for training than prediction.
\n\nPlease check the other great topics and posts!
\nFor example, https://www.kaggle.com/c/rsna-intracranial-hemorrhage-detection/discussion/111198641613
Hi @kambarakun - correct on the weight. The metric is written in C# and is not in any way related to scikit-learn's excellent implementation, apart from doing the same math. I'm glad it lines up so nicely!
", "votes": 4, "replies": [ { "id": 635702, "author_name": "kambarakun", "author_url": "", "post_date": "2019-09-28T04:16:38.893000", "content": "Hi @philculliton, thank you very much for your reply.
\nFirst of all, I'm really very very sorry if my posts prevent this competition.
And there are two requests.
\nThe first is that it doesn't matter even after the competition ended, so why did you not clarify the weight at the start of the competition and what is the significance of the weight (I'm MD and I think it is somehow related to clinical usage, or I think you expect high generalization models even for easy tasks).
\nSecond, please don't make big change (ex. change weights, delete metadata, etc.) in stage 2.
It would be a big surprise to re-weight the loss on stage 2 :D
", "votes": 2, "replies": [] }, { "id": 636870, "author_name": "kambarakun", "author_url": "", "post_date": "2019-09-30T09:37:42.497000", "content": "I sincerely hope that this will not happen.
", "votes": 0, "replies": [] }, { "id": 643521, "author_name": "Phil Culliton", "author_url": "", "post_date": "2019-10-07T15:30:26.007000", "content": "Hi, sorry I missed this, @kambarakun!
\n\nWe will not be making any changes to the weights or metadata in stage 2.
", "votes": 1, "replies": [] }, { "id": 644030, "author_name": "kambarakun", "author_url": "", "post_date": "2019-10-08T08:31:01.700000", "content": "@philculliton I am really happy to receive a reply.
\nI hope that it will be a good competition.
so it's like score = any*0.28 + disease1*0.14 + disease2*0.14 + disease3*0.14 + disease4*0.14 + disease5*0.14 @kambarakun ?
It is almost correct. Exactly 1/7 and 2/7.
\nThe LB score is currently rounded down. If it is the same as other competitions, the number of digits may increase after the deadline.
Hi @kambarakun. Could u explain me how [1, 1, 1, 1, 1, 2] turned into [1/7, 1/7, 1/7, 1/7, 1/7, 2/7]? Aren't these 2 supposed to be the same thing (weight 2 times bigger)?
", "votes": 0, "replies": [] }, { "id": 644037, "author_name": "kambarakun", "author_url": "", "post_date": "2019-10-08T08:35:25.697000", "content": "They are exactly the same.
\nI use the weights divided by 7 to adjust the sum of the logloss calculated for each label.
could you explain more on\"weights of any is x2 of others\"\nI can NOT follow you.\nThanks
", "votes": 0, "replies": [ { "id": 644043, "author_name": "kambarakun", "author_url": "", "post_date": "2019-10-08T08:42:46.050000", "content": "This means that the impact on the score is twice that of other labels.
\n\nFrom this, I think that any prediction method is worth trial and error.
\nFor example, is it a direct model prediction, probability calculation, or some post-processing?
Perhaps some people think that the influence of any is minor.
", "votes": 0, "replies": [] } ] }, { "id": 641855, "author_name": "", "author_url": "", "post_date": "2019-10-05T08:39:01.913000", "content": "", "votes": 0, "replies": [ { "id": 644045, "author_name": "kambarakun", "author_url": "", "post_date": "2019-10-08T08:44:04.330000", "content": "I think they are more important for training than prediction.
\n\nPlease check the other great topics and posts!
\nFor example, https://www.kaggle.com/c/rsna-intracranial-hemorrhage-detection/discussion/111198641613