{
  "id": 112387,
  "title": "Help with loss function",
  "url": "/competitions/rsna-intracranial-hemorrhage-detection/discussion/112387",
  "author_name": "bright",
  "post_date": "2019-10-12T12:59:09.874000",
  "votes": 3,
  "comment_count": 2,
  "views": 0,
  "content": "<p>Sorry if this has been mentioned before. I am finding a relatively good correlation to LB with\n<code>sklearn.metrics.log_loss(actual, predicted) where predicted = torch.sigmoid(logits)</code></p>\n\n<p>However BCEWithLogitsLoss is giving me much different results and where bce seems to converge during training and validation, but my LB score gets much worse the more epochs.</p>\n\n<p>I have been using torch.nn.BCEWithLogitsLoss() as my loss function and then getting metrics using the sklearn log loss and a metric from another discussion where</p>\n\n<p><code>\ndef my_loss(y_pred,y_true):\n    weights =[1.0, 1.0, 1.0, 1.0, 1.0, 2.0]\n    weights = torch.FloatTensor(weights).cuda()\n    return F.binary_cross_entropy_with_logits(y_pred.float(),\n                                  y_true.float(),\n                                  weights.repeat(y_pred.shape[0],1))\n</code></p>\n\n<p>Any help or advice would be appreciated</p>",
  "messages": [
    {
      "id": 647322,
      "postDate": "2019-10-12T12:59:09.873Z",
      "content": "<p>Sorry if this has been mentioned before. I am finding a relatively good correlation to LB with\n<code>sklearn.metrics.log_loss(actual, predicted) where predicted = torch.sigmoid(logits)</code></p>\n\n<p>However BCEWithLogitsLoss is giving me much different results and where bce seems to converge during training and validation, but my LB score gets much worse the more epochs.</p>\n\n<p>I have been using torch.nn.BCEWithLogitsLoss() as my loss function and then getting metrics using the sklearn log loss and a metric from another discussion where</p>\n\n<p><code>\ndef my_loss(y_pred,y_true):\n    weights =[1.0, 1.0, 1.0, 1.0, 1.0, 2.0]\n    weights = torch.FloatTensor(weights).cuda()\n    return F.binary_cross_entropy_with_logits(y_pred.float(),\n                                  y_true.float(),\n                                  weights.repeat(y_pred.shape[0],1))\n</code></p>\n\n<p>Any help or advice would be appreciated</p>",
      "rawMarkdown": "Sorry if this has been mentioned before. I am finding a relatively good correlation to LB with\n`sklearn.metrics.log_loss(actual, predicted) where predicted = torch.sigmoid(logits)`\n\nHowever BCEWithLogitsLoss is giving me much different results and where bce seems to converge during training and validation, but my LB score gets much worse the more epochs.\n\nI have been using torch.nn.BCEWithLogitsLoss() as my loss function and then getting metrics using the sklearn log loss and a metric from another discussion where\n\n```\ndef my_loss(y_pred,y_true):\n    weights =[1.0, 1.0, 1.0, 1.0, 1.0, 2.0]\n    weights = torch.FloatTensor(weights).cuda()\n    return F.binary_cross_entropy_with_logits(y_pred.float(),\n                                  y_true.float(),\n                                  weights.repeat(y_pred.shape[0],1))\n```\n\nAny help or advice would be appreciated",
      "votes": 3
    },
    {
      "id": 647371,
      "postDate": "2019-10-12T14:17:34.603Z",
      "content": "<p>You should be creating validation grouping on patients i.e. patient images should not overlap between train &amp; val</p>",
      "rawMarkdown": "You should be creating validation grouping on patients i.e. patient images should not overlap between train &amp; val",
      "votes": 2,
      "replies": [
        {
          "id": 647535,
          "postDate": "2019-10-12T19:34:57.293Z",
          "content": "<p>Thanks! Just ran a few experiments and it didn't change much, but is useful information for making my validation splits. Seems like it will be really useful for stage 2.</p>",
          "rawMarkdown": "Thanks! Just ran a few experiments and it didn't change much, but is useful information for making my validation splits. Seems like it will be really useful for stage 2."
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 647371,
      "author_name": "Nicholas Lyu",
      "author_url": "",
      "post_date": "2019-10-12T14:17:34.603000",
      "content": "<p>You should be creating validation grouping on patients i.e. patient images should not overlap between train &amp; val</p>",
      "votes": 2,
      "replies": [
        {
          "id": 647535,
          "author_name": "bright",
          "author_url": "",
          "post_date": "2019-10-12T19:34:57.293000",
          "content": "<p>Thanks! Just ran a few experiments and it didn't change much, but is useful information for making my validation splits. Seems like it will be really useful for stage 2.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "647322": "Sorry if this has been mentioned before. I am finding a relatively good correlation to LB with\n`sklearn.metrics.log_loss(actual, predicted) where predicted = torch.sigmoid(logits)`\n\nHowever BCEWithLogitsLoss is giving me much different results and where bce seems to converge during training and validation, but my LB score gets much worse the more epochs.\n\nI have been using torch.nn.BCEWithLogitsLoss() as my loss function and then getting metrics using the sklearn log loss and a metric from another discussion where\n\n```\ndef my_loss(y_pred,y_true):\n    weights =[1.0, 1.0, 1.0, 1.0, 1.0, 2.0]\n    weights = torch.FloatTensor(weights).cuda()\n    return F.binary_cross_entropy_with_logits(y_pred.float(),\n                                  y_true.float(),\n                                  weights.repeat(y_pred.shape[0],1))\n```\n\nAny help or advice would be appreciated",
    "647371": "You should be creating validation grouping on patients i.e. patient images should not overlap between train &amp; val"
  }
}