{
  "id": 378963,
  "title": "Overfitting problem",
  "url": "/competitions/rsna-breast-cancer-detection/discussion/378963",
  "author_name": "nicehzj",
  "post_date": "2023-01-17T17:00:02.205000",
  "votes": 3,
  "comment_count": 8,
  "views": 0,
  "content": "<p>When I use efficientnetb0, efficientnetv2_s to reproduce the code, I always get really strong overfitting…<br>\nI can only get high auc at first 3 epochs, then it goes down and the eval loss is increasing.<br>\nDo you have any suggestions on this?<br>\nThanks!</p>",
  "messages": [
    {
      "id": 2104172,
      "postDate": "2023-01-17T17:00:02.207Z",
      "content": "<p>When I use efficientnetb0, efficientnetv2_s to reproduce the code, I always get really strong overfitting…<br>\nI can only get high auc at first 3 epochs, then it goes down and the eval loss is increasing.<br>\nDo you have any suggestions on this?<br>\nThanks!</p>",
      "rawMarkdown": "When I use efficientnetb0, efficientnetv2_s to reproduce the code, I always get really strong overfitting...\nI can only get high auc at first 3 epochs, then it goes down and the eval loss is increasing.\nDo you have any suggestions on this?\nThanks!",
      "votes": 3
    },
    {
      "id": 2105888,
      "postDate": "2023-01-18T20:04:37.087Z",
      "content": "<p>I feel your pain.. the data is strange..</p>\n<p>The Devastator.</p>",
      "rawMarkdown": "I feel your pain.. the data is strange..\n\n\nThe Devastator.\n"
    },
    {
      "id": 2104467,
      "postDate": "2023-01-17T20:34:08.597Z",
      "content": "<p>Have you tried dropout? If so, increase the dropout rate. It will help with overfitting issue.</p>",
      "rawMarkdown": "Have you tried dropout? If so, increase the dropout rate. It will help with overfitting issue.",
      "replies": [
        {
          "id": 2104480,
          "postDate": "2023-01-17T20:43:58.103Z",
          "content": "<p>I set dropout=0.2 in timm model and fc layer, maybe I should increase it?</p>",
          "rawMarkdown": "I set dropout=0.2 in timm model and fc layer, maybe I should increase it?",
          "replies": [
            {
              "id": 2104500,
              "postDate": "2023-01-17T21:03:40.820Z",
              "content": "<p>Increase it to 0.5 and train again.</p>",
              "rawMarkdown": "Increase it to 0.5 and train again."
            }
          ]
        }
      ]
    },
    {
      "id": 2104361,
      "postDate": "2023-01-17T19:10:39.063Z",
      "content": "<p>Yes, this is beauty of this competition 😂 I have the same - training loss is going down for 3-4 epochs and then is going up … even my validation is better …. 😂 To be honest I will remember this competition for sure. </p>",
      "rawMarkdown": "Yes, this is beauty of this competition 😂 I have the same - training loss is going down for 3-4 epochs and then is going up ... even my validation is better .... 😂 To be honest I will remember this competition for sure. ",
      "replies": [
        {
          "id": 2104411,
          "postDate": "2023-01-17T19:54:27.583Z",
          "content": "<p>I tried many high score baseline codes in the discussion….but none of them can be reproduced….<br>\nI almost did everything… upsampling, gradient accumulation, decrease lr… but really hard to reach their score, even cannot close to them</p>",
          "rawMarkdown": "I tried many high score baseline codes in the discussion....but none of them can be reproduced....\nI almost did everything... upsampling, gradient accumulation, decrease lr... but really hard to reach their score, even cannot close to them",
          "votes": 1
        },
        {
          "id": 2104555,
          "postDate": "2023-01-17T22:07:48.563Z",
          "content": "<p>also may i ask which metric u used to choose the best model? auc?pf1?f1?<br>\nbecause I find that my auc is decreasing but pf1 is increasing…<br>\nthanks!</p>",
          "rawMarkdown": "also may i ask which metric u used to choose the best model? auc?pf1?f1?\nbecause I find that my auc is decreasing but pf1 is increasing...\nthanks!"
        },
        {
          "id": 2105196,
          "postDate": "2023-01-18T10:55:02.883Z",
          "content": "<p><a href=\"https://www.kaggle.com/remekkinas\" target=\"_blank\">@remekkinas</a> That does not really sound like overfitting because your training loss goes up after. It sounds like a different problem maybe even underfitting, you should play around with bigger models and see if you are able to overfit with more parameters otherwise there should be a different issue (e.g. huge data imbalance, data discrepancy, …)</p>",
          "rawMarkdown": "@remekkinas That does not really sound like overfitting because your training loss goes up after. It sounds like a different problem maybe even underfitting, you should play around with bigger models and see if you are able to overfit with more parameters otherwise there should be a different issue (e.g. huge data imbalance, data discrepancy, ...)",
          "votes": 1
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 2105888,
      "author_name": "The Devastator",
      "author_url": "",
      "post_date": "2023-01-18T20:04:37.087000",
      "content": "<p>I feel your pain.. the data is strange..</p>\n<p>The Devastator.</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2104467,
      "author_name": "Rasoul Mojtahedzadeh",
      "author_url": "",
      "post_date": "2023-01-17T20:34:08.597000",
      "content": "<p>Have you tried dropout? If so, increase the dropout rate. It will help with overfitting issue.</p>",
      "votes": 0,
      "replies": [
        {
          "id": 2104480,
          "author_name": "nicehzj",
          "author_url": "",
          "post_date": "2023-01-17T20:43:58.103000",
          "content": "<p>I set dropout=0.2 in timm model and fc layer, maybe I should increase it?</p>",
          "votes": 0,
          "replies": [
            {
              "id": 2104500,
              "author_name": "Rasoul Mojtahedzadeh",
              "author_url": "",
              "post_date": "2023-01-17T21:03:40.820000",
              "content": "<p>Increase it to 0.5 and train again.</p>",
              "votes": 0,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 2104361,
      "author_name": "Remek Kinas",
      "author_url": "",
      "post_date": "2023-01-17T19:10:39.063000",
      "content": "<p>Yes, this is beauty of this competition 😂 I have the same - training loss is going down for 3-4 epochs and then is going up … even my validation is better …. 😂 To be honest I will remember this competition for sure. </p>",
      "votes": 0,
      "replies": [
        {
          "id": 2104411,
          "author_name": "nicehzj",
          "author_url": "",
          "post_date": "2023-01-17T19:54:27.583000",
          "content": "<p>I tried many high score baseline codes in the discussion….but none of them can be reproduced….<br>\nI almost did everything… upsampling, gradient accumulation, decrease lr… but really hard to reach their score, even cannot close to them</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 2104555,
          "author_name": "nicehzj",
          "author_url": "",
          "post_date": "2023-01-17T22:07:48.563000",
          "content": "<p>also may i ask which metric u used to choose the best model? auc?pf1?f1?<br>\nbecause I find that my auc is decreasing but pf1 is increasing…<br>\nthanks!</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 2105196,
          "author_name": "Ali Abdin",
          "author_url": "",
          "post_date": "2023-01-18T10:55:02.883000",
          "content": "<p><a href=\"https://www.kaggle.com/remekkinas\" target=\"_blank\">@remekkinas</a> That does not really sound like overfitting because your training loss goes up after. It sounds like a different problem maybe even underfitting, you should play around with bigger models and see if you are able to overfit with more parameters otherwise there should be a different issue (e.g. huge data imbalance, data discrepancy, …)</p>",
          "votes": 1,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2104172": "When I use efficientnetb0, efficientnetv2_s to reproduce the code, I always get really strong overfitting...\nI can only get high auc at first 3 epochs, then it goes down and the eval loss is increasing.\nDo you have any suggestions on this?\nThanks!",
    "2105888": "I feel your pain.. the data is strange..\n\n\nThe Devastator.\n",
    "2104467": "Have you tried dropout? If so, increase the dropout rate. It will help with overfitting issue.",
    "2104361": "Yes, this is beauty of this competition 😂 I have the same - training loss is going down for 3-4 epochs and then is going up ... even my validation is better .... 😂 To be honest I will remember this competition for sure. "
  }
}