{
  "id": 383483,
  "title": "LB score is 0!",
  "url": "/competitions/rsna-breast-cancer-detection/discussion/383483",
  "author_name": "Lau2664",
  "post_date": "2023-02-04T00:40:00.176000",
  "votes": 1,
  "comment_count": 7,
  "views": 0,
  "content": "<p>I got this problem recently. I used my new-trained model in my submission notebook, which works well before, and my model got a pf1 score of 0.22 in validation dataset. But after many hours waiting, the final score is 0…<br><br>\nI tried some other submission notebooks which is copied from some public notebooks,  and all modification I did was simply replace the original model to my own, then I make another submission.<br><br>\nThe result is quite confusing. I got a pf1 score of 0.24 in one of the notebooks, meanwhile the others remained 0 score. How did that happen? Does anyone know what cause this problem please let me know🙏<br>\nI really don't want to waste many hours waiting for a 0 score😭 thanks!</p>",
  "messages": [
    {
      "id": 2129019,
      "postDate": "2023-02-04T09:02:03.763Z",
      "content": "<p>Is your <code>pf1 score of 0.24</code> based on validation data? If so have you considered overfitting on validation data or data discrepancy in your splits?</p>",
      "rawMarkdown": "Is your `pf1 score of 0.24` based on validation data? If so have you considered overfitting on validation data or data discrepancy in your splits?",
      "votes": 1,
      "replies": [
        {
          "id": 2129099,
          "postDate": "2023-02-04T10:25:32.640Z",
          "content": "<p>I got this score on test data</p>",
          "rawMarkdown": "I got this score on test data"
        },
        {
          "id": 2129231,
          "postDate": "2023-02-04T12:47:36.853Z",
          "content": "<p>Try to check your output for nan’s </p>",
          "rawMarkdown": "Try to check your output for nan’s ",
          "votes": 1
        }
      ]
    },
    {
      "id": 2128705,
      "postDate": "2023-02-04T00:40:00.177Z",
      "content": "<p>I got this problem recently. I used my new-trained model in my submission notebook, which works well before, and my model got a pf1 score of 0.22 in validation dataset. But after many hours waiting, the final score is 0…<br><br>\nI tried some other submission notebooks which is copied from some public notebooks,  and all modification I did was simply replace the original model to my own, then I make another submission.<br><br>\nThe result is quite confusing. I got a pf1 score of 0.24 in one of the notebooks, meanwhile the others remained 0 score. How did that happen? Does anyone know what cause this problem please let me know🙏<br>\nI really don't want to waste many hours waiting for a 0 score😭 thanks!</p>",
      "rawMarkdown": "I got this problem recently. I used my new-trained model in my submission notebook, which works well before, and my model got a pf1 score of 0.22 in validation dataset. But after many hours waiting, the final score is 0...<br/>\nI tried some other submission notebooks which is copied from some public notebooks,  and all modification I did was simply replace the original model to my own, then I make another submission.<br/>\nThe result is quite confusing. I got a pf1 score of 0.24 in one of the notebooks, meanwhile the others remained 0 score. How did that happen? Does anyone know what cause this problem please let me know🙏\nI really don't want to waste many hours waiting for a 0 score😭 thanks!",
      "votes": 1
    },
    {
      "id": 2140686,
      "postDate": "2023-02-12T04:02:42.850Z",
      "content": "<p>Thanks for you guys all! I guess the reason why I got 0 score is because I didn't use ROI test images, I simply use the original test image, but the model I use is trained on ROI images.</p>",
      "rawMarkdown": "Thanks for you guys all! I guess the reason why I got 0 score is because I didn't use ROI test images, I simply use the original test image, but the model I use is trained on ROI images."
    },
    {
      "id": 2137526,
      "postDate": "2023-02-10T05:18:03.537Z",
      "content": "<p>most likely is that you forget to normalised your iput <br>\ne.g. image = (image-mean)/std</p>",
      "rawMarkdown": "most likely is that you forget to normalised your iput \ne.g. image = (image-mean)/std"
    },
    {
      "id": 2137440,
      "postDate": "2023-02-10T02:16:55.313Z",
      "content": "<p>Hi, I met with same problem. My validation pf1 is 0.17 but after submission, my lb score is 0. Have you solved your problem?</p>",
      "rawMarkdown": "Hi, I met with same problem. My validation pf1 is 0.17 but after submission, my lb score is 0. Have you solved your problem?",
      "replies": [
        {
          "id": 2140688,
          "postDate": "2023-02-12T04:04:06.463Z",
          "content": "<p>Hi, I think you can copy the public inference notebook that works, then make some modifications on it. </p>",
          "rawMarkdown": "Hi, I think you can copy the public inference notebook that works, then make some modifications on it. "
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 2129019,
      "author_name": "ToriGlori",
      "author_url": "",
      "post_date": "2023-02-04T09:02:03.763000",
      "content": "<p>Is your <code>pf1 score of 0.24</code> based on validation data? If so have you considered overfitting on validation data or data discrepancy in your splits?</p>",
      "votes": 1,
      "replies": [
        {
          "id": 2129099,
          "author_name": "Lau2664",
          "author_url": "",
          "post_date": "2023-02-04T10:25:32.640000",
          "content": "<p>I got this score on test data</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 2129231,
          "author_name": "A.P.",
          "author_url": "",
          "post_date": "2023-02-04T12:47:36.853000",
          "content": "<p>Try to check your output for nan’s </p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 2140686,
      "author_name": "Lau2664",
      "author_url": "",
      "post_date": "2023-02-12T04:02:42.850000",
      "content": "<p>Thanks for you guys all! I guess the reason why I got 0 score is because I didn't use ROI test images, I simply use the original test image, but the model I use is trained on ROI images.</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2137526,
      "author_name": "hengck23",
      "author_url": "",
      "post_date": "2023-02-10T05:18:03.537000",
      "content": "<p>most likely is that you forget to normalised your iput <br>\ne.g. image = (image-mean)/std</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2137440,
      "author_name": "dingyan",
      "author_url": "",
      "post_date": "2023-02-10T02:16:55.313000",
      "content": "<p>Hi, I met with same problem. My validation pf1 is 0.17 but after submission, my lb score is 0. Have you solved your problem?</p>",
      "votes": 0,
      "replies": [
        {
          "id": 2140688,
          "author_name": "Lau2664",
          "author_url": "",
          "post_date": "2023-02-12T04:04:06.463000",
          "content": "<p>Hi, I think you can copy the public inference notebook that works, then make some modifications on it. </p>",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2129019": "Is your `pf1 score of 0.24` based on validation data? If so have you considered overfitting on validation data or data discrepancy in your splits?",
    "2128705": "I got this problem recently. I used my new-trained model in my submission notebook, which works well before, and my model got a pf1 score of 0.22 in validation dataset. But after many hours waiting, the final score is 0...<br/>\nI tried some other submission notebooks which is copied from some public notebooks,  and all modification I did was simply replace the original model to my own, then I make another submission.<br/>\nThe result is quite confusing. I got a pf1 score of 0.24 in one of the notebooks, meanwhile the others remained 0 score. How did that happen? Does anyone know what cause this problem please let me know🙏\nI really don't want to waste many hours waiting for a 0 score😭 thanks!",
    "2140686": "Thanks for you guys all! I guess the reason why I got 0 score is because I didn't use ROI test images, I simply use the original test image, but the model I use is trained on ROI images.",
    "2137526": "most likely is that you forget to normalised your iput \ne.g. image = (image-mean)/std",
    "2137440": "Hi, I met with same problem. My validation pf1 is 0.17 but after submission, my lb score is 0. Have you solved your problem?"
  }
}