{
  "id": 452208,
  "title": "CV/validation score vs LB",
  "url": "/competitions/UBC-OCEAN/discussion/452208",
  "author_name": "Andreu Arderiu",
  "post_date": "2023-11-01T10:04:12.524000",
  "votes": 3,
  "comment_count": 1,
  "views": 0,
  "content": "<p>Hello, after training some models with thumbnails and making few different submissions, It seems like in some cases an increase of CV/validation score doesn't correlate with LB score.</p>\n<p>What are you experience here, what local evaluation pipeline would you suggest to be able to identify improvements that will also result in LB improvements? Thank you in advance!</p>",
  "messages": [
    {
      "id": 2507883,
      "postDate": "2023-11-01T10:04:12.523Z",
      "content": "<p>Hello, after training some models with thumbnails and making few different submissions, It seems like in some cases an increase of CV/validation score doesn't correlate with LB score.</p>\n<p>What are you experience here, what local evaluation pipeline would you suggest to be able to identify improvements that will also result in LB improvements? Thank you in advance!</p>",
      "rawMarkdown": "Hello, after training some models with thumbnails and making few different submissions, It seems like in some cases an increase of CV/validation score doesn't correlate with LB score.\n\nWhat are you experience here, what local evaluation pipeline would you suggest to be able to identify improvements that will also result in LB improvements? Thank you in advance!",
      "votes": 3
    },
    {
      "id": 2507990,
      "postDate": "2023-11-01T11:20:38.330Z",
      "content": "<p>Same problem,<br>\nThis may be a problem because the test image was created at a different hospital scanner than the training image. And most of the test images are TMA.</p>",
      "rawMarkdown": "Same problem,\nThis may be a problem because the test image was created at a different hospital scanner than the training image. And most of the test images are TMA.",
      "votes": 1
    }
  ],
  "comments": [
    {
      "id": 2507990,
      "author_name": "devchopin",
      "author_url": "",
      "post_date": "2023-11-01T11:20:38.330000",
      "content": "<p>Same problem,<br>\nThis may be a problem because the test image was created at a different hospital scanner than the training image. And most of the test images are TMA.</p>",
      "votes": 1,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2507883": "Hello, after training some models with thumbnails and making few different submissions, It seems like in some cases an increase of CV/validation score doesn't correlate with LB score.\n\nWhat are you experience here, what local evaluation pipeline would you suggest to be able to identify improvements that will also result in LB improvements? Thank you in advance!",
    "2507990": "Same problem,\nThis may be a problem because the test image was created at a different hospital scanner than the training image. And most of the test images are TMA."
  }
}