{
  "id": 384853,
  "title": "Best threshold value for ensemble submissions",
  "url": "/competitions/rsna-breast-cancer-detection/discussion/384853",
  "author_name": "Naman Makkar",
  "post_date": "2023-02-09T19:40:31.608000",
  "votes": 2,
  "comment_count": 2,
  "views": 0,
  "content": "<p>I have carried out 5-fold cross validation and am trying to carry out inference with an ensemble model. I have the thresholds for the best f1 validation scores of all the 5 models. Should my final threshold for inference be the average of all thresholds or should it be the maximum of all the thresholds ? The thresholds have very little deviation varying from 0.73 to 0.88</p>",
  "messages": [
    {
      "id": 2137194,
      "postDate": "2023-02-09T19:40:31.610Z",
      "content": "<p>I have carried out 5-fold cross validation and am trying to carry out inference with an ensemble model. I have the thresholds for the best f1 validation scores of all the 5 models. Should my final threshold for inference be the average of all thresholds or should it be the maximum of all the thresholds ? The thresholds have very little deviation varying from 0.73 to 0.88</p>",
      "rawMarkdown": "I have carried out 5-fold cross validation and am trying to carry out inference with an ensemble model. I have the thresholds for the best f1 validation scores of all the 5 models. Should my final threshold for inference be the average of all thresholds or should it be the maximum of all the thresholds ? The thresholds have very little deviation varying from 0.73 to 0.88",
      "votes": 2
    },
    {
      "id": 2137367,
      "postDate": "2023-02-09T22:56:07.140Z",
      "content": "<p>Are you able to average the predictions from different models? That might make sense in some cases. You could also give weights to each of the models, if you have found out that some of the models in the ensemble of weak learners are to be trusted more or are to be punished for overconfidence?</p>",
      "rawMarkdown": "Are you able to average the predictions from different models? That might make sense in some cases. You could also give weights to each of the models, if you have found out that some of the models in the ensemble of weak learners are to be trusted more or are to be punished for overconfidence?",
      "replies": [
        {
          "id": 2137368,
          "postDate": "2023-02-09T23:00:21.350Z",
          "content": "<p>All of my models are giving me a validation F1 score of 0.3 - 0.33 so I just took the mean of the predictions and the mean of the thresholds.</p>",
          "rawMarkdown": "All of my models are giving me a validation F1 score of 0.3 - 0.33 so I just took the mean of the predictions and the mean of the thresholds."
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 2137367,
      "author_name": "Antti Isosalo",
      "author_url": "",
      "post_date": "2023-02-09T22:56:07.140000",
      "content": "<p>Are you able to average the predictions from different models? That might make sense in some cases. You could also give weights to each of the models, if you have found out that some of the models in the ensemble of weak learners are to be trusted more or are to be punished for overconfidence?</p>",
      "votes": 0,
      "replies": [
        {
          "id": 2137368,
          "author_name": "Naman Makkar",
          "author_url": "",
          "post_date": "2023-02-09T23:00:21.350000",
          "content": "<p>All of my models are giving me a validation F1 score of 0.3 - 0.33 so I just took the mean of the predictions and the mean of the thresholds.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2137194": "I have carried out 5-fold cross validation and am trying to carry out inference with an ensemble model. I have the thresholds for the best f1 validation scores of all the 5 models. Should my final threshold for inference be the average of all thresholds or should it be the maximum of all the thresholds ? The thresholds have very little deviation varying from 0.73 to 0.88",
    "2137367": "Are you able to average the predictions from different models? That might make sense in some cases. You could also give weights to each of the models, if you have found out that some of the models in the ensemble of weak learners are to be trusted more or are to be punished for overconfidence?"
  }
}