{
  "id": 432029,
  "title": "CV vs LB Scores",
  "url": "/competitions/rsna-2023-abdominal-trauma-detection/discussion/432029",
  "author_name": "Gunes Evitan",
  "post_date": "2023-08-15T19:25:32.573000",
  "votes": 31,
  "comment_count": 55,
  "views": 0,
  "content": "<p>I worked on my pipeline for 3 days and the result was disappointment.</p>\n<p>These are my OOF scores averaged across targets. My OOF mean sample weighted log loss is 0.5447 and this model scored 0.74 on public LB.<br>\nMy current approach is 2D and I'm using 5 fold cross-validation.</p>\n<table>\n<thead>\n<tr>\n<th>aggregation</th>\n<th>log_loss</th>\n<th>sample_weighted_log_loss</th>\n<th>accuracy</th>\n<th>precision</th>\n<th>recall</th>\n<th>specificity</th>\n<th>f1</th>\n<th>roc_auc</th>\n<th>average_precision</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>mean</td>\n<td>0.39403370027229895</td>\n<td>0.6088572296443904</td>\n<td>0.893971555932923</td>\n<td>0.15127715276303688</td>\n<td>0.16666666666666666</td>\n<td>1.0</td>\n<td>0.15857350589156527</td>\n<td>0.5520889782333832</td>\n<td>0.1335163957644465</td>\n</tr>\n</tbody>\n</table>\n<p>and these are my per target scores</p>\n<table>\n<thead>\n<tr>\n<th>target</th>\n<th>log_loss</th>\n<th>sample_weighted_log_loss</th>\n<th>accuracy</th>\n<th>precision</th>\n<th>recall</th>\n<th>specificity</th>\n<th>f1</th>\n<th>roc_auc</th>\n<th>average_precision</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>bowel_injury</td>\n<td>0.11192792245872818</td>\n<td>0.17707020379404662</td>\n<td>0.9779240076416896</td>\n<td>0.0</td>\n<td>0.0</td>\n<td>1.0</td>\n<td>0.0</td>\n<td>0.5657757425990549</td>\n<td>0.027722025805609667</td>\n</tr>\n<tr>\n<td>extravasation_injury</td>\n<td>0.40340889265515145</td>\n<td>0.5974000162252465</td>\n<td>0.9363192528125663</td>\n<td>0.0</td>\n<td>0.0</td>\n<td>1.0</td>\n<td>0.0</td>\n<td>0.5632324491800801</td>\n<td>0.07566879932761941</td>\n</tr>\n<tr>\n<td>any_injury</td>\n<td>0.5882653406365985</td>\n<td>0.9293227494263636</td>\n<td>0.7265973254086181</td>\n<td>0.0</td>\n<td>0.0</td>\n<td>1.0</td>\n<td>0.0</td>\n<td>0.5272587429210147</td>\n<td>0.2971583621601104</td>\n</tr>\n<tr>\n<td>kidney</td>\n<td>0.312776007936721</td>\n<td>0.5338212381898938</td>\n<td>0.939503290171938</td>\n<td>0.313167763390646</td>\n<td>0.3333333333333333</td>\n<td></td>\n<td>0.32293604757214256</td>\n<td></td>\n<td></td>\n</tr>\n<tr>\n<td>liver</td>\n<td>0.4177555413263836</td>\n<td>0.646181296215893</td>\n<td>0.8991721502865634</td>\n<td>0.2997240500955211</td>\n<td>0.3333333333333333</td>\n<td></td>\n<td>0.3156365262099028</td>\n<td></td>\n<td></td>\n</tr>\n<tr>\n<td>spleen</td>\n<td>0.5300684966202112</td>\n<td>0.7693478740148988</td>\n<td>0.8843133092761621</td>\n<td>0.29477110309205407</td>\n<td>0.3333333333333333</td>\n<td></td>\n<td>0.3128684615673463</td>\n<td></td>\n<td></td>\n</tr>\n</tbody>\n</table>\n<p>Does your validation score have any correlation with LB so far?</p>",
  "messages": [
    {
      "id": 2392644,
      "postDate": "2023-08-15T19:25:32.573Z",
      "content": "<p>I worked on my pipeline for 3 days and the result was disappointment.</p>\n<p>These are my OOF scores averaged across targets. My OOF mean sample weighted log loss is 0.5447 and this model scored 0.74 on public LB.<br>\nMy current approach is 2D and I'm using 5 fold cross-validation.</p>\n<table>\n<thead>\n<tr>\n<th>aggregation</th>\n<th>log_loss</th>\n<th>sample_weighted_log_loss</th>\n<th>accuracy</th>\n<th>precision</th>\n<th>recall</th>\n<th>specificity</th>\n<th>f1</th>\n<th>roc_auc</th>\n<th>average_precision</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>mean</td>\n<td>0.39403370027229895</td>\n<td>0.6088572296443904</td>\n<td>0.893971555932923</td>\n<td>0.15127715276303688</td>\n<td>0.16666666666666666</td>\n<td>1.0</td>\n<td>0.15857350589156527</td>\n<td>0.5520889782333832</td>\n<td>0.1335163957644465</td>\n</tr>\n</tbody>\n</table>\n<p>and these are my per target scores</p>\n<table>\n<thead>\n<tr>\n<th>target</th>\n<th>log_loss</th>\n<th>sample_weighted_log_loss</th>\n<th>accuracy</th>\n<th>precision</th>\n<th>recall</th>\n<th>specificity</th>\n<th>f1</th>\n<th>roc_auc</th>\n<th>average_precision</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>bowel_injury</td>\n<td>0.11192792245872818</td>\n<td>0.17707020379404662</td>\n<td>0.9779240076416896</td>\n<td>0.0</td>\n<td>0.0</td>\n<td>1.0</td>\n<td>0.0</td>\n<td>0.5657757425990549</td>\n<td>0.027722025805609667</td>\n</tr>\n<tr>\n<td>extravasation_injury</td>\n<td>0.40340889265515145</td>\n<td>0.5974000162252465</td>\n<td>0.9363192528125663</td>\n<td>0.0</td>\n<td>0.0</td>\n<td>1.0</td>\n<td>0.0</td>\n<td>0.5632324491800801</td>\n<td>0.07566879932761941</td>\n</tr>\n<tr>\n<td>any_injury</td>\n<td>0.5882653406365985</td>\n<td>0.9293227494263636</td>\n<td>0.7265973254086181</td>\n<td>0.0</td>\n<td>0.0</td>\n<td>1.0</td>\n<td>0.0</td>\n<td>0.5272587429210147</td>\n<td>0.2971583621601104</td>\n</tr>\n<tr>\n<td>kidney</td>\n<td>0.312776007936721</td>\n<td>0.5338212381898938</td>\n<td>0.939503290171938</td>\n<td>0.313167763390646</td>\n<td>0.3333333333333333</td>\n<td></td>\n<td>0.32293604757214256</td>\n<td></td>\n<td></td>\n</tr>\n<tr>\n<td>liver</td>\n<td>0.4177555413263836</td>\n<td>0.646181296215893</td>\n<td>0.8991721502865634</td>\n<td>0.2997240500955211</td>\n<td>0.3333333333333333</td>\n<td></td>\n<td>0.3156365262099028</td>\n<td></td>\n<td></td>\n</tr>\n<tr>\n<td>spleen</td>\n<td>0.5300684966202112</td>\n<td>0.7693478740148988</td>\n<td>0.8843133092761621</td>\n<td>0.29477110309205407</td>\n<td>0.3333333333333333</td>\n<td></td>\n<td>0.3128684615673463</td>\n<td></td>\n<td></td>\n</tr>\n</tbody>\n</table>\n<p>Does your validation score have any correlation with LB so far?</p>",
      "rawMarkdown": "I worked on my pipeline for 3 days and the result was disappointment.\n\nThese are my OOF scores averaged across targets. My OOF mean sample weighted log loss is 0.5447 and this model scored 0.74 on public LB.\nMy current approach is 2D and I'm using 5 fold cross-validation.\n\n|aggregation|log_loss           |sample_weighted_log_loss|accuracy         |precision          |recall             |specificity|f1                 |roc_auc           |average_precision |\n|-----------|-------------------|------------------------|-----------------|-------------------|-------------------|-----------|-------------------|------------------|------------------|\n|mean       |0.39403370027229895|0.6088572296443904      |0.893971555932923|0.15127715276303688|0.16666666666666666|1.0        |0.15857350589156527|0.5520889782333832|0.1335163957644465|\n\n\nand these are my per target scores\n\n\n|target              |log_loss           |sample_weighted_log_loss|accuracy          |precision          |recall            |specificity|f1                 |roc_auc           |average_precision   |\n|--------------------|-------------------|------------------------|------------------|-------------------|------------------|-----------|-------------------|------------------|--------------------|\n|bowel_injury        |0.11192792245872818|0.17707020379404662     |0.9779240076416896|0.0                |0.0               |1.0        |0.0                |0.5657757425990549|0.027722025805609667|\n|extravasation_injury|0.40340889265515145|0.5974000162252465      |0.9363192528125663|0.0                |0.0               |1.0        |0.0                |0.5632324491800801|0.07566879932761941 |\n|any_injury          |0.5882653406365985 |0.9293227494263636      |0.7265973254086181|0.0                |0.0               |1.0        |0.0                |0.5272587429210147|0.2971583621601104  |\n|kidney              |0.312776007936721  |0.5338212381898938      |0.939503290171938 |0.313167763390646  |0.3333333333333333|           |0.32293604757214256|                  |                    |\n|liver               |0.4177555413263836 |0.646181296215893       |0.8991721502865634|0.2997240500955211 |0.3333333333333333|           |0.3156365262099028 |                  |                    |\n|spleen              |0.5300684966202112 |0.7693478740148988      |0.8843133092761621|0.29477110309205407|0.3333333333333333|           |0.3128684615673463 |                  |                    |\n\n\nDoes your validation score have any correlation with LB so far?",
      "votes": 30
    },
    {
      "id": 2395294,
      "postDate": "2023-08-17T12:33:33.313Z",
      "content": "<p>I believe you still need to calculate any_injury score, and then calculate the mean of all 6, maybe that is why there is such a great gap between your calculated cv and lb</p>\n<p>I'm using the code from <a href=\"https://www.kaggle.com/code/metric/rsna-trauma-metric/notebook\" target=\"_blank\">here</a></p>",
      "rawMarkdown": "I believe you still need to calculate any_injury score, and then calculate the mean of all 6, maybe that is why there is such a great gap between your calculated cv and lb\n\nI'm using the code from [here](https://www.kaggle.com/code/metric/rsna-trauma-metric/notebook)",
      "votes": 5,
      "replies": [
        {
          "id": 2412071,
          "postDate": "2023-08-28T04:41:39.680Z",
          "content": "<p>Yes, that was the problem. I'm updating my scores with any injury included.</p>",
          "rawMarkdown": "Yes, that was the problem. I'm updating my scores with any injury included.",
          "votes": 1
        }
      ]
    },
    {
      "id": 2412154,
      "postDate": "2023-08-28T05:48:34.697Z",
      "content": "<p>3d Model: CV: 0.40 LB:0.46 (post-processed) CV: 0.42 LB: 0.52<br>\n2.5d Model: CV: 0.39 LB:0.45 (post-processed) CV: 0.45 LB: 0.56<br>\nEnsemble of both: CV:0.38 LB:0.44 (post-processed) CV: 0.41 LB:0.52</p>",
      "rawMarkdown": "3d Model: CV: 0.40 LB:0.46 (post-processed) CV: 0.42 LB: 0.52\n2.5d Model: CV: 0.39 LB:0.45 (post-processed) CV: 0.45 LB: 0.56\nEnsemble of both: CV:0.38 LB:0.44 (post-processed) CV: 0.41 LB:0.52",
      "votes": 4,
      "replies": [
        {
          "id": 2412288,
          "postDate": "2023-08-28T07:38:55.810Z",
          "content": "<p>if I may ask, what are the post processing steps you used?</p>",
          "rawMarkdown": "if I may ask, what are the post processing steps you used?",
          "replies": [
            {
              "id": 2412381,
              "postDate": "2023-08-28T08:43:37.057Z",
              "content": "<p>similar to weighted mean baseline</p>",
              "rawMarkdown": "similar to weighted mean baseline",
              "votes": 1
            },
            {
              "id": 2413745,
              "postDate": "2023-08-29T06:17:41.140Z",
              "content": "<p>I tried scaling model predictions and normalizing probabilities to 1 but my score always got worse.</p>",
              "rawMarkdown": "I tried scaling model predictions and normalizing probabilities to 1 but my score always got worse."
            },
            {
              "id": 2413953,
              "postDate": "2023-08-29T09:03:07.123Z",
              "content": "<p>I don't think you need to normalize after scaling, the hidden scoring mechanism will take care of that, see <a href=\"https://www.kaggle.com/code/metric/rsna-trauma-metric/notebook\" target=\"_blank\">https://www.kaggle.com/code/metric/rsna-trauma-metric/notebook</a></p>",
              "rawMarkdown": "I don't think you need to normalize after scaling, the hidden scoring mechanism will take care of that, see https://www.kaggle.com/code/metric/rsna-trauma-metric/notebook"
            },
            {
              "id": 2413977,
              "postDate": "2023-08-29T09:27:11.153Z",
              "content": "<p>Yeah, it's not needed while submitting. I was doing that for calculating my validation score properly. </p>",
              "rawMarkdown": "Yeah, it's not needed while submitting. I was doing that for calculating my validation score properly. "
            },
            {
              "id": 2413992,
              "postDate": "2023-08-29T09:38:55.550Z",
              "content": "<p>Then maybe you have to tune the scaling factors, in my experiments, 3d models need less scaling than 2.5d</p>",
              "rawMarkdown": "Then maybe you have to tune the scaling factors, in my experiments, 3d models need less scaling than 2.5d"
            },
            {
              "id": 2433743,
              "postDate": "2023-09-11T19:37:14.873Z",
              "content": "<p>Are you scaling output logits directly, or activated with softmax?</p>",
              "rawMarkdown": "Are you scaling output logits directly, or activated with softmax?"
            },
            {
              "id": 2463919,
              "postDate": "2023-10-01T17:26:36.717Z",
              "rawMarkdown": "",
              "isDeleted": true
            },
            {
              "id": 2463921,
              "postDate": "2023-10-01T17:27:35.953Z",
              "rawMarkdown": "",
              "isDeleted": true
            },
            {
              "id": 2463922,
              "postDate": "2023-10-01T17:28:11.163Z",
              "rawMarkdown": "",
              "isDeleted": true
            },
            {
              "id": 2463926,
              "postDate": "2023-10-01T17:32:02.993Z",
              "content": "<p>I have the same problem. How did you solve it</p>",
              "rawMarkdown": "I have the same problem. How did you solve it"
            }
          ]
        },
        {
          "id": 2419709,
          "postDate": "2023-09-02T07:04:09.557Z",
          "content": "<blockquote>\n  <p>3d Model: CV: 0.40 LB:0.46 (post-processed) CV: 0.42 LB: 0.52<br>\n  2.5d Model: CV: 0.39 LB:0.45 (post-processed) CV: 0.45 LB: 0.56<br>\n  Ensemble of both: CV:0.38 LB:0.44 (post-processed) CV: 0.41 LB:0.52</p>\n</blockquote>\n<p><a href=\"https://www.kaggle.com/fengqilong\" target=\"_blank\">@fengqilong</a> what do you mean by 2.5d model?</p>",
          "rawMarkdown": "> 3d Model: CV: 0.40 LB:0.46 (post-processed) CV: 0.42 LB: 0.52\n> 2.5d Model: CV: 0.39 LB:0.45 (post-processed) CV: 0.45 LB: 0.56\n> Ensemble of both: CV:0.38 LB:0.44 (post-processed) CV: 0.41 LB:0.52\n\n @fengqilong what do you mean by 2.5d model?\n",
          "replies": [
            {
              "id": 2419767,
              "postDate": "2023-09-02T08:00:50.510Z",
              "content": "<p>2d cnn on slices then aggregate</p>",
              "rawMarkdown": "2d cnn on slices then aggregate"
            },
            {
              "id": 2420722,
              "postDate": "2023-09-02T20:38:10.737Z",
              "content": "<p>Thank for your answer. <br>\nI'm new to this kind of data. I'd like to ask some questions. Regarding 3D model approach:</p>\n<ol>\n<li>do you stack slices as input of 3D model?</li>\n<li>Each patient ID should have hundreds of dicom images, how do you stack them?</li>\n</ol>",
              "rawMarkdown": "Thank for your answer. \nI'm new to this kind of data. I'd like to ask some questions. Regarding 3D model approach:\n1. do you stack slices as input of 3D model?\n2. Each patient ID should have hundreds of dicom images, how do you stack them?"
            },
            {
              "id": 2420808,
              "postDate": "2023-09-02T23:46:54.803Z",
              "content": "<p>Do you mean that you use 2D CNN as feature extractor, then aggregate all CNN feature you got from all of slices of a patient ID into one tensor?  <br>\nCan you elaborate how do you aggregate?  </p>",
              "rawMarkdown": "Do you mean that you use 2D CNN as feature extractor, then aggregate all CNN feature you got from all of slices of a patient ID into one tensor?  \nCan you elaborate how do you aggregate?  "
            }
          ]
        },
        {
          "id": 2419756,
          "postDate": "2023-09-02T07:58:38.303Z",
          "content": "<p>Are you using K folds or are these single fold results?</p>",
          "rawMarkdown": "Are you using K folds or are these single fold results?",
          "replies": [
            {
              "id": 2419764,
              "postDate": "2023-09-02T08:00:31.370Z",
              "content": "<p>4 folds cv</p>",
              "rawMarkdown": "4 folds cv"
            }
          ]
        },
        {
          "id": 2421036,
          "postDate": "2023-09-03T05:49:11.047Z",
          "content": "<p>could you provide any reference notebook link</p>",
          "rawMarkdown": "could you provide any reference notebook link"
        },
        {
          "id": 2422813,
          "postDate": "2023-09-04T09:04:28.560Z",
          "content": "<p>to understand 2.5/3D model any reference notebook</p>",
          "rawMarkdown": "to understand 2.5/3D model any reference notebook"
        }
      ]
    },
    {
      "id": 2444530,
      "postDate": "2023-09-18T10:13:46.030Z",
      "content": "<p>CV 0.407<br>\nLB 0.5 (only fold 0)</p>\n<p>I use the preprocessed data from <a href=\"https://www.kaggle.com/code/theoviel/get-started-quicker-dicom-png-conversion\" target=\"_blank\">here</a> :)</p>",
      "rawMarkdown": "CV 0.407\nLB 0.5 (only fold 0)\n\nI use the preprocessed data from [here](https://www.kaggle.com/code/theoviel/get-started-quicker-dicom-png-conversion) :)",
      "votes": 4,
      "replies": [
        {
          "id": 2444571,
          "postDate": "2023-09-18T10:48:05.270Z",
          "content": "<p>Damn you have a good model. I think your model and my preprocessing could reach to 0.25 :D</p>",
          "rawMarkdown": "Damn you have a good model. I think your model and my preprocessing could reach to 0.25 :D",
          "replies": [
            {
              "id": 2444685,
              "postDate": "2023-09-18T12:03:26.400Z",
              "content": "<p>Not sure my preprocessing is the bottleneck. Maybe it's bad I don't know 😅</p>",
              "rawMarkdown": "Not sure my preprocessing is the bottleneck. Maybe it's bad I don't know 😅"
            },
            {
              "id": 2444935,
              "postDate": "2023-09-18T14:47:09.157Z",
              "content": "<p>im using your preprocessing method as well🤣</p>",
              "rawMarkdown": "im using your preprocessing method as well🤣",
              "votes": 1
            }
          ]
        },
        {
          "id": 2447415,
          "postDate": "2023-09-20T04:29:51.807Z",
          "content": "<p><a href=\"https://www.kaggle.com/theoviel\" target=\"_blank\">@theoviel</a> <br>\nDo you have code to generate Fold 0 split?<br>\ni want to sync my fold split to yours and check my results. Thanks a lot</p>",
          "rawMarkdown": "@theoviel \nDo you have code to generate Fold 0 split?\ni want to sync my fold split to yours and check my results. Thanks a lot",
          "votes": 1,
          "replies": [
            {
              "id": 2447772,
              "postDate": "2023-09-20T08:38:33.990Z",
              "content": "<pre><code> os\n numpy  np\n pandas  pd\n\n\n ():\n    \n    cols = [\n        , , ,\n        , , , , \n    ]\n\n    df = pd.read_csv(data_path + )\n\n     iterstrat.ml_stratifiers  MultilabelStratifiedKFold\n    mskf = MultilabelStratifiedKFold(n_splits=k, shuffle=, random_state=)\n    splits = mskf.split(df, y=df[cols])\n\n    df[] = -\n     i, (_, val_idx)  (splits):\n        df.loc[val_idx, ] = i\n\n    df_folds = df[[, ]]\n    df_folds.to_csv(data_path + , index=)\n     df_folds\n</code></pre>",
              "rawMarkdown": "```\nimport os\nimport numpy as np\nimport pandas as pd\n\n\ndef prepare_folds(data_path=\"../input/\", k=4):\n    \"\"\"\n    Prepare data folds for cross-validation.\n    MultilabelStratifiedKFold is used.\n\n    Args:\n        data_path (str, optional): Path to the data directory. Defaults to \"../input/\".\n        k (int, optional): Number of cross-validation folds. Defaults to 4.\n\n    Returns:\n        pandas DataFrame: DataFrame containing the patient IDs and their respective fold assignments.\n    \"\"\"\n    cols = [\n        'bowel_injury', 'extravasation_injury', 'kidney_low',\n        'kidney_high', 'liver_low', 'liver_high', 'spleen_low', 'spleen_high'\n    ]\n\n    df = pd.read_csv(data_path + \"train.csv\")\n\n    from iterstrat.ml_stratifiers import MultilabelStratifiedKFold\n    mskf = MultilabelStratifiedKFold(n_splits=k, shuffle=True, random_state=42)\n    splits = mskf.split(df, y=df[cols])\n\n    df['fold'] = -1\n    for i, (_, val_idx) in enumerate(splits):\n        df.loc[val_idx, \"fold\"] = i\n\n    df_folds = df[[\"patient_id\", \"fold\"]]\n    df_folds.to_csv(data_path + f\"folds_{k}.csv\", index=False)\n    return df_folds\n```",
              "votes": 5
            },
            {
              "id": 2449486,
              "postDate": "2023-09-21T07:52:32.560Z",
              "content": "<p>after this what we need to do</p>",
              "rawMarkdown": "after this what we need to do\n",
              "votes": -1
            }
          ]
        }
      ]
    },
    {
      "id": 2480231,
      "postDate": "2023-10-13T07:13:44.683Z",
      "content": "<p>CV 0.39 LB 0.49 😭</p>",
      "rawMarkdown": "CV 0.39 LB 0.49 😭",
      "votes": 2,
      "replies": [
        {
          "id": 2480260,
          "postDate": "2023-10-13T07:33:16.933Z",
          "content": "<p>The gap is wider than usual. What's your CV method?</p>",
          "rawMarkdown": "The gap is wider than usual. What's your CV method?",
          "votes": 1,
          "replies": [
            {
              "id": 2480290,
              "postDate": "2023-10-13T08:01:42.353Z",
              "content": "<p><a href=\"https://www.kaggle.com/theoviel\" target=\"_blank\">@theoviel</a> 's 4 folds</p>",
              "rawMarkdown": "@theoviel 's 4 folds"
            },
            {
              "id": 2480301,
              "postDate": "2023-10-13T08:16:39.240Z",
              "content": "<p>Looks like you have a leak or a bug. I'm at 0.43 with such CV. </p>",
              "rawMarkdown": "Looks like you have a leak or a bug. I'm at 0.43 with such CV. ",
              "votes": 1
            },
            {
              "id": 2480302,
              "postDate": "2023-10-13T08:17:16.997Z",
              "content": "<p>I use MultilabelStratifiedKFold too but with 5 folds and I'm also stratifying patient scan count. My CV/LB score gap is always 0.06 but I'm not sure if it means anything or not.</p>",
              "rawMarkdown": "I use MultilabelStratifiedKFold too but with 5 folds and I'm also stratifying patient scan count. My CV/LB score gap is always 0.06 but I'm not sure if it means anything or not.",
              "votes": 1
            },
            {
              "id": 2480335,
              "postDate": "2023-10-13T08:59:12.260Z",
              "content": "<p>probably both! 🤣</p>",
              "rawMarkdown": "probably both! 🤣"
            },
            {
              "id": 2480403,
              "postDate": "2023-10-13T09:48:55.673Z",
              "content": "<p>Any post-processing? </p>",
              "rawMarkdown": "Any post-processing? "
            },
            {
              "id": 2480421,
              "postDate": "2023-10-13T09:59:36.067Z",
              "content": "<p>no post processing, only end to end training (with multiple stages -&gt; leaky?)</p>",
              "rawMarkdown": "no post processing, only end to end training (with multiple stages -> leaky?)",
              "votes": 1
            }
          ]
        }
      ]
    },
    {
      "id": 2432969,
      "postDate": "2023-09-11T08:41:45.467Z",
      "content": "<p>OOF CV: 0.362<br>\nFold 0 LB: 0.43<br>\n4 Folds LB: 0.39</p>",
      "rawMarkdown": "OOF CV: 0.362\nFold 0 LB: 0.43\n4 Folds LB: 0.39",
      "votes": 2,
      "replies": [
        {
          "id": 2433015,
          "postDate": "2023-09-11T09:31:50.437Z",
          "content": "<p>do you have code to generate Fold 0 split?<br>\ni want to sync my fold split to yours</p>",
          "rawMarkdown": "do you have code to generate Fold 0 split?\ni want to sync my fold split to yours\n",
          "votes": 1,
          "replies": [
            {
              "id": 2433100,
              "postDate": "2023-09-11T10:49:20.767Z",
              "content": "<p>You can find it <a href=\"https://www.kaggle.com/datasets/harshitsheoran/rsna-abd-2023-fold-info\" target=\"_blank\">here</a></p>",
              "rawMarkdown": "You can find it [here](https://www.kaggle.com/datasets/harshitsheoran/rsna-abd-2023-fold-info)",
              "votes": 2
            },
            {
              "id": 2433178,
              "postDate": "2023-09-11T11:40:46.267Z",
              "content": "<p>thanks a lot!</p>",
              "rawMarkdown": "thanks a lot!\n"
            }
          ]
        }
      ]
    },
    {
      "id": 2411947,
      "postDate": "2023-08-28T01:32:18.907Z",
      "content": "<p>I use simple 3d method, CV 0.55 and LB 0.62.<br>\nupdate: 2.5d model, CV 0.45, LB 0.51</p>",
      "rawMarkdown": "I use simple 3d method, CV 0.55 and LB 0.62.\nupdate: 2.5d model, CV 0.45, LB 0.51",
      "votes": 2,
      "replies": [
        {
          "id": 2412080,
          "postDate": "2023-08-28T04:48:44.323Z",
          "content": "<p>My final correctly calculated score was CV 0.6088 LB 0.74. You are the only one shared so I have to compare it with you. I guess there is a little bit correlation between our scores. If you don't mind can you share per target scores as well?</p>",
          "rawMarkdown": "My final correctly calculated score was CV 0.6088 LB 0.74. You are the only one shared so I have to compare it with you. I guess there is a little bit correlation between our scores. If you don't mind can you share per target scores as well?",
          "votes": 1,
          "replies": [
            {
              "id": 2412238,
              "postDate": "2023-08-28T06:58:23.390Z",
              "content": "<p>Of course, my score is still very weak and close to weight mean prediction.</p>\n<pre><code> .\n .\n .\n .\n .\n .\n: .\n</code></pre>",
              "rawMarkdown": "Of course, my score is still very weak and close to weight mean prediction.\n```\nbowel 0.23250129345708362\nextravasation 0.6069890864485395\nkidney 0.4289066552481692\nliver 0.529219169352519\nspleen 0.6563071886001313\nany_injury_weight 0.8416417573071429\nloss_score: 0.5492608584022642\n```",
              "votes": 1
            },
            {
              "id": 2412244,
              "postDate": "2023-08-28T07:02:12.720Z",
              "content": "<p>It looks like I can get slightly better scores on bowel and extravasation by using middle slice only but your kidney, liver and spleen scores are way better than mine.</p>",
              "rawMarkdown": "It looks like I can get slightly better scores on bowel and extravasation by using middle slice only but your kidney, liver and spleen scores are way better than mine."
            },
            {
              "id": 2412277,
              "postDate": "2023-08-28T07:27:15.803Z",
              "content": "<p>Our score seems too weak,  I got some information from public notebook:<br>\n| Training score with better scaling: 0.5793620565075116<br>\nI think we should try to get more information from data first.</p>",
              "rawMarkdown": "Our score seems too weak,  I got some information from public notebook:\n| Training score with better scaling: 0.5793620565075116\nI think we should try to get more information from data first.",
              "votes": 2
            },
            {
              "id": 2413306,
              "postDate": "2023-08-28T19:24:06.923Z",
              "content": "<p>I revisited that notebook after your comment and that score isn't quite right. They are also rescaling any_injury predictions but it can't be done on submissions. Score should be calculated after creating any_injury predictions like this</p>\n<pre><code>df[] = ( - df[[\n    , ,\n    , , \n]]).(axis=)\n</code></pre>\n<p>and I could get 0.609068 validation score after scaling other targets.</p>",
              "rawMarkdown": "I revisited that notebook after your comment and that score isn't quite right. They are also rescaling any_injury predictions but it can't be done on submissions. Score should be calculated after creating any_injury predictions like this\n```python\ndf['any_injury_prediction'] = (1 - df[[\n    'bowel_healthy_prediction', 'extravasation_healthy_prediction',\n    'kidney_healthy_prediction', 'liver_healthy_prediction', 'spleen_healthy_prediction'\n]]).max(axis=1)\n```\nand I could get 0.609068 validation score after scaling other targets.",
              "votes": 1
            },
            {
              "id": 2413315,
              "postDate": "2023-08-28T19:29:51.123Z",
              "content": "<p>nvm I also reached 0.579362 after scaling more lol</p>",
              "rawMarkdown": "nvm I also reached 0.579362 after scaling more lol",
              "votes": 1
            }
          ]
        },
        {
          "id": 2414226,
          "postDate": "2023-08-29T13:38:46.037Z",
          "content": "<p>May I ask if you apply any preprocessing methods for that simple 3d method? I'm trying to use 3d model and the only preprocessing method I use is to resample every image to the same spacing. But my LB keeps oscillating between 0.8 - 1.1 and I'm wondering if I should modify the model or apply some preprocessing approaches (such as making separate prediction of images from different protocols)</p>",
          "rawMarkdown": "May I ask if you apply any preprocessing methods for that simple 3d method? I'm trying to use 3d model and the only preprocessing method I use is to resample every image to the same spacing. But my LB keeps oscillating between 0.8 - 1.1 and I'm wondering if I should modify the model or apply some preprocessing approaches (such as making separate prediction of images from different protocols)",
          "replies": [
            {
              "id": 2417957,
              "postDate": "2023-09-01T03:09:24.167Z",
              "content": "<p>0.8 - 1.1  is probably using whole slice.<br>\n0.4 is using cropped organ i think</p>\n<p>if you visualise the CT scan  (e.g. mean image in the xy, xz,yz plane) you will see 3d rotation.<br>\nhence if you are using whole CT scan, you may need to do 3d augnmentation (like 3d affine, rotate, etc).<br>\ni think this is too expensive.</p>\n<hr>\n<p>on a side note: if you check past rsna competitions, top winning solutions usually used cropped solution.</p>\n<hr>\n<p>if you are using crop, you need some method to find inter-relationship features later (like transformer with cropped part as input). this may improve improve score better than 0.4</p>",
              "rawMarkdown": "0.8 - 1.1  is probably using whole slice.\n0.4 is using cropped organ i think\n\nif you visualise the CT scan  (e.g. mean image in the xy, xz,yz plane) you will see 3d rotation.\nhence if you are using whole CT scan, you may need to do 3d augnmentation (like 3d affine, rotate, etc).\ni think this is too expensive.\n\n---\non a side note: if you check past rsna competitions, top winning solutions usually used cropped solution.\n\n---\n\nif you are using crop, you need some method to find inter-relationship features later (like transformer with cropped part as input). this may improve improve score better than 0.4",
              "votes": 3
            },
            {
              "id": 2417972,
              "postDate": "2023-09-01T03:23:19.757Z",
              "content": "<p>3d augmentation doesn't work well in my test. I think you are right. Cropped or segmented organs are required.</p>",
              "rawMarkdown": "3d augmentation doesn't work well in my test. I think you are right. Cropped or segmented organs are required."
            },
            {
              "id": 2417994,
              "postDate": "2023-09-01T04:05:43.730Z",
              "content": "<p>\"cropped\" can also mean masked attention or \"masked pooling\". i.e. not physically cropped the scan by extract the relevant voxels in computation.</p>",
              "rawMarkdown": "\"cropped\" can also mean masked attention or \"masked pooling\". i.e. not physically cropped the scan by extract the relevant voxels in computation.",
              "votes": 2
            }
          ]
        },
        {
          "id": 2421035,
          "postDate": "2023-09-03T05:48:25.387Z",
          "content": "<p>any reference notebook to understand 3d /2.5 method for using ..</p>",
          "rawMarkdown": "any reference notebook to understand 3d /2.5 method for using .."
        }
      ]
    },
    {
      "id": 2444546,
      "postDate": "2023-09-18T10:24:33.420Z",
      "content": "<p>from kaggle_helper import *<br>\nfrom kaggle_metric import *<br>\nwhile importing it is showing no kaggle_helper module <br>\ncan anyone help</p>",
      "rawMarkdown": "from kaggle_helper import *\nfrom kaggle_metric import *\nwhile importing it is showing no kaggle_helper module \ncan anyone help"
    },
    {
      "id": 2396492,
      "postDate": "2023-08-18T09:38:25.933Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 2395294,
      "author_name": "Feng Qilong",
      "author_url": "",
      "post_date": "2023-08-17T12:33:33.313000",
      "content": "<p>I believe you still need to calculate any_injury score, and then calculate the mean of all 6, maybe that is why there is such a great gap between your calculated cv and lb</p>\n<p>I'm using the code from <a href=\"https://www.kaggle.com/code/metric/rsna-trauma-metric/notebook\" target=\"_blank\">here</a></p>",
      "votes": 5,
      "replies": [
        {
          "id": 2412071,
          "author_name": "Gunes Evitan",
          "author_url": "",
          "post_date": "2023-08-28T04:41:39.680000",
          "content": "<p>Yes, that was the problem. I'm updating my scores with any injury included.</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 2412154,
      "author_name": "Feng Qilong",
      "author_url": "",
      "post_date": "2023-08-28T05:48:34.697000",
      "content": "<p>3d Model: CV: 0.40 LB:0.46 (post-processed) CV: 0.42 LB: 0.52<br>\n2.5d Model: CV: 0.39 LB:0.45 (post-processed) CV: 0.45 LB: 0.56<br>\nEnsemble of both: CV:0.38 LB:0.44 (post-processed) CV: 0.41 LB:0.52</p>",
      "votes": 4,
      "replies": [
        {
          "id": 2412288,
          "author_name": "Priya Nagda",
          "author_url": "",
          "post_date": "2023-08-28T07:38:55.810000",
          "content": "<p>if I may ask, what are the post processing steps you used?</p>",
          "votes": 0,
          "replies": [
            {
              "id": 2412381,
              "author_name": "Feng Qilong",
              "author_url": "",
              "post_date": "2023-08-28T08:43:37.057000",
              "content": "<p>similar to weighted mean baseline</p>",
              "votes": 1,
              "replies": []
            },
            {
              "id": 2413745,
              "author_name": "Gunes Evitan",
              "author_url": "",
              "post_date": "2023-08-29T06:17:41.140000",
              "content": "<p>I tried scaling model predictions and normalizing probabilities to 1 but my score always got worse.</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 2413953,
              "author_name": "Feng Qilong",
              "author_url": "",
              "post_date": "2023-08-29T09:03:07.123000",
              "content": "<p>I don't think you need to normalize after scaling, the hidden scoring mechanism will take care of that, see <a href=\"https://www.kaggle.com/code/metric/rsna-trauma-metric/notebook\" target=\"_blank\">https://www.kaggle.com/code/metric/rsna-trauma-metric/notebook</a></p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 2413977,
              "author_name": "Gunes Evitan",
              "author_url": "",
              "post_date": "2023-08-29T09:27:11.153000",
              "content": "<p>Yeah, it's not needed while submitting. I was doing that for calculating my validation score properly. </p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 2413992,
              "author_name": "Feng Qilong",
              "author_url": "",
              "post_date": "2023-08-29T09:38:55.550000",
              "content": "<p>Then maybe you have to tune the scaling factors, in my experiments, 3d models need less scaling than 2.5d</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 2433743,
              "author_name": "AnatoliK",
              "author_url": "",
              "post_date": "2023-09-11T19:37:14.873000",
              "content": "<p>Are you scaling output logits directly, or activated with softmax?</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 2463919,
              "author_name": "",
              "author_url": "",
              "post_date": "2023-10-01T17:26:36.717000",
              "content": "",
              "votes": 0,
              "replies": []
            },
            {
              "id": 2463921,
              "author_name": "",
              "author_url": "",
              "post_date": "2023-10-01T17:27:35.953000",
              "content": "",
              "votes": 0,
              "replies": []
            },
            {
              "id": 2463922,
              "author_name": "",
              "author_url": "",
              "post_date": "2023-10-01T17:28:11.163000",
              "content": "",
              "votes": 0,
              "replies": []
            },
            {
              "id": 2463926,
              "author_name": "qiucen",
              "author_url": "",
              "post_date": "2023-10-01T17:32:02.993000",
              "content": "<p>I have the same problem. How did you solve it</p>",
              "votes": 0,
              "replies": []
            }
          ]
        },
        {
          "id": 2419709,
          "author_name": "GO FOR IT",
          "author_url": "",
          "post_date": "2023-09-02T07:04:09.557000",
          "content": "<blockquote>\n  <p>3d Model: CV: 0.40 LB:0.46 (post-processed) CV: 0.42 LB: 0.52<br>\n  2.5d Model: CV: 0.39 LB:0.45 (post-processed) CV: 0.45 LB: 0.56<br>\n  Ensemble of both: CV:0.38 LB:0.44 (post-processed) CV: 0.41 LB:0.52</p>\n</blockquote>\n<p><a href=\"https://www.kaggle.com/fengqilong\" target=\"_blank\">@fengqilong</a> what do you mean by 2.5d model?</p>",
          "votes": 0,
          "replies": [
            {
              "id": 2419767,
              "author_name": "Feng Qilong",
              "author_url": "",
              "post_date": "2023-09-02T08:00:50.510000",
              "content": "<p>2d cnn on slices then aggregate</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 2420722,
              "author_name": "GO FOR IT",
              "author_url": "",
              "post_date": "2023-09-02T20:38:10.737000",
              "content": "<p>Thank for your answer. <br>\nI'm new to this kind of data. I'd like to ask some questions. Regarding 3D model approach:</p>\n<ol>\n<li>do you stack slices as input of 3D model?</li>\n<li>Each patient ID should have hundreds of dicom images, how do you stack them?</li>\n</ol>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 2420808,
              "author_name": "GO FOR IT",
              "author_url": "",
              "post_date": "2023-09-02T23:46:54.803000",
              "content": "<p>Do you mean that you use 2D CNN as feature extractor, then aggregate all CNN feature you got from all of slices of a patient ID into one tensor?  <br>\nCan you elaborate how do you aggregate?  </p>",
              "votes": 0,
              "replies": []
            }
          ]
        },
        {
          "id": 2419756,
          "author_name": "Harshit Sheoran",
          "author_url": "",
          "post_date": "2023-09-02T07:58:38.303000",
          "content": "<p>Are you using K folds or are these single fold results?</p>",
          "votes": 0,
          "replies": [
            {
              "id": 2419764,
              "author_name": "Feng Qilong",
              "author_url": "",
              "post_date": "2023-09-02T08:00:31.370000",
              "content": "<p>4 folds cv</p>",
              "votes": 0,
              "replies": []
            }
          ]
        },
        {
          "id": 2421036,
          "author_name": "Satheesh Bhukya",
          "author_url": "",
          "post_date": "2023-09-03T05:49:11.047000",
          "content": "<p>could you provide any reference notebook link</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 2422813,
          "author_name": "Satheesh Bhukya",
          "author_url": "",
          "post_date": "2023-09-04T09:04:28.560000",
          "content": "<p>to understand 2.5/3D model any reference notebook</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 2444530,
      "author_name": "Theo Viel",
      "author_url": "",
      "post_date": "2023-09-18T10:13:46.030000",
      "content": "<p>CV 0.407<br>\nLB 0.5 (only fold 0)</p>\n<p>I use the preprocessed data from <a href=\"https://www.kaggle.com/code/theoviel/get-started-quicker-dicom-png-conversion\" target=\"_blank\">here</a> :)</p>",
      "votes": 4,
      "replies": [
        {
          "id": 2444571,
          "author_name": "Gunes Evitan",
          "author_url": "",
          "post_date": "2023-09-18T10:48:05.270000",
          "content": "<p>Damn you have a good model. I think your model and my preprocessing could reach to 0.25 :D</p>",
          "votes": 0,
          "replies": [
            {
              "id": 2444685,
              "author_name": "Theo Viel",
              "author_url": "",
              "post_date": "2023-09-18T12:03:26.400000",
              "content": "<p>Not sure my preprocessing is the bottleneck. Maybe it's bad I don't know 😅</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 2444935,
              "author_name": "Feng Qilong",
              "author_url": "",
              "post_date": "2023-09-18T14:47:09.157000",
              "content": "<p>im using your preprocessing method as well🤣</p>",
              "votes": 1,
              "replies": []
            }
          ]
        },
        {
          "id": 2447415,
          "author_name": "Halley",
          "author_url": "",
          "post_date": "2023-09-20T04:29:51.807000",
          "content": "<p><a href=\"https://www.kaggle.com/theoviel\" target=\"_blank\">@theoviel</a> <br>\nDo you have code to generate Fold 0 split?<br>\ni want to sync my fold split to yours and check my results. Thanks a lot</p>",
          "votes": 1,
          "replies": [
            {
              "id": 2447772,
              "author_name": "Theo Viel",
              "author_url": "",
              "post_date": "2023-09-20T08:38:33.990000",
              "content": "<pre><code> os\n numpy  np\n pandas  pd\n\n\n ():\n    \n    cols = [\n        , , ,\n        , , , , \n    ]\n\n    df = pd.read_csv(data_path + )\n\n     iterstrat.ml_stratifiers  MultilabelStratifiedKFold\n    mskf = MultilabelStratifiedKFold(n_splits=k, shuffle=, random_state=)\n    splits = mskf.split(df, y=df[cols])\n\n    df[] = -\n     i, (_, val_idx)  (splits):\n        df.loc[val_idx, ] = i\n\n    df_folds = df[[, ]]\n    df_folds.to_csv(data_path + , index=)\n     df_folds\n</code></pre>",
              "votes": 5,
              "replies": []
            },
            {
              "id": 2449486,
              "author_name": "Satheesh Bhukya",
              "author_url": "",
              "post_date": "2023-09-21T07:52:32.560000",
              "content": "<p>after this what we need to do</p>",
              "votes": -1,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 2480231,
      "author_name": "Optimo",
      "author_url": "",
      "post_date": "2023-10-13T07:13:44.683000",
      "content": "<p>CV 0.39 LB 0.49 😭</p>",
      "votes": 2,
      "replies": [
        {
          "id": 2480260,
          "author_name": "Gunes Evitan",
          "author_url": "",
          "post_date": "2023-10-13T07:33:16.933000",
          "content": "<p>The gap is wider than usual. What's your CV method?</p>",
          "votes": 1,
          "replies": [
            {
              "id": 2480290,
              "author_name": "Optimo",
              "author_url": "",
              "post_date": "2023-10-13T08:01:42.353000",
              "content": "<p><a href=\"https://www.kaggle.com/theoviel\" target=\"_blank\">@theoviel</a> 's 4 folds</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 2480301,
              "author_name": "Theo Viel",
              "author_url": "",
              "post_date": "2023-10-13T08:16:39.240000",
              "content": "<p>Looks like you have a leak or a bug. I'm at 0.43 with such CV. </p>",
              "votes": 1,
              "replies": []
            },
            {
              "id": 2480302,
              "author_name": "Gunes Evitan",
              "author_url": "",
              "post_date": "2023-10-13T08:17:16.997000",
              "content": "<p>I use MultilabelStratifiedKFold too but with 5 folds and I'm also stratifying patient scan count. My CV/LB score gap is always 0.06 but I'm not sure if it means anything or not.</p>",
              "votes": 1,
              "replies": []
            },
            {
              "id": 2480335,
              "author_name": "Optimo",
              "author_url": "",
              "post_date": "2023-10-13T08:59:12.260000",
              "content": "<p>probably both! 🤣</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 2480403,
              "author_name": "Ivan Panshin",
              "author_url": "",
              "post_date": "2023-10-13T09:48:55.673000",
              "content": "<p>Any post-processing? </p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 2480421,
              "author_name": "Optimo",
              "author_url": "",
              "post_date": "2023-10-13T09:59:36.067000",
              "content": "<p>no post processing, only end to end training (with multiple stages -&gt; leaky?)</p>",
              "votes": 1,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 2432969,
      "author_name": "Harshit Sheoran",
      "author_url": "",
      "post_date": "2023-09-11T08:41:45.467000",
      "content": "<p>OOF CV: 0.362<br>\nFold 0 LB: 0.43<br>\n4 Folds LB: 0.39</p>",
      "votes": 2,
      "replies": [
        {
          "id": 2433015,
          "author_name": "hengck23",
          "author_url": "",
          "post_date": "2023-09-11T09:31:50.437000",
          "content": "<p>do you have code to generate Fold 0 split?<br>\ni want to sync my fold split to yours</p>",
          "votes": 1,
          "replies": [
            {
              "id": 2433100,
              "author_name": "Harshit Sheoran",
              "author_url": "",
              "post_date": "2023-09-11T10:49:20.767000",
              "content": "<p>You can find it <a href=\"https://www.kaggle.com/datasets/harshitsheoran/rsna-abd-2023-fold-info\" target=\"_blank\">here</a></p>",
              "votes": 2,
              "replies": []
            },
            {
              "id": 2433178,
              "author_name": "hengck23",
              "author_url": "",
              "post_date": "2023-09-11T11:40:46.267000",
              "content": "<p>thanks a lot!</p>",
              "votes": 0,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 2411947,
      "author_name": "sheep",
      "author_url": "",
      "post_date": "2023-08-28T01:32:18.907000",
      "content": "<p>I use simple 3d method, CV 0.55 and LB 0.62.<br>\nupdate: 2.5d model, CV 0.45, LB 0.51</p>",
      "votes": 2,
      "replies": [
        {
          "id": 2412080,
          "author_name": "Gunes Evitan",
          "author_url": "",
          "post_date": "2023-08-28T04:48:44.323000",
          "content": "<p>My final correctly calculated score was CV 0.6088 LB 0.74. You are the only one shared so I have to compare it with you. I guess there is a little bit correlation between our scores. If you don't mind can you share per target scores as well?</p>",
          "votes": 1,
          "replies": [
            {
              "id": 2412238,
              "author_name": "sheep",
              "author_url": "",
              "post_date": "2023-08-28T06:58:23.390000",
              "content": "<p>Of course, my score is still very weak and close to weight mean prediction.</p>\n<pre><code> .\n .\n .\n .\n .\n .\n: .\n</code></pre>",
              "votes": 1,
              "replies": []
            },
            {
              "id": 2412244,
              "author_name": "Gunes Evitan",
              "author_url": "",
              "post_date": "2023-08-28T07:02:12.720000",
              "content": "<p>It looks like I can get slightly better scores on bowel and extravasation by using middle slice only but your kidney, liver and spleen scores are way better than mine.</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 2412277,
              "author_name": "sheep",
              "author_url": "",
              "post_date": "2023-08-28T07:27:15.803000",
              "content": "<p>Our score seems too weak,  I got some information from public notebook:<br>\n| Training score with better scaling: 0.5793620565075116<br>\nI think we should try to get more information from data first.</p>",
              "votes": 2,
              "replies": []
            },
            {
              "id": 2413306,
              "author_name": "Gunes Evitan",
              "author_url": "",
              "post_date": "2023-08-28T19:24:06.923000",
              "content": "<p>I revisited that notebook after your comment and that score isn't quite right. They are also rescaling any_injury predictions but it can't be done on submissions. Score should be calculated after creating any_injury predictions like this</p>\n<pre><code>df[] = ( - df[[\n    , ,\n    , , \n]]).(axis=)\n</code></pre>\n<p>and I could get 0.609068 validation score after scaling other targets.</p>",
              "votes": 1,
              "replies": []
            },
            {
              "id": 2413315,
              "author_name": "Gunes Evitan",
              "author_url": "",
              "post_date": "2023-08-28T19:29:51.123000",
              "content": "<p>nvm I also reached 0.579362 after scaling more lol</p>",
              "votes": 1,
              "replies": []
            }
          ]
        },
        {
          "id": 2414226,
          "author_name": "NorthM344",
          "author_url": "",
          "post_date": "2023-08-29T13:38:46.037000",
          "content": "<p>May I ask if you apply any preprocessing methods for that simple 3d method? I'm trying to use 3d model and the only preprocessing method I use is to resample every image to the same spacing. But my LB keeps oscillating between 0.8 - 1.1 and I'm wondering if I should modify the model or apply some preprocessing approaches (such as making separate prediction of images from different protocols)</p>",
          "votes": 0,
          "replies": [
            {
              "id": 2417957,
              "author_name": "hengck23",
              "author_url": "",
              "post_date": "2023-09-01T03:09:24.167000",
              "content": "<p>0.8 - 1.1  is probably using whole slice.<br>\n0.4 is using cropped organ i think</p>\n<p>if you visualise the CT scan  (e.g. mean image in the xy, xz,yz plane) you will see 3d rotation.<br>\nhence if you are using whole CT scan, you may need to do 3d augnmentation (like 3d affine, rotate, etc).<br>\ni think this is too expensive.</p>\n<hr>\n<p>on a side note: if you check past rsna competitions, top winning solutions usually used cropped solution.</p>\n<hr>\n<p>if you are using crop, you need some method to find inter-relationship features later (like transformer with cropped part as input). this may improve improve score better than 0.4</p>",
              "votes": 3,
              "replies": []
            },
            {
              "id": 2417972,
              "author_name": "NorthM344",
              "author_url": "",
              "post_date": "2023-09-01T03:23:19.757000",
              "content": "<p>3d augmentation doesn't work well in my test. I think you are right. Cropped or segmented organs are required.</p>",
              "votes": 0,
              "replies": []
            },
            {
              "id": 2417994,
              "author_name": "hengck23",
              "author_url": "",
              "post_date": "2023-09-01T04:05:43.730000",
              "content": "<p>\"cropped\" can also mean masked attention or \"masked pooling\". i.e. not physically cropped the scan by extract the relevant voxels in computation.</p>",
              "votes": 2,
              "replies": []
            }
          ]
        },
        {
          "id": 2421035,
          "author_name": "Satheesh Bhukya",
          "author_url": "",
          "post_date": "2023-09-03T05:48:25.387000",
          "content": "<p>any reference notebook to understand 3d /2.5 method for using ..</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 2444546,
      "author_name": "Satheesh Bhukya",
      "author_url": "",
      "post_date": "2023-09-18T10:24:33.420000",
      "content": "<p>from kaggle_helper import *<br>\nfrom kaggle_metric import *<br>\nwhile importing it is showing no kaggle_helper module <br>\ncan anyone help</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2396492,
      "author_name": "",
      "author_url": "",
      "post_date": "2023-08-18T09:38:25.933000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2392644": "I worked on my pipeline for 3 days and the result was disappointment.\n\nThese are my OOF scores averaged across targets. My OOF mean sample weighted log loss is 0.5447 and this model scored 0.74 on public LB.\nMy current approach is 2D and I'm using 5 fold cross-validation.\n\n|aggregation|log_loss           |sample_weighted_log_loss|accuracy         |precision          |recall             |specificity|f1                 |roc_auc           |average_precision |\n|-----------|-------------------|------------------------|-----------------|-------------------|-------------------|-----------|-------------------|------------------|------------------|\n|mean       |0.39403370027229895|0.6088572296443904      |0.893971555932923|0.15127715276303688|0.16666666666666666|1.0        |0.15857350589156527|0.5520889782333832|0.1335163957644465|\n\n\nand these are my per target scores\n\n\n|target              |log_loss           |sample_weighted_log_loss|accuracy          |precision          |recall            |specificity|f1                 |roc_auc           |average_precision   |\n|--------------------|-------------------|------------------------|------------------|-------------------|------------------|-----------|-------------------|------------------|--------------------|\n|bowel_injury        |0.11192792245872818|0.17707020379404662     |0.9779240076416896|0.0                |0.0               |1.0        |0.0                |0.5657757425990549|0.027722025805609667|\n|extravasation_injury|0.40340889265515145|0.5974000162252465      |0.9363192528125663|0.0                |0.0               |1.0        |0.0                |0.5632324491800801|0.07566879932761941 |\n|any_injury          |0.5882653406365985 |0.9293227494263636      |0.7265973254086181|0.0                |0.0               |1.0        |0.0                |0.5272587429210147|0.2971583621601104  |\n|kidney              |0.312776007936721  |0.5338212381898938      |0.939503290171938 |0.313167763390646  |0.3333333333333333|           |0.32293604757214256|                  |                    |\n|liver               |0.4177555413263836 |0.646181296215893       |0.8991721502865634|0.2997240500955211 |0.3333333333333333|           |0.3156365262099028 |                  |                    |\n|spleen              |0.5300684966202112 |0.7693478740148988      |0.8843133092761621|0.29477110309205407|0.3333333333333333|           |0.3128684615673463 |                  |                    |\n\n\nDoes your validation score have any correlation with LB so far?",
    "2395294": "I believe you still need to calculate any_injury score, and then calculate the mean of all 6, maybe that is why there is such a great gap between your calculated cv and lb\n\nI'm using the code from [here](https://www.kaggle.com/code/metric/rsna-trauma-metric/notebook)",
    "2412154": "3d Model: CV: 0.40 LB:0.46 (post-processed) CV: 0.42 LB: 0.52\n2.5d Model: CV: 0.39 LB:0.45 (post-processed) CV: 0.45 LB: 0.56\nEnsemble of both: CV:0.38 LB:0.44 (post-processed) CV: 0.41 LB:0.52",
    "2444530": "CV 0.407\nLB 0.5 (only fold 0)\n\nI use the preprocessed data from [here](https://www.kaggle.com/code/theoviel/get-started-quicker-dicom-png-conversion) :)",
    "2480231": "CV 0.39 LB 0.49 😭",
    "2432969": "OOF CV: 0.362\nFold 0 LB: 0.43\n4 Folds LB: 0.39",
    "2411947": "I use simple 3d method, CV 0.55 and LB 0.62.\nupdate: 2.5d model, CV 0.45, LB 0.51",
    "2444546": "from kaggle_helper import *\nfrom kaggle_metric import *\nwhile importing it is showing no kaggle_helper module \ncan anyone help",
    "2396492": ""
  }
}