{
  "id": 391341,
  "title": "19th Place Solution",
  "url": "/competitions/rsna-breast-cancer-detection/discussion/391341",
  "author_name": "Ivan Aerlic",
  "post_date": "2023-03-01T05:47:47.588000",
  "votes": 16,
  "comment_count": 0,
  "views": 0,
  "content": "<p><strong>19th Place Solution</strong></p>\n<p>Thanks to the team at RSNA and Kaggle for putting this competition together, and thanks to my teammates <a href=\"https://www.kaggle.com/ragnar123\" target=\"_blank\">@ragnar123</a> and <a href=\"https://www.kaggle.com/harshitsheoran\" target=\"_blank\">@harshitsheoran</a>.</p>\n<p>The final submission consisted of 2 CNN models : eca_nfnet_l0 and tf_efficientnet_b3_ns.</p>\n<table>\n<thead>\n<tr>\n<th>Model</th>\n<th>CV</th>\n<th>Best Public</th>\n<th>Best Private</th>\n<th>Used in Ensemble</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>eca_nfnet_l0 @ 1536 (Harshit)</td>\n<td>.488</td>\n<td>.6</td>\n<td>.47</td>\n<td>0</td>\n</tr>\n<tr>\n<td>eca_nfnet_l0 @ 1536 (Ivan)</td>\n<td>.4688</td>\n<td>.61</td>\n<td>.46</td>\n<td>1</td>\n</tr>\n<tr>\n<td>tf_efficientnet_b3_ns @ 1536 (Ivan)</td>\n<td>.491</td>\n<td>.63</td>\n<td>.48</td>\n<td>1</td>\n</tr>\n<tr>\n<td>tf_efficientnet_b3_ns @ 1920 by 1536 (Martin)</td>\n<td>.464</td>\n<td>.6</td>\n<td>.51</td>\n<td>0</td>\n</tr>\n</tbody>\n</table>\n<p><br></p>\n<p>When combined, these two models had a best Public LB of .65 and a best Private LB of .5. Unfortunately there was no correlation between public and private or CV and private. This led to us picking the wrong submission. In the end we had 12 submissions that would have gotten us into gold but had no way of telling if they were the correct ones. </p>\n<p><strong>What Worked?</strong></p>\n<ul>\n<li>Label Smoothing</li>\n<li>Auxiliary Classes (Bi-raids 2, Benign, Invasive, Biopsy)</li>\n<li>Weighted BCELoss</li>\n<li>Mosaic (with class max as target)</li>\n<li>Mixup (with class max as target)</li>\n<li>ROI Cropping (during training)</li>\n</ul>\n<p>For inference we were able to run more folds over the test data by first running 2 folds, seeing if it scored above a threshold (~0.03), and only run the rest of the folds if it was above that threshold. That trick worked well and it allowed us to use more folds on samples that were relevant.</p>\n<p>Another trick was using the Auxiliary class \"Difficult Negative Case\" to smooth out the Cancer class by using it in a weighted average. This gave a slight improvement on the CV, but nothing game changing.</p>\n<p>Thanks for reading, and good luck on your future Kaggle endeavors 👍</p>",
  "messages": [
    {
      "id": 2163846,
      "postDate": "2023-03-01T05:47:47.590Z",
      "content": "<p><strong>19th Place Solution</strong></p>\n<p>Thanks to the team at RSNA and Kaggle for putting this competition together, and thanks to my teammates <a href=\"https://www.kaggle.com/ragnar123\" target=\"_blank\">@ragnar123</a> and <a href=\"https://www.kaggle.com/harshitsheoran\" target=\"_blank\">@harshitsheoran</a>.</p>\n<p>The final submission consisted of 2 CNN models : eca_nfnet_l0 and tf_efficientnet_b3_ns.</p>\n<table>\n<thead>\n<tr>\n<th>Model</th>\n<th>CV</th>\n<th>Best Public</th>\n<th>Best Private</th>\n<th>Used in Ensemble</th>\n</tr>\n</thead>\n<tbody>\n<tr>\n<td>eca_nfnet_l0 @ 1536 (Harshit)</td>\n<td>.488</td>\n<td>.6</td>\n<td>.47</td>\n<td>0</td>\n</tr>\n<tr>\n<td>eca_nfnet_l0 @ 1536 (Ivan)</td>\n<td>.4688</td>\n<td>.61</td>\n<td>.46</td>\n<td>1</td>\n</tr>\n<tr>\n<td>tf_efficientnet_b3_ns @ 1536 (Ivan)</td>\n<td>.491</td>\n<td>.63</td>\n<td>.48</td>\n<td>1</td>\n</tr>\n<tr>\n<td>tf_efficientnet_b3_ns @ 1920 by 1536 (Martin)</td>\n<td>.464</td>\n<td>.6</td>\n<td>.51</td>\n<td>0</td>\n</tr>\n</tbody>\n</table>\n<p><br></p>\n<p>When combined, these two models had a best Public LB of .65 and a best Private LB of .5. Unfortunately there was no correlation between public and private or CV and private. This led to us picking the wrong submission. In the end we had 12 submissions that would have gotten us into gold but had no way of telling if they were the correct ones. </p>\n<p><strong>What Worked?</strong></p>\n<ul>\n<li>Label Smoothing</li>\n<li>Auxiliary Classes (Bi-raids 2, Benign, Invasive, Biopsy)</li>\n<li>Weighted BCELoss</li>\n<li>Mosaic (with class max as target)</li>\n<li>Mixup (with class max as target)</li>\n<li>ROI Cropping (during training)</li>\n</ul>\n<p>For inference we were able to run more folds over the test data by first running 2 folds, seeing if it scored above a threshold (~0.03), and only run the rest of the folds if it was above that threshold. That trick worked well and it allowed us to use more folds on samples that were relevant.</p>\n<p>Another trick was using the Auxiliary class \"Difficult Negative Case\" to smooth out the Cancer class by using it in a weighted average. This gave a slight improvement on the CV, but nothing game changing.</p>\n<p>Thanks for reading, and good luck on your future Kaggle endeavors 👍</p>",
      "rawMarkdown": "**19th Place Solution**\n\nThanks to the team at RSNA and Kaggle for putting this competition together, and thanks to my teammates @ragnar123 and @harshitsheoran.\n\nThe final submission consisted of 2 CNN models : eca_nfnet_l0 and tf_efficientnet_b3_ns.\n\n\n\n|  Model | CV | Best Public | Best Private | Used in Ensemble\n| --- | --- |\n| eca_nfnet_l0 @ 1536 (Harshit) | .488 | .6 | .47 | 0\n| eca_nfnet_l0 @ 1536 (Ivan) | .4688 | .61 | .46 | 1\n| tf_efficientnet_b3_ns @ 1536 (Ivan) | .491 | .63 | .48 | 1\n| tf_efficientnet_b3_ns @ 1920 by 1536 (Martin)| .464 | .6 | .51 | 0\n\n<br>\n\nWhen combined, these two models had a best Public LB of .65 and a best Private LB of .5. Unfortunately there was no correlation between public and private or CV and private. This led to us picking the wrong submission. In the end we had 12 submissions that would have gotten us into gold but had no way of telling if they were the correct ones. \n\n**What Worked?**\n\n- Label Smoothing\n- Auxiliary Classes (Bi-raids 2, Benign, Invasive, Biopsy)\n- Weighted BCELoss\n- Mosaic (with class max as target)\n- Mixup (with class max as target)\n- ROI Cropping (during training)\n\nFor inference we were able to run more folds over the test data by first running 2 folds, seeing if it scored above a threshold (~0.03), and only run the rest of the folds if it was above that threshold. That trick worked well and it allowed us to use more folds on samples that were relevant.\n\nAnother trick was using the Auxiliary class \"Difficult Negative Case\" to smooth out the Cancer class by using it in a weighted average. This gave a slight improvement on the CV, but nothing game changing.\n\nThanks for reading, and good luck on your future Kaggle endeavors 👍\n\n\n\n",
      "votes": 16
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "2163846": "**19th Place Solution**\n\nThanks to the team at RSNA and Kaggle for putting this competition together, and thanks to my teammates @ragnar123 and @harshitsheoran.\n\nThe final submission consisted of 2 CNN models : eca_nfnet_l0 and tf_efficientnet_b3_ns.\n\n\n\n|  Model | CV | Best Public | Best Private | Used in Ensemble\n| --- | --- |\n| eca_nfnet_l0 @ 1536 (Harshit) | .488 | .6 | .47 | 0\n| eca_nfnet_l0 @ 1536 (Ivan) | .4688 | .61 | .46 | 1\n| tf_efficientnet_b3_ns @ 1536 (Ivan) | .491 | .63 | .48 | 1\n| tf_efficientnet_b3_ns @ 1920 by 1536 (Martin)| .464 | .6 | .51 | 0\n\n<br>\n\nWhen combined, these two models had a best Public LB of .65 and a best Private LB of .5. Unfortunately there was no correlation between public and private or CV and private. This led to us picking the wrong submission. In the end we had 12 submissions that would have gotten us into gold but had no way of telling if they were the correct ones. \n\n**What Worked?**\n\n- Label Smoothing\n- Auxiliary Classes (Bi-raids 2, Benign, Invasive, Biopsy)\n- Weighted BCELoss\n- Mosaic (with class max as target)\n- Mixup (with class max as target)\n- ROI Cropping (during training)\n\nFor inference we were able to run more folds over the test data by first running 2 folds, seeing if it scored above a threshold (~0.03), and only run the rest of the folds if it was above that threshold. That trick worked well and it allowed us to use more folds on samples that were relevant.\n\nAnother trick was using the Auxiliary class \"Difficult Negative Case\" to smooth out the Cancer class by using it in a weighted average. This gave a slight improvement on the CV, but nothing game changing.\n\nThanks for reading, and good luck on your future Kaggle endeavors 👍\n\n\n\n"
  }
}