\nArtifacts
\n
The following is a summary of the key discussions of this competition.
\nI tried to add in all of the most important information, let me know If there is something that I missed.
Enjoy!
\nHeng CherKeng recently posted his experimental results.
\nThe post includes experiments with the followings:
\nRadek Osmulski shared a post about converting DICOM files to PNGs which resulted in an impressive decrease in file size from 314.72 GB to 921 MB.
\nRadek Osmulski shared an incredible post with us about 6 Computer Vision tricks for faster training and better models!
\nThese tricks can significantly reduce training time and improve model performance! Check out the post for more details!
\nRadek Osmulski discussed some insights on reaching 0.42.
\nRadek's observations from reaching LB 0.42
\nHeng CherKeng is having the best time of his life on this competition.
\nDoing what i have undone and undoing what i have done - Heng CherKeng
More joy and fun on the full thread
\ntomoo inubushi shared some interesting probing results in this notebook.
\nAll of these assumptions are TRUE:
\nTheo Viel has started a task of converting the DICOM images to PNGs/JPGs and has made his code public.
\nCode: Dicom -> Resized PNG/JPG. You can regenerate the data with the parameters of your choice.
\nAwsaf shared a post here about implementing the Probabilistic F Score in Tensorflow, PyTorch, and Numpy.
\nTensorflow:
\n ():\n preds = tf.clip_by_value(preds, , )\n y_true_count = tf.reduce_sum(labels)\n ctp = tf.reduce_sum(preds[labels==])\n cfp = tf.reduce_sum(preds[labels==])\n beta_squared = beta * beta\n c_precision = ctp / (ctp + cfp)\n c_recall = ctp / y_true_count\n (c_precision > c_recall > ):\n result = ( + beta_squared) * (c_precision * c_recall) / (beta_squared * c_precision + c_recall)\n result\n :\n \nPyTorch / Numpy:
\n ():\n preds = preds.clip(, )\n y_true_count = labels.()\n ctp = preds[labels==].()\n cfp = preds[labels==].()\n beta_squared = beta * beta\n c_precision = ctp / (ctp + cfp)\n c_recall = ctp / y_true_count\n (c_precision > c_recall > ):\n result = ( + beta_squared) * (c_precision * c_recall) / (beta_squared * c_precision + c_recall)\n result\n :\n \nTheo Viel shared his thoughts on the competition and also discussed his recent success in achieving a LB 0.24.
\nFrom the post:
\nAlso: Tricking the metric
\nAlso: Inference code is here
Remek Kinas discovered a way to improve Dicom processing speed by 1.6-2x.
\nGPU / 500 images (Parallel - 2 jobs):
\npydicom -> 396.74 sec
dicomsdl -> 243.39 sec
The estimation is that now processing 32.000 photos is about: 4h30min (but could be wrong)
The speedup was found here
As it turns out there has been a new addition to accelerator in Kaggle: TPU-1VM (local TPU)
\n\n\nDifferent then the older (remote-TPU)
\n
GCS_PATH anymore and it doesn't need internet access.Notebooks
\ntrain: RSNA-BCD: EfficientNet [TF][TPU-1VM][Train]
\ninfer: RSNA-BCD: EfficientNet [TF][TPU-1VM][Infer]
Some insights about this notebook:
\nROI + Rectangle Image
\nUpsample Cancer
\nAugmentations:
\nAswaf created a notebook to show how to convert Grad-CAM to bbox.
\n@kaggleqrdl recently shared VinDr-Mammo, a dataset sized 337.8 GB 5000 patients with full-field digital mammography with 5000 patients
\nLink here
\nFeatures:
\n\n\nHe also added that he traded emails with whom he reasonably believe is the author of the dataset (Nguyễn Quý Hà), and he said it would be OK to use this data with this RSNA Kaggle competition.
\n
\n
gradientBoost shared with us a breakdown of the RSNA dataset, as well as a list of external datasets that can be used to train models.
\nDataset Breakdown
\nCancer Presence Breakdown (patients)
\nList of External Datasets
\nVladimir Slaykovskiy shared some ideas that worked for them to achieve a LB score of 0.27.
\nNotebooks:
\n\nDavid Roberts recently shared a notebook showing why one might consider using windowing.
\nFrom the post:
\nSince most DICOM files have ranges greater than 0-255 (8 bit), applying standard normalization techniques results in loss of information. Since there is no way around this loss, the best we can do is to pick the \"most valuable\" range of pixels to normalize. This is what \"windowing\" is .. and it's critical to human readers being able to interpret images.\nSergey Saharovskiy has brought up an interesting discussion about identifying artifacts or anomalies in the data in this post.
\nHe shows multiplke types of artifacts:
\nArtifacts
\n
Lines and black square
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6259210%2F55160814a6734a9e61b06ab0476c71e6%2FScreenshot%20from%202022-12-05%2016-22-23.png?generation=1670275398422817&alt=media\")
White-like
\n
takuoko shared a benchmark using the openmmlab tool which achieved a score of 0.20 on the leaderboard.
This baseline can be used and modified to fit your needs by simply changing the configuration file settings such as the backbone and augmentation settings.
train (9913 patients) / val (2000 patients)
Network: EfficinetNetB3
Image size: 224
LB: 0.20 | Val without thresholding: 0.148 | Val with thresholding: 0.299
Michał Choiński discussed a very important topic in their post: how the way of resizing images can impact the results severely.
\nFor example in the paper Effect of the Pixel Interpolation Method for Downsampling Medical Images on Deep Learning Accuracy the authors applied several interpolation methods on the Chest X-ray images and measured their impact on the results of the trained Deep Learning models.
\nAlthough in this case the differences were not that gigantic, some clear patterns were observed:
\nHe created several datasets that contain png images resized to 256x256 pixels with different interpolation methods (as compared to bilinear that is used in OpenCV).
\nYou can find them below and experiment further with other sizes or interpolation methods:
Generated by the notebooks:
\nRemek Kinas noticed that in images of breast cancer there is a large variation in the arrangement of the object, and sometimes objects occupy only a small part of the image.
\nTo make the most efficient use of such images, Remek Kinas suggested the use of ROI (Region of Interest) extraction instead of just resizing the image.
By converting (resizing) without ROI extraction we have a very inefficient use of the reduced image. Most of the picture is blank.
\nSolution:
\nResult on train DS:
\nIn this post moth discussed the paper Focal Loss for Dense Object Detection and implemented Focal Loss in PyTorch using BCEWithLogitsLoss.
\nFrom the post:
\nThe Focal Loss function addresses class imbalance during training in tasks like image classification. It applies a modulating term to the cross entropy loss in order to focus learning on hard misclassified examples.
\nHere is the PyTorch implementation of the Focal Loss for BCE:
criterion = nn.BCEWithLogitsLoss(reduction=)\nGAMMA = \nALPHA = \n\n epoch epochs:\n\n ...\n\n images, labels train_data_loader:\n\n images = torch.tensor(images, device=device)\n labels = torch.tensor(labels, device=device)\n optimizer.zero_grad()\n out = model(images)\n \n bce_loss = criterion(out, labels.unsqueeze())\n probas = torch.sigmoid(out)\n loss = torch.where(labels >= ,\n ALPHA * (-probas)**GAMMA * bce_loss,\n (-ALPHA) * probas**GAMMA * bce_loss)\n loss = loss.mean()\n loss.backward()\n \n optimizer.step()\n\n ...\nIn this post, Wongi Park suggests that for a Breast Cancer Detection Task the best loss function to use should be one related to F1 score.
\nProposed F1 Loss:
\n torch\n\n ():\n tp = torch.(torch.tensor(y_true*y_pred).(), )\n tn = torch.(torch.tensor((-y_true)*(-y_pred)).(), )\n fp = torch.(torch.tensor((-y_true)*y_pred).(), )\n fn = torch.(torch.tensor(y_true*(-y_pred)).(), )\n\n p = tp / (tp + fp + )\n r = tp / (tp + fn + )\n\n f1 = * p * r / (p + r + 1e - )\n f1 = torch.where(torch.isnan(f1), torch.zeros_like(f1), f1)\n - torch.mean(f1)\nF1 Evaluation Function:
\n ():\n y_true_count = \n ctp = \n cfp = \n idx ((labels)):\n prediction = ((predictions[idx], ), )\n (labels[idx]):\n y_true_count += \n ctp += prediction\n cfp += - prediction\n :\n cfp += prediction\n beta_squared = beta * beta\n c_precision = ctp / (ctp + cfp)\n c_recall = ctp / y_true_count\n (c_precision > c_recall > ):\n result = ( + beta_squared) * (c_precision * c_recall) / (beta_squared * c_precision + c_recall)\n result\n : \nImportant Comment (By moth):
\nIn this post Wongi Park discussed using two alternatives to the F1 loss proposed before: Dice Loss and Soft F1 Score.
\nDice Loss:
\n ():\n \n bce = F.binary_cross_entropy_with_logits(pred, target, reduction=)\n\n pred = torch.sigmoid(pred)\n intersection = (pred * target).(dim=(,))\n union = pred.(dim=(,)) + target.(dim=(,))\n\n \n dice = * (intersection + smooth) / (union + smooth)\n\n \n dice_loss = - dice\n\n \n loss = bce + dice_loss\n\n loss.(), dice.()\nSoft F1 Score:
\n ():\n \n\n y = tf.cast(y, tf.float32)\n y_hat = tf.cast(y_hat, tf.float32)\n tp = tf.reduce_sum(y_hat * y, axis=)\n fp = tf.reduce_sum(y_hat * ( - y), axis=)\n fn = tf.reduce_sum(( - y_hat) * y, axis=)\n soft_f1 = *tp / (*tp + fn + fp + )\n cost = - soft_f1 \n macro_cost = tf.reduce_mean(cost) \n\n macro_cost\n\n\nI just want to add on this topic that all those trick losses are rarely seen in winning solutions.
\n
moth suggested an alternative to reduce memory and avoid OOM errors when training on 1024 pixel images.
\nThis suggestion is to freeze layers and only train a few layers at a time, which will reduce memory usage. This can be done by setting the trainable argument to False for all layers except the ones you want to train. This will greatly reduce the memory and computational requirements of the model, allowing it to run even on relatively low-end hardware.
How to freeze layers:
\nDetermine how many layers your model have:
\n i,(name, param) ((model.named_parameters())):\n (i,name)\nThis will print the layers of your model.
\nThen determine how many layers you want to freeze. You can experiment by freezing 100% model, 50%, 33%, 20%, etc.
NUM_FROZEN_LAYERS = \n i,(name, param) ((model.named_parameters())[:NUM_FROZEN_LAYERS]):\n param.requires_grad = \nMartin Kovacevic Buvinic has been training models with image size 512 and 1024 and found some interesting results.
\nThe main idea was to train with smaller image size and then if we get good results we can use bigger image size, nevertheless this experiment suggest that results from using image size 512 will not always have the same behaviour for image size 1024.
\nRemek Kinas created a notebook to provide inspiration on how to use Ray for parallel processing of DICOM files. This will help to speed up the processing and enable multiple tasks to be completed at the same time.
\nThis topic has interesting discussion in the comments about multiple architectures and their performance. (Good Read!)
\n27770 has some unusual mammograms and made a notebook to investigate this issue.moth discussed how to handle class imbalance in computer vision.
\nImportant Answer (By CPMP):
\nA good discussion about the topic can be found here.
\n\nNote: It is a REALLY good discussion with many people from the top of Kaggle proposing ideas! (Check it out!)
\n
Heng CherKeng shared an insightful post discussing possible reasons why ensemble models may not be working.
\nFrom the post:
\nYou may need to realigned your predicted probability (calibration) if:
\nYou probably needs to think of how to
\nDavid Roberts discussed a notebook for removing letter markers from images in this post. Letter markers are present in many images and can be distracting or cause confusion when analyzing the images.
\nThe notebook is right here.
\nIn this post moth asked about the vertical line which appears in many images.
\nImportant Answer (By David Roberts):
\nI believe the vertical lines on some images is the shadow of the compression paddle the machine uses to compress the breast. Possibly they are from biopsy paddles .. I am not certain. But, I think it's a safe bet that all the anatomy will be on one side of the line.\ngradientBoost recently published a paper in the IEEE Transactions on Medical Imaging titled “Deep Neural Networks Improve Radiologists Performance in Breast Cancer Screening”. The paper discusses the efficacy of using deep neural networks in the field of breast cancer screening, and how they can improve radiologists’ performance.
\nThe paper also explores how deep neural networks can be used to increase the accuracy of screening and reduce false positives.
@kaggleqrdl shared a post on how to make 2x faster jpeg LOSSLESS decoding. By using a combination of libjpeg-turbo and libmozjpeg the decoding of large images can be sped up significantly.
\nThe example provided shows that libjpeg-turbo is faster than libmozjpeg, but libmozjpeg produces better quality images. Thus, a combination of the two can provide both speed and quality.
\nmoth asked about the possibility of training a model on 1024px images using Kaggle kernels. It has been found that this can be difficult due to Out of Memory (OOM) errors.
\nThere are some methods you can try to reduce the memory of your training:
\nMethods from the post:
\nThe Post Set-up:
\nArchitecture: EfficientNetB2.
\nImage resolution: 1024px.
\nTrain Batch Size: 8 images per batch.
\nNumber of Frozen Layers: 100 out of 300 (~33% of the model).
\nConsumed GPU's VRAM: 4.7 out of 15.9 GB.
\nTraining time for 1 EPOCH: 43k images on ~1:03 hrs.
\nGradient Accumulation: True. Every 4 steps.
The following is a summary of the key discussions of this competition.
\nI tried to add in all of the most important information, let me know If there is something that I missed.
Enjoy!
\nHeng CherKeng recently posted his experimental results.
\nThe post includes experiments with the followings:
\nRadek Osmulski shared a post about converting DICOM files to PNGs which resulted in an impressive decrease in file size from 314.72 GB to 921 MB.
\nRadek Osmulski shared an incredible post with us about 6 Computer Vision tricks for faster training and better models!
\nThese tricks can significantly reduce training time and improve model performance! Check out the post for more details!
\nRadek Osmulski discussed some insights on reaching 0.42.
\nRadek's observations from reaching LB 0.42
\nHeng CherKeng is having the best time of his life on this competition.
\nDoing what i have undone and undoing what i have done - Heng CherKeng
More joy and fun on the full thread
\ntomoo inubushi shared some interesting probing results in this notebook.
\nAll of these assumptions are TRUE:
\nTheo Viel has started a task of converting the DICOM images to PNGs/JPGs and has made his code public.
\nCode: Dicom -> Resized PNG/JPG. You can regenerate the data with the parameters of your choice.
\nAwsaf shared a post here about implementing the Probabilistic F Score in Tensorflow, PyTorch, and Numpy.
\nTensorflow:
\n ():\n preds = tf.clip_by_value(preds, , )\n y_true_count = tf.reduce_sum(labels)\n ctp = tf.reduce_sum(preds[labels==])\n cfp = tf.reduce_sum(preds[labels==])\n beta_squared = beta * beta\n c_precision = ctp / (ctp + cfp)\n c_recall = ctp / y_true_count\n (c_precision > c_recall > ):\n result = ( + beta_squared) * (c_precision * c_recall) / (beta_squared * c_precision + c_recall)\n result\n :\n \nPyTorch / Numpy:
\n ():\n preds = preds.clip(, )\n y_true_count = labels.()\n ctp = preds[labels==].()\n cfp = preds[labels==].()\n beta_squared = beta * beta\n c_precision = ctp / (ctp + cfp)\n c_recall = ctp / y_true_count\n (c_precision > c_recall > ):\n result = ( + beta_squared) * (c_precision * c_recall) / (beta_squared * c_precision + c_recall)\n result\n :\n \nTheo Viel shared his thoughts on the competition and also discussed his recent success in achieving a LB 0.24.
\nFrom the post:
\nAlso: Tricking the metric
\nAlso: Inference code is here
Remek Kinas discovered a way to improve Dicom processing speed by 1.6-2x.
\nGPU / 500 images (Parallel - 2 jobs):
\npydicom -> 396.74 sec
dicomsdl -> 243.39 sec
The estimation is that now processing 32.000 photos is about: 4h30min (but could be wrong)
The speedup was found here
As it turns out there has been a new addition to accelerator in Kaggle: TPU-1VM (local TPU)
\n\n\nDifferent then the older (remote-TPU)
\n
GCS_PATH anymore and it doesn't need internet access.Notebooks
\ntrain: RSNA-BCD: EfficientNet [TF][TPU-1VM][Train]
\ninfer: RSNA-BCD: EfficientNet [TF][TPU-1VM][Infer]
Some insights about this notebook:
\nROI + Rectangle Image
\nUpsample Cancer
\nAugmentations:
\nAswaf created a notebook to show how to convert Grad-CAM to bbox.
\n@kaggleqrdl recently shared VinDr-Mammo, a dataset sized 337.8 GB 5000 patients with full-field digital mammography with 5000 patients
\nLink here
\nFeatures:
\n\n\nHe also added that he traded emails with whom he reasonably believe is the author of the dataset (Nguyễn Quý Hà), and he said it would be OK to use this data with this RSNA Kaggle competition.
\n
\n
gradientBoost shared with us a breakdown of the RSNA dataset, as well as a list of external datasets that can be used to train models.
\nDataset Breakdown
\nCancer Presence Breakdown (patients)
\nList of External Datasets
\nVladimir Slaykovskiy shared some ideas that worked for them to achieve a LB score of 0.27.
\nNotebooks:
\n\nDavid Roberts recently shared a notebook showing why one might consider using windowing.
\nFrom the post:
\nSince most DICOM files have ranges greater than 0-255 (8 bit), applying standard normalization techniques results in loss of information. Since there is no way around this loss, the best we can do is to pick the \"most valuable\" range of pixels to normalize. This is what \"windowing\" is .. and it's critical to human readers being able to interpret images.\nSergey Saharovskiy has brought up an interesting discussion about identifying artifacts or anomalies in the data in this post.
\nHe shows multiplke types of artifacts:
\nArtifacts
\n
Lines and black square
\n
White-like
\n
takuoko shared a benchmark using the openmmlab tool which achieved a score of 0.20 on the leaderboard.
This baseline can be used and modified to fit your needs by simply changing the configuration file settings such as the backbone and augmentation settings.
train (9913 patients) / val (2000 patients)
Network: EfficinetNetB3
Image size: 224
LB: 0.20 | Val without thresholding: 0.148 | Val with thresholding: 0.299
Michał Choiński discussed a very important topic in their post: how the way of resizing images can impact the results severely.
\nFor example in the paper Effect of the Pixel Interpolation Method for Downsampling Medical Images on Deep Learning Accuracy the authors applied several interpolation methods on the Chest X-ray images and measured their impact on the results of the trained Deep Learning models.
\nAlthough in this case the differences were not that gigantic, some clear patterns were observed:
\nHe created several datasets that contain png images resized to 256x256 pixels with different interpolation methods (as compared to bilinear that is used in OpenCV).
\nYou can find them below and experiment further with other sizes or interpolation methods:
Generated by the notebooks:
\nRemek Kinas noticed that in images of breast cancer there is a large variation in the arrangement of the object, and sometimes objects occupy only a small part of the image.
\nTo make the most efficient use of such images, Remek Kinas suggested the use of ROI (Region of Interest) extraction instead of just resizing the image.
By converting (resizing) without ROI extraction we have a very inefficient use of the reduced image. Most of the picture is blank.
\nSolution:
\nResult on train DS:
\nIn this post moth discussed the paper Focal Loss for Dense Object Detection and implemented Focal Loss in PyTorch using BCEWithLogitsLoss.
\nFrom the post:
\nThe Focal Loss function addresses class imbalance during training in tasks like image classification. It applies a modulating term to the cross entropy loss in order to focus learning on hard misclassified examples.
\nHere is the PyTorch implementation of the Focal Loss for BCE:
criterion = nn.BCEWithLogitsLoss(reduction=)\nGAMMA = \nALPHA = \n\n epoch epochs:\n\n ...\n\n images, labels train_data_loader:\n\n images = torch.tensor(images, device=device)\n labels = torch.tensor(labels, device=device)\n optimizer.zero_grad()\n out = model(images)\n \n bce_loss = criterion(out, labels.unsqueeze())\n probas = torch.sigmoid(out)\n loss = torch.where(labels >= ,\n ALPHA * (-probas)**GAMMA * bce_loss,\n (-ALPHA) * probas**GAMMA * bce_loss)\n loss = loss.mean()\n loss.backward()\n \n optimizer.step()\n\n ...\nIn this post, Wongi Park suggests that for a Breast Cancer Detection Task the best loss function to use should be one related to F1 score.
\nProposed F1 Loss:
\n torch\n\n ():\n tp = torch.(torch.tensor(y_true*y_pred).(), )\n tn = torch.(torch.tensor((-y_true)*(-y_pred)).(), )\n fp = torch.(torch.tensor((-y_true)*y_pred).(), )\n fn = torch.(torch.tensor(y_true*(-y_pred)).(), )\n\n p = tp / (tp + fp + )\n r = tp / (tp + fn + )\n\n f1 = * p * r / (p + r + 1e - )\n f1 = torch.where(torch.isnan(f1), torch.zeros_like(f1), f1)\n - torch.mean(f1)\nF1 Evaluation Function:
\n ():\n y_true_count = \n ctp = \n cfp = \n idx ((labels)):\n prediction = ((predictions[idx], ), )\n (labels[idx]):\n y_true_count += \n ctp += prediction\n cfp += - prediction\n :\n cfp += prediction\n beta_squared = beta * beta\n c_precision = ctp / (ctp + cfp)\n c_recall = ctp / y_true_count\n (c_precision > c_recall > ):\n result = ( + beta_squared) * (c_precision * c_recall) / (beta_squared * c_precision + c_recall)\n result\n : \nImportant Comment (By moth):
\nIn this post Wongi Park discussed using two alternatives to the F1 loss proposed before: Dice Loss and Soft F1 Score.
\nDice Loss:
\n ():\n \n bce = F.binary_cross_entropy_with_logits(pred, target, reduction=)\n\n pred = torch.sigmoid(pred)\n intersection = (pred * target).(dim=(,))\n union = pred.(dim=(,)) + target.(dim=(,))\n\n \n dice = * (intersection + smooth) / (union + smooth)\n\n \n dice_loss = - dice\n\n \n loss = bce + dice_loss\n\n loss.(), dice.()\nSoft F1 Score:
\n ():\n \n\n y = tf.cast(y, tf.float32)\n y_hat = tf.cast(y_hat, tf.float32)\n tp = tf.reduce_sum(y_hat * y, axis=)\n fp = tf.reduce_sum(y_hat * ( - y), axis=)\n fn = tf.reduce_sum(( - y_hat) * y, axis=)\n soft_f1 = *tp / (*tp + fn + fp + )\n cost = - soft_f1 \n macro_cost = tf.reduce_mean(cost) \n\n macro_cost\n\n\nI just want to add on this topic that all those trick losses are rarely seen in winning solutions.
\n
moth suggested an alternative to reduce memory and avoid OOM errors when training on 1024 pixel images.
\nThis suggestion is to freeze layers and only train a few layers at a time, which will reduce memory usage. This can be done by setting the trainable argument to False for all layers except the ones you want to train. This will greatly reduce the memory and computational requirements of the model, allowing it to run even on relatively low-end hardware.
How to freeze layers:
\nDetermine how many layers your model have:
\n i,(name, param) ((model.named_parameters())):\n (i,name)\nThis will print the layers of your model.
\nThen determine how many layers you want to freeze. You can experiment by freezing 100% model, 50%, 33%, 20%, etc.
NUM_FROZEN_LAYERS = \n i,(name, param) ((model.named_parameters())[:NUM_FROZEN_LAYERS]):\n param.requires_grad = \nMartin Kovacevic Buvinic has been training models with image size 512 and 1024 and found some interesting results.
\nThe main idea was to train with smaller image size and then if we get good results we can use bigger image size, nevertheless this experiment suggest that results from using image size 512 will not always have the same behaviour for image size 1024.
\nRemek Kinas created a notebook to provide inspiration on how to use Ray for parallel processing of DICOM files. This will help to speed up the processing and enable multiple tasks to be completed at the same time.
\nThis topic has interesting discussion in the comments about multiple architectures and their performance. (Good Read!)
\n27770 has some unusual mammograms and made a notebook to investigate this issue.moth discussed how to handle class imbalance in computer vision.
\nImportant Answer (By CPMP):
\nA good discussion about the topic can be found here.
\n\nNote: It is a REALLY good discussion with many people from the top of Kaggle proposing ideas! (Check it out!)
\n
Heng CherKeng shared an insightful post discussing possible reasons why ensemble models may not be working.
\nFrom the post:
\nYou may need to realigned your predicted probability (calibration) if:
\nYou probably needs to think of how to
\nDavid Roberts discussed a notebook for removing letter markers from images in this post. Letter markers are present in many images and can be distracting or cause confusion when analyzing the images.
\nThe notebook is right here.
\nIn this post moth asked about the vertical line which appears in many images.
\nImportant Answer (By David Roberts):
\nI believe the vertical lines on some images is the shadow of the compression paddle the machine uses to compress the breast. Possibly they are from biopsy paddles .. I am not certain. But, I think it's a safe bet that all the anatomy will be on one side of the line.\ngradientBoost recently published a paper in the IEEE Transactions on Medical Imaging titled “Deep Neural Networks Improve Radiologists Performance in Breast Cancer Screening”. The paper discusses the efficacy of using deep neural networks in the field of breast cancer screening, and how they can improve radiologists’ performance.
\nThe paper also explores how deep neural networks can be used to increase the accuracy of screening and reduce false positives.
@kaggleqrdl shared a post on how to make 2x faster jpeg LOSSLESS decoding. By using a combination of libjpeg-turbo and libmozjpeg the decoding of large images can be sped up significantly.
\nThe example provided shows that libjpeg-turbo is faster than libmozjpeg, but libmozjpeg produces better quality images. Thus, a combination of the two can provide both speed and quality.
\nmoth asked about the possibility of training a model on 1024px images using Kaggle kernels. It has been found that this can be difficult due to Out of Memory (OOM) errors.
\nThere are some methods you can try to reduce the memory of your training:
\nMethods from the post:
\nThe Post Set-up:
\nArchitecture: EfficientNetB2.
\nImage resolution: 1024px.
\nTrain Batch Size: 8 images per batch.
\nNumber of Frozen Layers: 100 out of 300 (~33% of the model).
\nConsumed GPU's VRAM: 4.7 out of 15.9 GB.
\nTraining time for 1 EPOCH: 43k images on ~1:03 hrs.
\nGradient Accumulation: True. Every 4 steps.
Really good summary! Thank you so much 🙏
", "rawMarkdown": "Really good summary! Thank you so much 🙏" }, { "id": 2097413, "postDate": "2023-01-12T16:53:56.947Z", "content": "Thanks for sharing!
", "rawMarkdown": "Thanks for sharing!" } ], "comments": [ { "id": 2082634, "author_name": "MohamedAmine SAIGHI", "author_url": "", "post_date": "2023-01-01T18:43:48.380000", "content": "Really good summary! Thank you so much 🙏
", "votes": 0, "replies": [] }, { "id": 2097413, "author_name": "SemenB", "author_url": "", "post_date": "2023-01-12T16:53:56.947000", "content": "Thanks for sharing!
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2076135": "### Summary of key discussions\n##### Marking the first month of the competition\n\nThe following is a summary of the key discussions of this competition. \nI tried to add in all of the most important information, let me know If there is something that I missed.\n\nEnjoy! \n\n_____\n\n##### [placeholder] LB 0.51 my experimental results ([Heng CherKeng](https://www.kaggle.com/hengck23))\n\n[Heng CherKeng](https://www.kaggle.com/hengck23) recently posted [his experimental results](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/370333).\n\n**The post includes experiments with the followings:**\n- SEResnext\n- EfficinetNetB2\n- EfficinetNetB4\n- Different Thresholds\n\n_____\n\n##### 📸 DICOM files converted to PNGs [314.72 GB -> 921 MB] ([Radek Osmulski](https://www.kaggle.com/radek1))\n\n[Radek Osmulski](https://www.kaggle.com/radek1) shared a post about converting DICOM files to PNGs which resulted in an impressive decrease in file size from 314.72 GB to 921 MB.\n- This was accomplished by using dcm2png from the dcm2jpg package, which is an open-source tool used for converting images from the DICOM format to the PNG format.\n- Additionally, the post also provided some tips on how to use dcm2png, such as setting the correct parameters and the importance of having the correct order in the PNG files.\n_____\n\n##### exploit the metric \"bug\" ([Heng CherKeng](https://www.kaggle.com/hengck23))\n\n- [Heng CherKeng](https://www.kaggle.com/hengck23) discussed an interesting \"bug\" or opportunity to exploit the metric and boost the score of the model without actually improving the accuracy.\n- **The main idea** is to take advantage of the metric given and artificially increase the leaderboard score.\n- [Heng CherKeng](https://www.kaggle.com/hengck23) emphasized that no metric is perfect and this \"bug\" can be used to improve the score.\n\n_____\n\n##### 17x dicom decode speedup on GPU for jpeg2000 encodings ([David Austin](https://www.kaggle.com/tivfrvqhs5))\n\n- [David Austin](https://www.kaggle.com/tivfrvqhs5) has discovered a way to hack the bitstream and extract/save the jp2 images which can then be decoded on GPU using [Nvidia DALI](https://docs.nvidia.com/deeplearning/dali/user-guide/docs/index.html) for a 17x speedup.\n- This is useful as approximately half of the dicoms in the dataset contain jpeg2000 encoded images.\n- Check out the [post](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/371534) for more information.\n\n_____\n\n##### 💡 6 Computer Vision tricks for faster training and better models 🚀🚀🚀 ([Radek Osmulski](https://www.kaggle.com/radek1))\n\n[Radek Osmulski](https://www.kaggle.com/radek1) shared an incredible post with us about 6 Computer Vision tricks for faster training and better models! \n\n- **Tricks:**\n 1. Train with half-precision (FP16)\n 2. Progressive resizing (start with smaller resolution of train images)\n 3. Upsample the lower represented class to improve performance\n 4. Use an LR Scheduler\n 5. User LR warmup\n 6. Image augmentations\n\nThese tricks can significantly reduce training time and improve model performance! Check out the [post](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/369155) for more details!\n\n_____\n\n##### Thoughts on reaching LB 0.42 ([Radek Osmulski](https://www.kaggle.com/radek1))\n\n[Radek Osmulski](https://www.kaggle.com/radek1) discussed some insights on reaching 0.42.\n\n**Radek's observations from reaching LB 0.42**\n- It is best to approach this competition like any other computer vision project but with two important characteristics.\n- The dataset is imbalanced which makes training hard (plus some important features become discernable only at higher resolutions)\n- Because of the class imbalance (there being very few positive examples) the metric is very noisy\nShared his notebook [here](https://www.kaggle.com/code/radek1/fast-ai-starter-pack-train-inference)\n_____\n\n##### Life is hard as a kaggler ([Heng CherKeng](https://www.kaggle.com/hengck23))\n\n[Heng CherKeng](https://www.kaggle.com/hengck23) is having the best time of his life on this competition.\n\n`Doing what i have undone and undoing what i have done` - [Heng CherKeng](https://www.kaggle.com/hengck23)\n\nMore joy and fun on the [full thread](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/371267)\n\n_____\n\n##### Some LB probing results to share ([tomoo inubushi](https://www.kaggle.com/tomooinubushi))\n\n[tomoo inubushi](https://www.kaggle.com/tomooinubushi) shared some interesting probing results in [this notebook](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/370341).\n\n**All of these assumptions are TRUE:**\n\n- There are no new site ID in test dataset.\n- Patient IDs in train and test sets do not overlap\n- Image IDs in train and test sets do not overlap\n- There are no new laterality values in test dataset.\n- There are new machine IDs in test dataset. (This is already raised by @abebe9849 in here)\n- There are no new view values in test dataset.\n- No. of images/patient are all >= 4\n- Site ID is always the same for each patient.\n- Age is always the same for each patient.\n- There are no overlap of machine IDs between two sites in test dataset.\n\n_____\n\n##### More PNG/JPG Datasets to Get Started ([Theo Viel](https://www.kaggle.com/theoviel))\n\n[Theo Viel](https://www.kaggle.com/theoviel) has started a task of converting the DICOM images to PNGs/JPGs and has made his code public.\n\n**Code:** [Dicom -> Resized PNG/JPG](https://www.kaggle.com/code/theoviel/dicom-resized-png-jpg/). You can regenerate the data with the parameters of your choice.\n- [256x256 pngs](https://www.kaggle.com/datasets/theoviel/rsna-breast-cancer-256-pngs) - to train your first models, or if you don't have a lot of compute power\n- [512x512 pngs](https://www.kaggle.com/datasets/theoviel/rsna-breast-cancer-512-pngs) - to build more competitive models.\n- [768x768 pngs](https://www.kaggle.com/datasets/theoviel/rsna-breast-cancer-768-pngs) - perhaps bigger images are better\n- [1024x1024 pngs](https://www.kaggle.com/datasets/theoviel/rsna-breast-cancer-1024-pngs) - if you don't know what to do with your compute\n_____\n\n##### ⭐️ ROI extracted dataset - resolution 768pix and 1024pix⭐️ ([Remek Kinas](https://www.kaggle.com/remekkinas))\n\n- [Remek Kinas](https://www.kaggle.com/remekkinas) created [ROI extracted datasets](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/369754) for this competition.\n- They come in two resolutions: 768pix and 1024pix and could potentially **improve the score.**\n\n_____\n\n##### Competition Metric in Tensorflow, PyTorch & Numpy ([Awsaf](https://www.kaggle.com/awsaf49))\n\n[Awsaf](https://www.kaggle.com/awsaf49) shared a post [here](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/369267) about implementing the Probabilistic F Score in Tensorflow, PyTorch, and Numpy.\n- The post gave a comparison of the speed when using matrix operations instead of for-loops.\n- The post also included a [notebook](https://www.kaggle.com/code/sohier/probabilistic-f-score) with the speed comparison.\n\n**Tensorflow:**\n\n```python\ndef pfbeta_tf(labels, preds, beta=1):\n preds = tf.clip_by_value(preds, 0, 1)\n y_true_count = tf.reduce_sum(labels)\n ctp = tf.reduce_sum(preds[labels==1])\n cfp = tf.reduce_sum(preds[labels==0])\n beta_squared = beta * beta\n c_precision = ctp / (ctp + cfp)\n c_recall = ctp / y_true_count\n if (c_precision > 0 and c_recall > 0):\n result = (1 + beta_squared) * (c_precision * c_recall) / (beta_squared * c_precision + c_recall)\n return result\n else:\n return 0.0\n```\n\n**PyTorch / Numpy:**\n\n```python\ndef pfbeta_torch(labels, preds, beta=1):\n preds = preds.clip(0, 1)\n y_true_count = labels.sum()\n ctp = preds[labels==1].sum()\n cfp = preds[labels==0].sum()\n beta_squared = beta * beta\n c_precision = ctp / (ctp + cfp)\n c_recall = ctp / y_true_count\n if (c_precision > 0 and c_recall > 0):\n result = (1 + beta_squared) * (c_precision * c_recall) / (beta_squared * c_precision + c_recall)\n return result\n else:\n return 0.0\n```\n_____\n\n##### Some Remarks for Achieving LB 0.24 (Updated) ([Theo Viel](https://www.kaggle.com/theoviel))\n\n[Theo Viel](https://www.kaggle.com/theoviel) shared his thoughts on the competition and also discussed his recent success in achieving a LB 0.24.\n\n**From the post:**\n- The problem is tough, but there seems to be signal in the data. 0.15 LB scores are already good models.\n- The metric is hard to increase. A 0.75 AUC model will score about 0.08 pF1.\n- Inferring the test set is super long, because dicom processing takes 6+ hours. Time to look into GPU accelerated dicom readers\n- 512x512 (link) scores better than 256x256 but you can experiment with 256px. Use Breast ROI cropping if you want to save training time though.\n\n**Also:** [Tricking the metric](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/369886)\n**Also:** Inference code is [here](https://www.kaggle.com/code/theoviel/rsna-breast-baseline-inference)\n_____\n\n##### Fast dicom export and processing (1.6-2x faster) 💪💪 ([Remek Kinas](https://www.kaggle.com/remekkinas))\n\n[Remek Kinas](https://www.kaggle.com/remekkinas) discovered a way to improve Dicom processing speed by 1.6-2x.\n\n**GPU / 500 images (Parallel - 2 jobs):**\n\n- pydicom -> 396.74 sec\n- dicomsdl -> 243.39 sec\n\n- The estimation is that now processing 32.000 photos is about: 4h30min (but could be wrong)\n- The speedup was found [here](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/369684#2057282)\n_____\n\n##### Faster Dicom Processing on GPU ([Theo Viel](https://www.kaggle.com/theoviel))\n\n- [Theo Viel](https://www.kaggle.com/theoviel) has created an inference notebook which can process medical images in Dicom format on the GPU and runs in just 4 hours!\n- The notebook includes a few tricks to boost the speed of the model, such as adjusting the batch size, using multiple GPUs and data augmentation.\n- [Theo Viel](https://www.kaggle.com/theoviel) also provides some code snippets that can be used to better optimize the model.\n- This is an incredibly useful resource for anyone looking to speed up their medical imaging processing!\n\n_____\n\n##### [LB: 0.37] Tensorflow Baseline with TPU-1VM ([Awsaf](https://www.kaggle.com/awsaf49))\n\n**As it turns out there has been a new addition to accelerator in Kaggle: TPU-1VM (local TPU)**\n\n> Different then the older (remote-TPU)\n\n- It doesn't require `GCS_PATH` anymore and it doesn't need `internet access`.\n- Unlike remote-TPU its first epoch is **not** slow.\n- Aswaf published a notebook to show how to train in TPU-1VM, this notebook also supports multi-GPU training.\n\nNotebooks\ntrain: [RSNA-BCD: EfficientNet [TF][TPU-1VM][Train]](https://www.kaggle.com/code/awsaf49/rsna-bcd-efficientnet-tf-tpu-1vm-train)\ninfer: [RSNA-BCD: EfficientNet [TF][TPU-1VM][Infer]](https://www.kaggle.com/code/awsaf49/rsna-bcd-efficientnet-tf-tpu-1vm-infer)\n\n**Some insights about this notebook:**\n\n**ROI + Rectangle Image**\n- This notebook will ROI (Region Of Interest) image instead of full image. ROI is extracted using OpenCV (binary + max contour).\n- This notebook will use rectangle image (width!=height; height/width=2.0) training to avoid distortion.\n\n**Upsample Cancer**\n- Upsample the cancer data 10x to reduce class_imbalance effect on loss. By the way, from initial experiment it seems upsample does helps.\n\n**Augmentations:**\n- Random - ScaleShiftRotate\n- Random - Horizontal Flip\n- Random - Brightness, Contrast, Hue, Saturation\n- Coarse Dropout\n- MixUp - soon\n\n_____\n\n##### Classfication to Object Detection [GradCAM -> BBox] ([Awsaf](https://www.kaggle.com/awsaf49))\n\n- [Awsaf](https://www.kaggle.com/awsaf49) was wondering if it is possible to convert a cancer detection problem from a classification problem to an object detection problem.\n- The advantages of using object detection would be the ability to optimize more easily using bounding boxes, as well as being able to focus more on cancer pixels which are only a small portion of the image.\n- We do have access to class **activation map (Grad-CAM)** which shows parts of an input image that most impact the classification score. What if we convert the grad-cam to bounding box (bbox)?\n\n[Aswaf](https://www.kaggle.com/awsaf49) created a [notebook](https://www.kaggle.com/code/awsaf49/rsna-bcd-gradcam-to-bbox) to show how to convert Grad-CAM to bbox.\n\n_____\n\n##### Easy load the image with nvJPEG2000(5x faster) ([Chenglu](https://www.kaggle.com/snaker))\n\n- [Chenglu](https://www.kaggle.com/snaker) wrote a [notebook](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/372275) to demonstrate how to load the dicom file with nvJPEG2000, which is 5x faster than other methods.\n- See the [notebook](https://www.kaggle.com/code/snaker/easy-load-the-image-with-nvjpeg2000) for full details.\n\n_____\n\n##### VinDr-Mammo: 337.8 GB 5000 patients with full-field digital mammography and yolov5 models trained on it ([@kaggleqrdl](https://www.kaggle.com/kaggleqrdl))\n\n[@kaggleqrdl](https://www.kaggle.com/kaggleqrdl) recently shared [VinDr-Mammo](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/373211), a dataset sized 337.8 GB 5000 patients with full-field digital mammography with 5000 patients\n\nLink [here](https://physionet.org/content/vindr-mammo/1.0.0/)\n\n**Features:**\n- FFDM / Full Field Digital Mammography / similar to RSNA dataset\n- 5000 patients / 337.8 GB dataset\n- All 20K images are marked with density classification\n- Findings annotations with bounding boxes around various types of marked regions.\n- 241 of these findings are marked BIRADS 5 (very high probability of malignancy)\n- 995 of the findings are marked BIRADS 4 (about 30% chance of cancer)\n- Remainder of bbox findings are BIRADS 3, so 2254 images have been annotated\n\n> He also added that he traded emails with whom he reasonably believe is the author of the dataset (Nguyễn Quý Hà), and he said it would be OK to use this data with this RSNA Kaggle competition.\n_____\n\n##### RSNA Dataset Breakdown, Compiled list of External Datasets ([gradientBoost](https://www.kaggle.com/quarkspark))\n\n[gradientBoost](https://www.kaggle.com/quarkspark) shared with us a breakdown of the RSNA dataset, as well as a list of external datasets that can be used to train models.\n\n**Dataset Breakdown**\n- Total Patients:11913\n- Total Unique Healthy Patients:11427\n- Total Unique Cancer Patients:486\n- Total Healthy Mammography Images: 53548\n- Total Training images having Cancer: 1158\n\n**Cancer Presence Breakdown (patients)**\n- Left Breast Only:242\n- Right Breast Only:238\n- Both Breasts:6\n- Each patient has a total of 1-14 images in the training dataset\n\n**List of External Datasets**\n- [King Abdulaziz University Mammogram Dataset](https://www.kaggle.com/datasets/asmaasaad/king-abdulaziz-university-mammogram-dataset)\n- [vindr.ai Dataset [5000 Images]](https://physionet.org/content/vindr-mammo/1.0.0/) and [this](https://vindr.ai/datasets/mammo)\n- [CBIS-DDSM Dataset](https://www.kaggle.com/datasets/awsaf49/cbis-ddsm-breast-cancer-image-dataset)\n- [Mini MIAS Dataset](http://peipa.essex.ac.uk/info/mias.html)\n- [17 different databases are listed for breast body part](https://portal.imaging.datacommons.cancer.gov/explore/) [Commented by [kaggleqrdl](https://www.kaggle.com/kaggleqrdl))\n- [DBT volumes for 5,060 patients. (1 TB)](https://sites.duke.edu/mazurowski/resources/digital-breast-tomosynthesis-database/) [Commented by [Heng](https://www.kaggle.com/hengck23))\n\n_____\n\n##### [LB:0.27] Pytorch+EffNetV2 some working ideas ([Vladimir Slaykovskiy](https://www.kaggle.com/vslaykovsky))\n\n[Vladimir Slaykovskiy](https://www.kaggle.com/vslaykovsky) shared some ideas that worked for them to achieve a LB score of 0.27.\n- Both thresholding and weight-balancing improved my CV. \"cancer\" targets are highly imbalanced, so I use weighted loss to counteract (Positive weight ~50). Thresholding works well for F1-ish metrics, so optimized threshold with evaluation set.\n- Used this preprocessed dataset of 512x512 png images to speed up dataloader by ~10x with no performance degradation.\n- Used additional classification auxilliary targets ['site_id', 'laterality', 'view', 'implant', 'biopsy', 'invasive', 'BIRADS', 'density', 'difficult_negative_case', 'machine_id', 'age']. Auxilliary targets help to learn combined distribution of *.CSV + *.PNG data which helps to improve performance of the main classifier.\n\n**Notebooks:**\n- [Training](https://www.kaggle.com/vslaykovsky/train-effnetv2-aux-targets-weighted-loss-thres)\n- [Inference](https://www.kaggle.com/vslaykovsky/infer-effnetv2-aux-targets-weighted-loss-thres)\n\n_____\n\n##### Why use \"windowing\" on mammography images? ([David Roberts](https://www.kaggle.com/davidbroberts))\n\n[David Roberts](https://www.kaggle.com/davidbroberts) recently shared a [notebook](https://www.kaggle.com/davidbroberts/mammography-apply-windowing) showing why one might consider using windowing.\n\n**From the post:**\n```\nSince most DICOM files have ranges greater than 0-255 (8 bit), applying standard normalization techniques results in loss of information. Since there is no way around this loss, the best we can do is to pick the \"most valuable\" range of pixels to normalize. This is what \"windowing\" is .. and it's critical to human readers being able to interpret images.\n```\n_____\n\n##### Artifacts or Anomalies (Finding Hard Examples) ([Sergey Saharovskiy](https://www.kaggle.com/sergiosaharovskiy))\n\n[Sergey Saharovskiy](https://www.kaggle.com/sergiosaharovskiy) has brought up an interesting discussion about identifying artifacts or anomalies in the data in [this post](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/370639). \n\n**He shows multiplke types of artifacts:**\n**Artifacts**\n\n\n**Lines and black square**\n\n\n**White-like**\n\n\n_____\n\n##### mmclassification benchmark (LB=0.20) ([takuoko](https://www.kaggle.com/takuok))\n\n- [takuoko](https://www.kaggle.com/takuok) shared a [benchmark](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/370508) using the openmmlab tool which achieved a score of 0.20 on the leaderboard.\n- This baseline can be used and modified to fit your needs by simply changing the configuration file settings such as the backbone and augmentation settings.\n\n- [mmclassification](https://github.com/open-mmlab/mmclassification)\n- train (9913 patients) / val (2000 patients)\n- **Network:** EfficinetNetB3\n- **Image size:** 224\n- **LB:** 0.20 | **Val without thresholding:** 0.148 | **Val with thresholding:** 0.299\n\n_____\n\n##### The way how you resize your images impacts the results! ([Michał Choiński](https://www.kaggle.com/mikecho))\n\n[Michał Choiński](https://www.kaggle.com/mikecho) discussed a very important topic in their [post](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/371717): how the way of resizing images can impact the results severely.\n\nFor example in the paper [Effect of the Pixel Interpolation Method for Downsampling Medical Images on Deep Learning Accuracy](https://www.scirp.org/journal/paperinformation.aspx?paperid=113621) the authors applied several interpolation methods on the Chest X-ray images and measured their impact on the results of the trained Deep Learning models.\n\nAlthough in this case the differences were not that gigantic, some clear patterns were observed:\n\n- **lanczos interpolation** always led to the worst results\n- **nearest neighbour interpolation** always led to the best results\n\nHe created several datasets that contain png images resized to 256x256 pixels with different interpolation methods (as compared to bilinear that is used in OpenCV).\nYou can find them below and experiment further with other sizes or interpolation methods:\n\n- **[RSNA Breast Cancer Detection - PNG/256/nearest](https://www.kaggle.com/datasets/mikecho/rsna-breast-cancer-detection-png256nearest)**\n- **[RSNA Breast Cancer Detection - PNG/256/bicubic](https://www.kaggle.com/datasets/mikecho/rsna-breast-cancer-detection-png256bicubic)**\n- **[RSNA Breast Cancer Detection - PNG/256/lanczos4](https://www.kaggle.com/datasets/mikecho/rsna-breast-cancer-detection-png256lanczos4)**\n\n**Generated by the notebooks:**\n\n- [RSNA Breast Cancer Dicom -> PNG - nearest](https://www.kaggle.com/code/mikecho/rsna-breast-cancer-dicom-png-nearest)\n- [RSNA Breast Cancer Dicom -> PNG - bicubic](https://www.kaggle.com/code/mikecho/rsna-breast-cancer-dicom-png-bicubic)\n- [RSNA Breast Cancer Dicom -> PNG - lanczos4](https://www.kaggle.com/code/mikecho/rsna-breast-cancer-dicom-png-lanczos4)\n\n_____\n\n##### ⭐️⭐️ Breast Cancer - ROI (brest) extractor ⭐️⭐️ ([Remek Kinas](https://www.kaggle.com/remekkinas))\n[Remek Kinas](https://www.kaggle.com/remekkinas) noticed that in images of breast cancer there is a large variation in the arrangement of the object, and sometimes objects occupy only a small part of the image.\nTo make the most efficient use of such images, [Remek Kinas](https://www.kaggle.com/remekkinas) suggested the use of ROI (Region of Interest) extraction instead of just resizing the image.\n\nBy converting (resizing) without ROI extraction we have a very inefficient use of the reduced image. Most of the picture is blank.\n\n**Solution:**\n\n- **Data Annotation** - [Remek Kinas](https://www.kaggle.com/remekkinas) annotated about 500 images in a human in the loop technique (3 models were created - [Remek Kinas](https://www.kaggle.com/remekkinas) started from 300 images and ended up about 500)\n- **Object detector training** - [Remek Kinas](https://www.kaggle.com/remekkinas) used yolov5 (small - balance between accuracy and speed)\n\n**Result on train DS:**\n\n- 54601 images processed successfully\n- 105 images - detection failed\n\n_____\n\n##### [PyTorch] Focal Loss using BCEWithLogitsLoss ([moth](https://www.kaggle.com/alejopaullier))\n\nIn this post [moth](https://www.kaggle.com/alejopaullier) discussed the paper [Focal Loss for Dense Object Detection](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/372054) and implemented Focal Loss in PyTorch using BCEWithLogitsLoss.\n\n**From the post:**\n\nThe Focal Loss function addresses class imbalance during training in tasks like image classification. It applies a modulating term to the cross entropy loss in order to focus learning on hard misclassified examples.\nHere is the PyTorch implementation of the Focal Loss for BCE:\n\n\n```python\n\n\ncriterion = nn.BCEWithLogitsLoss(reduction='none')\nGAMMA = 2 # default value\nALPHA = 0.25 # default value\n\nfor epoch in epochs:\n\n ...\n\n for images, labels in train_data_loader:\n\n images = torch.tensor(images, device=device)\n labels = torch.tensor(labels, device=device)\n optimizer.zero_grad()\n out = model(images)\n # ===== FOCAL LOSS ======\n bce_loss = criterion(out, labels.unsqueeze(1))\n probas = torch.sigmoid(out)\n loss = torch.where(labels >= 0.5,\n ALPHA * (1-probas)**GAMMA * bce_loss,\n (1-ALPHA) * probas**GAMMA * bce_loss)\n loss = loss.mean()\n loss.backward()\n # ======= END ========\n optimizer.step()\n\n ...\n\n\n```\n\n\n_____\n\n\n##### What's Best loss function in Breast Cancer Detection Task? [personal opinion] ([Wongi Park](https://www.kaggle.com/kalelpark))\n\nIn this [post](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/369400), [Wongi Park](https://www.kaggle.com/kalelpark) suggests that for a Breast Cancer Detection Task the best loss function to use should be one related to F1 score.\n\n**Proposed F1 Loss:**\n\n```python\nimport torch\n\ndef f1_loss(y_true, y_pred):\n tp = torch.sum(torch.tensor(y_true*y_pred).float(), 0)\n tn = torch.sum(torch.tensor((1-y_true)*(1-y_pred)).float(), 0)\n fp = torch.sum(torch.tensor((1-y_true)*y_pred).float(), 0)\n fn = torch.sum(torch.tensor(y_true*(1-y_pred)).float(), 0)\n\n p = tp / (tp + fp + 1e-7)\n r = tp / (tp + fn + 1e-7)\n\n f1 = 2 * p * r / (p + r + 1e - 7)\n f1 = torch.where(torch.isnan(f1), torch.zeros_like(f1), f1)\n return 1 - torch.mean(f1)\n```\n\n**F1 Evaluation Function:**\n\n```python\n\ndef pfbeta(labels, predictions, beta):\n y_true_count = 0\n ctp = 0\n cfp = 0\n for idx in range(len(labels)):\n prediction = min(max(predictions[idx], 0), 1)\n if (labels[idx]):\n y_true_count += 1\n ctp += prediction\n cfp += 1 - prediction\n else:\n cfp += prediction\n beta_squared = beta * beta\n c_precision = ctp / (ctp + cfp)\n c_recall = ctp / y_true_count\n if (c_precision > 0 and c_recall > 0):\n result = (1 + beta_squared) * (c_precision * c_recall) / (beta_squared * c_precision + c_recall)\n return result\n else: return 0\n\n```\n\n**Important Comment (By [moth](https://www.kaggle.com/alejopaullier)):**\n- F1-score is that it is not differentiable, thus it cannot be used as a loss function to compute gradients and update the weights when training the model.\n- We can however use differentiable approximations that can be used as loss functions. Like [Dice Loss](https://datascience.stackexchange.com/questions/66581/is-it-possible-to-make-f1-score-differentiable-and-use-it-directly-as-a-loss-fun) and soft [F1 Loss](https://towardsdatascience.com/the-unknown-benefits-of-using-a-soft-f1-loss-in-classification-systems-753902c0105d)\n\n_____\n\n##### Recommend Loss function for F1_Score Task. It's mainly Medical Image Analysis. ([Wongi Park](https://www.kaggle.com/kalelpark))\n\nIn this post [Wongi Park](https://www.kaggle.com/kalelpark) discussed using two alternatives to the F1 loss proposed before: **Dice Loss** and **Soft F1 Score**.\n\n**Dice Loss:**\n\n```python\n\ndef dice_loss(pred, target, smooth = 1e-5):\n # binary cross entropy loss\n bce = F.binary_cross_entropy_with_logits(pred, target, reduction='sum')\n\n pred = torch.sigmoid(pred)\n intersection = (pred * target).sum(dim=(2,3))\n union = pred.sum(dim=(2,3)) + target.sum(dim=(2,3))\n\n # dice coefficient\n dice = 2.0 * (intersection + smooth) / (union + smooth)\n\n # dice loss\n dice_loss = 1.0 - dice\n\n # total loss\n loss = bce + dice_loss\n\n return loss.sum(), dice.sum()\n\n```\n\n**Soft F1 Score:**\n\n\n```python\n\ndef macro_soft_f1(y, y_hat):\n \"\"\"Compute the macro soft F1-score as a cost.\n Average (1 - soft-F1) across all labels.\n Use probability values instead of binary predictions.\n\n Args:\n y (int32 Tensor): targets array of shape (BATCH_SIZE, N_LABELS)\n y_hat (float32 Tensor): probability matrix of shape (BATCH_SIZE, N_LABELS)\n\n Returns:\n cost (scalar Tensor): value of the cost function for the batch\n \"\"\"\n\n y = tf.cast(y, tf.float32)\n y_hat = tf.cast(y_hat, tf.float32)\n tp = tf.reduce_sum(y_hat * y, axis=0)\n fp = tf.reduce_sum(y_hat * (1 - y), axis=0)\n fn = tf.reduce_sum((1 - y_hat) * y, axis=0)\n soft_f1 = 2*tp / (2*tp + fn + fp + 1e-16)\n cost = 1 - soft_f1 # reduce 1 - soft-f1 in order to increase soft-f1\n macro_cost = tf.reduce_mean(cost) # average on all labels\n\n return macro_cost\n\n```\n\n> I just want to add on this topic that all those trick losses are rarely seen in winning solutions.\n\n_____\n\n\n##### Freezing Layers to Avoid OOM (1024px) ([moth](https://www.kaggle.com/alejopaullier))\n\n[moth](https://www.kaggle.com/alejopaullier) suggested an alternative to reduce memory and avoid OOM errors when training on 1024 pixel images.\nThis suggestion is to freeze layers and only train a few layers at a time, which will reduce memory usage. This can be done by setting the `trainable` argument to `False` for all layers except the ones you want to train. This will greatly reduce the memory and computational requirements of the model, allowing it to run even on relatively low-end hardware.\n\n**How to freeze layers:**\n\nDetermine how many layers your model have:\n\n```python\n\nfor i,(name, param) in enumerate(list(model.named_parameters())):\n print(i,name)\n\n```\n\nThis will print the layers of your model.\nThen determine how many layers you want to freeze. You can experiment by freezing 100% model, 50%, 33%, 20%, etc.\n\n\n```python\n\nNUM_FROZEN_LAYERS = 100 # how many layers you want to freeze\nfor i,(name, param) in enumerate(list(model.named_parameters())[0:NUM_FROZEN_LAYERS]):\n param.requires_grad = False\n\n```\n\n_____\n\n##### Training with size 512 vs 1024 experience ([Martin Kovacevic Buvinic](https://www.kaggle.com/ragnar123))\n\n[Martin Kovacevic Buvinic](https://www.kaggle.com/ragnar123) has been training models with image size 512 and 1024 and found some interesting results.\n\n- If we have backbone A and backbone B and train each backbone with the same hyperparammeters for image size 512, backbone A has a better CV compared to B.\n- What is interesting is that when the experiment was dont using image size 1024, backbone B has a much better CV compared to A.\n\nThe main idea was to train with smaller image size and then if we get good results we can use bigger image size, nevertheless this experiment suggest that **results from using image size 512 will not always have the same behaviour for image size 1024**.\n\n_____\n\n##### Ray - PARALLEL processing dicom files and ... do more tasks ([Remek Kinas](https://www.kaggle.com/remekkinas))\n[Remek Kinas](https://www.kaggle.com/remekkinas) created a [notebook](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/371093) to provide inspiration on how to use Ray for parallel processing of DICOM files. This will help to speed up the processing and enable multiple tasks to be completed at the same time.\n\n_____\n\n##### 16bits PNG dataset (1024) ([MPWARE](https://www.kaggle.com/mpware))\n\n- [MPWARE](https://www.kaggle.com/mpware) shared a post about creating a 16bits PNG dataset (1024) with the goal of improving the accuracy of the Santa2022 competition.\n- The post provides instructions on how to generate 16bits PNG dataset, which consists of a single folder containing all the images of the dataset. T\n- The post also explains the benefits of using 16bits PNG, such as improved accuracy and better compression.\n- Additionally, the post provides tips on how to make the dataset more efficient, such as setting image resolution and image size. Finally, the post includes code examples of how to generate the dataset.\n\n_____\n\n##### There's something about efficientnets ([James Howard](https://www.kaggle.com/jamesphoward))\n\n- [James Howard](https://www.kaggle.com/jamesphoward) recently discussed the better performance of EfficientNets on slices of lung tissue compared to Transformers in the HuBMAP competition 3 months ago.\n- EfficientNets achieved better results than transformers, but the reason behind this wasn't clear since they are a bit old. [James Howard](https://www.kaggle.com/jamesphoward) raised this question as a discussion topic.\n\n**This topic has interesting discussion in the comments about multiple architectures and their performance. (Good Read!)**\n\n_____\n\n##### Weird Mammograms in the Dataset ([outwrest](https://www.kaggle.com/outwrest))\n\n- [outwrest](https://www.kaggle.com/outwrest) has noticed that patient_id `27770` has some unusual mammograms and made a [notebook](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/373208) to investigate this issue.\n- The mammograms stand out from the other images due to a high amount of noise and no visible tissue.\n- This only affects 4 images, and two examples of this have been provided. [outwrest](https://www.kaggle.com/outwrest) is exploring other processing methods to see if there is an issue with the way the images are being processed.\n\n_____\n\n##### KerasCV + NoROI Baseline ([Awsaf](https://www.kaggle.com/awsaf49))\n\n- [Awsaf](https://www.kaggle.com/awsaf49) developed a KerasCV + NoROI Baseline which involves different techniques to improve the score.\n- Really good baseline! (Check it out!)\n\n_____\n\n\n##### Cropped datasets ([FabienDaniel](https://www.kaggle.com/fabiendaniel))\n\n- [FabienDaniel](https://www.kaggle.com/fabiendaniel) created cropped datasets of .png files, ranging from sizes 256 to 1024, which removes text from the images and crops them around the breast.\n- This was done by utilizing kernels from [theoviel](https://www.kaggle.com/theoviel) and [davidroberts](https://www.kaggle.com/davidroberts).\n- This can be really useful for training models with more focused and accurate results.\n\n_____\n\n##### How to Handle Class Imbalance in Computer Vision ([moth](https://www.kaggle.com/alejopaullier))\n\n[moth](https://www.kaggle.com/alejopaullier) discussed how to handle class imbalance in computer vision.\n\n**Important Answer (By [CPMP](https://www.kaggle.com/cpmpml)):**\nA good discussion about the topic can be found [here](https://www.kaggle.com/competitions/siim-isic-melanoma-classification/discussion/172892).\n\n> **Note:** It is a REALLY good discussion with many people from the top of Kaggle proposing ideas! (Check it out!)\n\n\n_____\n\n##### ensemble not working? ([](https://www.kaggle.com/hengck23))\n\n[Heng CherKeng](https://www.kaggle.com/hengck23) shared an insightful post discussing possible reasons why ensemble models may not be working.\n\n**From the post:**\n\n- The problem occurs because we want to find a single threshold to binarized. The main issue is not imbalanced but lack of pos samples. When there is lack of dataset, everything goes haywire.\n\n**You may need to realigned your predicted probability (calibration) if:**\n- You are using different train parameters (e.g. different weights in class weighted loss, different over sampling) for different fold\n- Even if you are using the same train parameters, the validation sample size is too small and biased for each fold\n- Even if you aligned and prove to work on public test set, it may not work on private set.\n\n**You probably needs to think of how to**\n- Use external data (then there is another problem of possibly domain shift)\n- Find better fold stratification and compute more reliable statistics\n- Probe your hidden test data for magic characteristics (you probably didn't solve the problem but employ competition tricks to stabilized your score)\n\n_____\n\n\n##### Notebook for removing letter markers ([David Roberts](https://www.kaggle.com/davidbroberts))\n\n[David Roberts](https://www.kaggle.com/davidbroberts) discussed a notebook for removing letter markers from images in [this post](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/370072). Letter markers are present in many images and can be distracting or cause confusion when analyzing the images.\n\nThe notebook is right [here](https://www.kaggle.com/code/davidbroberts/mammography-remove-letter-markers).\n\n_____\n\n\n##### Vertical line in images ([moth](https://www.kaggle.com/alejopaullier))\n\nIn [this post](https://www.kaggle.com/competitions/rsna-breast-cancer-detection/discussion/369291) [moth](https://www.kaggle.com/alejopaullier) asked about the vertical line which appears in many images.\n\n\n**Important Answer (By [David Roberts](https://www.kaggle.com/davidbroberts)):**\n\n\n```\n\nI believe the vertical lines on some images is the shadow of the compression paddle the machine uses to compress the breast. Possibly they are from biopsy paddles .. I am not certain. But, I think it's a safe bet that all the anatomy will be on one side of the line.\n\n```\n\n_____\n\n\n##### 2020 IEEE Breast Cancer Screening research paper ([gradientBoost](https://www.kaggle.com/quarkspark))\n\n[gradientBoost](https://www.kaggle.com/quarkspark) recently published a paper in the IEEE Transactions on Medical Imaging titled “Deep Neural Networks Improve Radiologists Performance in Breast Cancer Screening”. The paper discusses the efficacy of using deep neural networks in the field of breast cancer screening, and how they can improve radiologists’ performance.\nThe paper also explores how deep neural networks can be used to increase the accuracy of screening and reduce false positives.\n_____\n\n##### 2x faster jpeg LOSSLESS decoding ([@kaggleqrdl](https://www.kaggle.com/kaggleqrdl))\n\n[@kaggleqrdl](https://www.kaggle.com/kaggleqrdl) shared a post on how to make 2x faster jpeg LOSSLESS decoding. By using a combination of [libjpeg-turbo](https://libjpeg-turbo.org/) and [libmozjpeg](https://github.com/mozilla/mozjpeg) the decoding of large images can be sped up significantly.\n\n**The example provided shows that libjpeg-turbo is faster than libmozjpeg, but libmozjpeg produces better quality images. Thus, a combination of the two can provide both speed and quality.**\n\n_____\n\n##### How can we train models with 1024px images using Kaggle? ([moth](https://www.kaggle.com/alejopaullier))\n\n[moth](https://www.kaggle.com/alejopaullier) asked about the possibility of training a model on 1024px images using Kaggle kernels. It has been found that this can be difficult due to Out of Memory (OOM) errors.\n\nThere are some methods you can try to reduce the memory of your training:\n\n**Methods from the post:**\n\n- **Edit 1:** TPU accelerates training but not resolved OOM. Does not require much code refactoring but requires a lot of configuration regarding conflicting package versions and memory handling via gc.collect().\n- **Edit 2:** Freezing layers does help both with OOM and computation time:\n\n**The Post Set-up:**\n\n**Architecture:** EfficientNetB2.\n**Image resolution:** 1024px.\n**Train Batch Size:** 8 images per batch.\n**Number of Frozen Layers:** 100 out of 300 (~33% of the model).\n**Consumed GPU's VRAM:** 4.7 out of 15.9 GB.\n**Training time for 1 EPOCH:** 43k images on ~1:03 hrs.\n**Gradient Accumulation:** True. Every 4 steps.\n\n_____\n\n\n", "2082634": "Really good summary! Thank you so much 🙏", "2097413": "Thanks for sharing!" } }