![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F6102861%2Ff1139ecd39345e6e4992a1039aee5797%2Finput_images.png?generation=1667030306195626&alt=media\")
First of all, I would like to thank the kaggle staff and hosts for organizing and hosting this competition.
\nI would also like to thank my teammate @yuyuki11235. He suggested and implemented a lot of ideas and was really helpful.
\nWe hope to try again to win the gold medal next time.
We used a 2-stage approach, extracting the image-level features with CNN models and inferring the patient-level fracture probabilities with sequential models.
\nThe slice images were preprocessed with YOLOX cropping, 2.5D approach and a windowing method.
\nFor final submission, we used efficietnet-v2-l as a feature extractor and LSTM, GRU, and Conv1d for the sequence model.
We trained YOLOX-l model to detect vertebral positions.
\nBbox labels were created from the outer frame of the segmentation masks (similar to the 3rd place solution).
\nFor cropping, we only used the highest confidence bbox in the images with a confidence level of 0.7 or higher.
\n20 pixel margin was added to bbox when cropping.
As in previous RSNA competitions, we used a 2.5D approach using neighboring slices and a windowing technique.
\nFor windowing, we applied the values used in this paper.
We combined 2.5D and windowing as follows (s denotes a slice index).
\nch1 : s-1 slice with Standard bone window
\nch2 : s slice with Gross bone window
\nch3 : s+1 slice with Soft tissue window
An example of input images created by preprocessing is shown below.
\n
Our CNN models were based on @vslaykovsky's public baseline.
\nWe trained our models to infer fracture and vertebral positions.
\nThe main tips related to training are listed below.
Since the fracture and vertebral positions per image were not given in this competition, we tried to create the accurate image-level labels by multiple pseudo-labeling steps.
\nStarting with this notebook's labels, we first made pseudo-labels of fracture and vertebral positions from the oof predictions of efficientnet-b4 and use them as the labels for the next training (with some post-processings such as thresholding).
\nBy repeating the pseudo-labeling with larger models, we were able to achieve a better CV.
We trained patient-level classification models using image-level embeddings.
\nOur final ensemble were LSTM, GRU, and Conv1d models.
\nThe main processing steps are as follows:
Other tricks are as follows:
\nFirst of all, I would like to thank the kaggle staff and hosts for organizing and hosting this competition.
\nI would also like to thank my teammate @yuyuki11235. He suggested and implemented a lot of ideas and was really helpful.
\nWe hope to try again to win the gold medal next time.
We used a 2-stage approach, extracting the image-level features with CNN models and inferring the patient-level fracture probabilities with sequential models.
\nThe slice images were preprocessed with YOLOX cropping, 2.5D approach and a windowing method.
\nFor final submission, we used efficietnet-v2-l as a feature extractor and LSTM, GRU, and Conv1d for the sequence model.
We trained YOLOX-l model to detect vertebral positions.
\nBbox labels were created from the outer frame of the segmentation masks (similar to the 3rd place solution).
\nFor cropping, we only used the highest confidence bbox in the images with a confidence level of 0.7 or higher.
\n20 pixel margin was added to bbox when cropping.
As in previous RSNA competitions, we used a 2.5D approach using neighboring slices and a windowing technique.
\nFor windowing, we applied the values used in this paper.
We combined 2.5D and windowing as follows (s denotes a slice index).
\nch1 : s-1 slice with Standard bone window
\nch2 : s slice with Gross bone window
\nch3 : s+1 slice with Soft tissue window
An example of input images created by preprocessing is shown below.
\n
Our CNN models were based on @vslaykovsky's public baseline.
\nWe trained our models to infer fracture and vertebral positions.
\nThe main tips related to training are listed below.
Since the fracture and vertebral positions per image were not given in this competition, we tried to create the accurate image-level labels by multiple pseudo-labeling steps.
\nStarting with this notebook's labels, we first made pseudo-labels of fracture and vertebral positions from the oof predictions of efficientnet-b4 and use them as the labels for the next training (with some post-processings such as thresholding).
\nBy repeating the pseudo-labeling with larger models, we were able to achieve a better CV.
We trained patient-level classification models using image-level embeddings.
\nOur final ensemble were LSTM, GRU, and Conv1d models.
\nThe main processing steps are as follows:
Other tricks are as follows:
\nCongrats on your great finish!
\nI had a similar approach except for YOLO box cropping. Did you try without that? How much of an improvement do you get from it?
Congrats on your silver medal too!
\nIn our experiments, bbox cropping improved CV by 0.05 or more.
congratulations! How much did label cleaning increase LB? My team overfitted the CV due to cleaning, so please let me know if there is anything I should be careful about.
", "rawMarkdown": "congratulations! How much did label cleaning increase LB? My team overfitted the CV due to cleaning, so please let me know if there is anything I should be careful about." }, { "id": 2009156, "postDate": "2022-10-29T17:40:48.857Z", "content": "Congratulations! Can you give more details on your Phase 2 sequential models?
", "rawMarkdown": "Congratulations! Can you give more details on your Phase 2 sequential models?" } ], "comments": [ { "id": 2009081, "author_name": "Yerram Varun", "author_url": "", "post_date": "2022-10-29T16:10:19.153000", "content": "Congrats on your great finish!
\nI had a similar approach except for YOLO box cropping. Did you try without that? How much of an improvement do you get from it?
Congrats on your silver medal too!
\nIn our experiments, bbox cropping improved CV by 0.05 or more.
congratulations! How much did label cleaning increase LB? My team overfitted the CV due to cleaning, so please let me know if there is anything I should be careful about.
", "votes": 0, "replies": [] }, { "id": 2009156, "author_name": "SolverWorld", "author_url": "", "post_date": "2022-10-29T17:40:48.857000", "content": "Congratulations! Can you give more details on your Phase 2 sequential models?
", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2008869": "First of all, I would like to thank the kaggle staff and hosts for organizing and hosting this competition.\nI would also like to thank my teammate @yuyuki11235. He suggested and implemented a lot of ideas and was really helpful.\nWe hope to try again to win the gold medal next time.\n\n## Summary\nWe used a 2-stage approach, extracting the image-level features with CNN models and inferring the patient-level fracture probabilities with sequential models.\nThe slice images were preprocessed with YOLOX cropping, 2.5D approach and a windowing method.\nFor final submission, we used efficietnet-v2-l as a feature extractor and LSTM, GRU, and Conv1d for the sequence model.\n\n## Preprocess\n##### Bounding box cropping\nWe trained YOLOX-l model to detect vertebral positions.\nBbox labels were created from the outer frame of the segmentation masks (similar to the [3rd place solution](https://www.kaggle.com/competitions/rsna-2022-cervical-spine-fracture-detection/discussion/362643)).\nFor cropping, we only used the highest confidence bbox in the images with a confidence level of 0.7 or higher.\n20 pixel margin was added to bbox when cropping.\n\n##### Use neighbor slices and windowing\nAs in previous RSNA competitions, we used a 2.5D approach using neighboring slices and a windowing technique.\nFor windowing, we applied the values used in [this paper](https://arxiv.org/abs/2010.13336).\n- Standard bone window (w=500, c=1800)\n- Gross bone window (w=650, c=400)\n- Soft tissue window (w=300, c=80)\n\nWe combined 2.5D and windowing as follows (s denotes a slice index).\nch1 : s-1 slice with Standard bone window\nch2 : s slice with Gross bone window\nch3 : s+1 slice with Soft tissue window\n\nAn example of input images created by preprocessing is shown below.\n\n\n\n## CNN Model (1st stage)\nOur CNN models were based on @vslaykovsky's [public baseline](https://www.kaggle.com/code/vslaykovsky/train-pytorch-effnetv2-baseline-cv-0-49).\nWe trained our models to infer fracture and vertebral positions.\nThe main tips related to training are listed below.\n\n##### Augmentation\n- HorizontalFlip\n- GaussianBlur\n- GaussNoise\n- ShiftScaleRotate\n- CoarseDropout\n- RandomBrightnessContrast\n- OneOf([GridDistortion, ElasticTransform])\n\n##### Label cleaning\nSince the fracture and vertebral positions per image were not given in this competition, we tried to create the accurate image-level labels by multiple pseudo-labeling steps.\nStarting with [this notebook](https://www.kaggle.com/code/vslaykovsky/train-pytorch-effnetv2-baseline-cv-0-49)'s labels, we first made pseudo-labels of fracture and vertebral positions from the oof predictions of efficientnet-b4 and use them as the labels for the next training (with some post-processings such as thresholding).\nBy repeating the pseudo-labeling with larger models, we were able to achieve a better CV.\n\n## Sequential Model (2nd stage)\nWe trained patient-level classification models using image-level embeddings.\nOur final ensemble were LSTM, GRU, and Conv1d models.\nThe main processing steps are as follows:\n- concatenate image-level embedding for each patient (NxD, N: number of slices per patient, D: embedding dimension)\n- apply cv2.resize function to fix the length of the sequence (MxD, M: fixed sequence length) (use [this notebook](https://www.kaggle.com/competitions/rsna-str-pulmonary-embolism-detection/discussion/194145) idea)\n- input above features to each model\n\nOther tricks are as follows:\n- Mask only the C1~C7 areas by inferring the bone position (NxD→N'xD, N': number of slices of C1~C7)\n- Attention pooling\n- Randomly skip slices during training (fixed at skip 3 during inference, see Final Prediction section bellow)\n\n## Final Prediction\n- Use 1/3 slices as df[df[\"Slices\"] % 3 == 0] to reduce inference time\n- YOLOX-l bbox prediciotn (~1h inference time)\n- 4 out of 5 folds of efficienetnet-v2-l (below 2h inference time per fold)\n- image size = (512, 512)\n- 3 sequential models (LSTM, GRU, Conv1d) with difference seeds (4 seeds for each model)\n- fixed sequence length = 224\n", "2009081": "Congrats on your great finish!\nI had a similar approach except for YOLO box cropping. Did you try without that? How much of an improvement do you get from it?", "2046283": "congratulations! How much did label cleaning increase LB? My team overfitted the CV due to cleaning, so please let me know if there is anything I should be careful about.", "2009156": "Congratulations! Can you give more details on your Phase 2 sequential models?" } }