{ "id": 362641, "title": "Open sourcing my solution [2D Model, PyTorch]", "url": "/competitions/rsna-2022-cervical-spine-fracture-detection/discussion/362641", "author_name": "Samuel Cortinhas", "post_date": "2022-10-28T09:27:00.476000", "votes": 17, "comment_count": 2, "views": 0, "content": "

Even though our team dropped 350 places in private LB I still wanted to share our approach (& code) in the spirit of collaboration and helping others learn. First I want to say a huge thank you to my team mate @jirkaborovec, kaggle and all the hosts - this really was a fantastic competition.

\n

Model architecture

\n

\"2-D-model-architecture.png\"

\n

Our model was inspired by Vladimir's that most have seen and 1st place from the Melanoma Classification comp. We used image data and meta data simultaneously in the hopes of helping the model better predict which vertebrae is in each image.

\n

Feature engineering

\n

We extracted metadata (see my dataset here) from the dicom files including slice number, slice thickness and patient position. We then performed feature engineering on this:

\n\n

Learning rate scheduler

\n

\n

We used CosineAnnealingWarmupRestarts from the github here. Each cycle is one epoch. Optimisers tried included Adam and AdamW.

\n

Augmentations

\n

\n\n

A=Albumentations library. See links at the bottom for code.

\n

Torch, GPU, epochs

\n

We used PyTorch. It was actually my first time using it and I really like it. We only used kaggle GPU and kaggle notebooks to train our models. 1 epoch took roughly 10 hours. Best submission used 6 epochs.

\n

Code

\n

I am open sourcing all my code for anyone that wants to look. I've tried to comment it as best as possible. Happy to answer questions if anything is unclear.

\n
    \n
  1. RNSA - 2D model [Train] [PyTorch] (training script)
    2. \n
  2. RNSA - 2D model [Validate] [PyTorch] (evaluate on competition metric)
    3. \n
  3. RNSA - 2D model [Infer] [PyTorch] (submission notebook)
    4. \n
\n

Lessons learnt

\n", "messages": [ { "id": 2007507, "postDate": "2022-10-28T09:27:00.477Z", "content": "

Even though our team dropped 350 places in private LB I still wanted to share our approach (& code) in the spirit of collaboration and helping others learn. First I want to say a huge thank you to my team mate @jirkaborovec, kaggle and all the hosts - this really was a fantastic competition.

\n

Model architecture

\n

\"2-D-model-architecture.png\"

\n

Our model was inspired by Vladimir's that most have seen and 1st place from the Melanoma Classification comp. We used image data and meta data simultaneously in the hopes of helping the model better predict which vertebrae is in each image.

\n

Feature engineering

\n

We extracted metadata (see my dataset here) from the dicom files including slice number, slice thickness and patient position. We then performed feature engineering on this:

\n\n

Learning rate scheduler

\n

\n

We used CosineAnnealingWarmupRestarts from the github here. Each cycle is one epoch. Optimisers tried included Adam and AdamW.

\n

Augmentations

\n

\n\n

A=Albumentations library. See links at the bottom for code.

\n

Torch, GPU, epochs

\n

We used PyTorch. It was actually my first time using it and I really like it. We only used kaggle GPU and kaggle notebooks to train our models. 1 epoch took roughly 10 hours. Best submission used 6 epochs.

\n

Code

\n

I am open sourcing all my code for anyone that wants to look. I've tried to comment it as best as possible. Happy to answer questions if anything is unclear.

\n
    \n
  1. RNSA - 2D model [Train] [PyTorch] (training script)
    2. \n
  2. RNSA - 2D model [Validate] [PyTorch] (evaluate on competition metric)
    3. \n
  3. RNSA - 2D model [Infer] [PyTorch] (submission notebook)
    4. \n
\n

Lessons learnt

\n", "rawMarkdown": "Even though our team dropped 350 places in private LB I still wanted to share our approach (& code) in the spirit of collaboration and helping others learn. First I want to say a huge thank you to my team mate @jirkaborovec, kaggle and all the hosts - this really was a fantastic competition.\n\n## Model architecture\n\n[![2-D-model-architecture.png](https://i.postimg.cc/264GKjZp/2-D-model-architecture.png)](https://postimg.cc/mzg7csjV)\n\nOur model was inspired by [Vladimir's](https://www.kaggle.com/code/vslaykovsky/train-pytorch-effnetv2-baseline-cv-0-49) that most have seen and [1st place from the Melanoma Classification comp](https://www.kaggle.com/competitions/siim-isic-melanoma-classification/discussion/175412). We used image data and meta data simultaneously in the hopes of helping the model better predict which vertebrae is in each image.\n\n## Feature engineering\n\nWe extracted metadata (see my [dataset here](https://www.kaggle.com/datasets/samuelcortinhas/rsna-2022-spine-fracture-detection-metadata)) from the dicom files including slice number, slice thickness and patient position. We then performed feature engineering on this:\n\n* Calculate maximum slice number\n* Calculate ratio of slice number to max slice\n* Create indicator if images are reversed in z-axis (i.e. order of vertebrae is reversed).\n\n## Learning rate scheduler\n\n\n\nWe used CosineAnnealingWarmupRestarts from the [github here](https://github.com/katsura-jp/pytorch-cosine-annealing-with-warmup). Each cycle is one epoch. Optimisers tried included Adam and AdamW. \n\n## Augmentations\n\n\n\n* A.Resize(*(256,256), interpolation=cv2.INTER_LINEAR)\n* A.HorizontalFlip(p=0.35)\n* A.ShiftScaleRotate(..., p=0.4)\n* A.RandomBrightnessContrast(..., p=0.5)\n* A.OneOf([A.GridDistortion(p=1.0),A.ElasticTransform(p=1.0)], p=0.25)\n\nA=Albumentations library. See links at the bottom for code.\n\n## Torch, GPU, epochs\n\nWe used PyTorch. It was actually my first time using it and I really like it. We only used kaggle GPU and kaggle notebooks to train our models. 1 epoch took roughly 10 hours. Best submission used 6 epochs. \n\n## Code\n\nI am open sourcing all my code for anyone that wants to look. I've tried to comment it as best as possible. Happy to answer questions if anything is unclear.\n\n1. [RNSA - 2D model [Train] [PyTorch]](https://www.kaggle.com/samuelcortinhas/rnsa-2d-model-train-pytorch) (training script)\n2. [RNSA - 2D model [Validate] [PyTorch]](https://www.kaggle.com/samuelcortinhas/rnsa-2d-model-validate-pytorch) (evaluate on competition metric)\n3. [RNSA - 2D model [Infer] [PyTorch]](https://www.kaggle.com/samuelcortinhas/rnsa-2d-model-infer-pytorch) (submission notebook)\n\n## Lessons learnt\n\n* If there was more time I would have liked to train a segmentation model\n* Crop images to focus on region of interest\n* Look into 2.5D", "votes": 17 }, { "id": 2013398, "postDate": "2022-11-01T21:31:04.437Z", "content": "

Your solution is top notch @samuelcortinhas

", "rawMarkdown": "Your solution is top notch @samuelcortinhas ", "votes": 1 }, { "id": 2013559, "postDate": "2022-11-02T02:45:52.843Z", "content": "

@samuelcortinhas
\nI just started my data science journey on kaggle and thinking of participating in competitions. I have understanding of ml, dl, data cleaning and eda

\n

I have been following your impeccable work since I started my journey here.

\n

So, I want to ask you advice regarding,
\nWhat do you suggest me to start with to Ace in competitions ?

", "rawMarkdown": "@samuelcortinhas \nI just started my data science journey on kaggle and thinking of participating in competitions. I have understanding of ml, dl, data cleaning and eda\n\nI have been following your impeccable work since I started my journey here. \n\nSo, I want to ask you advice regarding, \nWhat do you suggest me to start with to Ace in competitions ?" } ], "comments": [ { "id": 2013398, "author_name": "Allena Venkata Sai Abhishek", "author_url": "", "post_date": "2022-11-01T21:31:04.437000", "content": "

Your solution is top notch @samuelcortinhas

", "votes": 1, "replies": [] }, { "id": 2013559, "author_name": "Allena Venkata Sai Abhishek", "author_url": "", "post_date": "2022-11-02T02:45:52.843000", "content": "

@samuelcortinhas
\nI just started my data science journey on kaggle and thinking of participating in competitions. I have understanding of ml, dl, data cleaning and eda

\n

I have been following your impeccable work since I started my journey here.

\n

So, I want to ask you advice regarding,
\nWhat do you suggest me to start with to Ace in competitions ?

", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2007507": "Even though our team dropped 350 places in private LB I still wanted to share our approach (& code) in the spirit of collaboration and helping others learn. First I want to say a huge thank you to my team mate @jirkaborovec, kaggle and all the hosts - this really was a fantastic competition.\n\n## Model architecture\n\n[![2-D-model-architecture.png](https://i.postimg.cc/264GKjZp/2-D-model-architecture.png)](https://postimg.cc/mzg7csjV)\n\nOur model was inspired by [Vladimir's](https://www.kaggle.com/code/vslaykovsky/train-pytorch-effnetv2-baseline-cv-0-49) that most have seen and [1st place from the Melanoma Classification comp](https://www.kaggle.com/competitions/siim-isic-melanoma-classification/discussion/175412). We used image data and meta data simultaneously in the hopes of helping the model better predict which vertebrae is in each image.\n\n## Feature engineering\n\nWe extracted metadata (see my [dataset here](https://www.kaggle.com/datasets/samuelcortinhas/rsna-2022-spine-fracture-detection-metadata)) from the dicom files including slice number, slice thickness and patient position. We then performed feature engineering on this:\n\n* Calculate maximum slice number\n* Calculate ratio of slice number to max slice\n* Create indicator if images are reversed in z-axis (i.e. order of vertebrae is reversed).\n\n## Learning rate scheduler\n\n\n\nWe used CosineAnnealingWarmupRestarts from the [github here](https://github.com/katsura-jp/pytorch-cosine-annealing-with-warmup). Each cycle is one epoch. Optimisers tried included Adam and AdamW. \n\n## Augmentations\n\n\n\n* A.Resize(*(256,256), interpolation=cv2.INTER_LINEAR)\n* A.HorizontalFlip(p=0.35)\n* A.ShiftScaleRotate(..., p=0.4)\n* A.RandomBrightnessContrast(..., p=0.5)\n* A.OneOf([A.GridDistortion(p=1.0),A.ElasticTransform(p=1.0)], p=0.25)\n\nA=Albumentations library. See links at the bottom for code.\n\n## Torch, GPU, epochs\n\nWe used PyTorch. It was actually my first time using it and I really like it. We only used kaggle GPU and kaggle notebooks to train our models. 1 epoch took roughly 10 hours. Best submission used 6 epochs. \n\n## Code\n\nI am open sourcing all my code for anyone that wants to look. I've tried to comment it as best as possible. Happy to answer questions if anything is unclear.\n\n1. [RNSA - 2D model [Train] [PyTorch]](https://www.kaggle.com/samuelcortinhas/rnsa-2d-model-train-pytorch) (training script)\n2. [RNSA - 2D model [Validate] [PyTorch]](https://www.kaggle.com/samuelcortinhas/rnsa-2d-model-validate-pytorch) (evaluate on competition metric)\n3. [RNSA - 2D model [Infer] [PyTorch]](https://www.kaggle.com/samuelcortinhas/rnsa-2d-model-infer-pytorch) (submission notebook)\n\n## Lessons learnt\n\n* If there was more time I would have liked to train a segmentation model\n* Crop images to focus on region of interest\n* Look into 2.5D", "2013398": "Your solution is top notch @samuelcortinhas ", "2013559": "@samuelcortinhas \nI just started my data science journey on kaggle and thinking of participating in competitions. I have understanding of ml, dl, data cleaning and eda\n\nI have been following your impeccable work since I started my journey here. \n\nSo, I want to ask you advice regarding, \nWhat do you suggest me to start with to Ace in competitions ?" } }