\n
![\"Patient](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16972845%2F7ffb464d82e78e29bca3dafe437a1532%2FDICOM1.png?generation=1699299484698499&alt=media\")
![\"Patient](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16972845%2F47a42e18f96eda654081f04d8eba4cc3%2FDICOM2.png?generation=1699299601100767&alt=media\")
\nPatient 10004 – record 21057 – IMG 1000.dcm, Patient 10004 – record 21057 – IMG 1029.dcm
Dataset provided for the contest consists of 3 main data sources:
\nOur initial aim of this project was to leverage all 3 parts together. Before going there, let us understand the data provided as images:
\nThe dataset provided consisted of 2 types of images (CT scan) - .dcm files and .ni files.
\n.dcm stands as an extension for DICOM files, an abbreviation for Digital Imaging and Communications in
\nMedicine. It is a set or sequence of X-ray images that CT scan is comprised of providing details on organ
\nhealth.
examples of DICOM images:
\n
\nPatient 10004 – record 21057 – IMG 1000.dcm, Patient 10004 – record 21057 – IMG 1029.dcm
For this problem, we decided to take an alternate route to the problem by converting the 3D lattice into 3 sets of lateral snapshots each having 2 axes fixed at 0 and 1 available for lateral traversal.
\nTo explain it in simple terms. We changed value of z while keeping x and y at 0. This produced slices parallel to x-y plane at regular intervals in z axis from z = 0 to z = max.
\nA visualisation of this lateral segment can be seen in the screenshot below. The screenshot is captured on a web-app available to public access via this link
\nexamples of NII files:
\n![\"Patient](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16972845%2Fde92fe3838a9035bcb54aa3b1ec46d33%2FNII_1.jpg?generation=1699299895059718&alt=media\")
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16972845%2Fa1b52b75588e1468665c0f2c5d578695%2FNII_2.jpg?generation=1699300536859629&alt=media\")
This was done using the mathematical expression:
\nnormalised-aortic-hu = (𝐚𝐨𝐫𝐭𝐢𝐜-𝐡𝐮 − 𝐥𝐨𝐰𝐞𝐬𝐭-𝐚𝐨𝐫𝐭𝐢𝐜-𝐡𝐮𝐞)/(𝐡𝐢𝐠𝐡𝐞𝐬𝐭-𝐚𝐨𝐫𝐭𝐢𝐜-𝐡𝐮𝐞 − 𝐥𝐨𝐰𝐞𝐬𝐭-𝐚𝐨𝐫𝐭𝐢𝐜-𝐡𝐮𝐞)
Redundant Full Black image cleanup
\nThe cleaning operation was primarily to remove the full black images from the nii file generated dataset. This was particularly difficult when we see that the number of such images in the dataset of nii file output was not the same. In some MRI images, first 12 images were fully black, in others first 16 were. This created an irregular sized set of images if we proceeded with this tactic. This required an alternate route of first finding
\nthe minimum such count of fully black images and then reduce that size from both sides of an image thus reducing the dataset, although keeping it uniform in all patient MRI images and in an optimised set size reducing irregularity and difficulty for model to process each frame.
So to talk numerically,
\nWe took 100 snapshots per axis of each MRI scan or each NII file.
\nSo, each NII file contributed along x = 100, y = 100, z = 100 => total 300 images.
\nWe found minimum non-full black image at 9th position. i.e. we decided to remove 8 images from each side
\non all axes. So, new dataset => x = 84 (100 - 8 - 8), y = 84, z = 84. => total = 252 images per NII file
\nSo effective dataset reduced by 16% after removal of redundant images.
Note: Feature Extraction could not be done well due to inaccurate slice ranges and image rotation in NII files unlike DICOM files.
\nRedundant Full Black image cleanup
\nInitial Approach of the dicom file processing was like NII file about removing redundant images, but it turned
\nout that nearly all of the images were significant and unskipable. This was great because now we had an
\noption to utilise full dataset and each individual image training was meaningful.
The second exploratory analysis process that we implemented was core feature extraction from images by localisation of organs. How exactly? As seen from the image below, the organs are localised to certain positions of CT scan or alternatively DICOM images. We tried to extract specific location of each organ with
\na 20% buffer border around each organ to adjust for any dislocation of organ due to natural causes like genetics or in body fat layers. This buffer was also to account for the organ movement due to diaphragm compression and relaxation during breathing.
We tried to localise this subsection of each organ and train it individually for each organ health splitting the DICOM based ML model into its subsections. Although this was successfully executed to separate specific organs from images separately with 20 buffers in each axis (10% on each border), there was still overlap.
\nIt so happened that the considered organs overlapped over each other’s specific sub-region images. So, an image for kidney health analysis contained a significant part of spleen as well, so if we went forward with this implementation, It was possible that damage on spleen could be reflected in damage on kidney by model predictions due to spleen sharing a significant portion in images.
\nFor testing of data on the dataset, we used 2 methods of testing excluding the public dataset-based testing.
\nInitial Approach:
\nAs discussed earlier we planned to utilise all the 3 types of data together. But there was a problem with this approach. The 3 datasets showed a lot of variations. If we were to treat MRI images and CT-SCAN images as a single input to the model, we were bound to face issues with training and model accuracy plunging down. To solve this issue, we decided to take an ensemble model like approach to the model where we would be treating each dataset separately with their own model and then we would combine the generated result
\nfrom each model with appropriate weights to decide on the best output to be returned as a result.
An illustration of the same can be seen as:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16972845%2F47fae49205d7d037c21eadfb3f126f25%2FMLDA_Model.jpg?generation=1699305429235805&alt=media\")
The data had disparity for each patient and not every patient had both CT scan (aka CAT scan in US) as well as MRI done while diagnostics. In such cases, we simply changed the weights for such patients and distributed the other weights proportionally.
\nFor example:
\nIf the decided ideal weights were: α=0.4, β=0.4, γ=0.2
\nIf a patient A only performed CT-SCAN and not MRI, we could simply set value of β=0 and redistribute α and γ proportionally as α = α/(α+ γ), γ = γ/(α+ γ)
\nAlthough this was our initial plan, we observed that the predicted values by the NII file model had a lot of discrepancy. Due to this, the idealistic β value would have been near 0. Thus, to save processing time, we eliminated the NII file processing segment and ML model entirely in final solution keeping just DICOM and Metadata.
To build this model, we have taken help of EfficientNet_B4 prebuilt model.
\nweights_path = \n\ndef build_model(num_classes):\n model = create_model(, =)\n\n os.path.exists(weights_path):\n model.load_state_dict(torch.load(weights_path, =), =) \n :\n (f)\n\n model.classifier = nn.Linear(model.classifier.in_features, num_classes)\n\n return model\n\ndevice = torch.device( torch.cuda.is_available() )\nmodel = build_model(len(train_df.columns) - 1).(device)\n\ncriterion = torch.nn.BCEWithLogitsLoss()\noptimizer = optim.Adam(model.parameters(), =0.001)\n\nnum_epochs = 3\n\n epoch range(num_epochs):\n model.train()\n running_loss = 0.0\n i, (inputs, labels) enumerate(train_loader):\n (i)\n inputs, labels = inputs.(device), labels.(device)\n optimizer.zero_grad()\n outputs = model(inputs)\n loss = criterion(outputs, labels)\n loss.backward()\n optimizer.()\n\n running_loss += loss.item()\n\n (f)\nIn summary, this process leverages DICOM images to train a deep learning model, fine-tunes its parameters, and refines its understanding of the dataset through dynamic weight assignments. By iteratively training, testing, and adjusting the model, we aim to achieve a high level of accuracy in its predictions. This approach represents a key component of our strategy for effectively utilizing DICOM images in our machine learning workflow.
\nIn the weighted baseline approach, we assign distinct weights to each of the nine target classes within the dataset. These weights reflect the relative significance of each class, taking into account the clinical importance of various medical conditions. For instance, we assign a weight of 2 to injuries like \"kidney low\" and \"liver low,\" indicating their moderate impact. Conditions such as \"spleen high\" and \"kidney high\" are assigned a weight of 4, reflecting their higher clinical significance. Additionally, we assign a weight of 6 to conditions like \"excavation high\" and \"excavation low,\" and set the base weight for no injury cases to 0. Please refer detailed solution linked at the end for code.
\nWhy EfficientNet_B4 and not other models?
\nTo explain this answer, it would be better to pictorially represent a study of comparable image detection ML Models like Resnet.
\n![\"efficientNet_b4](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16972845%2F891e0be6c537b1b858773f1459a0502e%2Fefficientnetb4%20comparision%20graph.png?generation=1699306350364423&alt=media\")
\nNOTE: EfficientNet BenchMarking Data belongs to the creator - AutoML MNAS. The above image is used only for educational purposes. The original Image is in Free to use MIT license.
Here, we can see that on same dataset benchmarking, the EfficientNet Model (all versions B0-B7) outperform other models like NASNet, Resnet, AmoebaNet etc. which are also a part of industry standard vision models. The graph although in a efficiency vs Parameter domain can be sought to have at par result in computational resources and dataset size due to its direct relation with number of parameters.
\nFor our model implementation, we used efficientNet_B4 prebuilt Model
\nEfficient net provides various versions ranging from B0 to B7. All these versions differ in model size, computational complexity, and accuracy. As the number of parameters increases the models become increasingly data-hungry requiring more data to train and tend to overfit to the data really fast if dataset size is inadequate for the model complexity and parameter count.
For our data set, B4 was most suitable. This was evident from when we compared computational complexity and accuracy, B4 was found to be more suitable for our model than B0, B7 or any other mode for same parameters when trained on the complete training set for out-sample testing. Hence, we have used Efficient_b4 model to train our model.
\nWe decided to go for individual weights for each organ where all values of the three model parameters α, β and γ from Result = αX + βY + γZ were different for each organ.
\nTaking Result = αX + βY + γZ we observe that the ideal values of parameters for organs are:
\nKidney: α = 0.43, β = 0.04, γ = 0.53
\nLiver: α = 0.49, β = 0.03, γ = 0.48
\nSpleen: α = 0.47, β = 0.03, γ = 0.50
\nExtravasation: α = 0.39, β = 0.00, γ = 0.61
\nBowel: α = 0.46, β = 0.03, γ = 0.51
\nAveraged: α = 0.44, β = 0.03, γ = 0.52
Note: As observed, the value of coefficient or Weigh NII Model is near 0 and thus regarded as insignificant contributor. In such case we thought that calculating NII would be a waste of computational resources and dropped it out of the final submission considering only Weighted baseline and DICOM model with adjusted accuracy similar to as seen above.
\nEpoch tuning (Epoch vs score curve [score α 1/accuracy] thus min = better) on Competition public dataset
\n![\"epoch](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16972845%2Fd55a58b1055ebdc1175c6eb8e7faac38%2Fepoch1.jpg?generation=1699306932413783&alt=media\")
From the above graph we can make out that the best accuracy (score) is obtained at epoch count = 3. Thus while submission to the competition, we decided to go ahead with 3 epoch model.
\nApart from the efficiency-epoch curve, we also used ROC-AUC curve and efficienct vs -log(λ) that was plotted for both data and observed to check for Overfit-underfit conditions and track result changes towards positive direction at each subsequent step.
\nThere was minimal amount of overfitting found for out-sample testing dataset, although it was minimised in full training dataset so overfit was self-controlled. Still to be double sure and reduce it further we implemented basic L1 regularisation. We also changed Batch-Normalisation marginally to suit the curve better.
\nNote: the problem was a multiclass output type problem, so the confusion matrix was multiclass and individually seen for all 13 predictable classes. we checked if expected value matches with a tolerance of 10% on both sides to take that prediction as correct.
\nFollowing the same logic:
TPtotal = TP1 + TP2 + …. + TP13
\nF1 SCORE: TP/(TP+ 1/2(FP+FN)) = 0.77
\nTesting accuracy: (TP + TN)/(TP + TN + FP + FN) = 76%
For Out-sample testing as well, the confusion matrix for all 13 parameters was evaluated similarly.
\nF1 SCORE: = 0.73
\nTesting accuracy: = 67%
Initially, we had planned to utilise the NII file dataset but later we discovered that no significant or notable value came from it. As evident in hyperparameteric tuning section, the beta coefficient value is ~0, indicating low contribution thus had to let it go.
We attempted to localise organs in dicom images and split the single Model into multiple organ specific models for better accuracy but failed due to image overlap and occlusion.
Dataset provided for the contest consists of 3 main data sources:
\nOur initial aim of this project was to leverage all 3 parts together. Before going there, let us understand the data provided as images:
\nThe dataset provided consisted of 2 types of images (CT scan) - .dcm files and .ni files.
\n.dcm stands as an extension for DICOM files, an abbreviation for Digital Imaging and Communications in
\nMedicine. It is a set or sequence of X-ray images that CT scan is comprised of providing details on organ
\nhealth.
examples of DICOM images:
\n
\nPatient 10004 – record 21057 – IMG 1000.dcm, Patient 10004 – record 21057 – IMG 1029.dcm
For this problem, we decided to take an alternate route to the problem by converting the 3D lattice into 3 sets of lateral snapshots each having 2 axes fixed at 0 and 1 available for lateral traversal.
\nTo explain it in simple terms. We changed value of z while keeping x and y at 0. This produced slices parallel to x-y plane at regular intervals in z axis from z = 0 to z = max.
\nA visualisation of this lateral segment can be seen in the screenshot below. The screenshot is captured on a web-app available to public access via this link
\nexamples of NII files:
\n
\n
This was done using the mathematical expression:
\nnormalised-aortic-hu = (𝐚𝐨𝐫𝐭𝐢𝐜-𝐡𝐮 − 𝐥𝐨𝐰𝐞𝐬𝐭-𝐚𝐨𝐫𝐭𝐢𝐜-𝐡𝐮𝐞)/(𝐡𝐢𝐠𝐡𝐞𝐬𝐭-𝐚𝐨𝐫𝐭𝐢𝐜-𝐡𝐮𝐞 − 𝐥𝐨𝐰𝐞𝐬𝐭-𝐚𝐨𝐫𝐭𝐢𝐜-𝐡𝐮𝐞)
Redundant Full Black image cleanup
\nThe cleaning operation was primarily to remove the full black images from the nii file generated dataset. This was particularly difficult when we see that the number of such images in the dataset of nii file output was not the same. In some MRI images, first 12 images were fully black, in others first 16 were. This created an irregular sized set of images if we proceeded with this tactic. This required an alternate route of first finding
\nthe minimum such count of fully black images and then reduce that size from both sides of an image thus reducing the dataset, although keeping it uniform in all patient MRI images and in an optimised set size reducing irregularity and difficulty for model to process each frame.
So to talk numerically,
\nWe took 100 snapshots per axis of each MRI scan or each NII file.
\nSo, each NII file contributed along x = 100, y = 100, z = 100 => total 300 images.
\nWe found minimum non-full black image at 9th position. i.e. we decided to remove 8 images from each side
\non all axes. So, new dataset => x = 84 (100 - 8 - 8), y = 84, z = 84. => total = 252 images per NII file
\nSo effective dataset reduced by 16% after removal of redundant images.
Note: Feature Extraction could not be done well due to inaccurate slice ranges and image rotation in NII files unlike DICOM files.
\nRedundant Full Black image cleanup
\nInitial Approach of the dicom file processing was like NII file about removing redundant images, but it turned
\nout that nearly all of the images were significant and unskipable. This was great because now we had an
\noption to utilise full dataset and each individual image training was meaningful.
The second exploratory analysis process that we implemented was core feature extraction from images by localisation of organs. How exactly? As seen from the image below, the organs are localised to certain positions of CT scan or alternatively DICOM images. We tried to extract specific location of each organ with
\na 20% buffer border around each organ to adjust for any dislocation of organ due to natural causes like genetics or in body fat layers. This buffer was also to account for the organ movement due to diaphragm compression and relaxation during breathing.
We tried to localise this subsection of each organ and train it individually for each organ health splitting the DICOM based ML model into its subsections. Although this was successfully executed to separate specific organs from images separately with 20 buffers in each axis (10% on each border), there was still overlap.
\nIt so happened that the considered organs overlapped over each other’s specific sub-region images. So, an image for kidney health analysis contained a significant part of spleen as well, so if we went forward with this implementation, It was possible that damage on spleen could be reflected in damage on kidney by model predictions due to spleen sharing a significant portion in images.
\nFor testing of data on the dataset, we used 2 methods of testing excluding the public dataset-based testing.
\nInitial Approach:
\nAs discussed earlier we planned to utilise all the 3 types of data together. But there was a problem with this approach. The 3 datasets showed a lot of variations. If we were to treat MRI images and CT-SCAN images as a single input to the model, we were bound to face issues with training and model accuracy plunging down. To solve this issue, we decided to take an ensemble model like approach to the model where we would be treating each dataset separately with their own model and then we would combine the generated result
\nfrom each model with appropriate weights to decide on the best output to be returned as a result.
An illustration of the same can be seen as:
\n
The data had disparity for each patient and not every patient had both CT scan (aka CAT scan in US) as well as MRI done while diagnostics. In such cases, we simply changed the weights for such patients and distributed the other weights proportionally.
\nFor example:
\nIf the decided ideal weights were: α=0.4, β=0.4, γ=0.2
\nIf a patient A only performed CT-SCAN and not MRI, we could simply set value of β=0 and redistribute α and γ proportionally as α = α/(α+ γ), γ = γ/(α+ γ)
\nAlthough this was our initial plan, we observed that the predicted values by the NII file model had a lot of discrepancy. Due to this, the idealistic β value would have been near 0. Thus, to save processing time, we eliminated the NII file processing segment and ML model entirely in final solution keeping just DICOM and Metadata.
To build this model, we have taken help of EfficientNet_B4 prebuilt model.
\nweights_path = \n\ndef build_model(num_classes):\n model = create_model(, =)\n\n os.path.exists(weights_path):\n model.load_state_dict(torch.load(weights_path, =), =) \n :\n (f)\n\n model.classifier = nn.Linear(model.classifier.in_features, num_classes)\n\n return model\n\ndevice = torch.device( torch.cuda.is_available() )\nmodel = build_model(len(train_df.columns) - 1).(device)\n\ncriterion = torch.nn.BCEWithLogitsLoss()\noptimizer = optim.Adam(model.parameters(), =0.001)\n\nnum_epochs = 3\n\n epoch range(num_epochs):\n model.train()\n running_loss = 0.0\n i, (inputs, labels) enumerate(train_loader):\n (i)\n inputs, labels = inputs.(device), labels.(device)\n optimizer.zero_grad()\n outputs = model(inputs)\n loss = criterion(outputs, labels)\n loss.backward()\n optimizer.()\n\n running_loss += loss.item()\n\n (f)\nIn summary, this process leverages DICOM images to train a deep learning model, fine-tunes its parameters, and refines its understanding of the dataset through dynamic weight assignments. By iteratively training, testing, and adjusting the model, we aim to achieve a high level of accuracy in its predictions. This approach represents a key component of our strategy for effectively utilizing DICOM images in our machine learning workflow.
\nIn the weighted baseline approach, we assign distinct weights to each of the nine target classes within the dataset. These weights reflect the relative significance of each class, taking into account the clinical importance of various medical conditions. For instance, we assign a weight of 2 to injuries like \"kidney low\" and \"liver low,\" indicating their moderate impact. Conditions such as \"spleen high\" and \"kidney high\" are assigned a weight of 4, reflecting their higher clinical significance. Additionally, we assign a weight of 6 to conditions like \"excavation high\" and \"excavation low,\" and set the base weight for no injury cases to 0. Please refer detailed solution linked at the end for code.
\nWhy EfficientNet_B4 and not other models?
\nTo explain this answer, it would be better to pictorially represent a study of comparable image detection ML Models like Resnet.
\n
\nNOTE: EfficientNet BenchMarking Data belongs to the creator - AutoML MNAS. The above image is used only for educational purposes. The original Image is in Free to use MIT license.
Here, we can see that on same dataset benchmarking, the EfficientNet Model (all versions B0-B7) outperform other models like NASNet, Resnet, AmoebaNet etc. which are also a part of industry standard vision models. The graph although in a efficiency vs Parameter domain can be sought to have at par result in computational resources and dataset size due to its direct relation with number of parameters.
\nFor our model implementation, we used efficientNet_B4 prebuilt Model
\nEfficient net provides various versions ranging from B0 to B7. All these versions differ in model size, computational complexity, and accuracy. As the number of parameters increases the models become increasingly data-hungry requiring more data to train and tend to overfit to the data really fast if dataset size is inadequate for the model complexity and parameter count.
For our data set, B4 was most suitable. This was evident from when we compared computational complexity and accuracy, B4 was found to be more suitable for our model than B0, B7 or any other mode for same parameters when trained on the complete training set for out-sample testing. Hence, we have used Efficient_b4 model to train our model.
\nWe decided to go for individual weights for each organ where all values of the three model parameters α, β and γ from Result = αX + βY + γZ were different for each organ.
\nTaking Result = αX + βY + γZ we observe that the ideal values of parameters for organs are:
\nKidney: α = 0.43, β = 0.04, γ = 0.53
\nLiver: α = 0.49, β = 0.03, γ = 0.48
\nSpleen: α = 0.47, β = 0.03, γ = 0.50
\nExtravasation: α = 0.39, β = 0.00, γ = 0.61
\nBowel: α = 0.46, β = 0.03, γ = 0.51
\nAveraged: α = 0.44, β = 0.03, γ = 0.52
Note: As observed, the value of coefficient or Weigh NII Model is near 0 and thus regarded as insignificant contributor. In such case we thought that calculating NII would be a waste of computational resources and dropped it out of the final submission considering only Weighted baseline and DICOM model with adjusted accuracy similar to as seen above.
\nEpoch tuning (Epoch vs score curve [score α 1/accuracy] thus min = better) on Competition public dataset
\n
From the above graph we can make out that the best accuracy (score) is obtained at epoch count = 3. Thus while submission to the competition, we decided to go ahead with 3 epoch model.
\nApart from the efficiency-epoch curve, we also used ROC-AUC curve and efficienct vs -log(λ) that was plotted for both data and observed to check for Overfit-underfit conditions and track result changes towards positive direction at each subsequent step.
\nThere was minimal amount of overfitting found for out-sample testing dataset, although it was minimised in full training dataset so overfit was self-controlled. Still to be double sure and reduce it further we implemented basic L1 regularisation. We also changed Batch-Normalisation marginally to suit the curve better.
\nNote: the problem was a multiclass output type problem, so the confusion matrix was multiclass and individually seen for all 13 predictable classes. we checked if expected value matches with a tolerance of 10% on both sides to take that prediction as correct.
\nFollowing the same logic:
TPtotal = TP1 + TP2 + …. + TP13
\nF1 SCORE: TP/(TP+ 1/2(FP+FN)) = 0.77
\nTesting accuracy: (TP + TN)/(TP + TN + FP + FN) = 76%
For Out-sample testing as well, the confusion matrix for all 13 parameters was evaluated similarly.
\nF1 SCORE: = 0.73
\nTesting accuracy: = 67%
Initially, we had planned to utilise the NII file dataset but later we discovered that no significant or notable value came from it. As evident in hyperparameteric tuning section, the beta coefficient value is ~0, indicating low contribution thus had to let it go.
We attempted to localise organs in dicom images and split the single Model into multiple organ specific models for better accuracy but failed due to image overlap and occlusion.