Welcome to the RSNA-STR Pulmonary Embolism Detection challenge.
\nIn this competition, you’ll be identifying the presence and type of pulmonary embolisms using image data. This competition is inference-only, meaning that your submitted kernels will not have access to the training set.
\nNote that the private test set is approximately 3x larger than the public test set (230GB vs. 70GB), so ensure that your kernels have enough time to finish their re-run. The training set includes 7279 studies, the public set 650, and the private set has 1517.
\nWe have a new resource available about debugging notebooks in code competitions. Please check it out for a description of the code competition process, as well as common issues you may run into.
\nAlso, be sure you are aware of this competition’s unique requirements for winners to (a) open source their code and (b) adhere to the defined label hierarchy in order to avoid disqualification and remain on the leaderboard.
\nBest of luck!
\nRemember: Kaggle is for everyone. Whether you're teaming up or sharing tips in the competition forum, we expect everyone to follow our Kaggle community guidelines.
", "messages": [ { "id": 1007174, "postDate": "2020-09-11T21:56:11.683Z", "content": "Welcome to the RSNA-STR Pulmonary Embolism Detection challenge.
\nIn this competition, you’ll be identifying the presence and type of pulmonary embolisms using image data. This competition is inference-only, meaning that your submitted kernels will not have access to the training set.
\nNote that the private test set is approximately 3x larger than the public test set (230GB vs. 70GB), so ensure that your kernels have enough time to finish their re-run. The training set includes 7279 studies, the public set 650, and the private set has 1517.
\nWe have a new resource available about debugging notebooks in code competitions. Please check it out for a description of the code competition process, as well as common issues you may run into.
\nAlso, be sure you are aware of this competition’s unique requirements for winners to (a) open source their code and (b) adhere to the defined label hierarchy in order to avoid disqualification and remain on the leaderboard.
\nBest of luck!
\nRemember: Kaggle is for everyone. Whether you're teaming up or sharing tips in the competition forum, we expect everyone to follow our Kaggle community guidelines.
", "rawMarkdown": "Welcome to the RSNA-STR Pulmonary Embolism Detection challenge.\n\nIn this competition, you’ll be identifying the presence and type of pulmonary embolisms using image data. This competition is **inference-only**, meaning that your submitted kernels will not have access to the training set.\n\nNote that the private test set is approximately 3x larger than the public test set (230GB vs. 70GB), so ensure that your kernels have enough time to finish their re-run. The training set includes 7279 studies, the public set 650, and the private set has 1517.\n\nWe have a new [resource](https://www.kaggle.com/code-competition-debugging) available about debugging notebooks in code competitions. Please check it out for a description of the code competition process, as well as common issues you may run into.\n\nAlso, be sure you are aware of this competition’s [unique requirements](https://www.kaggle.com/c/rsna-str-pulmonary-embolism-detection/overview/prizes) for winners to (a) open source their code and (b) adhere to the defined label hierarchy in order to avoid disqualification and remain on the leaderboard.\n\nBest of luck!\n\n**Remember**: Kaggle is for everyone. Whether you're teaming up or sharing tips in the competition forum, we expect everyone to follow our [Kaggle community guidelines](https://www.kaggle.com/community-guidelines).", "votes": 10 }, { "id": 1018798, "postDate": "2020-09-20T01:44:02.303Z", "content": "Is it possible to add GCDM in default Kaggle kernel environment like pydicom? Since we are not able to enable internet during inference, which make it quite difficult to install GCDM package.
", "rawMarkdown": "Is it possible to add GCDM in default Kaggle kernel environment like pydicom? Since we are not able to enable internet during inference, which make it quite difficult to install GCDM package.", "votes": 4 }, { "id": 1053795, "postDate": "2020-10-19T11:28:52.620Z", "content": "@juliaelliott @philculliton
\nIs it legal (for the final submission) to run locally prediction on the public test data and in the final submission notebook predict only the private data and merge the 2 CSVs?
If you're able to code your notebook submission to do this effectively without returning an error, then yes. However note that it would need to be able to dynamically predict on the private test set that you're not able to see.
", "rawMarkdown": "If you're able to code your notebook submission to do this effectively without returning an error, then yes. However note that it would need to be able to dynamically predict on the private test set that you're not able to see." }, { "id": 1055666, "postDate": "2020-10-21T01:55:26.590Z", "content": "@orkatz2 Were you able to successfully use cached predictions in the public test set? I have a Notebook Exceeded Allowed Compute when I've tried to do that. Could be unrelated but that was the only difference from a succesful submission
@philculliton In the prize section of the overview there is a sentence about conflicts:
\n\n\nWinning submissions will be inspected to ensure label predictions adhere to the expected label hierarchy defined by the diagram on the Data page. The metric intends to heavily penalize submissions which mis-predict in this manner, however due to the complexity of predictions at both image and study levels and as an extra precaution, the host will verify that prospective winners have not made conflicting label predictions. This includes:
\n
\n Submissions that make study predictions which conflict with that study's image predictions.
\n Submissions that make conflicting or disallowed sub-label predictions (i.e. simultaneously predicting RV/LV ratio >=1 and <1, more than one label for PE type, etc.)
But we are predicting probabilities, so we can predict RV/LV ratio >=1 ,0.4 and RV/LV ratio <1, 0.6 , but we can also predict RV/LV ratio >=1 ,0.4 and RV/LV ratio <1, 0.4 or even RV/LV ratio >=1 ,0.6 and RV/LV ratio <1, 0.6
Should the Negative for PE values be (1-sum(all pe values) or ?
\nIf Negative for PE in the the study prediction is 0.9 what should be the maximum values forPE Present on Image in that studies images?
I believe, this inspection needs to be defined better, in a mathematical way.
", "rawMarkdown": "@philculliton In the prize section of the overview there is a sentence about conflicts:\n> Winning submissions will be inspected to ensure label predictions adhere to the expected label hierarchy defined by the diagram on the Data page. The metric intends to heavily penalize submissions which mis-predict in this manner, however due to the complexity of predictions at both image and study levels and as an extra precaution, the host will verify that prospective winners have not made conflicting label predictions. This includes:\nSubmissions that make study predictions which conflict with that study's image predictions.\nSubmissions that make conflicting or disallowed sub-label predictions (i.e. simultaneously predicting RV/LV ratio >=1 and <1, more than one label for PE type, etc.)\n\nBut we are predicting probabilities, so we can predict ` RV/LV ratio >=1 ,0.4 and RV/LV ratio <1, 0.6` , but we can also predict ` RV/LV ratio >=1 ,0.4 and RV/LV ratio <1, 0.4` or even ` RV/LV ratio >=1 ,0.6 and RV/LV ratio <1, 0.6` \n\nShould the Negative for PE values be (1-sum(all pe values) or ?\n\nIf `Negative for PE` in the the study prediction is 0.9 what should be the maximum values for` PE Present on Image` in that studies images?\n\nI believe, this inspection needs to be defined better, in a mathematical way.", "votes": 2, "replies": [ { "id": 1011800, "postDate": "2020-09-15T17:23:57.573Z", "content": "@yuval6967 We're confirming this understanding with the host and will get back to you as soon as possible.
", "rawMarkdown": "@yuval6967 We're confirming this understanding with the host and will get back to you as soon as possible.", "votes": 1 }, { "id": 1013287, "postDate": "2020-09-16T15:59:20.710Z", "content": "I am also very interested in further details on this rule. Does it only apply if predictions are integers? If not, does it impose constraints on probabilities of different labels? If yes, what are they exactly?
", "rawMarkdown": "I am also very interested in further details on this rule. Does it only apply if predictions are integers? If not, does it impose constraints on probabilities of different labels? If yes, what are they exactly? " }, { "id": 1013448, "postDate": "2020-09-16T17:38:18.177Z", "content": "For the purpose of enforcing this rule on logical consistency we consider a label to be “predicted” if that label is assigned a probability of > 0.5. We require that the “predicted” labels be logically consistent. Note that we expect that logically inconsistent outputs will score poorly on the metric, and that high-scoring algorithms will naturally produce logically consistent outputs. However, due to the complexity of the metric and the difficulty in anticipating all corner cases, we have put this rule in place to reduce the ability to game the metric with nonsensical outputs.
\nSome specific examples of how this rule plays out:
\nAt the image level, any image with predicted probability > 0.5 is considered as being positive for PE will count as a positive image
\nAt the exam level, we have
\nSimilarly, if no image is positive (p > 0.5), then there must be one and only one negative or indeterminate with p > 0.5
\nRight, left, central -- if any image is predicted positive (p > 0.5) then at least one of these labels must be assigned p > 0.5; more than one of these labels may be assigned p > 0.5. When no images are predicted positive, then none of these labels may be assigned p > 0.5
RV/LV ratio. It can be only one of these and it must be present if at least one image is positive.
Is this currently enforced on Leaderboard Scores? As models are being developed, I'm sure we are not meeting a lot of these constraints.
\nHow would somebody know if their submission is violating constraints? Are individual predictions \"fixed\" to conform?
\nOr does this rule just take affect after the competition in validating the winners?
", "rawMarkdown": "Is this currently enforced on Leaderboard Scores? As models are being developed, I'm sure we are not meeting a lot of these constraints.\n\nHow would somebody know if their submission is violating constraints? Are individual predictions \"fixed\" to conform?\n\nOr does this rule just take affect after the competition in validating the winners?", "votes": 2 } ] }, { "id": 1013548, "postDate": "2020-09-16T18:55:06.663Z", "content": "@anthracene Thank for the important clarification ( @juliaelliott You might want to copy this to a new thread so no one will miss it)
", "rawMarkdown": "@anthracene Thank for the important clarification ( @juliaelliott You might want to copy this to a new thread so no one will miss it)", "votes": 1 }, { "id": 1035869, "postDate": "2020-10-03T07:05:58.910Z", "content": "Question on:
\n\n\nRV/LV ratio. It can be only one of these and it must be present if at least one image is positive.
\n
How is the RV/LV prediction handled for negative or indeterminate exams? I would think this ratio should be outputted for all exams, so any clarifications here would be helpful. (I.e. what if the patient had a large RV but does not have PE?) Is there a penalty for predicting RV/LV ratio greater than 1?
", "rawMarkdown": "Question on: \n\n> RV/LV ratio. It can be only one of these and it must be present if at least one image is positive.\n\nHow is the RV/LV prediction handled for negative or indeterminate exams? I would think this ratio should be outputted for all exams, so any clarifications here would be helpful. (I.e. what if the patient had a large RV but does not have PE?) Is there a penalty for predicting RV/LV ratio greater than 1?" }, { "id": 1041780, "postDate": "2020-10-07T23:05:19.243Z", "content": "RV/LV ratio tags are present only on positive exams. You should not make RV/LV ratio predictions on negative or indeterminate exams.
", "rawMarkdown": "RV/LV ratio tags are present only on positive exams. You should not make RV/LV ratio predictions on negative or indeterminate exams.", "votes": 1 } ] }, { "id": 1068422, "postDate": "2020-11-03T12:01:53.300Z", "content": "감사합니다. 많은 도움이 되었습니다.
", "rawMarkdown": "감사합니다. 많은 도움이 되었습니다." }, { "id": 1051709, "postDate": "2020-10-16T18:53:23.553Z", "content": "are we allowed to use pseudolabeling on the public test set?
", "rawMarkdown": "are we allowed to use pseudolabeling on the public test set?", "replies": [ { "id": 1054368, "postDate": "2020-10-19T21:26:55.573Z", "content": "@stephenliang Yes, as long as you the methods you're using are automated and not hand-labeling the test set.
", "rawMarkdown": "@stephenliang Yes, as long as you the methods you're using are automated and not hand-labeling the test set." } ] }, { "id": 1041904, "postDate": "2020-10-08T00:57:51.620Z", "content": "\n\n\n\n\n
", "rawMarkdown": "Hi @juliaelliott
\nCan we assume that when running inference against the private test set, there will also be a new 'test.csv' with updated study,instance,SOP IDs for the 1517 studies?
@yeeseng Yes, the test images folder and test.csv will be available in the private re-run but swapped out with the private set.
That's great thanks.
", "rawMarkdown": "That's great thanks." } ] }, { "id": 1024444, "postDate": "2020-09-23T21:44:27.460Z", "content": "\nThank you for your comprehensive response!
", "rawMarkdown": "Thank you for your comprehensive response!" }, { "id": 1024487, "postDate": "2020-09-23T23:04:46.267Z", "content": "Hi @juliaelliott, A quick follow up question for #2. What would be the deadline for posting on Kaggle as an additional open source dataset?
", "rawMarkdown": "Hi @juliaelliott, A quick follow up question for #2. What would be the deadline for posting on Kaggle as an additional open source dataset?" }, { "id": 1025673, "postDate": "2020-09-24T17:43:33.277Z", "content": "@aksg87 Ah, so if you are simply re-labeling features of the existing train set, then you are not obligated to declare or share publicly any of those new features, as long as you're entitled to be using them in your solution (i.e. they were obtained through payment or privately). The external data provision is simply designed to prohibit against participants using data or pre-trained models which are proprietary or only for private use in some way.
", "rawMarkdown": "@aksg87 Ah, so if you are simply re-labeling features of the existing train set, then you are not obligated to declare or share publicly any of those new features, as long as you're entitled to be using them in your solution (i.e. they were obtained through payment or privately). The external data provision is simply designed to prohibit against participants using data or pre-trained models which are proprietary or only for private use in some way." }, { "id": 1025920, "postDate": "2020-09-24T21:56:50.867Z", "content": "Okay thanks @juliaelliott
\nIt would be great to include these details in the competition rules but I suppose this discussion serves the same purpose! It might encourage creative solutions that focus on supplementing the training data in addition to model optimizations.
\nThanks again!
", "rawMarkdown": "Okay thanks @juliaelliott \n\nIt would be great to include these details in the competition rules but I suppose this discussion serves the same purpose! It might encourage creative solutions that focus on supplementing the training data in addition to model optimizations.\n\nThanks again!" } ] }, { "id": 1022102, "postDate": "2020-09-22T10:10:00.743Z", "content": "I have a small question regarding the rules:
\n\n\nThis is an inference-only code competition. Your submissions will not have access to the training images, so you must train your models elsewhere and incorporate them into your submission, without reference to the folder containing the train images.
\n
Does it also imply that train.csv is not available at the submission time or it only covers the images in train/?
@kozodoi train.csv will be available, but none of the train folder images will be available.
@philculliton
\nIn train.csv, rv_lv_ratio_gte_1, rv_lv_ratio_lt_1 leftsided_pe, rightsided_pe, central_pe, chronic_pe, acute_and_chronic_pe is ALL ZERO for non-positive exam.
\nIs it true for test time?
\n(updated question description for simplicity, 20200912)
@lisosia If the exam is negative for PE, then all of the corresponding study sub-labels would also be negative/zero. They cannot be left unpredicted and must be 0.
", "rawMarkdown": "@lisosia If the exam is negative for PE, then all of the corresponding study sub-labels would also be negative/zero. They cannot be left unpredicted and must be 0. ", "votes": 1 }, { "id": 1021943, "postDate": "2020-09-22T08:23:46.123Z", "content": "Thank you. I understand.
", "rawMarkdown": "Thank you. I understand." } ] }, { "id": 1011774, "postDate": "2020-09-15T17:14:57.730Z", "content": "Can you or someone else setup the find a teammate forum, thanks Phil.
", "rawMarkdown": "Can you or someone else setup the find a teammate forum, thanks Phil.", "replies": [ { "id": 1011799, "postDate": "2020-09-15T17:23:21.150Z", "content": "@bopengiowa Done, thanks for the reminder!
", "rawMarkdown": "@bopengiowa Done, thanks for the reminder!", "votes": 1 }, { "id": 1011805, "postDate": "2020-09-15T17:28:05.150Z", "content": "Thanks Julia! Great job!
", "rawMarkdown": "Thanks Julia! Great job!" } ] }, { "id": 1007379, "postDate": "2020-09-12T05:52:51.857Z", "content": "Great one
", "rawMarkdown": "Great one" }, { "id": 1064024, "postDate": "2020-10-29T16:01:10.877Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1041906, "postDate": "2020-10-08T00:58:43.917Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1041901, "postDate": "2020-10-08T00:57:15.673Z", "rawMarkdown": "", "isDeleted": true } ], "comments": [ { "id": 1018798, "author_name": "Tsai29", "author_url": "", "post_date": "2020-09-20T01:44:02.303000", "content": "Is it possible to add GCDM in default Kaggle kernel environment like pydicom? Since we are not able to enable internet during inference, which make it quite difficult to install GCDM package.
", "votes": 4, "replies": [] }, { "id": 1053795, "author_name": "OrKatz", "author_url": "", "post_date": "2020-10-19T11:28:52.620000", "content": "@juliaelliott @philculliton
\nIs it legal (for the final submission) to run locally prediction on the public test data and in the final submission notebook predict only the private data and merge the 2 CSVs?
If you're able to code your notebook submission to do this effectively without returning an error, then yes. However note that it would need to be able to dynamically predict on the private test set that you're not able to see.
", "votes": 0, "replies": [] }, { "id": 1055666, "author_name": "Ronaldo S.A. Batista", "author_url": "", "post_date": "2020-10-21T01:55:26.590000", "content": "@orkatz2 Were you able to successfully use cached predictions in the public test set? I have a Notebook Exceeded Allowed Compute when I've tried to do that. Could be unrelated but that was the only difference from a succesful submission
@philculliton In the prize section of the overview there is a sentence about conflicts:
\n\n\nWinning submissions will be inspected to ensure label predictions adhere to the expected label hierarchy defined by the diagram on the Data page. The metric intends to heavily penalize submissions which mis-predict in this manner, however due to the complexity of predictions at both image and study levels and as an extra precaution, the host will verify that prospective winners have not made conflicting label predictions. This includes:
\n
\n Submissions that make study predictions which conflict with that study's image predictions.
\n Submissions that make conflicting or disallowed sub-label predictions (i.e. simultaneously predicting RV/LV ratio >=1 and <1, more than one label for PE type, etc.)
But we are predicting probabilities, so we can predict RV/LV ratio >=1 ,0.4 and RV/LV ratio <1, 0.6 , but we can also predict RV/LV ratio >=1 ,0.4 and RV/LV ratio <1, 0.4 or even RV/LV ratio >=1 ,0.6 and RV/LV ratio <1, 0.6
Should the Negative for PE values be (1-sum(all pe values) or ?
\nIf Negative for PE in the the study prediction is 0.9 what should be the maximum values forPE Present on Image in that studies images?
I believe, this inspection needs to be defined better, in a mathematical way.
", "votes": 2, "replies": [ { "id": 1011800, "author_name": "Julia Elliott", "author_url": "", "post_date": "2020-09-15T17:23:57.573000", "content": "@yuval6967 We're confirming this understanding with the host and will get back to you as soon as possible.
", "votes": 1, "replies": [] }, { "id": 1013287, "author_name": "Nikita Kozodoi", "author_url": "", "post_date": "2020-09-16T15:59:20.710000", "content": "I am also very interested in further details on this rule. Does it only apply if predictions are integers? If not, does it impose constraints on probabilities of different labels? If yes, what are they exactly?
", "votes": 0, "replies": [] }, { "id": 1013448, "author_name": "John Mongan", "author_url": "", "post_date": "2020-09-16T17:38:18.177000", "content": "For the purpose of enforcing this rule on logical consistency we consider a label to be “predicted” if that label is assigned a probability of > 0.5. We require that the “predicted” labels be logically consistent. Note that we expect that logically inconsistent outputs will score poorly on the metric, and that high-scoring algorithms will naturally produce logically consistent outputs. However, due to the complexity of the metric and the difficulty in anticipating all corner cases, we have put this rule in place to reduce the ability to game the metric with nonsensical outputs.
\nSome specific examples of how this rule plays out:
\nAt the image level, any image with predicted probability > 0.5 is considered as being positive for PE will count as a positive image
\nAt the exam level, we have
\nSimilarly, if no image is positive (p > 0.5), then there must be one and only one negative or indeterminate with p > 0.5
\nRight, left, central -- if any image is predicted positive (p > 0.5) then at least one of these labels must be assigned p > 0.5; more than one of these labels may be assigned p > 0.5. When no images are predicted positive, then none of these labels may be assigned p > 0.5
RV/LV ratio. It can be only one of these and it must be present if at least one image is positive.
Is this currently enforced on Leaderboard Scores? As models are being developed, I'm sure we are not meeting a lot of these constraints.
\nHow would somebody know if their submission is violating constraints? Are individual predictions \"fixed\" to conform?
\nOr does this rule just take affect after the competition in validating the winners?
", "votes": 2, "replies": [] } ] }, { "id": 1013548, "author_name": "yuval reina", "author_url": "", "post_date": "2020-09-16T18:55:06.663000", "content": "@anthracene Thank for the important clarification ( @juliaelliott You might want to copy this to a new thread so no one will miss it)
", "votes": 1, "replies": [] }, { "id": 1035869, "author_name": "aksg87", "author_url": "", "post_date": "2020-10-03T07:05:58.910000", "content": "Question on:
\n\n\nRV/LV ratio. It can be only one of these and it must be present if at least one image is positive.
\n
How is the RV/LV prediction handled for negative or indeterminate exams? I would think this ratio should be outputted for all exams, so any clarifications here would be helpful. (I.e. what if the patient had a large RV but does not have PE?) Is there a penalty for predicting RV/LV ratio greater than 1?
", "votes": 0, "replies": [] }, { "id": 1041780, "author_name": "John Mongan", "author_url": "", "post_date": "2020-10-07T23:05:19.243000", "content": "RV/LV ratio tags are present only on positive exams. You should not make RV/LV ratio predictions on negative or indeterminate exams.
", "votes": 1, "replies": [] } ] }, { "id": 1068422, "author_name": "kimseajung", "author_url": "", "post_date": "2020-11-03T12:01:53.300000", "content": "감사합니다. 많은 도움이 되었습니다.
", "votes": 0, "replies": [] }, { "id": 1051709, "author_name": "Stephen Liang", "author_url": "", "post_date": "2020-10-16T18:53:23.553000", "content": "are we allowed to use pseudolabeling on the public test set?
", "votes": 0, "replies": [ { "id": 1054368, "author_name": "Julia Elliott", "author_url": "", "post_date": "2020-10-19T21:26:55.573000", "content": "@stephenliang Yes, as long as you the methods you're using are automated and not hand-labeling the test set.
", "votes": 0, "replies": [] } ] }, { "id": 1041904, "author_name": "Eng Rayan", "author_url": "", "post_date": "2020-10-08T00:57:51.620000", "content": "\n\n\n\n\n
", "votes": 0, "replies": [] }, { "id": 1031087, "author_name": "Yee Ng", "author_url": "", "post_date": "2020-09-29T08:29:09.410000", "content": "Hi @juliaelliott
\nCan we assume that when running inference against the private test set, there will also be a new 'test.csv' with updated study,instance,SOP IDs for the 1517 studies?
@yeeseng Yes, the test images folder and test.csv will be available in the private re-run but swapped out with the private set.
That's great thanks.
", "votes": 0, "replies": [] } ] }, { "id": 1024444, "author_name": "aksg87", "author_url": "", "post_date": "2020-09-23T21:44:27.460000", "content": "\nThank you for your comprehensive response!
", "votes": 0, "replies": [] }, { "id": 1024487, "author_name": "aksg87", "author_url": "", "post_date": "2020-09-23T23:04:46.267000", "content": "Hi @juliaelliott, A quick follow up question for #2. What would be the deadline for posting on Kaggle as an additional open source dataset?
", "votes": 0, "replies": [] }, { "id": 1025673, "author_name": "Julia Elliott", "author_url": "", "post_date": "2020-09-24T17:43:33.277000", "content": "@aksg87 Ah, so if you are simply re-labeling features of the existing train set, then you are not obligated to declare or share publicly any of those new features, as long as you're entitled to be using them in your solution (i.e. they were obtained through payment or privately). The external data provision is simply designed to prohibit against participants using data or pre-trained models which are proprietary or only for private use in some way.
", "votes": 0, "replies": [] }, { "id": 1025920, "author_name": "aksg87", "author_url": "", "post_date": "2020-09-24T21:56:50.867000", "content": "Okay thanks @juliaelliott
\nIt would be great to include these details in the competition rules but I suppose this discussion serves the same purpose! It might encourage creative solutions that focus on supplementing the training data in addition to model optimizations.
\nThanks again!
", "votes": 0, "replies": [] } ] }, { "id": 1022102, "author_name": "Nikita Kozodoi", "author_url": "", "post_date": "2020-09-22T10:10:00.743000", "content": "I have a small question regarding the rules:
\n\n\nThis is an inference-only code competition. Your submissions will not have access to the training images, so you must train your models elsewhere and incorporate them into your submission, without reference to the folder containing the train images.
\n
Does it also imply that train.csv is not available at the submission time or it only covers the images in train/?
@kozodoi train.csv will be available, but none of the train folder images will be available.
@philculliton
\nIn train.csv, rv_lv_ratio_gte_1, rv_lv_ratio_lt_1 leftsided_pe, rightsided_pe, central_pe, chronic_pe, acute_and_chronic_pe is ALL ZERO for non-positive exam.
\nIs it true for test time?
\n(updated question description for simplicity, 20200912)
@lisosia If the exam is negative for PE, then all of the corresponding study sub-labels would also be negative/zero. They cannot be left unpredicted and must be 0.
", "votes": 1, "replies": [] }, { "id": 1021943, "author_name": "yama", "author_url": "", "post_date": "2020-09-22T08:23:46.123000", "content": "Thank you. I understand.
", "votes": 0, "replies": [] } ] }, { "id": 1011774, "author_name": "Bo Peng", "author_url": "", "post_date": "2020-09-15T17:14:57.730000", "content": "Can you or someone else setup the find a teammate forum, thanks Phil.
", "votes": 0, "replies": [ { "id": 1011799, "author_name": "Julia Elliott", "author_url": "", "post_date": "2020-09-15T17:23:21.150000", "content": "@bopengiowa Done, thanks for the reminder!
", "votes": 1, "replies": [] }, { "id": 1011805, "author_name": "Bo Peng", "author_url": "", "post_date": "2020-09-15T17:28:05.150000", "content": "Thanks Julia! Great job!
", "votes": 0, "replies": [] } ] }, { "id": 1007379, "author_name": "R. Joseph Manoj, PhD", "author_url": "", "post_date": "2020-09-12T05:52:51.857000", "content": "Great one
", "votes": 0, "replies": [] }, { "id": 1064024, "author_name": "", "author_url": "", "post_date": "2020-10-29T16:01:10.877000", "content": "", "votes": 0, "replies": [] }, { "id": 1041906, "author_name": "", "author_url": "", "post_date": "2020-10-08T00:58:43.917000", "content": "", "votes": 0, "replies": [] }, { "id": 1041901, "author_name": "", "author_url": "", "post_date": "2020-10-08T00:57:15.673000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "1007174": "Welcome to the RSNA-STR Pulmonary Embolism Detection challenge.\n\nIn this competition, you’ll be identifying the presence and type of pulmonary embolisms using image data. This competition is **inference-only**, meaning that your submitted kernels will not have access to the training set.\n\nNote that the private test set is approximately 3x larger than the public test set (230GB vs. 70GB), so ensure that your kernels have enough time to finish their re-run. The training set includes 7279 studies, the public set 650, and the private set has 1517.\n\nWe have a new [resource](https://www.kaggle.com/code-competition-debugging) available about debugging notebooks in code competitions. Please check it out for a description of the code competition process, as well as common issues you may run into.\n\nAlso, be sure you are aware of this competition’s [unique requirements](https://www.kaggle.com/c/rsna-str-pulmonary-embolism-detection/overview/prizes) for winners to (a) open source their code and (b) adhere to the defined label hierarchy in order to avoid disqualification and remain on the leaderboard.\n\nBest of luck!\n\n**Remember**: Kaggle is for everyone. Whether you're teaming up or sharing tips in the competition forum, we expect everyone to follow our [Kaggle community guidelines](https://www.kaggle.com/community-guidelines).", "1018798": "Is it possible to add GCDM in default Kaggle kernel environment like pydicom? Since we are not able to enable internet during inference, which make it quite difficult to install GCDM package.", "1053795": "@juliaelliott @philculliton\nIs it legal (for the final submission) to run locally prediction on the public test data and in the final submission notebook predict only the private data and merge the 2 CSVs?\n", "1010407": "@philculliton In the prize section of the overview there is a sentence about conflicts:\n> Winning submissions will be inspected to ensure label predictions adhere to the expected label hierarchy defined by the diagram on the Data page. The metric intends to heavily penalize submissions which mis-predict in this manner, however due to the complexity of predictions at both image and study levels and as an extra precaution, the host will verify that prospective winners have not made conflicting label predictions. This includes:\nSubmissions that make study predictions which conflict with that study's image predictions.\nSubmissions that make conflicting or disallowed sub-label predictions (i.e. simultaneously predicting RV/LV ratio >=1 and <1, more than one label for PE type, etc.)\n\nBut we are predicting probabilities, so we can predict ` RV/LV ratio >=1 ,0.4 and RV/LV ratio <1, 0.6` , but we can also predict ` RV/LV ratio >=1 ,0.4 and RV/LV ratio <1, 0.4` or even ` RV/LV ratio >=1 ,0.6 and RV/LV ratio <1, 0.6` \n\nShould the Negative for PE values be (1-sum(all pe values) or ?\n\nIf `Negative for PE` in the the study prediction is 0.9 what should be the maximum values for` PE Present on Image` in that studies images?\n\nI believe, this inspection needs to be defined better, in a mathematical way.", "1068422": "감사합니다. 많은 도움이 되었습니다.", "1051709": "are we allowed to use pseudolabeling on the public test set?", "1041904": "