{ "id": 190181, "title": "Questions about the informational labels", "url": "/competitions/rsna-str-pulmonary-embolism-detection/discussion/190181", "author_name": "Alexander Soare", "post_date": "2020-10-10T13:59:55.856000", "votes": 0, "comment_count": 8, "views": 0, "content": "

Hi fellow Kagglers. Just catching myself up with this competition and after some time I still have some left over questions I hope you can help me with.

\n

Question 1

\n

One of the data fields is called flow_artifact and has no description in the official data description page.

\n

I presume this means there's an artifact that might look like PE but is not (similar in function to the true_filling_defect_not_pe label)

\n

Question 2

\n

Are the informational labels, per image or per study? After some quick EDA I found that the qa_motion and qa_contrast probably are per_study, flow_artifact seems to be per image, and true_filling_defect_not_pe I'm not sure because it's so rare.

", "messages": [ { "id": 1045400, "postDate": "2020-10-10T15:34:10.777Z", "content": "

@alexandersoare few of possible inconsistencies

\n

negative_exam_for_pe
\npe_present_on_image
\nI expected below to give no rows but it returns 2268 patients
\ntrain_df[(train_df.negative_exam_for_pe==0) & (train_df.pe_present_on_image==0) ].StudyInstanceUID.nunique()
\nideally pe_present_on_image should be 1 here as -ve exam for pe is positive

\n

train_df[(train_df.negative_exam_for_pe==1) & (train_df.pe_present_on_image==0) ].StudyInstanceUID.nunique()

\n

This gives as expected i think 4911

\n

train_df[(train_df.negative_exam_for_pe==1) & (train_df.pe_present_on_image==1) ].StudyInstanceUID.nunique()
\nthis also gives as expected no rows..

", "rawMarkdown": "@alexandersoare few of possible inconsistencies\n\nnegative_exam_for_pe\npe_present_on_image \nI expected below to give no rows but it returns 2268 patients \n`train_df[(train_df.negative_exam_for_pe==0) & (train_df.pe_present_on_image==0) ].StudyInstanceUID.nunique()`\nideally pe_present_on_image should be 1 here as -ve exam for pe is positive\n\n`train_df[(train_df.negative_exam_for_pe==1) & (train_df.pe_present_on_image==0) ].StudyInstanceUID.nunique()`\n\nThis gives as expected i think 4911\n\n`train_df[(train_df.negative_exam_for_pe==1) & (train_df.pe_present_on_image==1) ].StudyInstanceUID.nunique()`\nthis also gives as expected no rows..\n\n", "replies": [ { "id": 1045470, "postDate": "2020-10-10T17:13:22.747Z", "content": "

I think it's totally okay that train_df[(train_df.negative_exam_for_pe==0) & (train_df.pe_present_on_image==0) ].StudyInstanceUID.nunique() = 2268 . This just means that there exist patients where not every image has evidence of PE, but at least 1 does.

", "rawMarkdown": "I think it's totally okay that `train_df[(train_df.negative_exam_for_pe==0) & (train_df.pe_present_on_image==0) ].StudyInstanceUID.nunique()` = 2268 . This just means that there exist patients where not every image has evidence of PE, but at least 1 does." }, { "id": 1045473, "postDate": "2020-10-10T17:15:50.583Z", "content": "

yeah i realized that..

\n

but
\n in that case when i check the fields like left,right central pe have got label as 1 which i think are at study level. I m bit confused here .
\nDoes it means that for a study with say 300 slices or images then for whichever image if field pe present on image is 1 then the location of PE would be as per study level fields like left,right,central ,ratio fields ?

", "rawMarkdown": "yeah i realized that.. \n\nbut\n in that case when i check the fields like left,right central pe have got label as 1 which i think are at study level. I m bit confused here .\nDoes it means that for a study with say 300 slices or images then for whichever image if field pe present on image is 1 then the location of PE would be as per study level fields like left,right,central ,ratio fields ?" }, { "id": 1045478, "postDate": "2020-10-10T17:22:19.793Z", "content": "

I do think that leftside_pe, rightside_pe and central_pe are all study level labels (as advised here). I think the way to interpret this is: if there is at least one image with PE present in left/right/center then set leftside_pe/rightside_pe/central_pe to 1 (respectively).

", "rawMarkdown": "I do think that `leftside_pe`, `rightside_pe` and `central_pe` are all study level labels (as advised [here](https://www.kaggle.com/c/rsna-str-pulmonary-embolism-detection/data)). I think the way to interpret this is: if there is at least one image with PE present in left/right/center then set `leftside_pe`/`rightside_pe`/`central_pe` to 1 (respectively)." }, { "id": 1045479, "postDate": "2020-10-10T17:24:15.437Z", "content": "

any image with pe not present,will have all other fields at image level to be as 0 right from model stand point ?

", "rawMarkdown": "any image with pe not present,will have all other fields at image level to be as 0 right from model stand point ?" }, { "id": 1045483, "postDate": "2020-10-10T17:30:15.353Z", "content": "

I think the only image level label is pe_present_on_image. All other labels will pertain to the study, not the image.

", "rawMarkdown": "I think the only image level label is `pe_present_on_image`. All other labels will pertain to the study, not the image." }, { "id": 1048095, "postDate": "2020-10-13T07:30:50.057Z", "content": "

true_filling_defect_not_pe - informational, indicates a defect that is NOT PE

\n

this field is interesting..
\nwould it mean that such slices be ignored as defect found is not a PE…

", "rawMarkdown": "`true_filling_defect_not_pe - informational, indicates a defect that is NOT PE`\n\nthis field is interesting..\nwould it mean that such slices be ignored as defect found is not a PE..." }, { "id": 1048289, "postDate": "2020-10-13T11:07:40.207Z", "content": "

I suppose so. Yep

", "rawMarkdown": "I suppose so. Yep" } ] }, { "id": 1045303, "postDate": "2020-10-10T13:59:55.857Z", "content": "

Hi fellow Kagglers. Just catching myself up with this competition and after some time I still have some left over questions I hope you can help me with.

\n

Question 1

\n

One of the data fields is called flow_artifact and has no description in the official data description page.

\n

I presume this means there's an artifact that might look like PE but is not (similar in function to the true_filling_defect_not_pe label)

\n

Question 2

\n

Are the informational labels, per image or per study? After some quick EDA I found that the qa_motion and qa_contrast probably are per_study, flow_artifact seems to be per image, and true_filling_defect_not_pe I'm not sure because it's so rare.

", "rawMarkdown": "Hi fellow Kagglers. Just catching myself up with this competition and after some time I still have some left over questions I hope you can help me with.\n\n## Question 1\n\nOne of the data fields is called `flow_artifact` and has no description in the official [data description page](https://www.kaggle.com/c/rsna-str-pulmonary-embolism-detection/data).\n\nI presume this means there's an artifact that might look like PE but is not (similar in function to the `true_filling_defect_not_pe` label)\n\n## Question 2\n\nAre the _informational labels_, _per image_ or _per study_? After some quick EDA I found that the `qa_motion` and `qa_contrast` probably are _per_study_, `flow_artifact` seems to be _per image_, and `true_filling_defect_not_pe` I'm not sure because it's so rare." } ], "comments": [ { "id": 1045400, "author_name": "Jaideep", "author_url": "", "post_date": "2020-10-10T15:34:10.777000", "content": "

@alexandersoare few of possible inconsistencies

\n

negative_exam_for_pe
\npe_present_on_image
\nI expected below to give no rows but it returns 2268 patients
\ntrain_df[(train_df.negative_exam_for_pe==0) & (train_df.pe_present_on_image==0) ].StudyInstanceUID.nunique()
\nideally pe_present_on_image should be 1 here as -ve exam for pe is positive

\n

train_df[(train_df.negative_exam_for_pe==1) & (train_df.pe_present_on_image==0) ].StudyInstanceUID.nunique()

\n

This gives as expected i think 4911

\n

train_df[(train_df.negative_exam_for_pe==1) & (train_df.pe_present_on_image==1) ].StudyInstanceUID.nunique()
\nthis also gives as expected no rows..

", "votes": 0, "replies": [ { "id": 1045470, "author_name": "Alexander Soare", "author_url": "", "post_date": "2020-10-10T17:13:22.747000", "content": "

I think it's totally okay that train_df[(train_df.negative_exam_for_pe==0) & (train_df.pe_present_on_image==0) ].StudyInstanceUID.nunique() = 2268 . This just means that there exist patients where not every image has evidence of PE, but at least 1 does.

", "votes": 0, "replies": [] }, { "id": 1045473, "author_name": "Jaideep", "author_url": "", "post_date": "2020-10-10T17:15:50.583000", "content": "

yeah i realized that..

\n

but
\n in that case when i check the fields like left,right central pe have got label as 1 which i think are at study level. I m bit confused here .
\nDoes it means that for a study with say 300 slices or images then for whichever image if field pe present on image is 1 then the location of PE would be as per study level fields like left,right,central ,ratio fields ?

", "votes": 0, "replies": [] }, { "id": 1045478, "author_name": "Alexander Soare", "author_url": "", "post_date": "2020-10-10T17:22:19.793000", "content": "

I do think that leftside_pe, rightside_pe and central_pe are all study level labels (as advised here). I think the way to interpret this is: if there is at least one image with PE present in left/right/center then set leftside_pe/rightside_pe/central_pe to 1 (respectively).

", "votes": 0, "replies": [] }, { "id": 1045479, "author_name": "Jaideep", "author_url": "", "post_date": "2020-10-10T17:24:15.437000", "content": "

any image with pe not present,will have all other fields at image level to be as 0 right from model stand point ?

", "votes": 0, "replies": [] }, { "id": 1045483, "author_name": "Alexander Soare", "author_url": "", "post_date": "2020-10-10T17:30:15.353000", "content": "

I think the only image level label is pe_present_on_image. All other labels will pertain to the study, not the image.

", "votes": 0, "replies": [] }, { "id": 1048095, "author_name": "Jaideep", "author_url": "", "post_date": "2020-10-13T07:30:50.057000", "content": "

true_filling_defect_not_pe - informational, indicates a defect that is NOT PE

\n

this field is interesting..
\nwould it mean that such slices be ignored as defect found is not a PE…

", "votes": 0, "replies": [] }, { "id": 1048289, "author_name": "Alexander Soare", "author_url": "", "post_date": "2020-10-13T11:07:40.207000", "content": "

I suppose so. Yep

", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "1045400": "@alexandersoare few of possible inconsistencies\n\nnegative_exam_for_pe\npe_present_on_image \nI expected below to give no rows but it returns 2268 patients \n`train_df[(train_df.negative_exam_for_pe==0) & (train_df.pe_present_on_image==0) ].StudyInstanceUID.nunique()`\nideally pe_present_on_image should be 1 here as -ve exam for pe is positive\n\n`train_df[(train_df.negative_exam_for_pe==1) & (train_df.pe_present_on_image==0) ].StudyInstanceUID.nunique()`\n\nThis gives as expected i think 4911\n\n`train_df[(train_df.negative_exam_for_pe==1) & (train_df.pe_present_on_image==1) ].StudyInstanceUID.nunique()`\nthis also gives as expected no rows..\n\n", "1045303": "Hi fellow Kagglers. Just catching myself up with this competition and after some time I still have some left over questions I hope you can help me with.\n\n## Question 1\n\nOne of the data fields is called `flow_artifact` and has no description in the official [data description page](https://www.kaggle.com/c/rsna-str-pulmonary-embolism-detection/data).\n\nI presume this means there's an artifact that might look like PE but is not (similar in function to the `true_filling_defect_not_pe` label)\n\n## Question 2\n\nAre the _informational labels_, _per image_ or _per study_? After some quick EDA I found that the `qa_motion` and `qa_contrast` probably are _per_study_, `flow_artifact` seems to be _per image_, and `true_filling_defect_not_pe` I'm not sure because it's so rare." } }