{ "id": 432745, "title": "[Solved] Submission scoring error", "url": "/competitions/rsna-2023-abdominal-trauma-detection/discussion/432745", "author_name": "Priya Nagda", "post_date": "2023-08-18T18:07:37.373000", "votes": 1, "comment_count": 19, "views": 0, "content": "

After an hour of running :/
\nAny pointers on what could be the issue?
\n\"\"

\n

\"\"

", "messages": [ { "id": 2397255, "postDate": "2023-08-18T21:09:14.213Z", "content": "

Hi, probably is due to the exponent numbers inside the pd dataframe, you can try with this code:

\n

sub_df.to_csv(\"submission.csv\", index=False, float_format='%.20f')

", "rawMarkdown": "Hi, probably is due to the exponent numbers inside the pd dataframe, you can try with this code:\n\n``sub_df.to_csv(\"submission.csv\", index=False, float_format='%.20f')``", "votes": 1, "replies": [ { "id": 2397443, "postDate": "2023-08-19T03:25:46.763Z", "content": "

tried this, still gives the same error
\nalso for anyone else trying out the same thing, sub_df.to_csv(\"submission.csv\", index=False, float_format='%.20f') doesn't work for me. I had to use sub_df.round(3).to_csv(\"submission.csv\", index=False) just in case it doesn't work for you too,

", "rawMarkdown": "tried this, still gives the same error \nalso for anyone else trying out the same thing, `sub_df.to_csv(\"submission.csv\", index=False, float_format='%.20f')` doesn't work for me. I had to use `sub_df.round(3).to_csv(\"submission.csv\", index=False)` just in case it doesn't work for you too,", "votes": 1 } ] }, { "id": 2397106, "postDate": "2023-08-18T18:07:37.373Z", "content": "

After an hour of running :/
\nAny pointers on what could be the issue?
\n\"\"

\n

\"\"

", "rawMarkdown": "After an hour of running :/\nAny pointers on what could be the issue?\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F7037441%2F44071aea474c2c3ad0207a3f1f228447%2FScreenshot%202023-08-18%20at%2011.34.55%20PM.png?generation=1692382025922257&alt=media)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F7037441%2Fe46f25696c6ef1f16775e496937bdf1e%2FScreenshot%202023-08-18%20at%2011.34.24%20PM.png?generation=1692382039233228&alt=media)", "votes": 1 }, { "id": 2475786, "postDate": "2023-10-10T06:45:49.373Z", "content": "

hi, can you show us your result, I just want to see the format of the submission.csv, you could fill whatever values you what

", "rawMarkdown": "hi, can you show us your result, I just want to see the format of the submission.csv, you could fill whatever values you what", "replies": [ { "id": 2476048, "postDate": "2023-10-10T09:54:57.987Z", "content": "

As I figured it out…the format is same as sample_submission…no matter if you round values or not….

\n

MAIN ISSUE seems to be that EVERY SINGLE test patient should be present in the submission file!!!! Order also doesnt matter as the scoring is done on group weighted loss average(please have a look at the scoring script!)..
\nThe most likely reason (that's what was in my case)..if there is an exception in reading DICOM files , one might be skipping it…that means that patient-id is skipped altogether in submission file which LEADS to 'Submission Scoring Error'…because \"solution\" is coming in with all the test patients and submission file will not have the skipped patients… at least this was my case…

", "rawMarkdown": "As I figured it out...the format is same as sample_submission...no matter if you round values or not....\n\nMAIN ISSUE seems to be that EVERY SINGLE test patient should be present in the submission file!!!! Order also doesnt matter as the scoring is done on group weighted loss average(please have a look at the scoring script!)..\nThe most likely reason (that's what was in my case)..if there is an exception in reading DICOM files , one might be skipping it...that means that patient-id is skipped altogether in submission file which LEADS to 'Submission Scoring Error'...because \"solution\" is coming in with all the test patients and submission file will not have the skipped patients... at least this was my case...\n\n", "replies": [ { "id": 2476070, "postDate": "2023-10-10T10:13:37.113Z", "content": "

I use the file name as my patient_id, and check the number of series_id,I am not sure is it work, but I still have the Submission scoring error problems

", "rawMarkdown": "I use the file name as my patient_id, and check the number of series_id,I am not sure is it work, but I still have the Submission scoring error problems" } ] } ] }, { "id": 2404412, "postDate": "2023-08-23T08:37:06.327Z", "content": "

Hey @priyanagda
\nHow did you solve it
\nI am stuck for no reason

", "rawMarkdown": "Hey @priyanagda \nHow did you solve it\nI am stuck for no reason", "replies": [ { "id": 2404538, "postDate": "2023-08-23T09:56:10.793Z", "content": "

could you share your notebook?

", "rawMarkdown": "could you share your notebook?", "replies": [ { "id": 2405598, "postDate": "2023-08-24T02:26:55.130Z", "content": "
pd.set_option(, .)\ndf_sub=pd.read_csv()\ndf_sub[[col  col  df_sub.columns  col !=]]= df_sub[[col  col  df_sub.columns  col !=]].astype()\nsub_df = pd.read_csv()\nsub_df = sub_df[[]]\nsub_df = sub_df.merge(df_sub, on=, how=)\n\n\nsub_df.to_csv(,index=)\nsub_df.head()\n
\n

This is all what I did @priyanagda

", "rawMarkdown": "```python\npd.set_option('display.float_format', '{:.16f}'.format)\ndf_sub=pd.read_csv('/kaggle/input/rsna-submissions-hemanth/RSNA-ATD(1).csv')\ndf_sub[[col for col in df_sub.columns if col !=\"patient_id\"]]= df_sub[[col for col in df_sub.columns if col !=\"patient_id\"]].astype('float64')\nsub_df = pd.read_csv('/kaggle/input/rsna-2023-abdominal-trauma-detection/sample_submission.csv')\nsub_df = sub_df[['patient_id']]\nsub_df = sub_df.merge(df_sub, on='patient_id', how='left')\n\n# Store submission\nsub_df.to_csv('submission.csv',index=False)\nsub_df.head()\n```\n\nThis is all what I did @priyanagda " }, { "id": 2406010, "postDate": "2023-08-24T07:29:47.797Z", "content": "

Hard to say anything since I don't know what your submission file looks like. My assumptions:

\n", "rawMarkdown": "Hard to say anything since I don't know what your submission file looks like. My assumptions:\n- You probably have more than one entry/row for a single patient.\n- NaN values in predictions (try something like fillna(0) and submitting, if this solves it you know the error)\n- Keep only the patient_id column and the target cols to be on the safe side" } ] } ] }, { "id": 2397536, "postDate": "2023-08-19T05:33:18.163Z", "content": "

Better help possible if you make your notebook public and put a link here!

", "rawMarkdown": "Better help possible if you make your notebook public and put a link here!", "replies": [ { "id": 2398656, "postDate": "2023-08-19T19:48:46.533Z", "content": "

Hi, this is the notebook I am trying to submit.

", "rawMarkdown": "Hi, [this](https://www.kaggle.com/code/priyanagda/atd-inference) is the notebook I am trying to submit.", "replies": [ { "id": 2399062, "postDate": "2023-08-20T06:01:39.543Z", "content": "

Did not see anything obvious - was going to fork it and run a larger test set that I have created, but your model pth file is private - make it public also.

\n

As a first guess I would change the batch size. With the test set we have you can load all the images for a patient (of course there is only 1) and not happen upon a memory error that results in incomplete csv file being created. But a patient with 700 slices in a DICOM would likely cause memory error (if you stride =1).

", "rawMarkdown": "Did not see anything obvious - was going to fork it and run a larger [test set](https://www.kaggle.com/datasets/pcjimmmy/rsna-7-patients-test) that I have created, but your model pth file is private - make it public also.\n\nAs a first guess I would change the batch size. With the test set we have you can load all the images for a patient (of course there is only 1) and not happen upon a memory error that results in incomplete csv file being created. But a patient with 700 slices in a DICOM would likely cause memory error (if you stride =1)." }, { "id": 2399206, "postDate": "2023-08-20T07:52:46.877Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2399210, "postDate": "2023-08-20T07:53:49.793Z", "content": "

I believe the notebook is running fine, because if it would have been a memory error, it would have given Notebook threw an exception, isn't it?

", "rawMarkdown": "I believe the notebook is running fine, because if it would have been a memory error, it would have given Notebook threw an exception, isn't it?" }, { "id": 2400266, "postDate": "2023-08-21T00:01:40.183Z", "content": "

Hmm your notebook link is 404 error. Title now says Solved - assume that means you figured it out.

", "rawMarkdown": "Hmm your notebook link is 404 error. Title now says Solved - assume that means you figured it out." }, { "id": 2467929, "postDate": "2023-10-05T03:08:46.103Z", "content": "

And how did all solve the \"Submission Scoring Problem\" finally…I seem to have no 'luck' with that…deliberately using the term 'luck' as there is no specificity about the error…I seem to have more 'try…catch\" than the actual inference code…

", "rawMarkdown": "And how did all solve the \"Submission Scoring Problem\" finally...I seem to have no 'luck' with that...deliberately using the term 'luck' as there is no specificity about the error...I seem to have more 'try...catch\" than the actual inference code..." } ] } ] }, { "id": 2397475, "postDate": "2023-08-19T04:16:43.990Z", "content": "

Given its a submission scoring error, you can generate predictions for the train set, and then use this notebook (https://www.kaggle.com/code/metric/rsna-trauma-metric/notebook) to compute the score on the train set. Check if there are any errors.

", "rawMarkdown": "Given its a submission scoring error, you can generate predictions for the train set, and then use this notebook (https://www.kaggle.com/code/metric/rsna-trauma-metric/notebook) to compute the score on the train set. Check if there are any errors.", "replies": [ { "id": 2413667, "postDate": "2023-08-29T05:00:50.523Z", "content": "

I had this error and like you suggested. I was able to successfully make submission. Thank you for this comment!

", "rawMarkdown": "I had this error and like you suggested. I was able to successfully make submission. Thank you for this comment!", "votes": 1 } ] }, { "id": 2416753, "postDate": "2023-08-31T07:38:42.180Z", "content": "

del solution[row_id_column_name]
\ndel submission[row_id_column_name]

\n

what is row_id_column_name ???

", "rawMarkdown": "del solution[row_id_column_name]\ndel submission[row_id_column_name]\n\nwhat is row_id_column_name ??? ", "isDeleted": true } ], "comments": [ { "id": 2397255, "author_name": "Antonio Félix", "author_url": "", "post_date": "2023-08-18T21:09:14.213000", "content": "

Hi, probably is due to the exponent numbers inside the pd dataframe, you can try with this code:

\n

sub_df.to_csv(\"submission.csv\", index=False, float_format='%.20f')

", "votes": 1, "replies": [ { "id": 2397443, "author_name": "Priya Nagda", "author_url": "", "post_date": "2023-08-19T03:25:46.763000", "content": "

tried this, still gives the same error
\nalso for anyone else trying out the same thing, sub_df.to_csv(\"submission.csv\", index=False, float_format='%.20f') doesn't work for me. I had to use sub_df.round(3).to_csv(\"submission.csv\", index=False) just in case it doesn't work for you too,

", "votes": 1, "replies": [] } ] }, { "id": 2475786, "author_name": "john", "author_url": "", "post_date": "2023-10-10T06:45:49.373000", "content": "

hi, can you show us your result, I just want to see the format of the submission.csv, you could fill whatever values you what

", "votes": 0, "replies": [ { "id": 2476048, "author_name": "Sunil Krishnan", "author_url": "", "post_date": "2023-10-10T09:54:57.987000", "content": "

As I figured it out…the format is same as sample_submission…no matter if you round values or not….

\n

MAIN ISSUE seems to be that EVERY SINGLE test patient should be present in the submission file!!!! Order also doesnt matter as the scoring is done on group weighted loss average(please have a look at the scoring script!)..
\nThe most likely reason (that's what was in my case)..if there is an exception in reading DICOM files , one might be skipping it…that means that patient-id is skipped altogether in submission file which LEADS to 'Submission Scoring Error'…because \"solution\" is coming in with all the test patients and submission file will not have the skipped patients… at least this was my case…

", "votes": 0, "replies": [ { "id": 2476070, "author_name": "john", "author_url": "", "post_date": "2023-10-10T10:13:37.113000", "content": "

I use the file name as my patient_id, and check the number of series_id,I am not sure is it work, but I still have the Submission scoring error problems

", "votes": 0, "replies": [] } ] } ] }, { "id": 2404412, "author_name": "Hemanth Harikrishnan", "author_url": "", "post_date": "2023-08-23T08:37:06.327000", "content": "

Hey @priyanagda
\nHow did you solve it
\nI am stuck for no reason

", "votes": 0, "replies": [ { "id": 2404538, "author_name": "Priya Nagda", "author_url": "", "post_date": "2023-08-23T09:56:10.793000", "content": "

could you share your notebook?

", "votes": 0, "replies": [ { "id": 2405598, "author_name": "Hemanth Harikrishnan", "author_url": "", "post_date": "2023-08-24T02:26:55.130000", "content": "
pd.set_option(, .)\ndf_sub=pd.read_csv()\ndf_sub[[col  col  df_sub.columns  col !=]]= df_sub[[col  col  df_sub.columns  col !=]].astype()\nsub_df = pd.read_csv()\nsub_df = sub_df[[]]\nsub_df = sub_df.merge(df_sub, on=, how=)\n\n\nsub_df.to_csv(,index=)\nsub_df.head()\n
\n

This is all what I did @priyanagda

", "votes": 0, "replies": [] }, { "id": 2406010, "author_name": "Priya Nagda", "author_url": "", "post_date": "2023-08-24T07:29:47.797000", "content": "

Hard to say anything since I don't know what your submission file looks like. My assumptions:

\n", "votes": 0, "replies": [] } ] } ] }, { "id": 2397536, "author_name": "PC Jimmmy", "author_url": "", "post_date": "2023-08-19T05:33:18.163000", "content": "

Better help possible if you make your notebook public and put a link here!

", "votes": 0, "replies": [ { "id": 2398656, "author_name": "Priya Nagda", "author_url": "", "post_date": "2023-08-19T19:48:46.533000", "content": "

Hi, this is the notebook I am trying to submit.

", "votes": 0, "replies": [ { "id": 2399062, "author_name": "PC Jimmmy", "author_url": "", "post_date": "2023-08-20T06:01:39.543000", "content": "

Did not see anything obvious - was going to fork it and run a larger test set that I have created, but your model pth file is private - make it public also.

\n

As a first guess I would change the batch size. With the test set we have you can load all the images for a patient (of course there is only 1) and not happen upon a memory error that results in incomplete csv file being created. But a patient with 700 slices in a DICOM would likely cause memory error (if you stride =1).

", "votes": 0, "replies": [] }, { "id": 2399206, "author_name": "", "author_url": "", "post_date": "2023-08-20T07:52:46.877000", "content": "", "votes": 0, "replies": [] }, { "id": 2399210, "author_name": "Priya Nagda", "author_url": "", "post_date": "2023-08-20T07:53:49.793000", "content": "

I believe the notebook is running fine, because if it would have been a memory error, it would have given Notebook threw an exception, isn't it?

", "votes": 0, "replies": [] }, { "id": 2400266, "author_name": "PC Jimmmy", "author_url": "", "post_date": "2023-08-21T00:01:40.183000", "content": "

Hmm your notebook link is 404 error. Title now says Solved - assume that means you figured it out.

", "votes": 0, "replies": [] }, { "id": 2467929, "author_name": "Sunil Krishnan", "author_url": "", "post_date": "2023-10-05T03:08:46.103000", "content": "

And how did all solve the \"Submission Scoring Problem\" finally…I seem to have no 'luck' with that…deliberately using the term 'luck' as there is no specificity about the error…I seem to have more 'try…catch\" than the actual inference code…

", "votes": 0, "replies": [] } ] } ] }, { "id": 2397475, "author_name": "Jebastin Nadar", "author_url": "", "post_date": "2023-08-19T04:16:43.990000", "content": "

Given its a submission scoring error, you can generate predictions for the train set, and then use this notebook (https://www.kaggle.com/code/metric/rsna-trauma-metric/notebook) to compute the score on the train set. Check if there are any errors.

", "votes": 0, "replies": [ { "id": 2413667, "author_name": "Franklin Shih0617", "author_url": "", "post_date": "2023-08-29T05:00:50.523000", "content": "

I had this error and like you suggested. I was able to successfully make submission. Thank you for this comment!

", "votes": 1, "replies": [] } ] }, { "id": 2416753, "author_name": "", "author_url": "", "post_date": "2023-08-31T07:38:42.180000", "content": "

del solution[row_id_column_name]
\ndel submission[row_id_column_name]

\n

what is row_id_column_name ???

", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "2397255": "Hi, probably is due to the exponent numbers inside the pd dataframe, you can try with this code:\n\n``sub_df.to_csv(\"submission.csv\", index=False, float_format='%.20f')``", "2397106": "After an hour of running :/\nAny pointers on what could be the issue?\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F7037441%2F44071aea474c2c3ad0207a3f1f228447%2FScreenshot%202023-08-18%20at%2011.34.55%20PM.png?generation=1692382025922257&alt=media)\n\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F7037441%2Fe46f25696c6ef1f16775e496937bdf1e%2FScreenshot%202023-08-18%20at%2011.34.24%20PM.png?generation=1692382039233228&alt=media)", "2475786": "hi, can you show us your result, I just want to see the format of the submission.csv, you could fill whatever values you what", "2404412": "Hey @priyanagda \nHow did you solve it\nI am stuck for no reason", "2397536": "Better help possible if you make your notebook public and put a link here!", "2397475": "Given its a submission scoring error, you can generate predictions for the train set, and then use this notebook (https://www.kaggle.com/code/metric/rsna-trauma-metric/notebook) to compute the score on the train set. Check if there are any errors.", "2416753": "del solution[row_id_column_name]\ndel submission[row_id_column_name]\n\nwhat is row_id_column_name ??? " } }