i wonder has anyone done a local validation score for non-overlapping patent id?
\n\nor has anyone submitted only non-overlapping patient to the LB test server for probing?
\n\nmy experiments show that if there is totally on overlap, the drop in log loss core can be as much as 0.025, so do we expect a huge shakeup?
", "messages": [ { "id": 664008, "postDate": "2019-11-03T02:55:18.170Z", "content": "Unique PatientIDs:\nTraining set: 17079\nStage 1 test set: 2144\nConcatenated training + stage 1 test sets: 18938
\n\nOverlapping patientIDs between train and stage 1 test: 285 (1.5% overall)\n I am doing a stratified patient splitting by dominant patient label and I have a 0.01 improvement on LB compared to CV.
", "rawMarkdown": "**Unique PatientIDs:**\nTraining set: **17079**\nStage 1 test set: **2144**\nConcatenated training + stage 1 test sets: **18938**\n\nOverlapping patientIDs between train and stage 1 test: **285** (1.5% overall)\n I am doing a stratified patient splitting by dominant patient label and I have a 0.01 improvement on LB compared to CV.", "votes": 4 }, { "id": 663574, "postDate": "2019-11-02T10:44:03.163Z", "content": "i wonder has anyone done a local validation score for non-overlapping patent id?
\n\nor has anyone submitted only non-overlapping patient to the LB test server for probing?
\n\nmy experiments show that if there is totally on overlap, the drop in log loss core can be as much as 0.025, so do we expect a huge shakeup?
", "rawMarkdown": "i wonder has anyone done a local validation score for non-overlapping patent id?\n\nor has anyone submitted only non-overlapping patient to the LB test server for probing?\n\nmy experiments show that if there is totally on overlap, the drop in log loss core can be as much as 0.025, so do we expect a huge shakeup?\n\n\n\n", "votes": 2 }, { "id": 663750, "postDate": "2019-11-02T16:13:48.790Z", "content": "additional results:
\n\n288x288-desenset121: LB 0.73 (train with 40% of train data for quick experiment)\nlocal cross validation without patient id overlap 0.101
", "rawMarkdown": "additional results:\n\n288x288-desenset121: LB 0.73 (train with 40% of train data for quick experiment)\nlocal cross validation without patient id overlap 0.101\n" }, { "id": 663590, "postDate": "2019-11-02T11:37:23.007Z", "content": "In your experiments, are you also splitting up images within series? It seems like there is some patient overlap, but no series overlap (i.e., images from one scan are not split across train/test).
", "rawMarkdown": "In your experiments, are you also splitting up images within series? It seems like there is some patient overlap, but no series overlap (i.e., images from one scan are not split across train/test).", "replies": [ { "id": 663749, "postDate": "2019-11-02T16:12:23.393Z", "content": "no.
\n\nfor experiment with overlapping patient id, i just randomly split the images.
", "rawMarkdown": "no. \n\nfor experiment with overlapping patient id, i just randomly split the images." } ] } ], "comments": [ { "id": 664008, "author_name": "Alexandre Cadrin-Chênevert", "author_url": "", "post_date": "2019-11-03T02:55:18.170000", "content": "Unique PatientIDs:\nTraining set: 17079\nStage 1 test set: 2144\nConcatenated training + stage 1 test sets: 18938
\n\nOverlapping patientIDs between train and stage 1 test: 285 (1.5% overall)\n I am doing a stratified patient splitting by dominant patient label and I have a 0.01 improvement on LB compared to CV.
", "votes": 4, "replies": [] }, { "id": 663750, "author_name": "hengck23", "author_url": "", "post_date": "2019-11-02T16:13:48.790000", "content": "additional results:
\n\n288x288-desenset121: LB 0.73 (train with 40% of train data for quick experiment)\nlocal cross validation without patient id overlap 0.101
", "votes": 0, "replies": [] }, { "id": 663590, "author_name": "Ian Pan", "author_url": "", "post_date": "2019-11-02T11:37:23.007000", "content": "In your experiments, are you also splitting up images within series? It seems like there is some patient overlap, but no series overlap (i.e., images from one scan are not split across train/test).
", "votes": 0, "replies": [ { "id": 663749, "author_name": "hengck23", "author_url": "", "post_date": "2019-11-02T16:12:23.393000", "content": "no.
\n\nfor experiment with overlapping patient id, i just randomly split the images.
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "664008": "**Unique PatientIDs:**\nTraining set: **17079**\nStage 1 test set: **2144**\nConcatenated training + stage 1 test sets: **18938**\n\nOverlapping patientIDs between train and stage 1 test: **285** (1.5% overall)\n I am doing a stratified patient splitting by dominant patient label and I have a 0.01 improvement on LB compared to CV.", "663574": "i wonder has anyone done a local validation score for non-overlapping patent id?\n\nor has anyone submitted only non-overlapping patient to the LB test server for probing?\n\nmy experiments show that if there is totally on overlap, the drop in log loss core can be as much as 0.025, so do we expect a huge shakeup?\n\n\n\n", "663750": "additional results:\n\n288x288-desenset121: LB 0.73 (train with 40% of train data for quick experiment)\nlocal cross validation without patient id overlap 0.101\n", "663590": "In your experiments, are you also splitting up images within series? It seems like there is some patient overlap, but no series overlap (i.e., images from one scan are not split across train/test)." } }