{ "id": 155914, "title": "Which is the best? : classification, regression, ordinal regression etc...", "url": "/competitions/prostate-cancer-grade-assessment/discussion/155914", "author_name": "RabotniKuma", "post_date": "2020-06-03T14:01:27.520000", "votes": 24, "comment_count": 13, "views": 0, "content": "

This task can be solved in many ways: classification(e.g. cross entropy), regression(e.g. mae), ordinal regression(e.g. paper ) or others.

\n\n

So far I tried all of these three methods mentioned above and, to my surprise, cross entropy loss worked best. Just FYI, my best submission (public LB 0.90) is trained with cross entropy, single fold validation.

\n\n

I wonder how others deal with this task and which one performs best.

", "messages": [ { "id": 872790, "postDate": "2020-06-03T14:01:27.520Z", "content": "

This task can be solved in many ways: classification(e.g. cross entropy), regression(e.g. mae), ordinal regression(e.g. paper ) or others.

\n\n

So far I tried all of these three methods mentioned above and, to my surprise, cross entropy loss worked best. Just FYI, my best submission (public LB 0.90) is trained with cross entropy, single fold validation.

\n\n

I wonder how others deal with this task and which one performs best.

", "rawMarkdown": "This task can be solved in many ways: classification(e.g. cross entropy), regression(e.g. mae), ordinal regression(e.g. [paper](https://www.cv-foundation.org/openaccess/content_cvpr_2016/app/S21-20.pdf) ) or others.\n\nSo far I tried all of these three methods mentioned above and, to my surprise, cross entropy loss worked best. Just FYI, my best submission (public LB 0.90) is trained with cross entropy, single fold validation. \n\nI wonder how others deal with this task and which one performs best.", "votes": 24 }, { "id": 872828, "postDate": "2020-06-03T14:39:57.770Z", "content": "

Interesting topic.\nI use regression and scores 0.87 to 0.89 on LB.

\n\n

```\nfold cv rad kar lb epoch

\n\n
\n\n

0 0.89 0.85 0.90 0.87 28\n1 0.88 0.85 0.89 0.89 23 \n2 0.89 0.86 0.89 0.88 16\n3 0.89 0.86 0.89 0.88 29\n4 0.88 0.85 0.87 0.88 27\n```

\n\n

I haven't tried any other approaches yet but probably try them later. I kind of doubt they make a huge difference in terms of LB but could be a good option for ensembling as @drhabib mentioned.

", "rawMarkdown": "Interesting topic.\nI use regression and scores 0.87 to 0.89 on LB.\n\n```\nfold cv rad kar lb epoch \n---- ---- ---- ---- ---- ----\n0 0.89 0.85 0.90 0.87 28\n1 0.88 0.85 0.89 0.89 23 \n2 0.89 0.86 0.89 0.88 16\n3 0.89 0.86 0.89 0.88 29\n4 0.88 0.85 0.87 0.88 27\n```\n\nI haven't tried any other approaches yet but probably try them later. I kind of doubt they make a huge difference in terms of LB but could be a good option for ensembling as @drhabib mentioned.", "votes": 8, "replies": [ { "id": 873340, "postDate": "2020-06-04T04:04:23.233Z", "content": "

Thank you for sharing your results! \nYour current best score is fold ensemble or fold 1? \nIt seems fold ensemble is not working well for many people in terms of public LB.

", "rawMarkdown": "Thank you for sharing your results! \nYour current best score is fold ensemble or fold 1? \nIt seems fold ensemble is not working well for many people in terms of public LB." }, { "id": 873453, "postDate": "2020-06-04T06:48:13.690Z", "content": "

You are welcome. It's from fold 1.\nYes, averaging over folds does not yield good LB score on this competition. I got slightly worse LB by averaging them.

", "rawMarkdown": "You are welcome. It's from fold 1.\nYes, averaging over folds does not yield good LB score on this competition. I got slightly worse LB by averaging them.", "votes": 1 }, { "id": 875628, "postDate": "2020-06-06T03:20:32.653Z", "content": "

@appian Hi , As you mentioned about regression (MAE), may you put some link describing about how to use regression for classification. \nThanks .

", "rawMarkdown": "@appian Hi , As you mentioned about regression (MAE), may you put some link describing about how to use regression for classification. \nThanks .", "votes": 1 }, { "id": 875674, "postDate": "2020-06-06T04:37:33.377Z", "content": "

@rajnishe The point is that this task is not a typical classification. In this task classes are ordinal. (e.g. ISUP grade 4 is worse than grade 3, and grade 3 worse than grade 2.)

", "rawMarkdown": "@rajnishe The point is that this task is not a typical classification. In this task classes are **ordinal**. (e.g. ISUP grade 4 is worse than grade 3, and grade 3 worse than grade 2.)", "votes": 2 } ] }, { "id": 872814, "postDate": "2020-06-03T14:22:38.430Z", "content": "

Its a good topic of discussion. My best model is regression single fold and it scores 0.89 both cv and lb. I have used same set up to run classification and it results in cv and lb of 0.88. I would say so far classification and regression perform similar. However it will be interesting to know if CE and MSE loss force CNN to learn similar or different features. And there is also option of blending =)

", "rawMarkdown": "Its a good topic of discussion. My best model is regression single fold and it scores `0.89` both `cv` and `lb`. I have used same set up to run classification and it results in cv and lb of `0.88`. I would say so far classification and regression perform similar. However it will be interesting to know if `CE` and `MSE` loss force `CNN` to learn similar or different features. And there is also option of blending =)", "votes": 6, "replies": [ { "id": 873776, "postDate": "2020-06-04T12:20:21.743Z", "content": "

@drhabib May you please explain regression way, how to use regression on a classification.\nThanks

", "rawMarkdown": "@drhabib May you please explain regression way, how to use regression on a classification.\nThanks", "votes": -1 } ] }, { "id": 876736, "postDate": "2020-06-07T01:49:08.533Z", "content": "

My best submission is done with regression, although since I switched from layer 2 to layer 1 image I have not made a submission using classification

", "rawMarkdown": "My best submission is done with regression, although since I switched from layer 2 to layer 1 image I have not made a submission using classification", "votes": 2 }, { "id": 873342, "postDate": "2020-06-04T04:06:51.813Z", "content": "

I used regression on LB

", "rawMarkdown": "I used regression on LB" }, { "id": 878398, "postDate": "2020-06-08T13:52:43.743Z", "rawMarkdown": "", "isDeleted": true }, { "id": 874354, "postDate": "2020-06-04T21:37:11.963Z", "rawMarkdown": "", "isDeleted": true }, { "id": 872817, "postDate": "2020-06-03T14:27:00.307Z", "rawMarkdown": "", "isDeleted": true, "replies": [ { "id": 872824, "postDate": "2020-06-03T14:30:35.950Z", "rawMarkdown": "", "isDeleted": true } ] } ], "comments": [ { "id": 872828, "author_name": "Appian", "author_url": "", "post_date": "2020-06-03T14:39:57.770000", "content": "

Interesting topic.\nI use regression and scores 0.87 to 0.89 on LB.

\n\n

```\nfold cv rad kar lb epoch

\n\n
\n\n

0 0.89 0.85 0.90 0.87 28\n1 0.88 0.85 0.89 0.89 23 \n2 0.89 0.86 0.89 0.88 16\n3 0.89 0.86 0.89 0.88 29\n4 0.88 0.85 0.87 0.88 27\n```

\n\n

I haven't tried any other approaches yet but probably try them later. I kind of doubt they make a huge difference in terms of LB but could be a good option for ensembling as @drhabib mentioned.

", "votes": 8, "replies": [ { "id": 873340, "author_name": "RabotniKuma", "author_url": "", "post_date": "2020-06-04T04:04:23.233000", "content": "

Thank you for sharing your results! \nYour current best score is fold ensemble or fold 1? \nIt seems fold ensemble is not working well for many people in terms of public LB.

", "votes": 0, "replies": [] }, { "id": 873453, "author_name": "Appian", "author_url": "", "post_date": "2020-06-04T06:48:13.690000", "content": "

You are welcome. It's from fold 1.\nYes, averaging over folds does not yield good LB score on this competition. I got slightly worse LB by averaging them.

", "votes": 1, "replies": [] }, { "id": 875628, "author_name": "Rajnish Chauhan", "author_url": "", "post_date": "2020-06-06T03:20:32.653000", "content": "

@appian Hi , As you mentioned about regression (MAE), may you put some link describing about how to use regression for classification. \nThanks .

", "votes": 1, "replies": [] }, { "id": 875674, "author_name": "RabotniKuma", "author_url": "", "post_date": "2020-06-06T04:37:33.377000", "content": "

@rajnishe The point is that this task is not a typical classification. In this task classes are ordinal. (e.g. ISUP grade 4 is worse than grade 3, and grade 3 worse than grade 2.)

", "votes": 2, "replies": [] } ] }, { "id": 872814, "author_name": "DrHB", "author_url": "", "post_date": "2020-06-03T14:22:38.430000", "content": "

Its a good topic of discussion. My best model is regression single fold and it scores 0.89 both cv and lb. I have used same set up to run classification and it results in cv and lb of 0.88. I would say so far classification and regression perform similar. However it will be interesting to know if CE and MSE loss force CNN to learn similar or different features. And there is also option of blending =)

", "votes": 6, "replies": [ { "id": 873776, "author_name": "Rajnish Chauhan", "author_url": "", "post_date": "2020-06-04T12:20:21.743000", "content": "

@drhabib May you please explain regression way, how to use regression on a classification.\nThanks

", "votes": -1, "replies": [] } ] }, { "id": 876736, "author_name": "Shujun", "author_url": "", "post_date": "2020-06-07T01:49:08.533000", "content": "

My best submission is done with regression, although since I switched from layer 2 to layer 1 image I have not made a submission using classification

", "votes": 2, "replies": [] }, { "id": 873342, "author_name": "Abhishh1", "author_url": "", "post_date": "2020-06-04T04:06:51.813000", "content": "

I used regression on LB

", "votes": 0, "replies": [] }, { "id": 878398, "author_name": "", "author_url": "", "post_date": "2020-06-08T13:52:43.743000", "content": "", "votes": 0, "replies": [] }, { "id": 874354, "author_name": "", "author_url": "", "post_date": "2020-06-04T21:37:11.963000", "content": "", "votes": 0, "replies": [] }, { "id": 872817, "author_name": "", "author_url": "", "post_date": "2020-06-03T14:27:00.307000", "content": "", "votes": 0, "replies": [ { "id": 872824, "author_name": "", "author_url": "", "post_date": "2020-06-03T14:30:35.950000", "content": "", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "872790": "This task can be solved in many ways: classification(e.g. cross entropy), regression(e.g. mae), ordinal regression(e.g. [paper](https://www.cv-foundation.org/openaccess/content_cvpr_2016/app/S21-20.pdf) ) or others.\n\nSo far I tried all of these three methods mentioned above and, to my surprise, cross entropy loss worked best. Just FYI, my best submission (public LB 0.90) is trained with cross entropy, single fold validation. \n\nI wonder how others deal with this task and which one performs best.", "872828": "Interesting topic.\nI use regression and scores 0.87 to 0.89 on LB.\n\n```\nfold cv rad kar lb epoch \n---- ---- ---- ---- ---- ----\n0 0.89 0.85 0.90 0.87 28\n1 0.88 0.85 0.89 0.89 23 \n2 0.89 0.86 0.89 0.88 16\n3 0.89 0.86 0.89 0.88 29\n4 0.88 0.85 0.87 0.88 27\n```\n\nI haven't tried any other approaches yet but probably try them later. I kind of doubt they make a huge difference in terms of LB but could be a good option for ensembling as @drhabib mentioned.", "872814": "Its a good topic of discussion. My best model is regression single fold and it scores `0.89` both `cv` and `lb`. I have used same set up to run classification and it results in cv and lb of `0.88`. I would say so far classification and regression perform similar. However it will be interesting to know if `CE` and `MSE` loss force `CNN` to learn similar or different features. And there is also option of blending =)", "876736": "My best submission is done with regression, although since I switched from layer 2 to layer 1 image I have not made a submission using classification", "873342": "I used regression on LB", "878398": "", "874354": "", "872817": "" } }