\n
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F5221270%2Fe15e5476151a8710b3aef7cf3f5736c9%2Findex.jpeg?generation=1603130236285447&alt=media\")
Hi Kagglers,
\nHope it is not too late to share this new baseline (8 days to go), I tried to do it ASAP, till recently I get more availability to get the work done, wait for the local training results finished and finally successfully submitted the kernel… The kernel is also my latest submission result as you could see from my submission records…
\nFor people who are struggling with the competition, here is a good example to help you start with.
\nFor people who are affected by the public standing, I apologize for my new baseline model and hope it could help you do better ensembling.
I can see this competition gathers a lot of ideas\\discussions to automate the PE detection process. I'm not having a medical science background, but I've been participating OSIC, SIIMC in the past 2 competitions, which are both related to medical science. These 3 competitions are what I think one of the truly valuable parts of AI\\ML- to help humans in real life. Hope my work could be possibly contributing to both communities- Medical and Kaggle.
\nOk, back to the topic…
\nThe kernel I'm sharing contains:
\n1. cross-validation strategy: https://www.kaggle.com/khyeh0719/stratified-validation-strategy
\n2. competition metric as loss function for NN: https://www.kaggle.com/khyeh0719/0929-updated-rsna-competition-metric
\n3. Rule-based post-processing
\n4. 2-stage model training
The model weights are here: https://www.kaggle.com/khyeh0719/kh-rsna-model
\nReferencing the ideas from the last RSNA, it composes of 2 stages:
\n1. CNN model
\n2. RNN model to combine the CNN model's result
In my implementation, I train CNN models to get OOF predictions for stage 2 model learning.
\nI use very simple efficientnet b0 as a starter for quick prototyping. To make stage 2 model capture the properties for other exam-level targets, I also did a multitask learning in stage 1 model.
\nSingle-label CNN modeling:
\nlocal 5-fold performances:
epoch 0 train step 5596/5596, loss: 0.1157, acc: 0.9606, time: 7311.1298\nepoch 0 valid Step 1400/1400, loss: 0.1245, acc: 0.9610, time: 1290.3861\n\nepoch 0 train step 5593/5593, loss: 0.1164, acc: 0.9604, time: 7316.3813\nepoch 0 valid Step 1402/1402, loss: 0.1522, acc: 0.9581, time: 1282.8907\n\nepoch 0 train step 5595/5595, loss: 0.1149, acc: 0.9608, time: 7321.0060\nepoch 0 valid Step 1400/1400, loss: 0.1582, acc: 0.9607, time: 1284.0643\n\nepoch 0 train step 5599/5599, loss: 0.1141, acc: 0.9615, time: 7321.5100\nepoch 0 valid Step 1397/1397, loss: 0.1549, acc: 0.9565, time: 1275.1715\n\nepoch 0 train step 5597/5597, loss: 0.1160, acc: 0.9607, time: 7315.6827\nepoch 0 valid Step 1398/1398, loss: 0.1365, acc: 0.9608, time: 1282.4847\nMulti-label CNN modeling:
\nepoch 0 train Step 5596/5596, loss: 0.592, pe_pr: 0.9633, rv_lv: 0.9767, rv_lv: 0.9787, lefts: 0.9678, chron: 0.9950, right: 0.9657, acute: 0.9957, centr: 0.9851, indet: 0.9840, time: 13889.76\nepoch 0 valid Step 1400/1400, loss: 0.908, pe_pr: 0.9588, rv_lv: 0.9760, rv_lv: 0.9746, lefts: 0.9639, chron: 0.9956, right: 0.9616, acute: 0.9932, centr: 0.9840, indet: 0.9706, time: 1791.32\n\nepoch 0 train Step 5593/5593, loss: 0.596, pe_pr: 0.9631, rv_lv: 0.9772, rv_lv: 0.9783, lefts: 0.9676, chron: 0.9950, right: 0.9654, acute: 0.9949, centr: 0.9853, indet: 0.9845, time: 14452.85\nepoch 0 valid Step 1402/1402, loss: 0.885, pe_pr: 0.9596, rv_lv: 0.9734, rv_lv: 0.9770, lefts: 0.9647, chron: 0.9955, right: 0.9630, acute: 0.9966, centr: 0.9845, indet: 0.9769, time: 1953.95\n\nepoch 0 train Step 5595/5595, loss: 0.593, pe_pr: 0.9633, rv_lv: 0.9770, rv_lv: 0.9782, lefts: 0.9678, chron: 0.9952, right: 0.9656, acute: 0.9955, centr: 0.9854, indet: 0.9836, time: 14430.96\nepoch 0 valid Step 1400/1400, loss: 0.905, pe_pr: 0.9603, rv_lv: 0.9733, rv_lv: 0.9764, lefts: 0.9647, chron: 0.9949, right: 0.9633, acute: 0.9940, centr: 0.9822, indet: 0.9801, time: 1779.32\n\nepoch 0 train Step 5599/5599, loss: 0.592, pe_pr: 0.9639, rv_lv: 0.9776, rv_lv: 0.9785, lefts: 0.9684, chron: 0.9955, right: 0.9663, acute: 0.9950, centr: 0.9854, indet: 0.9837, time: 14050.73\nepoch 0 valid Step 1397/1397, loss: 0.966, pe_pr: 0.9547, rv_lv: 0.9710, rv_lv: 0.9751, lefts: 0.9600, chron: 0.9936, right: 0.9572, acute: 0.9959, centr: 0.9823, indet: 0.9790, time: 1779.47\n\nepoch 0 train Step 5597/5597, loss: 0.596, pe_pr: 0.9629, rv_lv: 0.9774, rv_lv: 0.9775, lefts: 0.9677, chron: 0.9952, right: 0.9653, acute: 0.9951, centr: 0.9852, indet: 0.9845, time: 13633.35\nepoch 0 valid Step 1398/1398, loss: 0.811, pe_pr: 0.9600, rv_lv: 0.9727, rv_lv: 0.9794, lefts: 0.9649, chron: 0.9941, right: 0.9624, acute: 0.9958, centr: 0.9824, indet: 0.9741, time: 1763.02\nUse the oof prediction from stage 1 to build a simple single-layer GRU model.
\nepoch 0 train step 2912/2912, loss: 0.2896, time: 579.4593\nepoch 0 valid Step 1455/1455, loss: 0.2913, time: 192.8254\nepoch 1 train step 2912/2912, loss: 0.2588, time: 579.0478\nepoch 1 valid Step 1455/1455, loss: 0.2889, time: 193.6239\n\nepoch 0 train step 2912/2912, loss: 0.2934, time: 578.8298\nepoch 0 valid Step 1456/1456, loss: 0.2943, time: 194.7364\nepoch 1 train step 2912/2912, loss: 0.2615, time: 579.9668\nepoch 1 valid Step 1456/1456, loss: 0.2770, time: 194.4227\n\nepoch 0 train step 2912/2912, loss: 0.2921, time: 579.3877\nepoch 0 valid Step 1456/1456, loss: 0.2976, time: 191.9975\nepoch 1 train step 2912/2912, loss: 0.2586, time: 579.5109\nepoch 1 valid Step 1456/1456, loss: 0.2909, time: 190.9924\n\nepoch 0 train step 2912/2912, loss: 0.2941, time: 579.8269\nepoch 0 valid Step 1456/1456, loss: 0.2904, time: 187.3076\nepoch 1 train step 2912/2912, loss: 0.2608, time: 580.1677\nepoch 1 valid Step 1456/1456, loss: 0.2759, time: 187.2792\n\nepoch 0 train step 2912/2912, loss: 0.2921, time: 578.8625\nepoch 0 valid Step 1456/1456, loss: 0.2907, time: 190.8983\nepoch 1 train step 2912/2912, loss: 0.2612, time: 579.0903\nepoch 1 valid Step 1456/1456, loss: 0.2811, time: 190.6105\nI also did rule-based post-processing in the shared stage 2 kernel, it does not affect my cv much.
\nAre you crazy to public such kernel of silver level before 1 week of compettion finish? what is your goal? destroy results of the people who work hard?
", "rawMarkdown": "Are you crazy to public such kernel of silver level before 1 week of compettion finish? what is your goal? destroy results of the people who work hard?", "votes": 22 }, { "id": 1053980, "postDate": "2020-10-19T14:46:49.713Z", "content": "Hi Kagglers,
\nHope it is not too late to share this new baseline (8 days to go), I tried to do it ASAP, till recently I get more availability to get the work done, wait for the local training results finished and finally successfully submitted the kernel… The kernel is also my latest submission result as you could see from my submission records…
\nFor people who are struggling with the competition, here is a good example to help you start with.
\nFor people who are affected by the public standing, I apologize for my new baseline model and hope it could help you do better ensembling.
I can see this competition gathers a lot of ideas\\discussions to automate the PE detection process. I'm not having a medical science background, but I've been participating OSIC, SIIMC in the past 2 competitions, which are both related to medical science. These 3 competitions are what I think one of the truly valuable parts of AI\\ML- to help humans in real life. Hope my work could be possibly contributing to both communities- Medical and Kaggle.
\nOk, back to the topic…
\nThe kernel I'm sharing contains:
\n1. cross-validation strategy: https://www.kaggle.com/khyeh0719/stratified-validation-strategy
\n2. competition metric as loss function for NN: https://www.kaggle.com/khyeh0719/0929-updated-rsna-competition-metric
\n3. Rule-based post-processing
\n4. 2-stage model training
The model weights are here: https://www.kaggle.com/khyeh0719/kh-rsna-model
\nReferencing the ideas from the last RSNA, it composes of 2 stages:
\n1. CNN model
\n2. RNN model to combine the CNN model's result
In my implementation, I train CNN models to get OOF predictions for stage 2 model learning.
\nI use very simple efficientnet b0 as a starter for quick prototyping. To make stage 2 model capture the properties for other exam-level targets, I also did a multitask learning in stage 1 model.
\nSingle-label CNN modeling:
\nlocal 5-fold performances:
epoch 0 train step 5596/5596, loss: 0.1157, acc: 0.9606, time: 7311.1298\nepoch 0 valid Step 1400/1400, loss: 0.1245, acc: 0.9610, time: 1290.3861\n\nepoch 0 train step 5593/5593, loss: 0.1164, acc: 0.9604, time: 7316.3813\nepoch 0 valid Step 1402/1402, loss: 0.1522, acc: 0.9581, time: 1282.8907\n\nepoch 0 train step 5595/5595, loss: 0.1149, acc: 0.9608, time: 7321.0060\nepoch 0 valid Step 1400/1400, loss: 0.1582, acc: 0.9607, time: 1284.0643\n\nepoch 0 train step 5599/5599, loss: 0.1141, acc: 0.9615, time: 7321.5100\nepoch 0 valid Step 1397/1397, loss: 0.1549, acc: 0.9565, time: 1275.1715\n\nepoch 0 train step 5597/5597, loss: 0.1160, acc: 0.9607, time: 7315.6827\nepoch 0 valid Step 1398/1398, loss: 0.1365, acc: 0.9608, time: 1282.4847\nMulti-label CNN modeling:
\nepoch 0 train Step 5596/5596, loss: 0.592, pe_pr: 0.9633, rv_lv: 0.9767, rv_lv: 0.9787, lefts: 0.9678, chron: 0.9950, right: 0.9657, acute: 0.9957, centr: 0.9851, indet: 0.9840, time: 13889.76\nepoch 0 valid Step 1400/1400, loss: 0.908, pe_pr: 0.9588, rv_lv: 0.9760, rv_lv: 0.9746, lefts: 0.9639, chron: 0.9956, right: 0.9616, acute: 0.9932, centr: 0.9840, indet: 0.9706, time: 1791.32\n\nepoch 0 train Step 5593/5593, loss: 0.596, pe_pr: 0.9631, rv_lv: 0.9772, rv_lv: 0.9783, lefts: 0.9676, chron: 0.9950, right: 0.9654, acute: 0.9949, centr: 0.9853, indet: 0.9845, time: 14452.85\nepoch 0 valid Step 1402/1402, loss: 0.885, pe_pr: 0.9596, rv_lv: 0.9734, rv_lv: 0.9770, lefts: 0.9647, chron: 0.9955, right: 0.9630, acute: 0.9966, centr: 0.9845, indet: 0.9769, time: 1953.95\n\nepoch 0 train Step 5595/5595, loss: 0.593, pe_pr: 0.9633, rv_lv: 0.9770, rv_lv: 0.9782, lefts: 0.9678, chron: 0.9952, right: 0.9656, acute: 0.9955, centr: 0.9854, indet: 0.9836, time: 14430.96\nepoch 0 valid Step 1400/1400, loss: 0.905, pe_pr: 0.9603, rv_lv: 0.9733, rv_lv: 0.9764, lefts: 0.9647, chron: 0.9949, right: 0.9633, acute: 0.9940, centr: 0.9822, indet: 0.9801, time: 1779.32\n\nepoch 0 train Step 5599/5599, loss: 0.592, pe_pr: 0.9639, rv_lv: 0.9776, rv_lv: 0.9785, lefts: 0.9684, chron: 0.9955, right: 0.9663, acute: 0.9950, centr: 0.9854, indet: 0.9837, time: 14050.73\nepoch 0 valid Step 1397/1397, loss: 0.966, pe_pr: 0.9547, rv_lv: 0.9710, rv_lv: 0.9751, lefts: 0.9600, chron: 0.9936, right: 0.9572, acute: 0.9959, centr: 0.9823, indet: 0.9790, time: 1779.47\n\nepoch 0 train Step 5597/5597, loss: 0.596, pe_pr: 0.9629, rv_lv: 0.9774, rv_lv: 0.9775, lefts: 0.9677, chron: 0.9952, right: 0.9653, acute: 0.9951, centr: 0.9852, indet: 0.9845, time: 13633.35\nepoch 0 valid Step 1398/1398, loss: 0.811, pe_pr: 0.9600, rv_lv: 0.9727, rv_lv: 0.9794, lefts: 0.9649, chron: 0.9941, right: 0.9624, acute: 0.9958, centr: 0.9824, indet: 0.9741, time: 1763.02\nUse the oof prediction from stage 1 to build a simple single-layer GRU model.
\nepoch 0 train step 2912/2912, loss: 0.2896, time: 579.4593\nepoch 0 valid Step 1455/1455, loss: 0.2913, time: 192.8254\nepoch 1 train step 2912/2912, loss: 0.2588, time: 579.0478\nepoch 1 valid Step 1455/1455, loss: 0.2889, time: 193.6239\n\nepoch 0 train step 2912/2912, loss: 0.2934, time: 578.8298\nepoch 0 valid Step 1456/1456, loss: 0.2943, time: 194.7364\nepoch 1 train step 2912/2912, loss: 0.2615, time: 579.9668\nepoch 1 valid Step 1456/1456, loss: 0.2770, time: 194.4227\n\nepoch 0 train step 2912/2912, loss: 0.2921, time: 579.3877\nepoch 0 valid Step 1456/1456, loss: 0.2976, time: 191.9975\nepoch 1 train step 2912/2912, loss: 0.2586, time: 579.5109\nepoch 1 valid Step 1456/1456, loss: 0.2909, time: 190.9924\n\nepoch 0 train step 2912/2912, loss: 0.2941, time: 579.8269\nepoch 0 valid Step 1456/1456, loss: 0.2904, time: 187.3076\nepoch 1 train step 2912/2912, loss: 0.2608, time: 580.1677\nepoch 1 valid Step 1456/1456, loss: 0.2759, time: 187.2792\n\nepoch 0 train step 2912/2912, loss: 0.2921, time: 578.8625\nepoch 0 valid Step 1456/1456, loss: 0.2907, time: 190.8983\nepoch 1 train step 2912/2912, loss: 0.2612, time: 579.0903\nepoch 1 valid Step 1456/1456, loss: 0.2811, time: 190.6105\nI also did rule-based post-processing in the shared stage 2 kernel, it does not affect my cv much.
\nAs much as I think releasing baselines is helpful, is it really a great idea to release a top 20 kernel in the last week of a competition?
", "rawMarkdown": "As much as I think releasing baselines is helpful, is it really a great idea to release a top 20 kernel in the last week of a competition? ", "votes": 14, "replies": [ { "id": 1054071, "postDate": "2020-10-19T16:27:57.403Z", "content": "I guess you make your point as well. As in the past competition, there will be a warning of not sharing a high-scoring kernel within one week, I don't see it when I publish it (8th day before the competition ends…)
", "rawMarkdown": "I guess you make your point as well. As in the past competition, there will be a warning of not sharing a high-scoring kernel within one week, I don't see it when I publish it (8th day before the competition ends...)", "votes": -8 }, { "id": 1054098, "postDate": "2020-10-19T17:05:23.597Z", "content": "Myself, and tons of others, have worked for a month to get to this score. Releasing a kernel with a month left makes sense - people can improve and understand the code. Releasing with a week left (whether 7 or 8 days) defeats the purpose. Most people will just submit the public kernel, and now people who didn't participate, but forked your code, have a medal. I understand this isn't the intention, but it is an unfortunate consequence.
\nEdit: Meme for condolences to everyone not in the top 27
\n
Unfortunately, now it could become a game of the person/team with large compute available wins silver or may be gold who knows ? :(. But I think now whatever we say or do cant change what is going to happen in this comp . However , thank you for the idea , could really have been helpful 2/3 weeks ago ..
", "rawMarkdown": "Unfortunately, now it could become a game of the person/team with large compute available wins silver or may be gold who knows ? :(. But I think now whatever we say or do cant change what is going to happen in this comp . However , thank you for the idea , could really have been helpful 2/3 weeks ago ..", "votes": 2 }, { "id": 1054123, "postDate": "2020-10-19T17:38:42.427Z", "rawMarkdown": "", "votes": -3, "isDeleted": true }, { "id": 1054133, "postDate": "2020-10-19T17:54:26.633Z", "content": "@stanleyjzheng @phoenix9032 Get your point now, I'm sorry to cause the unfortunate results :(
", "rawMarkdown": "@stanleyjzheng @phoenix9032 Get your point now, I'm sorry to cause the unfortunate results :(" }, { "id": 1054347, "postDate": "2020-10-19T20:45:37.163Z", "content": "@khyeh0719 You've been registered at kaggle for 3 years but behave as a novice… Do you really think that there is any difference between posting it in 7 or 8 days before the competition end? If you never participated in competitions, it would be probably understandable, but you intentionally ruin the competition trying to post the last moment high performing kernel and get upvotes (like some novices do by posting blends and forks not caring about their reputation).
\nSuch a kernel would be awesome if posted 2 month before the competition end, so all participants would appreciate the work you have done, and it could really boost the interest to this competition as well as motivate new approaches. Such a kernel would be ok if posted 1 month before the end: some people would complain, but you would still make a meaningful contribution to the competition. But think what you have done just posting it in a week before the end…
I completely agree. Such actions kill the competitive spirit and motivation. He's just a conceited youngster.
", "rawMarkdown": "I completely agree. Such actions kill the competitive spirit and motivation. He's just a conceited youngster.", "votes": 1 }, { "id": 1054523, "postDate": "2020-10-20T02:08:06.403Z", "content": "I agree with @iafoss and this is really unfortunate, and it seems this type of thing is happening more and more often. In OpenVaccine, many top solutions used almost the same architecture from high scoring kernels (many of which were published over a week before the competition deadline) + additional data tricks. I hope there would be some policy change to punish this type of behavior, as it is extremely detrimental to the competitive spirit of the community.
", "rawMarkdown": "I agree with @iafoss and this is really unfortunate, and it seems this type of thing is happening more and more often. In OpenVaccine, many top solutions used almost the same architecture from high scoring kernels (many of which were published over a week before the competition deadline) + additional data tricks. I hope there would be some policy change to punish this type of behavior, as it is extremely detrimental to the competitive spirit of the community.", "votes": 4 }, { "id": 1054545, "postDate": "2020-10-20T02:33:14.440Z", "content": "@shujun717 This issue has been discussed for many times at kaggle, and they added the possibility to report unhealthy kernels (3 dots in the upper left corner) for further investigation. Though, to completely prevent such behavior, good option could be complete suppression of publication and updating any public kernels in 2 or even 4 weeks before the end of the competition. In this competition just training the models could take a week, and people would need several more weeks to explore the posted approach, not just copy-past and get a medal if one is lucky or has good GPU resources to quickly train several big models not even modifying the code… Though, public kernels published at the early stage of the competition are really great both for getting started and for learning. If this kernel was posted earlier, it would be extremely helpful.
", "rawMarkdown": "@shujun717 This issue has been discussed for many times at kaggle, and they added the possibility to report unhealthy kernels (3 dots in the upper left corner) for further investigation. Though, to completely prevent such behavior, good option could be complete suppression of publication and updating any public kernels in 2 or even 4 weeks before the end of the competition. In this competition just training the models could take a week, and people would need several more weeks to explore the posted approach, not just copy-past and get a medal if one is lucky or has good GPU resources to quickly train several big models not even modifying the code... Though, public kernels published at the early stage of the competition are really great both for getting started and for learning. If this kernel was posted earlier, it would be extremely helpful.", "votes": 10 }, { "id": 1054630, "postDate": "2020-10-20T04:11:49.357Z", "content": "@iafoss Thanks for pointing out the option to report… I think it would be a good option to suppress public notebooks in the final two weeks as well. Hopefully they implement something like that
", "rawMarkdown": "@iafoss Thanks for pointing out the option to report... I think it would be a good option to suppress public notebooks in the final two weeks as well. Hopefully they implement something like that", "votes": 1 }, { "id": 1054751, "postDate": "2020-10-20T06:11:27.880Z", "content": "@khyeh0719 One thing you can do is to make your model files dataset private. So that copy and edit submissions can be stopped.
", "rawMarkdown": "@khyeh0719 One thing you can do is to make your model files dataset private. So that copy and edit submissions can be stopped. ", "votes": 1 }, { "id": 1054829, "postDate": "2020-10-20T07:44:27.110Z", "content": "@ratan123 Nope, that is definitely not a good option. If @khyeh0719 makes it private now, there will be a division between those already downloaded the model and those not yet. To make this game fair, he must leave it available to all the participants.
", "rawMarkdown": "@ratan123 Nope, that is definitely not a good option. If @khyeh0719 makes it private now, there will be a division between those already downloaded the model and those not yet. To make this game fair, he must leave it available to all the participants.", "votes": 11 } ] }, { "id": 1054128, "postDate": "2020-10-19T17:49:09.917Z", "content": "You can call it a baseline all you want, but this is not one. A baseline is supposed to be a simple model, something that is easy enough to beat, but robust enough to ensure that any model that surpasses its accuracy actually \"works\". Releasing a solid solution in the last week is basically undermining the efforts of the participants who have spent the last several weeks on this competition.
", "rawMarkdown": "You can call it a baseline all you want, but this is not one. A baseline is supposed to be a simple model, something that is easy enough to beat, but robust enough to ensure that any model that surpasses its accuracy actually \"works\". Releasing a solid solution in the last week is basically undermining the efforts of the participants who have spent the last several weeks on this competition.", "votes": 15, "replies": [ { "id": 1054135, "postDate": "2020-10-19T17:55:14.307Z", "content": "Thanks for pointing out, I don't mean to cause the unfortunate results :(
", "rawMarkdown": "Thanks for pointing out, I don't mean to cause the unfortunate results :(", "votes": -4 }, { "id": 1054141, "postDate": "2020-10-19T18:00:51.697Z", "content": "@khyeh0719 as least don't make your models public….
", "rawMarkdown": "@khyeh0719 as least don't make your models public...." }, { "id": 1054152, "postDate": "2020-10-19T18:09:55.767Z", "content": "Even our best model we have improved for a month brings score down when ensembled 😂
\nWe also don't have crazy compute to tune this in only a week. I guess we can suffer together
![\"\"](\"https://i.kym-cdn.com/entries/icons/facebook/000/012/073/7686178464_fdc8ea66c7.jpg\"){kind=icon}
@stanleyjzheng don't give up :D. Did you make a dummy sub? That LB climb was fast af.
", "rawMarkdown": "@stanleyjzheng don't give up :D. Did you make a dummy sub? That LB climb was fast af.", "votes": 1 }, { "id": 1054157, "postDate": "2020-10-19T18:17:02.310Z", "content": "@underwearfitting Yep, dummy csv only sub, our highest working sub is still 0.275. Thanks for the encouragement, I guess we'll see what we can do in a short time.
", "rawMarkdown": "@underwearfitting Yep, dummy csv only sub, our highest working sub is still 0.275. Thanks for the encouragement, I guess we'll see what we can do in a short time." }, { "id": 1054158, "postDate": "2020-10-19T18:17:39.827Z", "content": "@stanleyjzheng I see you're really bummed out about this. Maybe the hosts can disqualify ALL submissions that are really similar to any public baseline.
", "rawMarkdown": "@stanleyjzheng I see you're really bummed out about this. Maybe the hosts can disqualify ALL submissions that are really similar to any public baseline.", "votes": 1 }, { "id": 1054161, "postDate": "2020-10-19T18:21:28.487Z", "content": "@darkcube Thanks for the kind words. I wish… but it'll never happen, high scoring public notebooks just a part of Kaggle that I guess we all have to accept.
", "rawMarkdown": "@darkcube Thanks for the kind words. I wish... but it'll never happen, high scoring public notebooks just a part of Kaggle that I guess we all have to accept. ", "votes": 2 } ] }, { "id": 1054772, "postDate": "2020-10-20T06:38:29.577Z", "content": "em… I think it's better to divide your code into inference, and training. You can public your training code and hide your own model, and say \"I use this model to get a good performance in public LB\". The assiduous Kagglers still will be learning a lot from your code. Because for me I spend a lot of time on the training model and just get about 0.290-0.300 public scores, but there is a group of people just watch on Best Score, they using one single submit and exceed all my model, I will feel unfair. Anyway, I'll keep learning your code, and thanks your idea sharing (not for 0.233 solution :) )
", "rawMarkdown": "em... I think it's better to divide your code into inference, and training. You can public your training code and **hide** your own model, and say \"I use this model to get a good performance in public LB\". The assiduous Kagglers still will be learning a lot from your code. Because for me I spend a lot of time on the training model and just get about 0.290-0.300 public scores, but there is a group of people just watch on **Best Score**, they using one single submit and exceed all my model, I will feel unfair. Anyway, I'll keep learning your code, and thanks your idea sharing (not for 0.233 solution :) )", "votes": 10 }, { "id": 1061149, "postDate": "2020-10-26T19:16:28.750Z", "content": "\nThis is a really great kernel and I wanted to express appreciation for you sharing it. I know a lot of people gave you slack for it; but the reality of the matter is:
\nIf anything, the only shocking part of the three kernels you pushed out was that you also shared the weights… :-). I think in the past, most other competitors would have withheld those; but at the end, you probably ended up saving the environment from a lot of throw-away electricity / carbon offset.
\nI wanted to share some notes about this kernel specifically because there were critics who claimed it wasn't fair and that people with superior compute would be able to take more advantage. That is 100% accurate. More compute means you have more RoI for time to conduct more experiments. That is true if you have 1 week to go or 3 months to go. No one claims it isn't fair that there are GM's that get paid by NVIDIA or H2O to work full time on Kaggle and get DGX boxes? Why complain here?
\nWhat more, there's something to be said about taking a public kernel and optimizing it. I won't get into the data science aspect, but I'll talk strictly about software engineering. These kernels take a long time to run and can be optimized to run many orders of magnitude faster by just making slight alterations to do code. For example, people notes that the multi-output ran significantly slower than the single output one even though there were no substantive changes. Did anyone here bother to profile it? The culprit was multiple ilocs on an unfiltered dataframe. Simple converting over to numpy at dataset load makes this kitty pur 15x (1500%) faster on my desktop machine and my dual gpu's remain well fed at +90% utilization. Other tricks include sampling on stage 1 to reduce overfitting, since some of the patients have >1000 slices and all the slices are very well correlated axially.
Anyhow, just wanted to say thanks for the kernel, most of the publicly shared resources were pretty stale or just mean target by slice submissions. Cheers.
", "rawMarkdown": "@khyeh0719 \n\nThis is a really great kernel and I wanted to express appreciation for you sharing it. I know a lot of people gave you slack for it; but the reality of the matter is:\n\n1. You shared the kernel t-8 days before the competition deadline, which is an additional 24hours in surplus of what the community collectively requested Kaggle set as the etiquette guideline for publishing high scoring kernels\n2. A lot of the tricks you shared were code realizations from last year's brain hemorrhage competition\n\nIf anything, the only shocking part of the three kernels you pushed out was that you also shared the weights... :-). I think in the past, most other competitors would have withheld those; but at the end, you probably ended up saving the environment from a lot of throw-away electricity / carbon offset.\n\nI wanted to share some notes about this kernel specifically because there were critics who claimed it wasn't fair and that people with superior compute would be able to take more advantage. That is 100% accurate. More compute means you have more RoI for time to conduct more experiments. That is true if you have 1 week to go or 3 months to go. No one claims it isn't fair that there are GM's that get paid by NVIDIA or H2O to work full time on Kaggle and get DGX boxes? Why complain here?\n\nWhat more, there's something to be said about taking a public kernel and optimizing it. I won't get into the data science aspect, but I'll talk strictly about software engineering. These kernels take a long time to run and can be optimized to run many orders of magnitude faster by just making slight alterations to do code. For example, people notes that the multi-output ran significantly slower than the single output one even though there were no substantive changes. Did anyone here bother to profile it? The culprit was multiple `ilocs` on an unfiltered dataframe. Simple converting over to numpy at dataset load makes this kitty pur 15x (1500%) faster on my desktop machine and my dual gpu's remain well fed at +90% utilization. Other tricks include sampling on stage 1 to reduce overfitting, since some of the patients have >1000 slices and all the slices are very well correlated axially.\n\nAnyhow, just wanted to say thanks for the kernel, most of the publicly shared resources were pretty stale or just mean target by slice submissions. Cheers.", "votes": 7, "replies": [ { "id": 1061159, "postDate": "2020-10-26T19:20:10.160Z", "content": "I echo these sentiments - I didn't have the resources to do the data science optimization aspect, but I spent many hours optimizing the code and got inference to run in 1/4 of the time, then adding TTA is my current score. I am still very much behind where I was before the kernel however, and spent a ton of time I could have used studying for exams. I have to admit, I did learn a ton from the code and reading it, so thanks to Kun for that. My apologies for the harsh criticism at release, this was very frustrating, until I could see the improbability. Kaggle is about learning, not about the medals after all.
", "rawMarkdown": "I echo these sentiments - I didn't have the resources to do the data science optimization aspect, but I spent many hours optimizing the code and got inference to run in 1/4 of the time, then adding TTA is my current score. I am still very much behind where I was before the kernel however, and spent a ton of time I could have used studying for exams. I have to admit, I did learn a ton from the code and reading it, so thanks to Kun for that. My apologies for the harsh criticism at release, this was very frustrating, until I could see the improbability. Kaggle is about learning, not about the medals after all. ", "votes": 2 }, { "id": 1061165, "postDate": "2020-10-26T19:26:45.200Z", "content": "When all the dust settles, there will be a very small handful of winners. And then there's going to be the remaining 98% of us. If we don't set proper expectations and goals from the get-go (like you were successfully able to do above @stanleyjzheng) then we risk taking damage to our mental health :-). I hope you place favorably on the private LB and above all, the effort you invested + knowledge you acquired is yours for the keeping.
", "rawMarkdown": "When all the dust settles, there will be a very small handful of winners. And then there's going to be the remaining 98% of us. If we don't set proper expectations and goals from the get-go (like you were successfully able to do above @stanleyjzheng) then we risk taking damage to our mental health :-). I hope you place favorably on the private LB and above all, the effort you invested + knowledge you acquired is yours for the keeping.", "votes": 3 }, { "id": 1061192, "postDate": "2020-10-26T19:45:09.517Z", "content": "@authman Thanks a lot for the kind words:)
", "rawMarkdown": "@authman Thanks a lot for the kind words:)", "votes": 3 } ] }, { "id": 1056492, "postDate": "2020-10-21T18:46:39.817Z", "content": "I'm torn to be honest. On one hand I agree with the commenters that don't think Kun should have released it so close to the end. But on the other I'm grateful to him to be able to look through his Effnet training code and compare it to mine to learn where I went wrong. I'm spent so many hours trying to train that model on GPU and never got it faster than 45mins/epoch (for 6500 patients).
\nTo be fair though the skill of the man where solving my massive problem is just one throw away line for him \"I use very simple efficientnet b0 as a starter for quick prototyping.\" makes me:
\na) want to cry at the gap between our skill levels
\nb) makes me happy that these things are solvable and I can learn
\n\nepoch 0 train Step 5596/5596, loss: 0.592, pe_pr: 0.9633, rv_lv: 0.9767, rv_lv: 0.9787, lefts: 0.9678, chron: 0.9950, right: 0.9657, acute: 0.9957, centr: 0.9851, indet: 0.9840, time: 13889.76
\n
Am I missing something? 45min per epoch is still legendary.
", "rawMarkdown": "> epoch 0 train Step 5596/5596, loss: 0.592, pe_pr: 0.9633, rv_lv: 0.9767, rv_lv: 0.9787, lefts: 0.9678, chron: 0.9950, right: 0.9657, acute: 0.9957, centr: 0.9851, indet: 0.9840, **time: 13889.76**\n\nAm I missing something? 45min per epoch is still legendary.", "votes": 3 }, { "id": 1057807, "postDate": "2020-10-23T03:07:59.073Z", "rawMarkdown": "", "isDeleted": true } ] }, { "id": 1054115, "postDate": "2020-10-19T17:31:09.487Z", "content": "I think you published your kernel very early
\nEntering the game one day before the end would be enough 😑
Thanks for the idea. Especially the rule-based PP. Bad timing tho :)
", "rawMarkdown": "Thanks for the idea. Especially the rule-based PP. Bad timing tho :)", "votes": 3 }, { "id": 1054191, "postDate": "2020-10-19T18:58:25.343Z", "content": "If you were already in the medal range before the kernel was released, chances are you can improve your score by re-using what was shared.
\nThe most straigh-forward approach is just to blend your model with it. This should be enough to beat all the people that will effortlessly fork the kernel. Hence still do a decent finish.
If you trained stage 1 models, you can use them to train stage 2 models as @khyeh0719 did. This is what I am planning to do and I am pretty confident this will be enough to improve this (way too strong) baseline. There is a lot of room for improvement, and if you spent the last month experimenting you have extra knowledge people who \"join & fork\" don't have.
", "rawMarkdown": "If you were already in the medal range before the kernel was released, chances are you can improve your score by re-using what was shared.\nThe most straigh-forward approach is just to blend your model with it. This should be enough to beat all the people that will effortlessly fork the kernel. Hence still do a decent finish.\n\nIf you trained stage 1 models, you can use them to train stage 2 models as @khyeh0719 did. This is what I am planning to do and I am pretty confident this will be enough to improve this (way too strong) baseline. There is a lot of room for improvement, and if you spent the last month experimenting you have extra knowledge people who \"join & fork\" don't have.\n", "votes": 4, "replies": [ { "id": 1054198, "postDate": "2020-10-19T19:03:41.913Z", "content": "Good point. But I don't think you are 100% right about blending. The max inference time is 9 hours here. From what we've done so far, we cannot afford blending with his results.
", "rawMarkdown": "Good point. But I don't think you are 100% right about blending. The max inference time is 9 hours here. From what we've done so far, we cannot afford blending with his results.", "votes": 4 }, { "id": 1054205, "postDate": "2020-10-19T19:06:25.607Z", "content": "Great advice.
\nI agree with Sin about blending. This notebook takes a significant amount of time to run, and even blending with our best model (0.275) in a dummy submission, we still see a decrease in score. We're also using tensorflow, so existing models are also worthless. Still some great advice, and maybe a week is enough to train in pytorch and salvage a silver medal.
Actually, a two stage model approach will cost mostly 8.5 hours to finish the re-run, the inference speed it's the key,there's not much space for ensembling and blending. So be careful about the balance between the model complexity and compute bottleneck
", "rawMarkdown": "Actually, a two stage model approach will cost mostly 8.5 hours to finish the re-run, the inference speed it's the key,there's not much space for ensembling and blending. So be careful about the balance between the model complexity and compute bottleneck", "votes": 2 }, { "id": 1054334, "postDate": "2020-10-19T20:29:54.070Z", "content": "Indeed, I did not know such models were so long to run. Sorry about that. Looks like the best bet is to retrain a first level model then.
", "rawMarkdown": "Indeed, I did not know such models were so long to run. Sorry about that. Looks like the best bet is to retrain a first level model then.", "votes": 1 }, { "id": 1054349, "postDate": "2020-10-19T20:46:54.577Z", "content": "@theoviel
\nInference takes almost 9 hours. You can't really blend here.
As many here, I forked to submit this (strong) baseline, while still working on my model aside. Because I'm working too hard to set up and train my model, and it would be too easy for newcomers who fork and submit it to be above me, given the model weights are publicly shared too .
", "rawMarkdown": "@theoviel \nInference takes almost 9 hours. You can't really blend here. \n\nAs many here, I forked to submit this (strong) baseline, while still working on my model aside. Because I'm working too hard to set up and train my model, and it would be too easy for newcomers who fork and submit it to be above me, given the model weights are publicly shared too .", "votes": 4 }, { "id": 1054350, "postDate": "2020-10-19T20:55:13.350Z", "content": "A little bit more sad part is the person in question has way more credential and experience in kaggle than me and lot of other people that I didnt think a kernel gold this way would interest him .. :(
", "rawMarkdown": "A little bit more sad part is the person in question has way more credential and experience in kaggle than me and lot of other people that I didnt think a kernel gold this way would interest him .. :( ", "votes": 2 }, { "id": 1054355, "postDate": "2020-10-19T20:59:30.747Z", "content": "Exactly! I don't really see the motivation. Kun deserves the utmost respect for his amazing competition finishes, and is one solo gold away from Kaggle's greatest honour. However, he still chooses to release kernels in the last week… No hard feelings to anyone in this thread, just suprising to see from such an esteemed member.
", "rawMarkdown": "Exactly! I don't really see the motivation. Kun deserves the utmost respect for his amazing competition finishes, and is one solo gold away from Kaggle's greatest honour. However, he still chooses to release kernels in the last week... No hard feelings to anyone in this thread, just suprising to see from such an esteemed member.", "votes": 2 }, { "id": 1054360, "postDate": "2020-10-19T21:15:34.237Z", "content": "I also have a lot of respect for Kun, and he obviously doesn't have any bad intentions in sharing his great notebooks with the community, and which undoubtedly took him a lot of effort.
\nHowever, this competition is computationnaly heavy and sadly many people won't see it as a baseline to beat, but rather an easy way to gain medal, particularly with just a week left.
", "rawMarkdown": "I also have a lot of respect for Kun, and he obviously doesn't have any bad intentions in sharing his great notebooks with the community, and which undoubtedly took him a lot of effort.\n\nHowever, this competition is computationnaly heavy and sadly many people won't see it as a baseline to beat, but rather an easy way to gain medal, particularly with just a week left. \n\n", "votes": 1 }, { "id": 1058848, "postDate": "2020-10-24T11:04:50.887Z", "content": "How are you calculating the inference time of this notebook? Execution Info says 7787.2 seconds
", "rawMarkdown": "How are you calculating the inference time of this notebook? Execution Info says 7787.2 seconds" } ] }, { "id": 1054568, "postDate": "2020-10-20T02:45:50.673Z", "content": "well… I didn't see that coming
", "rawMarkdown": "well... I didn't see that coming", "votes": 4 }, { "id": 1054147, "postDate": "2020-10-19T18:04:56.380Z", "content": "On the plus side, the work you did is great, and a lot of people will learn from it.
\nHowever, I am quite happy I did not join this competition earlier that 2 days ago.
", "rawMarkdown": "On the plus side, the work you did is great, and a lot of people will learn from it.\n\nHowever, I am quite happy I did not join this competition earlier that 2 days ago.", "votes": 2, "replies": [ { "id": 1054160, "postDate": "2020-10-19T18:19:57.733Z", "content": "What are you planning ..lol :D
", "rawMarkdown": "What are you planning ..lol :D", "votes": 2 } ] }, { "id": 1058788, "postDate": "2020-10-24T09:10:35.400Z", "content": "Great kernel but not the best timing. 8 days before deadline and just 1 day before the last week… You could have just posted it either a month ago or when the competition finished, but now you turned the medal race into a fork and blend competition for many participants (including myself). Next time please about other people's efforts
", "rawMarkdown": "Great kernel but not the best timing. 8 days before deadline and just 1 day before the last week... You could have just posted it either a month ago or when the competition finished, but now you turned the medal race into a fork and blend competition for many participants (including myself). Next time please about other people's efforts", "votes": 1 }, { "id": 1057859, "postDate": "2020-10-23T04:59:55.103Z", "content": "Hi, @khyeh0719 , When i train your Stage2 RNN Model , this kernel, i got below loss.
\nepoch 0 train step 2912/2912, loss: 0.4568, time: 4443.1336
\nepoch 0 valid Step 1455/1455, loss: 0.5876, time: 1570.2982
\nepoch 1 train step 2912/2912, loss: 0.4145, time: 4734.2051
\nepoch 1 valid Step 1455/1455, loss: 0.5958, time: 1444.1449
Can you tell me What could be wrong?
", "rawMarkdown": "Hi, @khyeh0719 , When i train your Stage2 RNN Model , this [kernel](https://www.kaggle.com/khyeh0719/cnn-gru-baseline-stage2-train-inference), i got below loss.\n\nepoch 0 train step 2912/2912, loss: 0.4568, time: 4443.1336\nepoch 0 valid Step 1455/1455, loss: 0.5876, time: 1570.2982\nepoch 1 train step 2912/2912, loss: 0.4145, time: 4734.2051\nepoch 1 valid Step 1455/1455, loss: 0.5958, time: 1444.1449\n\nCan you tell me What could be wrong?", "votes": 1 }, { "id": 1057200, "postDate": "2020-10-22T13:17:09.030Z", "content": "@khyeh0719 with 5596 mini batches, are you training with 256 BS? If you don't mind me asking, what hardware are you running this on?
", "rawMarkdown": "@khyeh0719 with 5596 mini batches, are you training with 256 BS? If you don't mind me asking, what hardware are you running this on?", "votes": 1, "replies": [ { "id": 1058043, "postDate": "2020-10-23T09:28:09.787Z", "content": "I had the same doubt - I concluded that either it is a much larger BS or he is training on only partial dataset. Most probably it is the latter.
", "rawMarkdown": "I had the same doubt - I concluded that either it is a much larger BS or he is training on only partial dataset. Most probably it is the latter.", "votes": 1 } ] }, { "id": 1058613, "postDate": "2020-10-24T01:06:31.453Z", "content": "The key problem is that most of the participants may not have enough GPU resources to reproduce the whole process in a short time. I guess that simply replacing efficientnet-b0 with efficientnet-b1 or efficienetnet-b5 can achieve better results…
", "rawMarkdown": "The key problem is that most of the participants may not have enough GPU resources to reproduce the whole process in a short time. I guess that simply replacing efficientnet-b0 with efficientnet-b1 or efficienetnet-b5 can achieve better results..." }, { "id": 1054150, "postDate": "2020-10-19T18:09:17.150Z", "content": "Thank you so much for sharing your work. I'm kind of a latecomer myself (I only started participating 5 days ago). So your notebook is really going to help.
\nDespite being a latecomer, I kinda agree with @stanleyjzheng that you probably shouldn't have shared it. I strongly believe that people just copy other people's work to make a submission don't even deserve a stand in the leaderboard. Maybe just providing the ideas you used would be a far greater help for everyone?
", "rawMarkdown": "Thank you so much for sharing your work. I'm kind of a latecomer myself (I only started participating 5 days ago). So your notebook is really going to help. \n\nDespite being a latecomer, I kinda agree with @stanleyjzheng that you probably shouldn't have shared it. I strongly believe that people just copy other people's work to make a submission don't even deserve a stand in the leaderboard. Maybe just providing the *ideas* you used would be a far greater help for everyone?" }, { "id": 1054854, "postDate": "2020-10-20T08:20:24.500Z", "content": "While I understand the point others are making in this thread, @khyeh0719 , I'm very thankful to you that you shared these notebook.
\nEspecially for me, I'll be able to learn a lot about best practices, resolve many issues I was facing in writing PyTorch/Fastai code.
\nAfter the competition, I would have lost the steam and drive to focus on that. During competition, I want to now understand each and every line of code and see where I was going wrong.
Thanks again!
", "rawMarkdown": "While I understand the point others are making in this thread, @khyeh0719 , I'm very thankful to you that you shared these notebook.\nEspecially for me, I'll be able to learn a lot about best practices, resolve many issues I was facing in writing PyTorch/Fastai code.\nAfter the competition, I would have lost the steam and drive to focus on that. During competition, I want to now understand each and every line of code and see where I was going wrong.\n\nThanks again!", "votes": -4 }, { "id": 1054782, "postDate": "2020-10-20T06:56:13.720Z", "content": "good one thanks for sharing with others nice work keep going
", "rawMarkdown": "good one thanks for sharing with others nice work keep going", "votes": -5 }, { "id": 1054610, "postDate": "2020-10-20T03:33:12.133Z", "content": "I dont see the problems here if the kernel don't change gold range rank order. maybe I am wrong.
", "rawMarkdown": "I dont see the problems here if the kernel don't change gold range rank order. maybe I am wrong.", "votes": -10, "replies": [ { "id": 1054631, "postDate": "2020-10-20T04:12:47.257Z", "content": "Oh yea, nothing wrong. Just a week to go through 1600 lines of code, ruining the competition for 670 teams. I don't see any problems either.
\nI do see where you are coming from - none of the cash positions are changed, but this still causes frustration for the 670 of us who are not in gold medal position.
", "rawMarkdown": "Oh yea, nothing wrong. Just a week to go through 1600 lines of code, ruining the competition for 670 teams. I don't see any problems either.\n\nI do see where you are coming from - none of the cash positions are changed, but this still causes frustration for the 670 of us who are not in gold medal position.", "votes": 6 }, { "id": 1054643, "postDate": "2020-10-20T04:24:22.063Z", "content": "Ok, so if we are not here to get gold range rank, we are worthless.🤕🤒
", "rawMarkdown": "Ok, so if we are not here to get gold range rank, we are worthless.🤕🤒", "votes": 8 }, { "id": 1054748, "postDate": "2020-10-20T06:08:32Z", "content": "OK, maybe I'm wrong in some way. I apologize.
", "rawMarkdown": "OK, maybe I'm wrong in some way. I apologize.", "votes": 1 }, { "id": 1056610, "postDate": "2020-10-21T22:35:00.470Z", "content": "@stanleyjzheng ok, I know this really sounds that it should be obvious but are you telling me that going through 1600 lines of code in a single week is unrealistic?!!! I'm here scolding myself for not understanding it from first glance lol.
", "rawMarkdown": "@stanleyjzheng ok, I know this really sounds that it should be obvious but are you telling me that going through 1600 lines of code in a single week is unrealistic?!!! I'm here scolding myself for not understanding it from first glance lol.", "votes": -2 }, { "id": 1056618, "postDate": "2020-10-21T22:59:51.093Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1057080, "postDate": "2020-10-22T10:49:34.773Z", "content": "are you sure there are no bits of performance you can squeeze in? TensorFlow really amazes me from that point. Whenever I think my code is as efficient as it can get, the guide always proves me wrong.
", "rawMarkdown": "are you sure there are no bits of performance you can squeeze in? TensorFlow really amazes me from that point. Whenever I think my code is as efficient as it can get, the guide always proves me wrong.", "votes": -1 }, { "id": 1058405, "postDate": "2020-10-23T16:53:15.600Z", "content": "how are you supposed to use tensorflow in a pytorch model
", "rawMarkdown": "how are you supposed to use tensorflow in a pytorch model ", "votes": 2 }, { "id": 1058799, "postDate": "2020-10-24T09:45:48.937Z", "content": "He said they were using TensorFlow. And there are scripts that can convert them to TensorFlow. I'm gonna provide a dataset with the converted models in a few hours.
", "rawMarkdown": "He said they were using TensorFlow. And there are scripts that can convert them to TensorFlow. I'm gonna provide a dataset with the converted models in a few hours." }, { "id": 1059260, "postDate": "2020-10-24T20:41:08.743Z", "content": "Okay, that was a living hell.
\nhere is the script
\nhere is a few models converted to onnx which you can then convert back to basically any major ML framework (or at least that's what they claim to do).
\nAlso the input is (C, H, W) instead of (H, W, C), and for some reason the converted keras model doesn't work for cpu. You can convert the rest, I'm done.
to be fair people this is hardly NOT a baseline. I mean all he did was implement last year's solution and added:
\nAnd that's it really. like, come on people. I'm 16 and the only new thing I found in his submission is the 2-stage training since you can't train an end-to-end model due to lack of resources. (if you want to do that you have to feed the entire 220 scans into the model and compute the training step on that which simply would take a huge amount of memory).
", "rawMarkdown": "to be fair people this is hardly **NOT** a baseline. I mean all he did was implement last year's solution and added:\n1. the stratified validation strategy which you should always use anyways.\n2. the competition's metric as a loss function which 1)we aren't really sure if it would increase the LB much, 2)it's easy to implement and 3)easy to come up with. @jaideepvalani mentioned it already [here](https://www.kaggle.com/c/rsna-str-pulmonary-embolism-detection/discussion/190879).\n3. the rule-based pp which I'm not really sure what it is but, yeah.\n\nAnd that's it really. like, come on people. I'm 16 and the only new thing I found in his submission is the 2-stage training since you can't train an end-to-end model due to lack of resources. (if you want to do that you have to feed the entire 220 scans into the model and compute the training step on that which simply would take a huge amount of memory).", "votes": -7, "replies": [ { "id": 1058388, "postDate": "2020-10-23T16:34:31.610Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1058400, "postDate": "2020-10-23T16:50:16.677Z", "content": "I'm not really sure what model would take 8 hours per epoch (unless you're training an ensemble) but have you considered training your models on a subset of the data? maybe include all negative_exam_for_pe = 0 and indeterminate = 1 and maybe double that with data from negative_exam_for_pe = 1. Most of what you'd learn from negative exams after about say 2000 data points I think is marginal.
\nAlso, WE'RE THE SAME AGE????? that's awesome man. I never thought I'd find another 16 y/o on this website.
", "rawMarkdown": "I'm not really sure what model would take 8 hours per epoch (unless you're training an ensemble) but have you considered training your models on a subset of the data? maybe include all negative_exam_for_pe = 0 and indeterminate = 1 and maybe double that with data from negative_exam_for_pe = 1. Most of what you'd learn from negative exams after about say 2000 data points I think is marginal.\n\nAlso, WE'RE THE SAME AGE????? that's awesome man. I never thought I'd find another 16 y/o on this website.", "votes": -3 }, { "id": 1058413, "postDate": "2020-10-23T16:59:32.390Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1058431, "postDate": "2020-10-23T17:19:44.813Z", "content": "if he can train on a batch of 256 then why did he implement 2 stage training? I imagine training on and end-to-end model would yield far more robust results. As opposed to 2 stage training being basically transfer learning.
", "rawMarkdown": "if he can train on a batch of 256 then why did he implement 2 stage training? I imagine training on and end-to-end model would yield far more robust results. As opposed to 2 stage training being basically transfer learning." } ] }, { "id": 1057984, "postDate": "2020-10-23T08:14:52.120Z", "content": "👍Good kernal
\nJust a little late.There is Less free time to study your model, especially calculated quantities are too much!🤔
Are you crazy to public such kernel of silver level before 1 week of compettion finish? what is your goal? destroy results of the people who work hard?
", "votes": 22, "replies": [] }, { "id": 1054061, "author_name": "Stanley Zheng", "author_url": "", "post_date": "2020-10-19T16:16:51.673000", "content": "As much as I think releasing baselines is helpful, is it really a great idea to release a top 20 kernel in the last week of a competition?
", "votes": 14, "replies": [ { "id": 1054071, "author_name": "khyeh", "author_url": "", "post_date": "2020-10-19T16:27:57.403000", "content": "I guess you make your point as well. As in the past competition, there will be a warning of not sharing a high-scoring kernel within one week, I don't see it when I publish it (8th day before the competition ends…)
", "votes": -8, "replies": [] }, { "id": 1054098, "author_name": "Stanley Zheng", "author_url": "", "post_date": "2020-10-19T17:05:23.597000", "content": "Myself, and tons of others, have worked for a month to get to this score. Releasing a kernel with a month left makes sense - people can improve and understand the code. Releasing with a week left (whether 7 or 8 days) defeats the purpose. Most people will just submit the public kernel, and now people who didn't participate, but forked your code, have a medal. I understand this isn't the intention, but it is an unfortunate consequence.
\nEdit: Meme for condolences to everyone not in the top 27
\n
Unfortunately, now it could become a game of the person/team with large compute available wins silver or may be gold who knows ? :(. But I think now whatever we say or do cant change what is going to happen in this comp . However , thank you for the idea , could really have been helpful 2/3 weeks ago ..
", "votes": 2, "replies": [] }, { "id": 1054123, "author_name": "", "author_url": "", "post_date": "2020-10-19T17:38:42.427000", "content": "", "votes": -3, "replies": [] }, { "id": 1054133, "author_name": "khyeh", "author_url": "", "post_date": "2020-10-19T17:54:26.633000", "content": "@stanleyjzheng @phoenix9032 Get your point now, I'm sorry to cause the unfortunate results :(
", "votes": 0, "replies": [] }, { "id": 1054347, "author_name": "Iafoss", "author_url": "", "post_date": "2020-10-19T20:45:37.163000", "content": "@khyeh0719 You've been registered at kaggle for 3 years but behave as a novice… Do you really think that there is any difference between posting it in 7 or 8 days before the competition end? If you never participated in competitions, it would be probably understandable, but you intentionally ruin the competition trying to post the last moment high performing kernel and get upvotes (like some novices do by posting blends and forks not caring about their reputation).
\nSuch a kernel would be awesome if posted 2 month before the competition end, so all participants would appreciate the work you have done, and it could really boost the interest to this competition as well as motivate new approaches. Such a kernel would be ok if posted 1 month before the end: some people would complain, but you would still make a meaningful contribution to the competition. But think what you have done just posting it in a week before the end…
I completely agree. Such actions kill the competitive spirit and motivation. He's just a conceited youngster.
", "votes": 1, "replies": [] }, { "id": 1054523, "author_name": "Shujun", "author_url": "", "post_date": "2020-10-20T02:08:06.403000", "content": "I agree with @iafoss and this is really unfortunate, and it seems this type of thing is happening more and more often. In OpenVaccine, many top solutions used almost the same architecture from high scoring kernels (many of which were published over a week before the competition deadline) + additional data tricks. I hope there would be some policy change to punish this type of behavior, as it is extremely detrimental to the competitive spirit of the community.
", "votes": 4, "replies": [] }, { "id": 1054545, "author_name": "Iafoss", "author_url": "", "post_date": "2020-10-20T02:33:14.440000", "content": "@shujun717 This issue has been discussed for many times at kaggle, and they added the possibility to report unhealthy kernels (3 dots in the upper left corner) for further investigation. Though, to completely prevent such behavior, good option could be complete suppression of publication and updating any public kernels in 2 or even 4 weeks before the end of the competition. In this competition just training the models could take a week, and people would need several more weeks to explore the posted approach, not just copy-past and get a medal if one is lucky or has good GPU resources to quickly train several big models not even modifying the code… Though, public kernels published at the early stage of the competition are really great both for getting started and for learning. If this kernel was posted earlier, it would be extremely helpful.
", "votes": 10, "replies": [] }, { "id": 1054630, "author_name": "Shujun", "author_url": "", "post_date": "2020-10-20T04:11:49.357000", "content": "@iafoss Thanks for pointing out the option to report… I think it would be a good option to suppress public notebooks in the final two weeks as well. Hopefully they implement something like that
", "votes": 1, "replies": [] }, { "id": 1054751, "author_name": "ratan rohith", "author_url": "", "post_date": "2020-10-20T06:11:27.880000", "content": "@khyeh0719 One thing you can do is to make your model files dataset private. So that copy and edit submissions can be stopped.
", "votes": 1, "replies": [] }, { "id": 1054829, "author_name": "RabotniKuma", "author_url": "", "post_date": "2020-10-20T07:44:27.110000", "content": "@ratan123 Nope, that is definitely not a good option. If @khyeh0719 makes it private now, there will be a division between those already downloaded the model and those not yet. To make this game fair, he must leave it available to all the participants.
", "votes": 11, "replies": [] } ] }, { "id": 1054128, "author_name": "Vibhu Agrawal", "author_url": "", "post_date": "2020-10-19T17:49:09.917000", "content": "You can call it a baseline all you want, but this is not one. A baseline is supposed to be a simple model, something that is easy enough to beat, but robust enough to ensure that any model that surpasses its accuracy actually \"works\". Releasing a solid solution in the last week is basically undermining the efforts of the participants who have spent the last several weeks on this competition.
", "votes": 15, "replies": [ { "id": 1054135, "author_name": "khyeh", "author_url": "", "post_date": "2020-10-19T17:55:14.307000", "content": "Thanks for pointing out, I don't mean to cause the unfortunate results :(
", "votes": -4, "replies": [] }, { "id": 1054141, "author_name": "sin", "author_url": "", "post_date": "2020-10-19T18:00:51.697000", "content": "@khyeh0719 as least don't make your models public….
", "votes": 0, "replies": [] }, { "id": 1054152, "author_name": "Stanley Zheng", "author_url": "", "post_date": "2020-10-19T18:09:55.767000", "content": "Even our best model we have improved for a month brings score down when ensembled 😂
\nWe also don't have crazy compute to tune this in only a week. I guess we can suffer together
@stanleyjzheng don't give up :D. Did you make a dummy sub? That LB climb was fast af.
", "votes": 1, "replies": [] }, { "id": 1054157, "author_name": "Stanley Zheng", "author_url": "", "post_date": "2020-10-19T18:17:02.310000", "content": "@underwearfitting Yep, dummy csv only sub, our highest working sub is still 0.275. Thanks for the encouragement, I guess we'll see what we can do in a short time.
", "votes": 0, "replies": [] }, { "id": 1054158, "author_name": "DarkCube", "author_url": "", "post_date": "2020-10-19T18:17:39.827000", "content": "@stanleyjzheng I see you're really bummed out about this. Maybe the hosts can disqualify ALL submissions that are really similar to any public baseline.
", "votes": 1, "replies": [] }, { "id": 1054161, "author_name": "Stanley Zheng", "author_url": "", "post_date": "2020-10-19T18:21:28.487000", "content": "@darkcube Thanks for the kind words. I wish… but it'll never happen, high scoring public notebooks just a part of Kaggle that I guess we all have to accept.
", "votes": 2, "replies": [] } ] }, { "id": 1054772, "author_name": "Dewei Chen", "author_url": "", "post_date": "2020-10-20T06:38:29.577000", "content": "em… I think it's better to divide your code into inference, and training. You can public your training code and hide your own model, and say \"I use this model to get a good performance in public LB\". The assiduous Kagglers still will be learning a lot from your code. Because for me I spend a lot of time on the training model and just get about 0.290-0.300 public scores, but there is a group of people just watch on Best Score, they using one single submit and exceed all my model, I will feel unfair. Anyway, I'll keep learning your code, and thanks your idea sharing (not for 0.233 solution :) )
", "votes": 10, "replies": [] }, { "id": 1061149, "author_name": "عثمان", "author_url": "", "post_date": "2020-10-26T19:16:28.750000", "content": "\nThis is a really great kernel and I wanted to express appreciation for you sharing it. I know a lot of people gave you slack for it; but the reality of the matter is:
\nIf anything, the only shocking part of the three kernels you pushed out was that you also shared the weights… :-). I think in the past, most other competitors would have withheld those; but at the end, you probably ended up saving the environment from a lot of throw-away electricity / carbon offset.
\nI wanted to share some notes about this kernel specifically because there were critics who claimed it wasn't fair and that people with superior compute would be able to take more advantage. That is 100% accurate. More compute means you have more RoI for time to conduct more experiments. That is true if you have 1 week to go or 3 months to go. No one claims it isn't fair that there are GM's that get paid by NVIDIA or H2O to work full time on Kaggle and get DGX boxes? Why complain here?
\nWhat more, there's something to be said about taking a public kernel and optimizing it. I won't get into the data science aspect, but I'll talk strictly about software engineering. These kernels take a long time to run and can be optimized to run many orders of magnitude faster by just making slight alterations to do code. For example, people notes that the multi-output ran significantly slower than the single output one even though there were no substantive changes. Did anyone here bother to profile it? The culprit was multiple ilocs on an unfiltered dataframe. Simple converting over to numpy at dataset load makes this kitty pur 15x (1500%) faster on my desktop machine and my dual gpu's remain well fed at +90% utilization. Other tricks include sampling on stage 1 to reduce overfitting, since some of the patients have >1000 slices and all the slices are very well correlated axially.
Anyhow, just wanted to say thanks for the kernel, most of the publicly shared resources were pretty stale or just mean target by slice submissions. Cheers.
", "votes": 7, "replies": [ { "id": 1061159, "author_name": "Stanley Zheng", "author_url": "", "post_date": "2020-10-26T19:20:10.160000", "content": "I echo these sentiments - I didn't have the resources to do the data science optimization aspect, but I spent many hours optimizing the code and got inference to run in 1/4 of the time, then adding TTA is my current score. I am still very much behind where I was before the kernel however, and spent a ton of time I could have used studying for exams. I have to admit, I did learn a ton from the code and reading it, so thanks to Kun for that. My apologies for the harsh criticism at release, this was very frustrating, until I could see the improbability. Kaggle is about learning, not about the medals after all.
", "votes": 2, "replies": [] }, { "id": 1061165, "author_name": "عثمان", "author_url": "", "post_date": "2020-10-26T19:26:45.200000", "content": "When all the dust settles, there will be a very small handful of winners. And then there's going to be the remaining 98% of us. If we don't set proper expectations and goals from the get-go (like you were successfully able to do above @stanleyjzheng) then we risk taking damage to our mental health :-). I hope you place favorably on the private LB and above all, the effort you invested + knowledge you acquired is yours for the keeping.
", "votes": 3, "replies": [] }, { "id": 1061192, "author_name": "khyeh", "author_url": "", "post_date": "2020-10-26T19:45:09.517000", "content": "@authman Thanks a lot for the kind words:)
", "votes": 3, "replies": [] } ] }, { "id": 1056492, "author_name": "Mark P", "author_url": "", "post_date": "2020-10-21T18:46:39.817000", "content": "I'm torn to be honest. On one hand I agree with the commenters that don't think Kun should have released it so close to the end. But on the other I'm grateful to him to be able to look through his Effnet training code and compare it to mine to learn where I went wrong. I'm spent so many hours trying to train that model on GPU and never got it faster than 45mins/epoch (for 6500 patients).
\nTo be fair though the skill of the man where solving my massive problem is just one throw away line for him \"I use very simple efficientnet b0 as a starter for quick prototyping.\" makes me:
\na) want to cry at the gap between our skill levels
\nb) makes me happy that these things are solvable and I can learn
\n\nepoch 0 train Step 5596/5596, loss: 0.592, pe_pr: 0.9633, rv_lv: 0.9767, rv_lv: 0.9787, lefts: 0.9678, chron: 0.9950, right: 0.9657, acute: 0.9957, centr: 0.9851, indet: 0.9840, time: 13889.76
\n
Am I missing something? 45min per epoch is still legendary.
", "votes": 3, "replies": [] }, { "id": 1057807, "author_name": "", "author_url": "", "post_date": "2020-10-23T03:07:59.073000", "content": "", "votes": 0, "replies": [] } ] }, { "id": 1054115, "author_name": "Yaroslav Isaienkov", "author_url": "", "post_date": "2020-10-19T17:31:09.487000", "content": "I think you published your kernel very early
\nEntering the game one day before the end would be enough 😑
Thanks for the idea. Especially the rule-based PP. Bad timing tho :)
", "votes": 3, "replies": [] }, { "id": 1054191, "author_name": "Theo Viel", "author_url": "", "post_date": "2020-10-19T18:58:25.343000", "content": "If you were already in the medal range before the kernel was released, chances are you can improve your score by re-using what was shared.
\nThe most straigh-forward approach is just to blend your model with it. This should be enough to beat all the people that will effortlessly fork the kernel. Hence still do a decent finish.
If you trained stage 1 models, you can use them to train stage 2 models as @khyeh0719 did. This is what I am planning to do and I am pretty confident this will be enough to improve this (way too strong) baseline. There is a lot of room for improvement, and if you spent the last month experimenting you have extra knowledge people who \"join & fork\" don't have.
", "votes": 4, "replies": [ { "id": 1054198, "author_name": "sin", "author_url": "", "post_date": "2020-10-19T19:03:41.913000", "content": "Good point. But I don't think you are 100% right about blending. The max inference time is 9 hours here. From what we've done so far, we cannot afford blending with his results.
", "votes": 4, "replies": [] }, { "id": 1054205, "author_name": "Stanley Zheng", "author_url": "", "post_date": "2020-10-19T19:06:25.607000", "content": "Great advice.
\nI agree with Sin about blending. This notebook takes a significant amount of time to run, and even blending with our best model (0.275) in a dummy submission, we still see a decrease in score. We're also using tensorflow, so existing models are also worthless. Still some great advice, and maybe a week is enough to train in pytorch and salvage a silver medal.
Actually, a two stage model approach will cost mostly 8.5 hours to finish the re-run, the inference speed it's the key,there's not much space for ensembling and blending. So be careful about the balance between the model complexity and compute bottleneck
", "votes": 2, "replies": [] }, { "id": 1054334, "author_name": "Theo Viel", "author_url": "", "post_date": "2020-10-19T20:29:54.070000", "content": "Indeed, I did not know such models were so long to run. Sorry about that. Looks like the best bet is to retrain a first level model then.
", "votes": 1, "replies": [] }, { "id": 1054349, "author_name": "Serigne ", "author_url": "", "post_date": "2020-10-19T20:46:54.577000", "content": "@theoviel
\nInference takes almost 9 hours. You can't really blend here.
As many here, I forked to submit this (strong) baseline, while still working on my model aside. Because I'm working too hard to set up and train my model, and it would be too easy for newcomers who fork and submit it to be above me, given the model weights are publicly shared too .
", "votes": 4, "replies": [] }, { "id": 1054350, "author_name": "Nirjhar Roy", "author_url": "", "post_date": "2020-10-19T20:55:13.350000", "content": "A little bit more sad part is the person in question has way more credential and experience in kaggle than me and lot of other people that I didnt think a kernel gold this way would interest him .. :(
", "votes": 2, "replies": [] }, { "id": 1054355, "author_name": "Stanley Zheng", "author_url": "", "post_date": "2020-10-19T20:59:30.747000", "content": "Exactly! I don't really see the motivation. Kun deserves the utmost respect for his amazing competition finishes, and is one solo gold away from Kaggle's greatest honour. However, he still chooses to release kernels in the last week… No hard feelings to anyone in this thread, just suprising to see from such an esteemed member.
", "votes": 2, "replies": [] }, { "id": 1054360, "author_name": "Serigne ", "author_url": "", "post_date": "2020-10-19T21:15:34.237000", "content": "I also have a lot of respect for Kun, and he obviously doesn't have any bad intentions in sharing his great notebooks with the community, and which undoubtedly took him a lot of effort.
\nHowever, this competition is computationnaly heavy and sadly many people won't see it as a baseline to beat, but rather an easy way to gain medal, particularly with just a week left.
", "votes": 1, "replies": [] }, { "id": 1058848, "author_name": "Philip Dhingra", "author_url": "", "post_date": "2020-10-24T11:04:50.887000", "content": "How are you calculating the inference time of this notebook? Execution Info says 7787.2 seconds
", "votes": 0, "replies": [] } ] }, { "id": 1054568, "author_name": "Marsh", "author_url": "", "post_date": "2020-10-20T02:45:50.673000", "content": "well… I didn't see that coming
", "votes": 4, "replies": [] }, { "id": 1054147, "author_name": "Theo Viel", "author_url": "", "post_date": "2020-10-19T18:04:56.380000", "content": "On the plus side, the work you did is great, and a lot of people will learn from it.
\nHowever, I am quite happy I did not join this competition earlier that 2 days ago.
", "votes": 2, "replies": [ { "id": 1054160, "author_name": "Nirjhar Roy", "author_url": "", "post_date": "2020-10-19T18:19:57.733000", "content": "What are you planning ..lol :D
", "votes": 2, "replies": [] } ] }, { "id": 1058788, "author_name": "Alberto Benayas", "author_url": "", "post_date": "2020-10-24T09:10:35.400000", "content": "Great kernel but not the best timing. 8 days before deadline and just 1 day before the last week… You could have just posted it either a month ago or when the competition finished, but now you turned the medal race into a fork and blend competition for many participants (including myself). Next time please about other people's efforts
", "votes": 1, "replies": [] }, { "id": 1057859, "author_name": "saatuo", "author_url": "", "post_date": "2020-10-23T04:59:55.103000", "content": "Hi, @khyeh0719 , When i train your Stage2 RNN Model , this kernel, i got below loss.
\nepoch 0 train step 2912/2912, loss: 0.4568, time: 4443.1336
\nepoch 0 valid Step 1455/1455, loss: 0.5876, time: 1570.2982
\nepoch 1 train step 2912/2912, loss: 0.4145, time: 4734.2051
\nepoch 1 valid Step 1455/1455, loss: 0.5958, time: 1444.1449
Can you tell me What could be wrong?
", "votes": 1, "replies": [] }, { "id": 1057200, "author_name": "عثمان", "author_url": "", "post_date": "2020-10-22T13:17:09.030000", "content": "@khyeh0719 with 5596 mini batches, are you training with 256 BS? If you don't mind me asking, what hardware are you running this on?
", "votes": 1, "replies": [ { "id": 1058043, "author_name": "Vee", "author_url": "", "post_date": "2020-10-23T09:28:09.787000", "content": "I had the same doubt - I concluded that either it is a much larger BS or he is training on only partial dataset. Most probably it is the latter.
", "votes": 1, "replies": [] } ] }, { "id": 1058613, "author_name": "wcysysu", "author_url": "", "post_date": "2020-10-24T01:06:31.453000", "content": "The key problem is that most of the participants may not have enough GPU resources to reproduce the whole process in a short time. I guess that simply replacing efficientnet-b0 with efficientnet-b1 or efficienetnet-b5 can achieve better results…
", "votes": 0, "replies": [] }, { "id": 1054150, "author_name": "DarkCube", "author_url": "", "post_date": "2020-10-19T18:09:17.150000", "content": "Thank you so much for sharing your work. I'm kind of a latecomer myself (I only started participating 5 days ago). So your notebook is really going to help.
\nDespite being a latecomer, I kinda agree with @stanleyjzheng that you probably shouldn't have shared it. I strongly believe that people just copy other people's work to make a submission don't even deserve a stand in the leaderboard. Maybe just providing the ideas you used would be a far greater help for everyone?
", "votes": 0, "replies": [] }, { "id": 1054854, "author_name": "imnishantg", "author_url": "", "post_date": "2020-10-20T08:20:24.500000", "content": "While I understand the point others are making in this thread, @khyeh0719 , I'm very thankful to you that you shared these notebook.
\nEspecially for me, I'll be able to learn a lot about best practices, resolve many issues I was facing in writing PyTorch/Fastai code.
\nAfter the competition, I would have lost the steam and drive to focus on that. During competition, I want to now understand each and every line of code and see where I was going wrong.
Thanks again!
", "votes": -4, "replies": [] }, { "id": 1054782, "author_name": "srinivassunk", "author_url": "", "post_date": "2020-10-20T06:56:13.720000", "content": "good one thanks for sharing with others nice work keep going
", "votes": -5, "replies": [] }, { "id": 1054610, "author_name": "Marcus Lin", "author_url": "", "post_date": "2020-10-20T03:33:12.133000", "content": "I dont see the problems here if the kernel don't change gold range rank order. maybe I am wrong.
", "votes": -10, "replies": [ { "id": 1054631, "author_name": "Stanley Zheng", "author_url": "", "post_date": "2020-10-20T04:12:47.257000", "content": "Oh yea, nothing wrong. Just a week to go through 1600 lines of code, ruining the competition for 670 teams. I don't see any problems either.
\nI do see where you are coming from - none of the cash positions are changed, but this still causes frustration for the 670 of us who are not in gold medal position.
", "votes": 6, "replies": [] }, { "id": 1054643, "author_name": "sin", "author_url": "", "post_date": "2020-10-20T04:24:22.063000", "content": "Ok, so if we are not here to get gold range rank, we are worthless.🤕🤒
", "votes": 8, "replies": [] }, { "id": 1054748, "author_name": "Marcus Lin", "author_url": "", "post_date": "2020-10-20T06:08:32", "content": "OK, maybe I'm wrong in some way. I apologize.
", "votes": 1, "replies": [] }, { "id": 1056610, "author_name": "DarkCube", "author_url": "", "post_date": "2020-10-21T22:35:00.470000", "content": "@stanleyjzheng ok, I know this really sounds that it should be obvious but are you telling me that going through 1600 lines of code in a single week is unrealistic?!!! I'm here scolding myself for not understanding it from first glance lol.
", "votes": -2, "replies": [] }, { "id": 1056618, "author_name": "", "author_url": "", "post_date": "2020-10-21T22:59:51.093000", "content": "", "votes": 0, "replies": [] }, { "id": 1057080, "author_name": "DarkCube", "author_url": "", "post_date": "2020-10-22T10:49:34.773000", "content": "are you sure there are no bits of performance you can squeeze in? TensorFlow really amazes me from that point. Whenever I think my code is as efficient as it can get, the guide always proves me wrong.
", "votes": -1, "replies": [] }, { "id": 1058405, "author_name": "Shubham Thapa", "author_url": "", "post_date": "2020-10-23T16:53:15.600000", "content": "how are you supposed to use tensorflow in a pytorch model
", "votes": 2, "replies": [] }, { "id": 1058799, "author_name": "DarkCube", "author_url": "", "post_date": "2020-10-24T09:45:48.937000", "content": "He said they were using TensorFlow. And there are scripts that can convert them to TensorFlow. I'm gonna provide a dataset with the converted models in a few hours.
", "votes": 0, "replies": [] }, { "id": 1059260, "author_name": "DarkCube", "author_url": "", "post_date": "2020-10-24T20:41:08.743000", "content": "Okay, that was a living hell.
\nhere is the script
\nhere is a few models converted to onnx which you can then convert back to basically any major ML framework (or at least that's what they claim to do).
\nAlso the input is (C, H, W) instead of (H, W, C), and for some reason the converted keras model doesn't work for cpu. You can convert the rest, I'm done.
to be fair people this is hardly NOT a baseline. I mean all he did was implement last year's solution and added:
\nAnd that's it really. like, come on people. I'm 16 and the only new thing I found in his submission is the 2-stage training since you can't train an end-to-end model due to lack of resources. (if you want to do that you have to feed the entire 220 scans into the model and compute the training step on that which simply would take a huge amount of memory).
", "votes": -7, "replies": [ { "id": 1058388, "author_name": "", "author_url": "", "post_date": "2020-10-23T16:34:31.610000", "content": "", "votes": 0, "replies": [] }, { "id": 1058400, "author_name": "DarkCube", "author_url": "", "post_date": "2020-10-23T16:50:16.677000", "content": "I'm not really sure what model would take 8 hours per epoch (unless you're training an ensemble) but have you considered training your models on a subset of the data? maybe include all negative_exam_for_pe = 0 and indeterminate = 1 and maybe double that with data from negative_exam_for_pe = 1. Most of what you'd learn from negative exams after about say 2000 data points I think is marginal.
\nAlso, WE'RE THE SAME AGE????? that's awesome man. I never thought I'd find another 16 y/o on this website.
", "votes": -3, "replies": [] }, { "id": 1058413, "author_name": "", "author_url": "", "post_date": "2020-10-23T16:59:32.390000", "content": "", "votes": 0, "replies": [] }, { "id": 1058431, "author_name": "DarkCube", "author_url": "", "post_date": "2020-10-23T17:19:44.813000", "content": "if he can train on a batch of 256 then why did he implement 2 stage training? I imagine training on and end-to-end model would yield far more robust results. As opposed to 2 stage training being basically transfer learning.
", "votes": 0, "replies": [] } ] }, { "id": 1057984, "author_name": "cxt", "author_url": "", "post_date": "2020-10-23T08:14:52.120000", "content": "👍Good kernal
\nJust a little late.There is Less free time to study your model, especially calculated quantities are too much!🤔