Am I the only one struggling with the running time limit of 9 hours of the notebook under submission?
\nI am performing a preprocessing over the segmented volumes in order to train a segmentation CNN that will allow me to train over segmented ROIs into the competition's labels. However, the preprocessing and the inference of a volume take around 10 seconds, that's without taking into account the train time which is around 2 hours, so 10 seconds * 3000 volumes (train+test that I would need to apply it), it would take around 8 hours, and remember this is only processing the data to obtain the ROIs for another training pipeline to classify.
\nObviously this makes this type of pipelines (segmentation + classification) almost impossible, I think one could spend a lot of time trying to optimize a little bit, but it is incredible far from a feasible time which would be taking less than 2-3 seconds per volume. It is so frustrating that you cannot use your idea due to a time limitation in the notebook.
\nProbably it is part of the challenge that you cannot easily use an optimal pipeline and must go through workarounds to fit the submission into the time limitation.
", "messages": [ { "id": 1909398, "postDate": "2022-08-22T15:33:59.037Z", "content": "I get my inference done in less than 2 hours, I am doing a bone classification, a bone ROI segmentation and a fracture classification… I think it would be more than enough if you are good enough with your optimizations, you will have to optimize it to the limit if you want this type of speed but its very much not impossible…
", "rawMarkdown": "I get my inference done in less than 2 hours, I am doing a bone classification, a bone ROI segmentation and a fracture classification... I think it would be more than enough if you are good enough with your optimizations, you will have to optimize it to the limit if you want this type of speed but its very much not impossible...", "votes": 4, "replies": [ { "id": 1917768, "postDate": "2022-08-29T03:54:05.657Z", "content": "what labels you used for bone classification? or how do you classify the bone, by knn or what?
", "rawMarkdown": "what labels you used for bone classification? or how do you classify the bone, by knn or what?" }, { "id": 1917795, "postDate": "2022-08-29T04:48:41.327Z", "content": "My current bone classifier technique is something I would not like to tell at this point, but I am planning to change it anyways by exploring public notebooks because it is apparently not as fast or good as even machine learning public notebook (ref to .88 acc one)…
", "rawMarkdown": "My current bone classifier technique is something I would not like to tell at this point, but I am planning to change it anyways by exploring public notebooks because it is apparently not as fast or good as even machine learning public notebook (ref to .88 acc one)..." } ] }, { "id": 1909239, "postDate": "2022-08-22T13:01:59.920Z", "content": "Am I the only one struggling with the running time limit of 9 hours of the notebook under submission?
\nI am performing a preprocessing over the segmented volumes in order to train a segmentation CNN that will allow me to train over segmented ROIs into the competition's labels. However, the preprocessing and the inference of a volume take around 10 seconds, that's without taking into account the train time which is around 2 hours, so 10 seconds * 3000 volumes (train+test that I would need to apply it), it would take around 8 hours, and remember this is only processing the data to obtain the ROIs for another training pipeline to classify.
\nObviously this makes this type of pipelines (segmentation + classification) almost impossible, I think one could spend a lot of time trying to optimize a little bit, but it is incredible far from a feasible time which would be taking less than 2-3 seconds per volume. It is so frustrating that you cannot use your idea due to a time limitation in the notebook.
\nProbably it is part of the challenge that you cannot easily use an optimal pipeline and must go through workarounds to fit the submission into the time limitation.
", "rawMarkdown": "Am I the only one struggling with the running time limit of 9 hours of the notebook under submission?\n\nI am performing a preprocessing over the segmented volumes in order to train a segmentation CNN that will allow me to train over segmented ROIs into the competition's labels. However, the preprocessing and the inference of a volume take around 10 seconds, that's without taking into account the train time which is around 2 hours, so 10 seconds * 3000 volumes (train+test that I would need to apply it), it would take around 8 hours, and remember this is only processing the data to obtain the ROIs for another training pipeline to classify. \n\nObviously this makes this type of pipelines (segmentation + classification) almost impossible, I think one could spend a lot of time trying to optimize a little bit, but it is incredible far from a feasible time which would be taking less than 2-3 seconds per volume. It is so frustrating that you cannot use your idea due to a time limitation in the notebook. \n\nProbably it is part of the challenge that you cannot easily use an optimal pipeline and must go through workarounds to fit the submission into the time limitation. \n", "votes": 1 }, { "id": 1909843, "postDate": "2022-08-23T01:33:55.653Z", "content": "Train and infer in different notebooks. Save the best weights of the trained model. And use that in infernece.
", "rawMarkdown": "Train and infer in different notebooks. Save the best weights of the trained model. And use that in infernece.", "votes": 2, "replies": [ { "id": 1910065, "postDate": "2022-08-23T06:34:35.850Z", "content": "You mean that I can preproccess the train dataset and train in my home pc, and then upload the weights with just the inference code and submit that?? I though that you have to submit everything and they create a clean kernel and run every cell so they train in there and then run inference.
\nIf it is that way then is pretty easy to stay within the constrain of 9 hours.
", "rawMarkdown": "You mean that I can preproccess the train dataset and train in my home pc, and then upload the weights with just the inference code and submit that?? I though that you have to submit everything and they create a clean kernel and run every cell so they train in there and then run inference.\n\nIf it is that way then is pretty easy to stay within the constrain of 9 hours. " }, { "id": 1910331, "postDate": "2022-08-23T10:53:42.097Z", "content": "Yeah that's the essence
", "rawMarkdown": "Yeah that's the essence " } ] }, { "id": 1909443, "postDate": "2022-08-22T16:02:58.543Z", "content": "@harshitsheoran I think I'm swapping to only 2D networks because I thought to use a 3D CNN to classify ROIs obtained by an UNET. But for me at least, it seems impossible to obtain a whole 3D segmentations in time and save in disk ROIs to train.
", "rawMarkdown": "@harshitsheoran I think I'm swapping to only 2D networks because I thought to use a 3D CNN to classify ROIs obtained by an UNET. But for me at least, it seems impossible to obtain a whole 3D segmentations in time and save in disk ROIs to train. \n\n", "replies": [ { "id": 1909450, "postDate": "2022-08-22T16:07:16.527Z", "content": "Whatever works for you, I am only telling you that it is possible, but getting it to work might take a really long time (still worth it imo) if you are not very well experienced in optimization.
", "rawMarkdown": "Whatever works for you, I am only telling you that it is possible, but getting it to work might take a really long time (still worth it imo) if you are not very well experienced in optimization." }, { "id": 1909468, "postDate": "2022-08-22T16:21:19.457Z", "content": "Ok, I will try to optimize as much as I can, thank you for the help mate.
", "rawMarkdown": "Ok, I will try to optimize as much as I can, thank you for the help mate. " } ] } ], "comments": [ { "id": 1909398, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2022-08-22T15:33:59.037000", "content": "I get my inference done in less than 2 hours, I am doing a bone classification, a bone ROI segmentation and a fracture classification… I think it would be more than enough if you are good enough with your optimizations, you will have to optimize it to the limit if you want this type of speed but its very much not impossible…
", "votes": 4, "replies": [ { "id": 1917768, "author_name": "dragon zhang", "author_url": "", "post_date": "2022-08-29T03:54:05.657000", "content": "what labels you used for bone classification? or how do you classify the bone, by knn or what?
", "votes": 0, "replies": [] }, { "id": 1917795, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2022-08-29T04:48:41.327000", "content": "My current bone classifier technique is something I would not like to tell at this point, but I am planning to change it anyways by exploring public notebooks because it is apparently not as fast or good as even machine learning public notebook (ref to .88 acc one)…
", "votes": 0, "replies": [] } ] }, { "id": 1909843, "author_name": "Collide_Conquer_19", "author_url": "", "post_date": "2022-08-23T01:33:55.653000", "content": "Train and infer in different notebooks. Save the best weights of the trained model. And use that in infernece.
", "votes": 2, "replies": [ { "id": 1910065, "author_name": "Javier Vera", "author_url": "", "post_date": "2022-08-23T06:34:35.850000", "content": "You mean that I can preproccess the train dataset and train in my home pc, and then upload the weights with just the inference code and submit that?? I though that you have to submit everything and they create a clean kernel and run every cell so they train in there and then run inference.
\nIf it is that way then is pretty easy to stay within the constrain of 9 hours.
", "votes": 0, "replies": [] }, { "id": 1910331, "author_name": "Collide_Conquer_19", "author_url": "", "post_date": "2022-08-23T10:53:42.097000", "content": "Yeah that's the essence
", "votes": 0, "replies": [] } ] }, { "id": 1909443, "author_name": "Javier Vera", "author_url": "", "post_date": "2022-08-22T16:02:58.543000", "content": "@harshitsheoran I think I'm swapping to only 2D networks because I thought to use a 3D CNN to classify ROIs obtained by an UNET. But for me at least, it seems impossible to obtain a whole 3D segmentations in time and save in disk ROIs to train.
", "votes": 0, "replies": [ { "id": 1909450, "author_name": "Harshit Sheoran", "author_url": "", "post_date": "2022-08-22T16:07:16.527000", "content": "Whatever works for you, I am only telling you that it is possible, but getting it to work might take a really long time (still worth it imo) if you are not very well experienced in optimization.
", "votes": 0, "replies": [] }, { "id": 1909468, "author_name": "Javier Vera", "author_url": "", "post_date": "2022-08-22T16:21:19.457000", "content": "Ok, I will try to optimize as much as I can, thank you for the help mate.
", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "1909398": "I get my inference done in less than 2 hours, I am doing a bone classification, a bone ROI segmentation and a fracture classification... I think it would be more than enough if you are good enough with your optimizations, you will have to optimize it to the limit if you want this type of speed but its very much not impossible...", "1909239": "Am I the only one struggling with the running time limit of 9 hours of the notebook under submission?\n\nI am performing a preprocessing over the segmented volumes in order to train a segmentation CNN that will allow me to train over segmented ROIs into the competition's labels. However, the preprocessing and the inference of a volume take around 10 seconds, that's without taking into account the train time which is around 2 hours, so 10 seconds * 3000 volumes (train+test that I would need to apply it), it would take around 8 hours, and remember this is only processing the data to obtain the ROIs for another training pipeline to classify. \n\nObviously this makes this type of pipelines (segmentation + classification) almost impossible, I think one could spend a lot of time trying to optimize a little bit, but it is incredible far from a feasible time which would be taking less than 2-3 seconds per volume. It is so frustrating that you cannot use your idea due to a time limitation in the notebook. \n\nProbably it is part of the challenge that you cannot easily use an optimal pipeline and must go through workarounds to fit the submission into the time limitation. \n", "1909843": "Train and infer in different notebooks. Save the best weights of the trained model. And use that in infernece.", "1909443": "@harshitsheoran I think I'm swapping to only 2D networks because I thought to use a 3D CNN to classify ROIs obtained by an UNET. But for me at least, it seems impossible to obtain a whole 3D segmentations in time and save in disk ROIs to train. \n\n" } }