![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F930277%2F57ffaac4efc88b8c3acf4449adc83556%2FScreenshot%20from%202023-02-26%2013-51-14.png?generation=1677444818785444&alt=media\")
New Kaggler here… So I built a pipeline to use a radiomics-based approach on the RSNA challenge, specifically using the pyradiomics Python module.
\nIt runs perfectly on my home computer and also in the Kaggle notebook… until I set \"internet off\" mode. Then, when I try to import the pyradiomics package with
\n!pip install pyradiomics
I instead get:
\n\n\nERROR: Could not find a version that satisfies the requirement pyradiomics
\n
and also:
\n\n\nModuleNotFoundError: No module named 'radiomics'
\n
…It appears that PyRadiomics is not part of the environment previously set up by Kaggle, so with internet off it hits a wall trying to download it. So is there any way to pre-load a module that you need, before submitting in \"internet off\" mode?
\nIt would be almost unbelievable if a) my time on this is completely wasted to the point that I cannot even *submit *the work; and b) it is not permitted to use a radiomics package on what is very clearly a prime use case for a radiomics approach!
\nI still have a day and a half to submit something. Any advice appreciated ASAP! (Fortunately the pipeline runs relatively fast.)
", "messages": [ { "id": 2160434, "postDate": "2023-02-26T17:11:13.650Z", "content": "Did you find a working solution? I am making and copying the wheels on the Kaggle platform (e.g., !pip wheel pyradiomics) into a personal Kaggle dataset and adding them from there. (There might be many whl files created and you could try to see which of the dependencies are already satisfied to make your zip to be uploaded as a dataset smaller.) Here is a good tutorial: https://www.kaggle.com/code/jiny333/creating-whl-files-to-install-external-libraries
It would be interesting to hear about this pyradiomics-based solution if it some day becomes public. 👍
", "rawMarkdown": "It would be interesting to hear about this pyradiomics-based solution if it some day becomes public. 👍" }, { "id": 2160658, "postDate": "2023-02-26T20:38:57.240Z", "content": "Thanks Antti. I might try this \"wheels\" approach. So far i have been trying to just bring in the packages i need from Github, store them in local folder, and then use pip to install them from there instead of the internet. But this leads to a string of dependencies i have to install too… then dependencies for the dependencies… and here we are still 😒
", "rawMarkdown": "Thanks Antti. I might try this \"wheels\" approach. So far i have been trying to just bring in the packages i need from Github, store them in local folder, and then use pip to install them from there instead of the internet. But this leads to a string of dependencies i have to install too... then dependencies for the dependencies... and here we are still 😒", "votes": 1, "replies": [ { "id": 2160665, "postDate": "2023-02-26T20:56:32.327Z", "content": "Also when I install dependencies i often find i can't even look in the folders i downloaded. They just hang forever:
\n
Previously, the whole system got sick when i tried to run the first cell that sets up the environment:
\n![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F930277%2Fb1303793e01c813afcdca22c3f910495%2FScreenshot%20from%202023-02-26%2007-27-23_kaggleMalfunction.png?generation=1677444968857888&alt=media\")
Like, who is in charge here? I am basically being kept from even submitting my work by absurd technical issues.
", "rawMarkdown": "Also when I install dependencies i often find i can't even look in the folders i downloaded. They just hang forever:\n\n\n\nPreviously, the whole system got sick when i tried to run the first cell that sets up the environment: \n\n\n\nLike, who is in charge here? I am basically being kept from even *submitting my work* by absurd technical issues." }, { "id": 2160668, "postDate": "2023-02-26T21:06:56.087Z", "content": "Let's be in charge today. Fork this https://www.kaggle.com/code/anttiisosalo/create-pyradiomics-wheel/notebook > download zip, upload as private dataset, and add data, and you should be good to go.
", "rawMarkdown": "Let's be in charge today. Fork this https://www.kaggle.com/code/anttiisosalo/create-pyradiomics-wheel/notebook > download zip, upload as private dataset, and add data, and you should be good to go.", "votes": 1 }, { "id": 2160669, "postDate": "2023-02-26T21:08:13.577Z", "content": "And see here https://www.kaggle.com/code/anttiisosalo/16-bit-post-processing-rsna-bc-detection?scriptVersionId=119182142&cellId=12 how I am accessing private resources.
", "rawMarkdown": "And see here https://www.kaggle.com/code/anttiisosalo/16-bit-post-processing-rsna-bc-detection?scriptVersionId=119182142&cellId=12 how I am accessing private resources.", "votes": 1 }, { "id": 2160690, "postDate": "2023-02-26T22:09:27.720Z", "content": "I like your attitude 😀. But i think i have got it working without having to mess with repos or wheels.
\nThe thing i didn't see is the \"persistence\" pull down. So I did the following:
\nWith internet on, do usual !pip install pyradiomics
Set \"persistence\" to \"variables and files\". also set \"environment\" to \"always use latest environment\"
Now turn off internet and submit.
It looks like by doing this, you can tell the Kaggle notebook to simply remember whatever libraries you already had installed and open before the internet switched off. So since i already installed pyradiomics, it remembers to hang on to it.
\nWe will see if it finishes… so far running about an hour with no \"import error\", far longer than it was before… else i will follow your advice…
", "rawMarkdown": "I like your attitude 😀. But i think i have got it working without having to mess with repos or wheels.\n\nThe thing i didn't see is the \"persistence\" pull down. So I did the following:\n\n1. With internet on, do usual !pip install pyradiomics\n\n2. Set \"persistence\" to \"variables and files\". also set \"environment\" to \"always use latest environment\"\n\n3. Now turn off internet and submit. \n\nIt looks like by doing this, you can tell the Kaggle notebook to simply *remember* whatever libraries you already had installed and open before the internet switched off. So since i already installed pyradiomics, it remembers to hang on to it. \n\nWe will see if it finishes... so far running about an hour with no \"import error\", far longer than it was before... else i will follow your advice...", "votes": 1 }, { "id": 2160696, "postDate": "2023-02-26T22:21:12.250Z", "rawMarkdown": "", "isDeleted": true }, { "id": 2160785, "postDate": "2023-02-27T01:35:57.540Z", "content": "Well, I tried just about everything. But the wheels don't work either. I can bring them in as a zip file, but when I try to actually use them to build pyradiomics, I just get
\n\n\n\"ERROR: [packagename].whl is not a supported wheel on this platform.\"
\n
It's really odd that the competition organizers seem to have made it so hard for anyone to use any packages but the ones they already thought of. And it has nothing to do with the competition rules, since I can run this on my own home computer without internet, no problem.
\nMaybe will try looking for exact versions of the wheels to match the Python version Kaggle is using? I'm about out of ideas.
\n**UPDATE: I was downloading wheels for the wrong version of Python (3.8, while Kaggle uses 3.7.12). So now the wheels install fine and pyradiomics is installing + running without internet!
\n(Still think it should be made much simpler for people to use unusual python packages on here.)**
", "rawMarkdown": "Well, I tried just about everything. But the wheels don't work either. I can bring them in as a zip file, but when I try to actually use them to build pyradiomics, I just get \n\n>\"ERROR: [packagename].whl is not a supported wheel on this platform.\"\n\nIt's really odd that the competition organizers seem to have made it so hard for anyone to use any packages but the ones they already thought of. And it has nothing to do with the competition rules, since I can run this on my own home computer without internet, no problem.\n\nMaybe will try looking for exact versions of the wheels to match the Python version Kaggle is using? I'm about out of ideas.\n\n**UPDATE: I was downloading wheels for the wrong version of Python (3.8, while Kaggle uses 3.7.12). So now the wheels install fine and pyradiomics is installing + running without internet! \n\n(Still think it should be made much simpler for people to use unusual python packages on here.)**" } ] } ] }, { "id": 2160328, "postDate": "2023-02-26T15:29:51.510Z", "content": "Yea. In my experience you need \"internet on\" to install anything that's not standard in Kaggle notebooks (numpy, pandas, etc.) I am not familiar with an pre-download methods, but that does not mean it doesn't exist.
", "rawMarkdown": "Yea. In my experience you need \"internet on\" to install anything that's not standard in Kaggle notebooks (numpy, pandas, etc.) I am not familiar with an pre-download methods, but that does not mean it doesn't exist.", "replies": [ { "id": 2160344, "postDate": "2023-02-26T15:49:25.233Z", "content": "I think I may have an idea (fingers crossed), for anyone curious. The key is mentioned here:
\nGist of what you do is:
\nNew Kaggler here… So I built a pipeline to use a radiomics-based approach on the RSNA challenge, specifically using the pyradiomics Python module.
\nIt runs perfectly on my home computer and also in the Kaggle notebook… until I set \"internet off\" mode. Then, when I try to import the pyradiomics package with
\n!pip install pyradiomics
I instead get:
\n\n\nERROR: Could not find a version that satisfies the requirement pyradiomics
\n
and also:
\n\n\nModuleNotFoundError: No module named 'radiomics'
\n
…It appears that PyRadiomics is not part of the environment previously set up by Kaggle, so with internet off it hits a wall trying to download it. So is there any way to pre-load a module that you need, before submitting in \"internet off\" mode?
\nIt would be almost unbelievable if a) my time on this is completely wasted to the point that I cannot even *submit *the work; and b) it is not permitted to use a radiomics package on what is very clearly a prime use case for a radiomics approach!
\nI still have a day and a half to submit something. Any advice appreciated ASAP! (Fortunately the pipeline runs relatively fast.)
", "rawMarkdown": "New Kaggler here... So I built a pipeline to use a radiomics-based approach on the RSNA challenge, specifically using the [pyradiomics ](https://pyradiomics.readthedocs.io/en/latest/) Python module. \n\nIt runs perfectly on my home computer and also in the Kaggle notebook... until I set \"internet off\" mode. Then, when I try to import the pyradiomics package with\n\n`!pip install pyradiomics`\n\nI instead get: \n>ERROR: Could not find a version that satisfies the requirement pyradiomics\n\nand also:\n>ModuleNotFoundError: No module named 'radiomics'\n\n...It appears that PyRadiomics is not part of the environment previously set up by Kaggle, so with internet off it hits a wall trying to download it. So is there any way to pre-load a module that you need, before submitting in \"internet off\" mode?\n\nIt would be almost unbelievable if a) my time on this is completely wasted to the point that I cannot even *submit *the work; and b) it is not permitted to use a radiomics package on what is very clearly a prime use case for a radiomics approach!\n\nI still have a day and a half to submit something. Any advice appreciated ASAP! (Fortunately the pipeline runs relatively fast.)" } ], "comments": [ { "id": 2160434, "author_name": "Antti Isosalo", "author_url": "", "post_date": "2023-02-26T17:11:13.650000", "content": "Did you find a working solution? I am making and copying the wheels on the Kaggle platform (e.g., !pip wheel pyradiomics) into a personal Kaggle dataset and adding them from there. (There might be many whl files created and you could try to see which of the dependencies are already satisfied to make your zip to be uploaded as a dataset smaller.) Here is a good tutorial: https://www.kaggle.com/code/jiny333/creating-whl-files-to-install-external-libraries
It would be interesting to hear about this pyradiomics-based solution if it some day becomes public. 👍
", "votes": 0, "replies": [] }, { "id": 2160658, "author_name": "gsharpminor", "author_url": "", "post_date": "2023-02-26T20:38:57.240000", "content": "Thanks Antti. I might try this \"wheels\" approach. So far i have been trying to just bring in the packages i need from Github, store them in local folder, and then use pip to install them from there instead of the internet. But this leads to a string of dependencies i have to install too… then dependencies for the dependencies… and here we are still 😒
", "votes": 1, "replies": [ { "id": 2160665, "author_name": "gsharpminor", "author_url": "", "post_date": "2023-02-26T20:56:32.327000", "content": "Also when I install dependencies i often find i can't even look in the folders i downloaded. They just hang forever:
\n
Previously, the whole system got sick when i tried to run the first cell that sets up the environment:
\n
Like, who is in charge here? I am basically being kept from even submitting my work by absurd technical issues.
", "votes": 0, "replies": [] }, { "id": 2160668, "author_name": "Antti Isosalo", "author_url": "", "post_date": "2023-02-26T21:06:56.087000", "content": "Let's be in charge today. Fork this https://www.kaggle.com/code/anttiisosalo/create-pyradiomics-wheel/notebook > download zip, upload as private dataset, and add data, and you should be good to go.
", "votes": 1, "replies": [] }, { "id": 2160669, "author_name": "Antti Isosalo", "author_url": "", "post_date": "2023-02-26T21:08:13.577000", "content": "And see here https://www.kaggle.com/code/anttiisosalo/16-bit-post-processing-rsna-bc-detection?scriptVersionId=119182142&cellId=12 how I am accessing private resources.
", "votes": 1, "replies": [] }, { "id": 2160690, "author_name": "gsharpminor", "author_url": "", "post_date": "2023-02-26T22:09:27.720000", "content": "I like your attitude 😀. But i think i have got it working without having to mess with repos or wheels.
\nThe thing i didn't see is the \"persistence\" pull down. So I did the following:
\nWith internet on, do usual !pip install pyradiomics
Set \"persistence\" to \"variables and files\". also set \"environment\" to \"always use latest environment\"
Now turn off internet and submit.
It looks like by doing this, you can tell the Kaggle notebook to simply remember whatever libraries you already had installed and open before the internet switched off. So since i already installed pyradiomics, it remembers to hang on to it.
\nWe will see if it finishes… so far running about an hour with no \"import error\", far longer than it was before… else i will follow your advice…
", "votes": 1, "replies": [] }, { "id": 2160696, "author_name": "", "author_url": "", "post_date": "2023-02-26T22:21:12.250000", "content": "", "votes": 0, "replies": [] }, { "id": 2160785, "author_name": "gsharpminor", "author_url": "", "post_date": "2023-02-27T01:35:57.540000", "content": "Well, I tried just about everything. But the wheels don't work either. I can bring them in as a zip file, but when I try to actually use them to build pyradiomics, I just get
\n\n\n\"ERROR: [packagename].whl is not a supported wheel on this platform.\"
\n
It's really odd that the competition organizers seem to have made it so hard for anyone to use any packages but the ones they already thought of. And it has nothing to do with the competition rules, since I can run this on my own home computer without internet, no problem.
\nMaybe will try looking for exact versions of the wheels to match the Python version Kaggle is using? I'm about out of ideas.
\n**UPDATE: I was downloading wheels for the wrong version of Python (3.8, while Kaggle uses 3.7.12). So now the wheels install fine and pyradiomics is installing + running without internet!
\n(Still think it should be made much simpler for people to use unusual python packages on here.)**
", "votes": 0, "replies": [] } ] } ] }, { "id": 2160328, "author_name": "Kyle Graupe", "author_url": "", "post_date": "2023-02-26T15:29:51.510000", "content": "Yea. In my experience you need \"internet on\" to install anything that's not standard in Kaggle notebooks (numpy, pandas, etc.) I am not familiar with an pre-download methods, but that does not mean it doesn't exist.
", "votes": 0, "replies": [ { "id": 2160344, "author_name": "gsharpminor", "author_url": "", "post_date": "2023-02-26T15:49:25.233000", "content": "I think I may have an idea (fingers crossed), for anyone curious. The key is mentioned here:
\nGist of what you do is:
\n