","metadata":{}},{"cell_type":"code","source":"from IPython.display import HTML\nhtmlCodeHIDE = \"\"\"\n\n\n\n\"\"\"\nHTML(htmlCodeHIDE)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-09-26T08:34:10.121147Z","iopub.execute_input":"2023-09-26T08:34:10.121582Z","iopub.status.idle":"2023-09-26T08:34:10.167067Z","shell.execute_reply.started":"2023-09-26T08:34:10.121527Z","shell.execute_reply":"2023-09-26T08:34:10.166085Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"When you are working for a competition that doesn't allow you to use the internet, you will run in a trouble finding for a method to download the essential libraries for the competition. \nI had to search for few hours to find the one right method that works. \nBelow is the accumulation of my search.
\n***Simply copy paste below code at the bottom cell and run all*** \n```\n!echo 'desired_library_name' > requirements.txt \n!pip download -r requirements.txt -d wheelhouse \n```\n\nIf this method doesn't work because it will download `.tar.gz` file, you can either use `gzip` to unzip it or you can simply use below method to install. \nFirst, perform the usual pip install `!pip install a-library-that-doesn't-work-with-the-method-above`. \nThen find the temporary directory path from the output code, concatenate with the file name. \nI will take `torch-scatter` for example. \nInput code as below: \n```\n!pip install torch-scatter\n```\nThe output is as below: \n```\nCollecting torch-scatter\nUsing cached torch_scatter-2.1.1.tar.gz (107 kB)\nPreparing metadata (setup.py) ... done\nBuilding wheels for collected packages: torch-scatter\nBuilding wheel for torch-scatter (setup.py) ... done\nCreated wheel for torch-scatter: filename=torch_scatter-2.1.1-cp310-cp310-linux_x86_64.whl size=492589 sha256=30bf4c5e645c34cdb201e3be99f59ad9726f51769dbba5bc6b4552e1b44075e7\nStored in directory: /root/.cache/pip/wheels/ef/67/58/6566a3b61c6ec0f2ca0c2c324cd035ef2955601f0fb3197d5f\nSuccessfully built torch-scatter\nInstalling collected packages: torch-scatter\nSuccessfully installed torch-scatter-2.1.1\n```\nFrom above, you will take Stored in directory: `/root/.cache/pip/wheels/ef/67/58/6566a3b61c6ec0f2ca0c2c324cd035ef2955601f0fb3197d5f` *filename=* `torch_scatter-2.1.1-cp310-cp310-linux_x86_64.whl` \n \nThen add `!cp` infront of all these copied location and file name, and add a space ` `, add `/kaggle/working` to specify the output directory. \nAs below : \n\n`!cp /root/.cache/pip/wheels/ef/67/58/6566a3b61c6ec0f2ca0c2c324cd035ef2955601f0fb3197d5f/torch_scatter-2.1.1-cp310-cp310-linux_x86_64.whl /kaggle/working`\n \n#### Save version, nothing more to do here. \n### Go to your working notebook\nPress `Add Data` \nType \"Wheel Downloader\" or whatever name you give to this notebook and add the outputs to your notebook \n \nThen simply copy paste the code below: \n`!pip install -q /kaggle/input/wheel-downloader/wheelhouse/whatever-version-of-your-desired-library.whl`\n \n## Done\n","metadata":{"_kg_hide-input":false,"_kg_hide-output":false}},{"cell_type":"code","source":"from IPython.display import HTML\nhtmlCodeHide = \"\"\"\n\n\n\n\"\"\"\nHTML(htmlCodeHide)","metadata":{"_kg_hide-input":true,"execution":{"iopub.status.busy":"2023-09-26T08:34:10.168997Z","iopub.execute_input":"2023-09-26T08:34:10.169961Z","iopub.status.idle":"2023-09-26T08:34:10.180552Z","shell.execute_reply.started":"2023-09-26T08:34:10.169921Z","shell.execute_reply":"2023-09-26T08:34:10.178962Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"\n인터넷 사용이 허용되지 않는 대부분의 대회에 참가할 때, 작업에 필요한 라이브러리를 다운로드하는 방법을 찾는 데 어려움을 겪게 됩니다. \n저 또한 이 부분에서 어려움을 느꼈고, 제대로 된 방법을 찾기 위해 몇 시간 동안이나 검색해봐야 했습니다. \n정보가 너무 산개되어 있기에 본 노트북에 모두 정리하여 두었으니 캐글 대회를 처음 해보시는 분들께서는 참고하시면 될 것 같습니다! \n***내용을 숙지하신 후 필요하신 라이브러리 추가 후 Run All을 누르시면 바로 사용 가능하실 겁니다!***\n\n먼저, 가장 간단한 방법은 아래의 코드를 복사하셔서 라이브러리 이름만 바꿔서 복붙해주시는 겁니다!\n```\n!echo '다운받으실 라이브러리' > requirements.txt\n!pip download -r requirements.txt -d wheelhouse\n```\n\n`.tar.gz` 파일을 다운로드하기 때문에 위 방법이 작동하지 않는 경우, `gzip`을 활용하여 압축을 풀어서 사용하셔도 되고, 아래 방법을 사용하여 간단히 설치하실 수도 있습니다. \n먼저, 일반적인 pip 설치 방법인 `!pip install a-library-that-does-work-with-the-method-as-a-method`를 수행합니다. \n그런 다음 출력 코드에서 임시 디렉토리 경로를 찾아 파일 이름과 연결합니다.\n`torch-scatter`를 예로 들어보겠습니다. \n아래와 같이 입력하여 `torch-scatter`를 다운받습니다.: \n```\n!pip install torch-scatter\n```\n출력은 아래와 같이 나올겁니다!: \n```\nCollecting torch-scatter\nUsing cached torch_scatter-2.1.1.tar.gz (107 kB)\nPreparing metadata (setup.py) ... done\nBuilding wheels for collected packages: torch-scatter\nBuilding wheel for torch-scatter (setup.py) ... done\nCreated wheel for torch-scatter: filename=torch_scatter-2.1.1-cp310-cp310-linux_x86_64.whl size=492589 sha256=30bf4c5e645c34cdb201e3be99f59ad9726f51769dbba5bc6b4552e1b44075e7\nStored in directory: /root/.cache/pip/wheels/ef/67/58/6566a3b61c6ec0f2ca0c2c324cd035ef2955601f0fb3197d5f\nSuccessfully built torch-scatter\nInstalling collected packages: torch-scatter\nSuccessfully installed torch-scatter-2.1.1\n```\n여기서 \nStored in directory: \n`/root/.cache/pip/wheels/ef/67/58/6566a3b61c6ec0f2ca0c2c324cd035ef2955601f0fb3197d5f` 경로를 먼저 복사하신 후 \n파일 이름인 `torch_scatter-2.1.1-cp310-cp310-linux_x86_64.whl`을 복사하셔서 아래와 같이 만들어줍니다. \n`/root/.cache/pip/wheels/ef/67/58/6566a3b61c6ec0f2ca0c2c324cd035ef2955601f0fb3197d5f/torch_scatter-2.1.1-cp310-cp310-linux_x86_64.whl`
\n그런 후 맨 앞에 `!cp`를 추가 후 한칸 `스페이스`, 맨 뒤에 한 칸 `스페이스` 후 `/kaggle/working/wheelhouse`를 추가하여 주시면 됩니다!
최종적으로는 아래와 같이 나오게 붙여넣어 주시면 됩니다. \n`!cp /root/.cache/pip/wheels/ef/67/58/6566a3b61c6ec0f2ca0c2c324cd035ef2955601f0fb3197d5f/torch_scatter-2.1.1-cp310-cp310-linux_x86_64.whl /kaggle/working`\n\n#### 더이상 여기에서는 하실 게 없습니다. Output도 함께 저장이 되도록 Save Version을 눌러주세요. \n### 작업하실 / 하고 계신 노트북으로 이동해 주세요!\n`Add Data`를 눌러주세요. \n이 노트북의 제목인 \"Wheel Downloader\" 혹은 설정하신 제목을 입력해주세요. \n그 후 `+` 버튼을 눌러 노트북을 `Input`에 추가하여 주세요.\n \n그런 다음 다운 받아두신 라이브러리를 아래와 같은 간단한 명령어로 불러오실 수 있습니다. \n`!pip install -q /kaggle/input/wheel-downloader/wheelhouse/whatever-version-of-your-desired-library.whl`\n \n## 끝! \n내용이 도움이 되셨다면 저어어어어 위에 숫자 옆에 있는 ***Upvote*** (좋아요) 버튼 한번 부탁드리겠습니당ㅎㅎㅎㅎ","metadata":{"_kg_hide-input":false,"_kg_hide-output":false}},{"cell_type":"code","source":"from IPython.display import HTML\nhtmlCodeHIDE = \"\"\"\n\n\n\n\"\"\"\nHTML(htmlCodeHIDE)","metadata":{"_kg_hide-input":true,"_kg_hide-output":false,"execution":{"iopub.status.busy":"2023-09-26T08:34:10.18213Z","iopub.execute_input":"2023-09-26T08:34:10.182854Z","iopub.status.idle":"2023-09-26T08:34:10.205372Z","shell.execute_reply.started":"2023-09-26T08:34:10.182807Z","shell.execute_reply":"2023-09-26T08:34:10.203239Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"当你为一个不允许你使用互联网的比赛工作时，你会在寻找下载比赛必备Library的方法时遇到麻烦。 \n我搜索了几个小时，才找到一个正确有效的方法。\n \n以下是我的搜索结果。
\n***只需将下面的Code复制粘贴到底部Cell并全部运行即可*** \n```\n!echo 'desired_library_name' > requirements.txt \n!pip download -r requirements.txt -d wheelhouse \n```\n\n如果此方法无法正常工作，因为它会下载 `.tar.gz` 文件，你可以使用 `gzip` 解压缩，或者直接使用下面的方法安装。 \n首先，执行常规的`!pip install a-library-that-doesn't-work-with-the-method-above`. \n然后从输出代码中找到临时目录路径，与文件名连接。 \n我以 `torch-scatter` 为例。 \n输入Code如下： \n```\n!pip install torch-scatter\n```\n输出结果如下: \n```\nCollecting torch-scatter\nUsing cached torch_scatter-2.1.1.tar.gz (107 kB)\nPreparing metadata (setup.py) ... done\nBuilding wheels for collected packages: torch-scatter\nBuilding wheel for torch-scatter (setup.py) ... done\nCreated wheel for torch-scatter: filename=torch_scatter-2.1.1-cp310-cp310-linux_x86_64.whl size=492589 sha256=30bf4c5e645c34cdb201e3be99f59ad9726f51769dbba5bc6b4552e1b44075e7\nStored in directory: /root/.cache/pip/wheels/ef/67/58/6566a3b61c6ec0f2ca0c2c324cd035ef2955601f0fb3197d5f\nSuccessfully built torch-scatter\nInstalling collected packages: torch-scatter\nSuccessfully installed torch-scatter-2.1.1\n```\n从上Code中，您将看到 *Stored in directory:* `/root/.cache/pip/wheels/ef/67/58/6566a3b61c6ec0f2ca0c2c324cd035ef2955601f0fb3197d5f` *filename=* `torch_scatter-2.1.1-cp310-cp310-linux_x86_64.whl` \n \n然后在所有这些复制的位置和文件名前添加`!cp`，并添加空格 ` `，添加`/kaggle/working`指定输出 directory。\n \n如下所示： \n\n`!cp /root/.cache/pip/wheels/ef/67/58/6566a3b61c6ec0f2ca0c2c324cd035ef2955601f0fb3197d5f/torch_scatter-2.1.1-cp310-cp310-linux_x86_64.whl /kaggle/working`\n \n#### Save version, 这里无需再做其他操作。 \n### 回到您的工作笔记本\n请按 `Add Data` \n键入 \"Wheel Downloader\" 或您为该Notebook起的任何名称，然后将输出添加到您的工作Notebook中。 \n \n然后只需复制粘贴下面的Code: \n`!pip install -q /kaggle/input/wheel-downloader/wheelhouse/whatever-version-of-your-desired-library.whl`\n \n## 结束\n","metadata":{}},{"cell_type":"markdown","source":"\n\n## Find my other notebooks!\n### Individual Research\n[🀄 Ideogram-based vs. Phonogram-based Language](https://www.kaggle.com/code/jasonheesanglee/ideogram-based-vs-phonogram-based-language) \n-> Research on processing different types of languages.\n \n\n### Competition Notebooks\n***ICR*** -> [EDA on Greeks](https://www.kaggle.com/code/jasonheesanglee/updated-beginner-eda-on-greeks) \n*This was my first in depth notebook which I tried something others didn't try. \nI later found out that there were reasons why other people didn't try.* \n\n***RSNA*** -> [RSNA23 | scale_h implementation](https://www.kaggle.com/code/jasonheesanglee/rsna23-scale-h-implementation) \n*Implemented scale_h and played with numbers based on [Jake S. Brusca's notebook](https://www.kaggle.com/code/jakebrusca/rsna23-weighted-mean-baseline)* \n\n***CommonLit*** \n-> [SpellCheck Tool Comparison](https://www.kaggle.com/code/jasonheesanglee/spellcheck-tool-comparison) \n*In the beginning of this competition, I thought finding the best SpellCheck tool will be crucial to get better score. \nI later found out this [discussion](https://www.kaggle.com/competitions/commonlit-evaluate-student-summaries/discussion/431432), competition host saying spelling and grammar mistakes are not part of the rubric... \nWell... At least I tried.*\n\n-> [MobileBERT Implementation](https://www.kaggle.com/code/jasonheesanglee/en-mobilebert-implementation?scriptVersionId=143760569) \n*In this notebook, I tried to implement MobileBERT to match the purpose of the competition; Making teachers' job easier. \nHowever, I failed for the first attempt, and currently buidling the teacher model on my other working station. \nI will update the notebook with the teacher model once completed*","metadata":{}},{"cell_type":"markdown","source":"# wheelhouse method\nWe can create more folders inside for classification :)","metadata":{}},{"cell_type":"code","source":"###################################################\n####### Unless you know what you are doing ########\n############ do not modify this cell ##############\n############ 뭔가 알고 하시는게 아니라면 ################\n############ 이 부분은 건드리지 말아주세요 ###############\n############# 除非您知道自己在做什么 #################\n############### 请不要改这个 Cell ###################\n###################################################\n\n!mkdir wheelhouse\n# !mkdir wheelhouse/ML\n# !mkdir wheelhouse/DL\n!mkdir wheelhouse/NLP\n# !mkdir wheelhouse/CV","metadata":{"execution":{"iopub.status.busy":"2024-07-02T13:48:46.048218Z","iopub.execute_input":"2024-07-02T13:48:46.04876Z","iopub.status.idle":"2024-07-02T13:48:48.314467Z","shell.execute_reply.started":"2024-07-02T13:48:46.048708Z","shell.execute_reply":"2024-07-02T13:48:48.312878Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !echo 'python_gdcm' > requirements.txt\n# !cat requirements.txt\n# !pip download -r requirements.txt -d wheelhouse/CV","metadata":{"execution":{"iopub.status.busy":"2023-09-26T08:34:15.985917Z","iopub.execute_input":"2023-09-26T08:34:15.986371Z","iopub.status.idle":"2023-09-26T08:34:21.860538Z","shell.execute_reply.started":"2023-09-26T08:34:15.986333Z","shell.execute_reply":"2023-09-26T08:34:21.858937Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !echo 'pylibjpeg' > requirements.txt\n# !pip download -r requirements.txt -d wheelhouse/CV","metadata":{"execution":{"iopub.status.busy":"2023-09-26T08:34:21.862703Z","iopub.execute_input":"2023-09-26T08:34:21.863197Z","iopub.status.idle":"2023-09-26T08:34:26.42405Z","shell.execute_reply.started":"2023-09-26T08:34:21.863149Z","shell.execute_reply":"2023-09-26T08:34:26.423004Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !echo 'symspellpy' > requirements.txt\n# !pip download -r requirements.txt -d wheelhouse/NLP","metadata":{"execution":{"iopub.status.busy":"2023-09-26T08:34:26.428112Z","iopub.execute_input":"2023-09-26T08:34:26.428515Z","iopub.status.idle":"2023-09-26T08:34:50.126517Z","shell.execute_reply.started":"2023-09-26T08:34:26.428475Z","shell.execute_reply":"2023-09-26T08:34:50.125359Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !echo 'pyspellchecker' > requirements.txt\n# !pip download -r requirements.txt -d wheelhouse/NLP","metadata":{"execution":{"iopub.status.busy":"2023-09-26T08:34:50.128041Z","iopub.execute_input":"2023-09-26T08:34:50.128537Z","iopub.status.idle":"2023-09-26T08:34:53.811373Z","shell.execute_reply.started":"2023-09-26T08:34:50.128489Z","shell.execute_reply":"2023-09-26T08:34:53.809592Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !echo 'dicomsdl' > requirements.txt\n# !pip download -r requirements.txt -d wheelhouse/CV","metadata":{"execution":{"iopub.status.busy":"2023-09-26T08:34:53.813632Z","iopub.execute_input":"2023-09-26T08:34:53.814874Z","iopub.status.idle":"2023-09-26T08:34:57.665069Z","shell.execute_reply.started":"2023-09-26T08:34:53.814831Z","shell.execute_reply":"2023-09-26T08:34:57.663374Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"!echo 'immutabledict' > requirements.txt\n!pip download -r requirements.txt -d wheelhouse/NLP\n!echo 'sentencepiece' > requirements.txt\n!pip download -r requirements.txt -d wheelhouse/NLP\n!git clone https://github.com/google/gemma_pytorch.git\n!mkdir /kaggle/working/gemma/\n!mv /kaggle/working/gemma_pytorch/gemma/* /kaggle/working/gemma/\n","metadata":{"execution":{"iopub.status.busy":"2024-07-02T13:51:36.406148Z","iopub.execute_input":"2024-07-02T13:51:36.40677Z","iopub.status.idle":"2024-07-02T13:51:48.159082Z","shell.execute_reply.started":"2024-07-02T13:51:36.406722Z","shell.execute_reply":"2024-07-02T13:51:48.15712Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"# !pip install method\n\n***Note***: You will need to `!pip install ~~~` in a single cell to check the path and the file name. \nThen move on to the next cell and do `!cp`","metadata":{}},{"cell_type":"code","source":"# !pip install torch-geometric\n# !cp /root/.cache/pip/wheels/ac/dc/30/e2874821ff308ee67dcd7a66dbde912411e19e35a1addda028/torch_geometric-2.3.1-py3-none-any.whl /kaggle/working/wheelhouse/DL","metadata":{"execution":{"iopub.status.busy":"2023-09-26T08:35:01.249158Z","iopub.execute_input":"2023-09-26T08:35:01.24958Z","iopub.status.idle":"2023-09-26T08:35:34.981117Z","shell.execute_reply.started":"2023-09-26T08:35:01.249544Z","shell.execute_reply":"2023-09-26T08:35:34.97976Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !pip install torch-scatter\n# !cp /root/.cache/pip/wheels/ef/67/58/6566a3b61c6ec0f2ca0c2c324cd035ef2955601f0fb3197d5f/torch_scatter-2.1.1-cp310-cp310-linux_x86_64.whl /kaggle/working/wheelhouse/DL","metadata":{"execution":{"iopub.status.busy":"2023-09-26T08:35:34.982934Z","iopub.execute_input":"2023-09-26T08:35:34.983429Z","iopub.status.idle":"2023-09-26T08:40:18.55428Z","shell.execute_reply.started":"2023-09-26T08:35:34.98338Z","shell.execute_reply":"2023-09-26T08:40:18.552386Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"# !pip install editdistpy\n# !cp /root/.cache/pip/wheels/88/6a/a6/a1283cc145323a1fb3d475bd158ee60b248ab1985230d266fc/editdistpy-0.1.3-cp310-cp310-linux_x86_64.whl /kaggle/working/wheelhouse/NLP","metadata":{"execution":{"iopub.status.busy":"2023-09-26T08:40:18.556753Z","iopub.execute_input":"2023-09-26T08:40:18.558128Z","iopub.status.idle":"2023-09-26T08:40:57.039248Z","shell.execute_reply.started":"2023-09-26T08:40:18.557965Z","shell.execute_reply":"2023-09-26T08:40:57.037585Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}