{ "id": 109520, "title": "The API is now working for downloads!", "url": "/competitions/rsna-intracranial-hemorrhage-detection/discussion/109520", "author_name": "Phil Culliton", "post_date": "2019-09-19T19:13:20.998000", "votes": 19, "comment_count": 67, "views": 0, "content": "

Plaese make sure to update the API library (to version kaggle-1.5.6, thanks @seshurajup!) with:\npip install --upgrade kaggle

\n\n

It should now download the full ZIP via the command line.

\n\n

Thanks for all of your reports! Let us know if any problems occur.

", "messages": [ { "id": 832649, "postDate": "2020-05-04T10:12:11.727Z", "content": "

```\nThe fix: !pip install --upgrade --force-reinstall --no-deps kaggle

\n\n

```

\n\n

The underlying problem: we install both py2 and py3 packages, and (for historical reasons) the py2 packages are installed second. kaggle is a wrapper installed by the kaggle python package; since we do py2 second, the py2 wrapper is in /usr/local/bin, and happens to be an older version.

\n\n", "rawMarkdown": "\n```\nThe fix: !pip install --upgrade --force-reinstall --no-deps kaggle\n\n```\n\n*The underlying problem: we install both py2 and py3 packages, and (for historical reasons) the py2 packages are installed second. kaggle is a wrapper installed by the kaggle python package; since we do py2 second, the py2 wrapper is in /usr/local/bin, and happens to be an older version.*\n\n\n\n* 1) Go to your kaggle acount and find the dataset you are trying to download\n* 2) in the data tab, you see API command and download all button\n* 3) click download all button, which will prompt you to the rules tab if you have not accepted terms and conditions\n* 4) After you accepted terms, go back to data tab where you can either click download all (download a zip file to your HDD) or copy & paste API command to your Colab env.**", "votes": 22, "replies": [ { "id": 873244, "postDate": "2020-06-04T00:39:46.470Z", "content": "

Thanks work for me!!!

", "rawMarkdown": "Thanks work for me!!!" }, { "id": 963198, "postDate": "2020-08-08T18:38:02.437Z", "content": "

Thanks it worked for me. Regards!

", "rawMarkdown": "Thanks it worked for me. Regards!" }, { "id": 989357, "postDate": "2020-08-28T18:27:44.443Z", "content": "

Can confirm. This method works!

", "rawMarkdown": "Can confirm. This method works!" }, { "id": 1086070, "postDate": "2020-11-21T10:42:28.587Z", "content": "

thanks.. worked

", "rawMarkdown": "thanks.. worked" } ] }, { "id": 650253, "postDate": "2019-10-16T08:15:02.247Z", "content": "

Colab users.\nPlease try the following.

\n\n

\n!pip uninstall -y kaggle\n!pip install --upgrade pip\n!pip install kaggle==1.5.6\n!kaggle -v\n

\n\n

Then\n!kaggle competitions download -c rsna-intracranial-hemorrhage-detection\nto download.

", "rawMarkdown": "Colab users.\nPlease try the following. \n\n```\n!pip uninstall -y kaggle\n!pip install --upgrade pip\n!pip install kaggle==1.5.6\n!kaggle -v\n```\n\nThen\n`!kaggle competitions download -c rsna-intracranial-hemorrhage-detection`\nto download.", "votes": 31, "replies": [ { "id": 650267, "postDate": "2019-10-16T08:32:52.133Z", "content": "

Hi Higepon,

\n\n

I'll try thanks a lot for your help!

\n\n

Cheers,\nSaph

", "rawMarkdown": "Hi Higepon, \n\nI'll try thanks a lot for your help!\n\nCheers,\nSaph", "votes": 1 }, { "id": 650285, "postDate": "2019-10-16T08:41:40.010Z", "content": "

Sure. Let me if it works for you :)

", "rawMarkdown": "Sure. Let me if it works for you :)", "votes": 1 }, { "id": 655336, "postDate": "2019-10-23T00:00:48.997Z", "content": "

Hi @higepon, it works great! Thanks so much , otherwise this time I am really clueless :D\nHowever, after thatm, I faced two dumb problems which I don’t know how to work around — if you have some ideas please let me know :

\n\n

(1) After downloaded zip file. The unzip process makes my Chrome browser out of memory. (too many output?)

\n\n

(2) Even unzip OK (still not the case), SSD of 320 GB is not enough for (a) original zip file (>160GB — somehow bigger than stating in Kaggle) and (b) unzipped files (another 160GB)

\n\n

Do you solve these two issues?

", "rawMarkdown": "Hi @higepon, it works great! Thanks so much , otherwise this time I am really clueless :D\nHowever, after thatm, I faced two dumb problems which I don’t know how to work around — if you have some ideas please let me know :\n\n(1) After downloaded zip file. The unzip process makes my Chrome browser out of memory. (too many output?)\n\n(2) Even unzip OK (still not the case), SSD of 320 GB is not enough for (a) original zip file (>160GB — somehow bigger than stating in Kaggle) and (b) unzipped files (another 160GB)\n\nDo you solve these two issues?", "votes": 1 }, { "id": 655674, "postDate": "2019-10-23T10:56:50.997Z", "content": "

Hi Neuron Engineer. Using Fuze-zip might solve your problem. It makes it possible to access zipped files without having to unzip them. This great tip was mentioned by @guiferviz in this thread. His colab notebook explains how to set up fuze-zip: https://colab.research.google.com/gist/guiferviz/50912a681776d5afe012b1a9259bd637/resize-dataset.ipynb

", "rawMarkdown": "Hi Neuron Engineer. Using Fuze-zip might solve your problem. It makes it possible to access zipped files without having to unzip them. This great tip was mentioned by @guiferviz in [this](https://www.kaggle.com/c/rsna-intracranial-hemorrhage-detection/discussion/109978#latest-655201) thread. His colab notebook explains how to set up fuze-zip: https://colab.research.google.com/gist/guiferviz/50912a681776d5afe012b1a9259bd637/resize-dataset.ipynb", "votes": 1 }, { "id": 655708, "postDate": "2019-10-23T12:04:29.597Z", "content": "

Hi Neuron Engineer,\nFor (1) I think unzip -q may help to reduce output.\nFor (2) I actually suffered from this and I'm switching to GCP.

", "rawMarkdown": "Hi Neuron Engineer,\nFor (1) I think unzip -q may help to reduce output.\nFor (2) I actually suffered from this and I'm switching to GCP.", "votes": 1 }, { "id": 656219, "postDate": "2019-10-24T02:47:03.703Z", "content": "

@vbookshelf @higepon Thanks so much, I appreciate all your great helps!!!

", "rawMarkdown": "@vbookshelf @higepon Thanks so much, I appreciate all your great helps!!!" }, { "id": 659681, "postDate": "2019-10-28T04:35:45.013Z", "content": "

Oh my god thank you you are a lifesaver

", "rawMarkdown": "Oh my god thank you you are a lifesaver" }, { "id": 1086073, "postDate": "2020-11-21T10:42:51.967Z", "content": "

appreciate.. thanks

", "rawMarkdown": "appreciate.. thanks" }, { "id": 1123134, "postDate": "2020-12-23T00:12:19.283Z", "content": "

Hi Higepon,

\n

Your suggestion worked perfectly for me,

\n

I had the latest version of 1.5.10, but I followed your advice verbatim down to 1.5.6,
\nAnd it's downloading fine on Colab

\n

Thanks!!

", "rawMarkdown": "Hi Higepon,\n\nYour suggestion worked perfectly for me,\n\nI had the latest version of 1.5.10, but I followed your advice verbatim down to 1.5.6,\nAnd it's downloading fine on Colab\n\nThanks!!\n" } ] }, { "id": 1124921, "postDate": "2020-12-24T09:16:03.753Z", "content": "

while using in google colab below command worked for me

\n

!pip install --upgrade --force-reinstall --no-deps kaggle

", "rawMarkdown": "while using in google colab below command worked for me\n\n!pip install --upgrade --force-reinstall --no-deps kaggle\n\n", "votes": 7 }, { "id": 630128, "postDate": "2019-09-19T19:13:21Z", "content": "

Plaese make sure to update the API library (to version kaggle-1.5.6, thanks @seshurajup!) with:\npip install --upgrade kaggle

\n\n

It should now download the full ZIP via the command line.

\n\n

Thanks for all of your reports! Let us know if any problems occur.

", "rawMarkdown": "Plaese make sure to update the API library (to version kaggle-1.5.6, thanks @seshurajup!) with:\n`pip install --upgrade kaggle`\n\nIt should now download the full ZIP via the command line.\n\nThanks for all of your reports! Let us know if any problems occur.", "votes": 17 }, { "id": 850291, "postDate": "2020-05-16T13:42:48.303Z", "content": "

I just faced the same problem (kaggle API cannot be updated to 1.5.6). Just use the following command\n!pip install --upgrade --force-reinstall --no-deps kaggle

", "rawMarkdown": "I just faced the same problem (kaggle API cannot be updated to 1.5.6). Just use the following command\n!pip install --upgrade --force-reinstall --no-deps kaggle", "votes": 3 }, { "id": 631461, "postDate": "2019-09-22T04:23:56.493Z", "content": "

Still getting 429 errors

\n\n

```\n$ kaggle --version\n> Kaggle API 1.5.6

\n\n

$ kaggle competitions download -c rsna-intracranial-hemorrhage-detection\n> 429 - Too Many Requests\n```

", "rawMarkdown": "Still getting 429 errors\n\n```\n$ kaggle --version\n> Kaggle API 1.5.6\n\n$ kaggle competitions download -c rsna-intracranial-hemorrhage-detection\n> 429 - Too Many Requests\n```", "votes": 3 }, { "id": 792567, "postDate": "2020-03-31T10:36:43.663Z", "content": "

!kaggle competitions download -c expedia-personalized-sort

\n\n

Output:-

\n\n

403 - Forbidden

\n\n

plz tell to solve it

", "rawMarkdown": "!kaggle competitions download -c expedia-personalized-sort\n\n# Output:-\n403 - Forbidden\n\nplz tell to solve it\n", "votes": 1 }, { "id": 653688, "postDate": "2019-10-20T20:22:02.097Z", "content": "

Can't you just upload the data in a Google cloud bucket so everyone can access it directly without having to download it or do tedious data management in google colab ?

\n\n

Would be great

", "rawMarkdown": "Can't you just upload the data in a Google cloud bucket so everyone can access it directly without having to download it or do tedious data management in google colab ? \n\nWould be great", "votes": 1 }, { "id": 647760, "postDate": "2019-10-13T08:27:23.490Z", "content": "

I'm using VS2019/C#. I have 110 GB of diskspace. Is there any way I can download files on demand? That way I can extract information, convert images and store on my disk? The ten available images are of healthy heads, if I'm correct.

", "rawMarkdown": "I'm using VS2019/C#. I have 110 GB of diskspace. Is there any way I can download files on demand? That way I can extract information, convert images and store on my disk? The ten available images are of healthy heads, if I'm correct.", "votes": 1, "replies": [ { "id": 665024, "postDate": "2019-11-04T14:45:51.917Z", "content": "

you can use @jhoward 's dataset of jpegs images, it's clean and super useful in your case

", "rawMarkdown": "you can use @jhoward 's dataset of jpegs images, it's clean and super useful in your case", "votes": 1 } ] }, { "id": 644305, "postDate": "2019-10-08T15:47:25.867Z", "content": "

yep, update the kaggle-ver to the latest one.\nregards,\nA

", "rawMarkdown": "yep, update the kaggle-ver to the latest one.\nregards,\nA", "votes": 1 }, { "id": 639977, "postDate": "2019-10-03T18:25:23.080Z", "content": "

Download Problems\nIf you are at the other side of the globe, the bandwidth is a bit reduced. So it can be happen that the download takes more than 24 hours for this bunch of data and some provider swap their IP for private connections every 24 hours. In this case the \"kaggle\" downloader is hanging and the the connection is broken. I have not found a way to restart the process at the failed position.

\n\n

Please try to arrange the data in 2 or better 3 zips if possible. This will increase the opportunity to get the data. Thanks a lot.

", "rawMarkdown": "**Download Problems**\nIf you are at the other side of the globe, the bandwidth is a bit reduced. So it can be happen that the download takes more than 24 hours for this bunch of data and some provider swap their IP for private connections every 24 hours. In this case the \"kaggle\" downloader is hanging and the the connection is broken. I have not found a way to restart the process at the failed position.\n\nPlease try to arrange the data in 2 or better 3 zips if possible. This will increase the opportunity to get the data. Thanks a lot. ", "votes": 1 }, { "id": 644611, "postDate": "2019-10-09T04:47:40.043Z", "content": "

Is there a way to download into a google cloud bucket?

", "rawMarkdown": "Is there a way to download into a google cloud bucket?", "votes": 2 }, { "id": 630154, "postDate": "2019-09-19T20:00:25.113Z", "content": "

Its working now with kaggle-1.5.6 version. Thanks @philculliton

\n\n

To check your kaggle version from command line\n\n$kaggle --version\nKaggle API 1.5.6\n

", "rawMarkdown": "Its working now with kaggle-1.5.6 version. Thanks @philculliton \n\nTo check your kaggle version from command line\n```\n$kaggle --version\nKaggle API 1.5.6\n```", "votes": 2 }, { "id": 646382, "postDate": "2019-10-11T07:39:24.730Z", "content": "

Its downloading now. Google Colab users please upgrade pip, uninstall kaggle and then reinstall it with --upgrade option. If you upgrade directly it doesn't upgrades to 1.5.6, it just uses the 1.5.4 version.

", "rawMarkdown": "Its downloading now. Google Colab users please upgrade pip, uninstall kaggle and then reinstall it with --upgrade option. If you upgrade directly it doesn't upgrades to 1.5.6, it just uses the 1.5.4 version.", "replies": [ { "id": 647224, "postDate": "2019-10-12T07:43:56.273Z", "content": "

are you sure colab can download such large datasets?!

", "rawMarkdown": "are you sure colab can download such large datasets?!" }, { "id": 647429, "postDate": "2019-10-12T15:55:37.030Z", "content": "

Yeah bro....Colab can download it. Its just the extraction that it is struggling with. Colab offers around 450 gb space in its notebooks and I think the extracted dataset is well beyond it, with the .zip file included . Anyone with a fix for this please help me out.

", "rawMarkdown": "Yeah bro....Colab can download it. Its just the extraction that it is struggling with. Colab offers around 450 gb space in its notebooks and I think the extracted dataset is well beyond it, with the .zip file included . Anyone with a fix for this please help me out.", "votes": 1 }, { "id": 653220, "postDate": "2019-10-20T04:21:15.407Z", "content": "

I ran into the same problem. I even tried just unzipping the the training set and still went over the allotted space.

", "rawMarkdown": "I ran into the same problem. I even tried just unzipping the the training set and still went over the allotted space. ", "votes": 1 }, { "id": 653690, "postDate": "2019-10-20T20:22:48.640Z", "content": "

Did you manage to fix the issue ?

", "rawMarkdown": "Did you manage to fix the issue ? " } ] }, { "id": 638541, "postDate": "2019-10-02T02:48:31.920Z", "content": "

Just a report. Today, I tried to download the dataset through API but failed 2 times in a row.

\n\n
\n

$ kaggle competitions download -c rsna-intracranial-hemorrhage-detection

\n
\n\n

1st try stopped downloading at 62.0G/156G \n2nd try stopped downloading at 68.7G/156G

\n\n
\n

$ kaggle --version\n Kaggle API 1.5.6

\n
\n\n

This is similar to what was reported in the past.\nNow I think I will just use web API to download as it looks like a way.

", "rawMarkdown": "Just a report. Today, I tried to download the dataset through API but failed 2 times in a row.\n\n> $ kaggle competitions download -c rsna-intracranial-hemorrhage-detection\n\n1st try stopped downloading at 62.0G/156G \n2nd try stopped downloading at 68.7G/156G\n\n> $ kaggle --version\nKaggle API 1.5.6\n\nThis is similar to what was reported in the past.\nNow I think I will just use web API to download as it looks like a way.\n", "votes": -1 }, { "id": 1221565, "postDate": "2021-03-01T05:14:31.533Z", "content": "

I am unable to load this file to my Google Drive using Colab. I am getting a recurring error that says that 146.68 GB out of 147.15 GB have been used then the download stops at 64% of the data set.

\n

I have 2 TB in my Google Drive and have Colabs Pro.

\n

I used this command before trying to install it and confirmed my version is kaggle-1.5.10:
\n!pip install --upgrade --force-reinstall --no-deps kaggle

\n

Any suggestions on how to fix this?

", "rawMarkdown": "I am unable to load this file to my Google Drive using Colab. I am getting a recurring error that says that 146.68 GB out of 147.15 GB have been used then the download stops at 64% of the data set.\n\nI have 2 TB in my Google Drive and have Colabs Pro.\n\nI used this command before trying to install it and confirmed my version is kaggle-1.5.10: \n!pip install --upgrade --force-reinstall --no-deps kaggle\n\nAny suggestions on how to fix this?" }, { "id": 848112, "postDate": "2020-05-14T18:40:42.940Z", "content": "

I have installed kaggle API to download the dataset, though I now see that the dataset is no longer available for download on Kaggle. Does anyone have any leads on another source where the original DICOM dataset may be downloaded?

", "rawMarkdown": "I have installed kaggle API to download the dataset, though I now see that the dataset is no longer available for download on Kaggle. Does anyone have any leads on another source where the original DICOM dataset may be downloaded?" }, { "id": 832652, "postDate": "2020-05-04T10:18:32.630Z", "content": "

I'm working on google colab and I've been through the same problem. but i did two tings .

\n\n

First you have to register your mobile number along with your country code.\nSecond you have to click on last submission on the kaggle dataset page\nThen download kaggle.json file from kaggle.\nAfter that on google colab\n\n!pip install -q kaggle\n!mkdir -p ~/.kaggle\n!cp kaggle.json ~/.kaggle/\n!chmod 600 ~/.kaggle/kaggle.json\n!kaggle competitions download -c web-traffic-time-series-forecasting\n

", "rawMarkdown": "I'm working on google colab and I've been through the same problem. but i did two tings .\n\nFirst you have to register your mobile number along with your country code.\nSecond you have to click on last submission on the kaggle dataset page\nThen download kaggle.json file from kaggle.\nAfter that on google colab\n```\n!pip install -q kaggle\n!mkdir -p ~/.kaggle\n!cp kaggle.json ~/.kaggle/\n!chmod 600 ~/.kaggle/kaggle.json\n!kaggle competitions download -c web-traffic-time-series-forecasting\n```" }, { "id": 665448, "postDate": "2019-11-05T02:47:03.623Z", "content": "

kaggle competitions download -c rsna-intracranial-hemorrhage-detection -f stage_2_test_images returns 404. There is an open thread for this: https://www.kaggle.com/c/rsna-intracranial-hemorrhage-detection/discussion/115726#latest-665432

", "rawMarkdown": "`kaggle competitions download -c rsna-intracranial-hemorrhage-detection -f stage_2_test_images` returns 404. There is an open thread for this: https://www.kaggle.com/c/rsna-intracranial-hemorrhage-detection/discussion/115726#latest-665432\n" }, { "id": 665022, "postDate": "2019-11-04T14:41:51.930Z", "content": "

even if we can download we can't unzip it because it's too big to be unzipped in colab

", "rawMarkdown": "even if we can download we can't unzip it because it's too big to be unzipped in colab", "replies": [ { "id": 665074, "postDate": "2019-11-04T15:52:08.367Z", "content": "

use Fuze-zip instead unzip :) @rashidhaffadi

", "rawMarkdown": "use Fuze-zip instead unzip :) @rashidhaffadi", "votes": 1 }, { "id": 665135, "postDate": "2019-11-04T17:08:03.043Z", "content": "
\n

even if we can download we can't unzip it because it's too big to be unzipped in colab

\n
\n\n

You can still use fuzezip to load the data into your model. You don't have to unzip it. I tried using Colab and for some reason the training is much longer than Kaggle.

", "rawMarkdown": "> even if we can download we can't unzip it because it's too big to be unzipped in colab\n\nYou can still use fuzezip to load the data into your model. You don't have to unzip it. I tried using Colab and for some reason the training is much longer than Kaggle. ", "votes": 1 }, { "id": 665317, "postDate": "2019-11-04T21:46:27.867Z", "content": "

thank you, i never worked with fuzezip can you provide some documentation or tips please. @dskswu , @seshurajup

", "rawMarkdown": "thank you, i never worked with fuzezip can you provide some documentation or tips please. @dskswu , @seshurajup " }, { "id": 665349, "postDate": "2019-11-04T22:53:22.320Z", "content": "

@rashidhaffadi In Colab you can right click the folder to get the path.

", "rawMarkdown": "@rashidhaffadi In Colab you can right click the folder to get the path. ", "votes": 1 }, { "id": 665442, "postDate": "2019-11-05T02:36:48.567Z", "content": "

so, what I'm going to do with the path

", "rawMarkdown": "so, what I'm going to do with the path" }, { "id": 665560, "postDate": "2019-11-05T05:33:21.067Z", "content": "

This is what I did in Colab\n```

\n\n

30-45min in Google Colab.

\n\n

raw_data_dir = \"input/raw\"\n!kaggle competitions download -c rsna-intracranial-hemorrhage-detection -p {raw_data_dir}\n```

\n\n

```\n%%time

\n\n

Around 10 min in Google Colab.

\n\n

input_dir = \"input/raw/kaggle_data/\"

\n\n

!mkdir {input_dir}

\n\n

!fuse-zip input/raw/rsna-intracranial-hemorrhage-detection.zip {input_dir}\n```

\n\n

\nDATA_DIR = 'input/raw/kaggle_data/'\nTEST_IMAGES_DIR = DATA_DIR + 'stage_2_test_images/'\nTRAIN_IMAGES_DIR = DATA_DIR + 'stage_2_train_images/'\n

\n\n

This is the same as of guiferviz's Colab notebook. Except, I switched the folders around.

", "rawMarkdown": "This is what I did in Colab\n```\n# 30-45min in Google Colab.\nraw_data_dir = \"input/raw\"\n!kaggle competitions download -c rsna-intracranial-hemorrhage-detection -p {raw_data_dir}\n```\n\n```\n%%time\n# Around 10 min in Google Colab.\n\ninput_dir = \"input/raw/kaggle_data/\"\n#!mkdir {input_dir}\n!fuse-zip input/raw/rsna-intracranial-hemorrhage-detection.zip {input_dir}\n```\n\n```\nDATA_DIR = 'input/raw/kaggle_data/'\nTEST_IMAGES_DIR = DATA_DIR + 'stage_2_test_images/'\nTRAIN_IMAGES_DIR = DATA_DIR + 'stage_2_train_images/'\n```\n\nThis is the same as of [guiferviz's Colab notebook](https://colab.research.google.com/gist/guiferviz/50912a681776d5afe012b1a9259bd637/resize-dataset.ipynb). Except, I switched the folders around. ", "votes": 1 }, { "id": 666101, "postDate": "2019-11-05T18:19:39.090Z", "content": "

thank you very much

", "rawMarkdown": "thank you very much" } ] }, { "id": 645096, "postDate": "2019-10-09T19:20:52.290Z", "content": "

Hello everyone!\nI'm trying to download the API from colab and not enough files are just a few samples, is anyone getting it?\nNote: I have already done the pip install --upgrade kaggle and all the settings and yet through the api link only some images are downloaded.

", "rawMarkdown": "Hello everyone!\nI'm trying to download the API from colab and not enough files are just a few samples, is anyone getting it?\nNote: I have already done the pip install --upgrade kaggle and all the settings and yet through the api link only some images are downloaded." }, { "id": 644836, "postDate": "2019-10-09T12:12:27.550Z", "content": "

\"\"\nPlease provide a fix for the google colab environment. While upgrading the kaggle api it shows that it has been upgraded to 1.5.6 but while downloading the dataset it still uses the earlier version. Is there any way to fix this. Please forgive me if I sound dumb\n\"\"

", "rawMarkdown": "![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3825518%2F70f3e72cfc2675976c95924ecde6b0ee%2Fupgrading.png?generation=1570624403202481&alt=media)\nPlease provide a fix for the google colab environment. While upgrading the kaggle api it shows that it has been upgraded to 1.5.6 but while downloading the dataset it still uses the earlier version. Is there any way to fix this. Please forgive me if I sound dumb\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3825518%2F8aa07a5a20f5471a8781b1072635e8a0%2Fdownloading.png?generation=1570624536629115&alt=media)\n", "replies": [ { "id": 705776, "postDate": "2019-12-29T13:08:15.650Z", "content": "

I am facing the same problem here. Please can someone help us here

", "rawMarkdown": "I am facing the same problem here. Please can someone help us here " } ] }, { "id": 634498, "postDate": "2019-09-26T11:26:00.587Z", "content": "

Hi Phil,\nIs there a way to download only test data. The API is downloading both the (train + test) data. I have already downloaded train data. So don't want to do that again

", "rawMarkdown": "Hi Phil,\nIs there a way to download **only** test data. The API is downloading both the (train + test) data. I have already downloaded train data. So don't want to do that again", "replies": [ { "id": 634711, "postDate": "2019-09-26T16:47:20.957Z", "content": "

Hi @satti007 - at the moment the fix for the API is restricted to downloading everything. Sorry about that! We're continuing to work on the API, but for the near future this will be the state of things.

", "rawMarkdown": "Hi @satti007 - at the moment the fix for the API is restricted to downloading everything. Sorry about that! We're continuing to work on the API, but for the near future this will be the state of things.", "votes": 2 }, { "id": 635514, "postDate": "2019-09-27T18:23:33.113Z", "content": "

okay.. thanks for the reply.

", "rawMarkdown": "okay.. thanks for the reply. " }, { "id": 643815, "postDate": "2019-10-08T00:46:46.280Z", "content": "

ahhh by mistake I deleted test images from my desktop.. so now I have to download whole 156 gb data again ? Please let me know if there is any way to download only test images ?

", "rawMarkdown": "ahhh by mistake I deleted test images from my desktop.. so now I have to download whole 156 gb data again ? Please let me know if there is any way to download only test images ?" }, { "id": 651125, "postDate": "2019-10-17T04:20:13.093Z", "content": "

I ran into a similar situation as you. Here is what I found and I'm unzipping just the training set.

\n\n

How to unzip specific folder from a .zip with Python

\n\n

```\nimport zipfile

\n\n

archive = zipfile.ZipFile('archive.zip')

\n\n

for file in archive.namelist():\n if file.startswith('foo/'):\n archive.extract(file, 'destination_path')\n```

", "rawMarkdown": "I ran into a similar situation as you. Here is what I found and I'm unzipping just the training set. \n\n[How to unzip specific folder from a .zip with Python](https://stackoverflow.com/questions/13765486/how-to-unzip-specific-folder-from-a-zip-with-python)\n\n```\nimport zipfile\n\narchive = zipfile.ZipFile('archive.zip')\n\nfor file in archive.namelist():\n if file.startswith('foo/'):\n archive.extract(file, 'destination_path')\n```", "votes": 1 } ] }, { "id": 1123133, "postDate": "2020-12-23T00:09:50.887Z", "rawMarkdown": "", "isDeleted": true }, { "id": 659919, "postDate": "2019-10-28T12:44:55.260Z", "rawMarkdown": "", "isDeleted": true }, { "id": 647348, "postDate": "2019-10-12T13:38:30.060Z", "content": "

Can anyone tell how to save the downloaded zip dataset to google drive from google colaboratory?

", "rawMarkdown": "Can anyone tell how to save the downloaded zip dataset to google drive from google colaboratory?", "isDeleted": true, "replies": [ { "id": 832623, "postDate": "2020-05-04T09:51:45.623Z", "content": "

click on save copy to drive option in file menu

", "rawMarkdown": "click on save copy to drive option in file menu" } ] }, { "id": 647228, "postDate": "2019-10-12T07:53:08.053Z", "content": "

The complete dataset is not downloading for me. Google colaboratory showing this warning:

\n\n

Warning: Looks like you're using an outdated API Version, please consider updating (server 1.5.6 / client 1.5.4)

\n\n

How to update this?

\n\n

By the way, the kaggle version is 1.5.6 which I have installed

", "rawMarkdown": "The complete dataset is not downloading for me. Google colaboratory showing this warning: \n\n**Warning: Looks like you're using an outdated API Version, please consider updating (server 1.5.6 / client 1.5.4)**\n\nHow to update this?\n\nBy the way, the kaggle version is 1.5.6 which I have installed", "isDeleted": true }, { "id": 641366, "postDate": "2019-10-04T16:14:33.673Z", "content": "

I tried again with the given API. It's still downloading only 20 train images and 20 test images. And by the way, I am working with Google Colaboratory.

", "rawMarkdown": "I tried again with the given API. It's still downloading only 20 train images and 20 test images. And by the way, I am working with Google Colaboratory.", "votes": 1, "isDeleted": true, "replies": [ { "id": 641417, "postDate": "2019-10-04T17:11:06.527Z", "content": "

Thanks! You're running this command

\n\n
\n

kaggle competitions download -c rsna-intracranial-hemorrhage-detection

\n
\n\n

And it's not downloading a very large zip? Have you updated your API using this command?

\n\n
\n

pip install --upgrade kaggle

\n
\n\n

Please let me know!

", "rawMarkdown": "Thanks! You're running this command\n> kaggle competitions download -c rsna-intracranial-hemorrhage-detection\n\nAnd it's not downloading a very large zip? Have you updated your API using this command?\n\n> pip install --upgrade kaggle\n\nPlease let me know!" }, { "id": 644827, "postDate": "2019-10-09T12:04:24.100Z", "content": "

Yeah. I have updated the kaggle api and its still not downloading the complete dataset. Could you please zip it in a single zip file which we could download and extract?

", "rawMarkdown": "Yeah. I have updated the kaggle api and its still not downloading the complete dataset. Could you please zip it in a single zip file which we could download and extract?" }, { "id": 646907, "postDate": "2019-10-11T21:27:11.100Z", "content": "

Hi Phil,

\n\n

I'm new in python 3, I upgrade the kaggle API, do I have to execute this command \"kaggle competitions download -c rsna-intracranial-hemorrhage-detection\" after ? If yes nothing happened I just get an error \"invalid syntax\", could you help me or anybody else ?

\n\n

Cheers !\nSaphane

", "rawMarkdown": "Hi Phil,\n\nI'm new in python 3, I upgrade the kaggle API, do I have to execute this command \"kaggle competitions download -c rsna-intracranial-hemorrhage-detection\" after ? If yes nothing happened I just get an error \"invalid syntax\", could you help me or anybody else ?\n\nCheers !\nSaphane" }, { "id": 652047, "postDate": "2019-10-18T09:01:26.193Z", "content": "

Hey. You have to use !kaggle competitions download -c rsna-intracranial-hemorrhage-detection. Then it would work.

", "rawMarkdown": "Hey. You have to use !kaggle competitions download -c rsna-intracranial-hemorrhage-detection. Then it would work. ", "votes": 1 } ] }, { "id": 639833, "postDate": "2019-10-03T15:59:56.617Z", "content": "

Only 20 train images and 20 test images are downloading. Can anyone tell how to get the whole thing with API ?

", "rawMarkdown": "Only 20 train images and 20 test images are downloading. Can anyone tell how to get the whole thing with API ?", "isDeleted": true, "replies": [ { "id": 639902, "postDate": "2019-10-03T17:20:42.303Z", "content": "

Use this command to download ( size 156GB )\nkaggle competitions download -c rsna-intracranial-hemorrhage-detection @siddhardhan2317

", "rawMarkdown": "Use this command to download ( **size 156GB** )\n`kaggle competitions download -c rsna-intracranial-hemorrhage-detection` @siddhardhan2317 ", "votes": -1 } ] }, { "id": 630133, "postDate": "2019-09-19T19:26:19.977Z", "rawMarkdown": "", "isDeleted": true }, { "id": 1086068, "postDate": "2020-11-21T10:42:05.243Z", "content": "

Thanks for kaggle API warning and fix

", "rawMarkdown": "Thanks for kaggle API warning and fix" }, { "id": 646817, "postDate": "2019-10-11T18:29:51.717Z", "content": "

Ahhh! :-) Thank you!

", "rawMarkdown": "Ahhh! :-) Thank you!" } ], "comments": [ { "id": 832649, "author_name": "Ramanjaneyulu thanniru", "author_url": "", "post_date": "2020-05-04T10:12:11.727000", "content": "

```\nThe fix: !pip install --upgrade --force-reinstall --no-deps kaggle

\n\n

```

\n\n

The underlying problem: we install both py2 and py3 packages, and (for historical reasons) the py2 packages are installed second. kaggle is a wrapper installed by the kaggle python package; since we do py2 second, the py2 wrapper is in /usr/local/bin, and happens to be an older version.

\n\n", "votes": 22, "replies": [ { "id": 873244, "author_name": "Douglas Magalhães de Oliveira", "author_url": "", "post_date": "2020-06-04T00:39:46.470000", "content": "

Thanks work for me!!!

", "votes": 0, "replies": [] }, { "id": 963198, "author_name": "Rakesh", "author_url": "", "post_date": "2020-08-08T18:38:02.437000", "content": "

Thanks it worked for me. Regards!

", "votes": 0, "replies": [] }, { "id": 989357, "author_name": "Kiryl Ziusko", "author_url": "", "post_date": "2020-08-28T18:27:44.443000", "content": "

Can confirm. This method works!

", "votes": 0, "replies": [] }, { "id": 1086070, "author_name": "VikasKM", "author_url": "", "post_date": "2020-11-21T10:42:28.587000", "content": "

thanks.. worked

", "votes": 0, "replies": [] } ] }, { "id": 650253, "author_name": "higepon", "author_url": "", "post_date": "2019-10-16T08:15:02.247000", "content": "

Colab users.\nPlease try the following.

\n\n

\n!pip uninstall -y kaggle\n!pip install --upgrade pip\n!pip install kaggle==1.5.6\n!kaggle -v\n

\n\n

Then\n!kaggle competitions download -c rsna-intracranial-hemorrhage-detection\nto download.

", "votes": 31, "replies": [ { "id": 650267, "author_name": "Saph", "author_url": "", "post_date": "2019-10-16T08:32:52.133000", "content": "

Hi Higepon,

\n\n

I'll try thanks a lot for your help!

\n\n

Cheers,\nSaph

", "votes": 1, "replies": [] }, { "id": 650285, "author_name": "higepon", "author_url": "", "post_date": "2019-10-16T08:41:40.010000", "content": "

Sure. Let me if it works for you :)

", "votes": 1, "replies": [] }, { "id": 655336, "author_name": "Neuron Engineer", "author_url": "", "post_date": "2019-10-23T00:00:48.997000", "content": "

Hi @higepon, it works great! Thanks so much , otherwise this time I am really clueless :D\nHowever, after thatm, I faced two dumb problems which I don’t know how to work around — if you have some ideas please let me know :

\n\n

(1) After downloaded zip file. The unzip process makes my Chrome browser out of memory. (too many output?)

\n\n

(2) Even unzip OK (still not the case), SSD of 320 GB is not enough for (a) original zip file (>160GB — somehow bigger than stating in Kaggle) and (b) unzipped files (another 160GB)

\n\n

Do you solve these two issues?

", "votes": 1, "replies": [] }, { "id": 655674, "author_name": "vbookshelf", "author_url": "", "post_date": "2019-10-23T10:56:50.997000", "content": "

Hi Neuron Engineer. Using Fuze-zip might solve your problem. It makes it possible to access zipped files without having to unzip them. This great tip was mentioned by @guiferviz in this thread. His colab notebook explains how to set up fuze-zip: https://colab.research.google.com/gist/guiferviz/50912a681776d5afe012b1a9259bd637/resize-dataset.ipynb

", "votes": 1, "replies": [] }, { "id": 655708, "author_name": "higepon", "author_url": "", "post_date": "2019-10-23T12:04:29.597000", "content": "

Hi Neuron Engineer,\nFor (1) I think unzip -q may help to reduce output.\nFor (2) I actually suffered from this and I'm switching to GCP.

", "votes": 1, "replies": [] }, { "id": 656219, "author_name": "Neuron Engineer", "author_url": "", "post_date": "2019-10-24T02:47:03.703000", "content": "

@vbookshelf @higepon Thanks so much, I appreciate all your great helps!!!

", "votes": 0, "replies": [] }, { "id": 659681, "author_name": "madmax0404", "author_url": "", "post_date": "2019-10-28T04:35:45.013000", "content": "

Oh my god thank you you are a lifesaver

", "votes": 0, "replies": [] }, { "id": 1086073, "author_name": "VikasKM", "author_url": "", "post_date": "2020-11-21T10:42:51.967000", "content": "

appreciate.. thanks

", "votes": 0, "replies": [] }, { "id": 1123134, "author_name": "Lawrence Alaso Krukrubo", "author_url": "", "post_date": "2020-12-23T00:12:19.283000", "content": "

Hi Higepon,

\n

Your suggestion worked perfectly for me,

\n

I had the latest version of 1.5.10, but I followed your advice verbatim down to 1.5.6,
\nAnd it's downloading fine on Colab

\n

Thanks!!

", "votes": 0, "replies": [] } ] }, { "id": 1124921, "author_name": "harishml", "author_url": "", "post_date": "2020-12-24T09:16:03.753000", "content": "

while using in google colab below command worked for me

\n

!pip install --upgrade --force-reinstall --no-deps kaggle

", "votes": 7, "replies": [] }, { "id": 850291, "author_name": "Chung-Hsien Tsai", "author_url": "", "post_date": "2020-05-16T13:42:48.303000", "content": "

I just faced the same problem (kaggle API cannot be updated to 1.5.6). Just use the following command\n!pip install --upgrade --force-reinstall --no-deps kaggle

", "votes": 3, "replies": [] }, { "id": 631461, "author_name": "Karl Hornlund", "author_url": "", "post_date": "2019-09-22T04:23:56.493000", "content": "

Still getting 429 errors

\n\n

```\n$ kaggle --version\n> Kaggle API 1.5.6

\n\n

$ kaggle competitions download -c rsna-intracranial-hemorrhage-detection\n> 429 - Too Many Requests\n```

", "votes": 3, "replies": [] }, { "id": 792567, "author_name": "Prasad Desai", "author_url": "", "post_date": "2020-03-31T10:36:43.663000", "content": "

!kaggle competitions download -c expedia-personalized-sort

\n\n

Output:-

\n\n

403 - Forbidden

\n\n

plz tell to solve it

", "votes": 1, "replies": [] }, { "id": 653688, "author_name": "Cyril Equilbec", "author_url": "", "post_date": "2019-10-20T20:22:02.097000", "content": "

Can't you just upload the data in a Google cloud bucket so everyone can access it directly without having to download it or do tedious data management in google colab ?

\n\n

Would be great

", "votes": 1, "replies": [] }, { "id": 647760, "author_name": "Tord Malmgren", "author_url": "", "post_date": "2019-10-13T08:27:23.490000", "content": "

I'm using VS2019/C#. I have 110 GB of diskspace. Is there any way I can download files on demand? That way I can extract information, convert images and store on my disk? The ten available images are of healthy heads, if I'm correct.

", "votes": 1, "replies": [ { "id": 665024, "author_name": "Rashid Haffadi (Not Active)", "author_url": "", "post_date": "2019-11-04T14:45:51.917000", "content": "

you can use @jhoward 's dataset of jpegs images, it's clean and super useful in your case

", "votes": 1, "replies": [] } ] }, { "id": 644305, "author_name": "Adrian Zinovei", "author_url": "", "post_date": "2019-10-08T15:47:25.867000", "content": "

yep, update the kaggle-ver to the latest one.\nregards,\nA

", "votes": 1, "replies": [] }, { "id": 639977, "author_name": "Takeo", "author_url": "", "post_date": "2019-10-03T18:25:23.080000", "content": "

Download Problems\nIf you are at the other side of the globe, the bandwidth is a bit reduced. So it can be happen that the download takes more than 24 hours for this bunch of data and some provider swap their IP for private connections every 24 hours. In this case the \"kaggle\" downloader is hanging and the the connection is broken. I have not found a way to restart the process at the failed position.

\n\n

Please try to arrange the data in 2 or better 3 zips if possible. This will increase the opportunity to get the data. Thanks a lot.

", "votes": 1, "replies": [] }, { "id": 644611, "author_name": "rego-luna", "author_url": "", "post_date": "2019-10-09T04:47:40.043000", "content": "

Is there a way to download into a google cloud bucket?

", "votes": 2, "replies": [] }, { "id": 630154, "author_name": "SeshuRaju 🧘‍♂️", "author_url": "", "post_date": "2019-09-19T20:00:25.113000", "content": "

Its working now with kaggle-1.5.6 version. Thanks @philculliton

\n\n

To check your kaggle version from command line\n\n$kaggle --version\nKaggle API 1.5.6\n

", "votes": 2, "replies": [] }, { "id": 646382, "author_name": "Shubham Kumar Singh", "author_url": "", "post_date": "2019-10-11T07:39:24.730000", "content": "

Its downloading now. Google Colab users please upgrade pip, uninstall kaggle and then reinstall it with --upgrade option. If you upgrade directly it doesn't upgrades to 1.5.6, it just uses the 1.5.4 version.

", "votes": 0, "replies": [ { "id": 647224, "author_name": "pupil3", "author_url": "", "post_date": "2019-10-12T07:43:56.273000", "content": "

are you sure colab can download such large datasets?!

", "votes": 0, "replies": [] }, { "id": 647429, "author_name": "Shubham Kumar Singh", "author_url": "", "post_date": "2019-10-12T15:55:37.030000", "content": "

Yeah bro....Colab can download it. Its just the extraction that it is struggling with. Colab offers around 450 gb space in its notebooks and I think the extracted dataset is well beyond it, with the .zip file included . Anyone with a fix for this please help me out.

", "votes": 1, "replies": [] }, { "id": 653220, "author_name": "William Green", "author_url": "", "post_date": "2019-10-20T04:21:15.407000", "content": "

I ran into the same problem. I even tried just unzipping the the training set and still went over the allotted space.

", "votes": 1, "replies": [] }, { "id": 653690, "author_name": "Cyril Equilbec", "author_url": "", "post_date": "2019-10-20T20:22:48.640000", "content": "

Did you manage to fix the issue ?

", "votes": 0, "replies": [] } ] }, { "id": 638541, "author_name": "Appian", "author_url": "", "post_date": "2019-10-02T02:48:31.920000", "content": "

Just a report. Today, I tried to download the dataset through API but failed 2 times in a row.

\n\n
\n

$ kaggle competitions download -c rsna-intracranial-hemorrhage-detection

\n
\n\n

1st try stopped downloading at 62.0G/156G \n2nd try stopped downloading at 68.7G/156G

\n\n
\n

$ kaggle --version\n Kaggle API 1.5.6

\n
\n\n

This is similar to what was reported in the past.\nNow I think I will just use web API to download as it looks like a way.

", "votes": -1, "replies": [] }, { "id": 1221565, "author_name": "DataScienceMD", "author_url": "", "post_date": "2021-03-01T05:14:31.533000", "content": "

I am unable to load this file to my Google Drive using Colab. I am getting a recurring error that says that 146.68 GB out of 147.15 GB have been used then the download stops at 64% of the data set.

\n

I have 2 TB in my Google Drive and have Colabs Pro.

\n

I used this command before trying to install it and confirmed my version is kaggle-1.5.10:
\n!pip install --upgrade --force-reinstall --no-deps kaggle

\n

Any suggestions on how to fix this?

", "votes": 0, "replies": [] }, { "id": 848112, "author_name": "john comer", "author_url": "", "post_date": "2020-05-14T18:40:42.940000", "content": "

I have installed kaggle API to download the dataset, though I now see that the dataset is no longer available for download on Kaggle. Does anyone have any leads on another source where the original DICOM dataset may be downloaded?

", "votes": 0, "replies": [] }, { "id": 832652, "author_name": "Ramanjaneyulu thanniru", "author_url": "", "post_date": "2020-05-04T10:18:32.630000", "content": "

I'm working on google colab and I've been through the same problem. but i did two tings .

\n\n

First you have to register your mobile number along with your country code.\nSecond you have to click on last submission on the kaggle dataset page\nThen download kaggle.json file from kaggle.\nAfter that on google colab\n\n!pip install -q kaggle\n!mkdir -p ~/.kaggle\n!cp kaggle.json ~/.kaggle/\n!chmod 600 ~/.kaggle/kaggle.json\n!kaggle competitions download -c web-traffic-time-series-forecasting\n

", "votes": 0, "replies": [] }, { "id": 665448, "author_name": "Kerem Turgutlu", "author_url": "", "post_date": "2019-11-05T02:47:03.623000", "content": "

kaggle competitions download -c rsna-intracranial-hemorrhage-detection -f stage_2_test_images returns 404. There is an open thread for this: https://www.kaggle.com/c/rsna-intracranial-hemorrhage-detection/discussion/115726#latest-665432

", "votes": 0, "replies": [] }, { "id": 665022, "author_name": "Rashid Haffadi (Not Active)", "author_url": "", "post_date": "2019-11-04T14:41:51.930000", "content": "

even if we can download we can't unzip it because it's too big to be unzipped in colab

", "votes": 0, "replies": [ { "id": 665074, "author_name": "SeshuRaju 🧘‍♂️", "author_url": "", "post_date": "2019-11-04T15:52:08.367000", "content": "

use Fuze-zip instead unzip :) @rashidhaffadi

", "votes": 1, "replies": [] }, { "id": 665135, "author_name": "William Green", "author_url": "", "post_date": "2019-11-04T17:08:03.043000", "content": "
\n

even if we can download we can't unzip it because it's too big to be unzipped in colab

\n
\n\n

You can still use fuzezip to load the data into your model. You don't have to unzip it. I tried using Colab and for some reason the training is much longer than Kaggle.

", "votes": 1, "replies": [] }, { "id": 665317, "author_name": "Rashid Haffadi (Not Active)", "author_url": "", "post_date": "2019-11-04T21:46:27.867000", "content": "

thank you, i never worked with fuzezip can you provide some documentation or tips please. @dskswu , @seshurajup

", "votes": 0, "replies": [] }, { "id": 665349, "author_name": "William Green", "author_url": "", "post_date": "2019-11-04T22:53:22.320000", "content": "

@rashidhaffadi In Colab you can right click the folder to get the path.

", "votes": 1, "replies": [] }, { "id": 665442, "author_name": "Rashid Haffadi (Not Active)", "author_url": "", "post_date": "2019-11-05T02:36:48.567000", "content": "

so, what I'm going to do with the path

", "votes": 0, "replies": [] }, { "id": 665560, "author_name": "William Green", "author_url": "", "post_date": "2019-11-05T05:33:21.067000", "content": "

This is what I did in Colab\n```

\n\n

30-45min in Google Colab.

\n\n

raw_data_dir = \"input/raw\"\n!kaggle competitions download -c rsna-intracranial-hemorrhage-detection -p {raw_data_dir}\n```

\n\n

```\n%%time

\n\n

Around 10 min in Google Colab.

\n\n

input_dir = \"input/raw/kaggle_data/\"

\n\n

!mkdir {input_dir}

\n\n

!fuse-zip input/raw/rsna-intracranial-hemorrhage-detection.zip {input_dir}\n```

\n\n

\nDATA_DIR = 'input/raw/kaggle_data/'\nTEST_IMAGES_DIR = DATA_DIR + 'stage_2_test_images/'\nTRAIN_IMAGES_DIR = DATA_DIR + 'stage_2_train_images/'\n

\n\n

This is the same as of guiferviz's Colab notebook. Except, I switched the folders around.

", "votes": 1, "replies": [] }, { "id": 666101, "author_name": "Rashid Haffadi (Not Active)", "author_url": "", "post_date": "2019-11-05T18:19:39.090000", "content": "

thank you very much

", "votes": 0, "replies": [] } ] }, { "id": 645096, "author_name": "Emerson CM Correa", "author_url": "", "post_date": "2019-10-09T19:20:52.290000", "content": "

Hello everyone!\nI'm trying to download the API from colab and not enough files are just a few samples, is anyone getting it?\nNote: I have already done the pip install --upgrade kaggle and all the settings and yet through the api link only some images are downloaded.

", "votes": 0, "replies": [] }, { "id": 644836, "author_name": "Shubham Kumar Singh", "author_url": "", "post_date": "2019-10-09T12:12:27.550000", "content": "

\"\"\nPlease provide a fix for the google colab environment. While upgrading the kaggle api it shows that it has been upgraded to 1.5.6 but while downloading the dataset it still uses the earlier version. Is there any way to fix this. Please forgive me if I sound dumb\n\"\"

", "votes": 0, "replies": [ { "id": 705776, "author_name": "Madu Victor ", "author_url": "", "post_date": "2019-12-29T13:08:15.650000", "content": "

I am facing the same problem here. Please can someone help us here

", "votes": 0, "replies": [] } ] }, { "id": 634498, "author_name": "Satish Golla", "author_url": "", "post_date": "2019-09-26T11:26:00.587000", "content": "

Hi Phil,\nIs there a way to download only test data. The API is downloading both the (train + test) data. I have already downloaded train data. So don't want to do that again

", "votes": 0, "replies": [ { "id": 634711, "author_name": "Phil Culliton", "author_url": "", "post_date": "2019-09-26T16:47:20.957000", "content": "

Hi @satti007 - at the moment the fix for the API is restricted to downloading everything. Sorry about that! We're continuing to work on the API, but for the near future this will be the state of things.

", "votes": 2, "replies": [] }, { "id": 635514, "author_name": "Satish Golla", "author_url": "", "post_date": "2019-09-27T18:23:33.113000", "content": "

okay.. thanks for the reply.

", "votes": 0, "replies": [] }, { "id": 643815, "author_name": "Mayank Pathak", "author_url": "", "post_date": "2019-10-08T00:46:46.280000", "content": "

ahhh by mistake I deleted test images from my desktop.. so now I have to download whole 156 gb data again ? Please let me know if there is any way to download only test images ?

", "votes": 0, "replies": [] }, { "id": 651125, "author_name": "William Green", "author_url": "", "post_date": "2019-10-17T04:20:13.093000", "content": "

I ran into a similar situation as you. Here is what I found and I'm unzipping just the training set.

\n\n

How to unzip specific folder from a .zip with Python

\n\n

```\nimport zipfile

\n\n

archive = zipfile.ZipFile('archive.zip')

\n\n

for file in archive.namelist():\n if file.startswith('foo/'):\n archive.extract(file, 'destination_path')\n```

", "votes": 1, "replies": [] } ] }, { "id": 1123133, "author_name": "", "author_url": "", "post_date": "2020-12-23T00:09:50.887000", "content": "", "votes": 0, "replies": [] }, { "id": 659919, "author_name": "", "author_url": "", "post_date": "2019-10-28T12:44:55.260000", "content": "", "votes": 0, "replies": [] }, { "id": 647348, "author_name": "", "author_url": "", "post_date": "2019-10-12T13:38:30.060000", "content": "

Can anyone tell how to save the downloaded zip dataset to google drive from google colaboratory?

", "votes": 0, "replies": [ { "id": 832623, "author_name": "Ramanjaneyulu thanniru", "author_url": "", "post_date": "2020-05-04T09:51:45.623000", "content": "

click on save copy to drive option in file menu

", "votes": 0, "replies": [] } ] }, { "id": 647228, "author_name": "", "author_url": "", "post_date": "2019-10-12T07:53:08.053000", "content": "

The complete dataset is not downloading for me. Google colaboratory showing this warning:

\n\n

Warning: Looks like you're using an outdated API Version, please consider updating (server 1.5.6 / client 1.5.4)

\n\n

How to update this?

\n\n

By the way, the kaggle version is 1.5.6 which I have installed

", "votes": 0, "replies": [] }, { "id": 641366, "author_name": "", "author_url": "", "post_date": "2019-10-04T16:14:33.673000", "content": "

I tried again with the given API. It's still downloading only 20 train images and 20 test images. And by the way, I am working with Google Colaboratory.

", "votes": 1, "replies": [ { "id": 641417, "author_name": "Phil Culliton", "author_url": "", "post_date": "2019-10-04T17:11:06.527000", "content": "

Thanks! You're running this command

\n\n
\n

kaggle competitions download -c rsna-intracranial-hemorrhage-detection

\n
\n\n

And it's not downloading a very large zip? Have you updated your API using this command?

\n\n
\n

pip install --upgrade kaggle

\n
\n\n

Please let me know!

", "votes": 0, "replies": [] }, { "id": 644827, "author_name": "Shubham Kumar Singh", "author_url": "", "post_date": "2019-10-09T12:04:24.100000", "content": "

Yeah. I have updated the kaggle api and its still not downloading the complete dataset. Could you please zip it in a single zip file which we could download and extract?

", "votes": 0, "replies": [] }, { "id": 646907, "author_name": "Saph", "author_url": "", "post_date": "2019-10-11T21:27:11.100000", "content": "

Hi Phil,

\n\n

I'm new in python 3, I upgrade the kaggle API, do I have to execute this command \"kaggle competitions download -c rsna-intracranial-hemorrhage-detection\" after ? If yes nothing happened I just get an error \"invalid syntax\", could you help me or anybody else ?

\n\n

Cheers !\nSaphane

", "votes": 0, "replies": [] }, { "id": 652047, "author_name": "Manoj Prabhakar", "author_url": "", "post_date": "2019-10-18T09:01:26.193000", "content": "

Hey. You have to use !kaggle competitions download -c rsna-intracranial-hemorrhage-detection. Then it would work.

", "votes": 1, "replies": [] } ] }, { "id": 639833, "author_name": "", "author_url": "", "post_date": "2019-10-03T15:59:56.617000", "content": "

Only 20 train images and 20 test images are downloading. Can anyone tell how to get the whole thing with API ?

", "votes": 0, "replies": [ { "id": 639902, "author_name": "SeshuRaju 🧘‍♂️", "author_url": "", "post_date": "2019-10-03T17:20:42.303000", "content": "

Use this command to download ( size 156GB )\nkaggle competitions download -c rsna-intracranial-hemorrhage-detection @siddhardhan2317

", "votes": -1, "replies": [] } ] }, { "id": 630133, "author_name": "", "author_url": "", "post_date": "2019-09-19T19:26:19.977000", "content": "", "votes": 0, "replies": [] }, { "id": 1086068, "author_name": "", "author_url": "", "post_date": "2020-11-21T10:42:05.243000", "content": "", "votes": 0, "replies": [] }, { "id": 646817, "author_name": "", "author_url": "", "post_date": "2019-10-11T18:29:51.717000", "content": "", "votes": 0, "replies": [] } ], "raw_markdown_by_id": { "832649": "\n```\nThe fix: !pip install --upgrade --force-reinstall --no-deps kaggle\n\n```\n\n*The underlying problem: we install both py2 and py3 packages, and (for historical reasons) the py2 packages are installed second. kaggle is a wrapper installed by the kaggle python package; since we do py2 second, the py2 wrapper is in /usr/local/bin, and happens to be an older version.*\n\n\n\n* 1) Go to your kaggle acount and find the dataset you are trying to download\n* 2) in the data tab, you see API command and download all button\n* 3) click download all button, which will prompt you to the rules tab if you have not accepted terms and conditions\n* 4) After you accepted terms, go back to data tab where you can either click download all (download a zip file to your HDD) or copy & paste API command to your Colab env.**", "650253": "Colab users.\nPlease try the following. \n\n```\n!pip uninstall -y kaggle\n!pip install --upgrade pip\n!pip install kaggle==1.5.6\n!kaggle -v\n```\n\nThen\n`!kaggle competitions download -c rsna-intracranial-hemorrhage-detection`\nto download.", "1124921": "while using in google colab below command worked for me\n\n!pip install --upgrade --force-reinstall --no-deps kaggle\n\n", "630128": "Plaese make sure to update the API library (to version kaggle-1.5.6, thanks @seshurajup!) with:\n`pip install --upgrade kaggle`\n\nIt should now download the full ZIP via the command line.\n\nThanks for all of your reports! Let us know if any problems occur.", "850291": "I just faced the same problem (kaggle API cannot be updated to 1.5.6). Just use the following command\n!pip install --upgrade --force-reinstall --no-deps kaggle", "631461": "Still getting 429 errors\n\n```\n$ kaggle --version\n> Kaggle API 1.5.6\n\n$ kaggle competitions download -c rsna-intracranial-hemorrhage-detection\n> 429 - Too Many Requests\n```", "792567": "!kaggle competitions download -c expedia-personalized-sort\n\n# Output:-\n403 - Forbidden\n\nplz tell to solve it\n", "653688": "Can't you just upload the data in a Google cloud bucket so everyone can access it directly without having to download it or do tedious data management in google colab ? \n\nWould be great", "647760": "I'm using VS2019/C#. I have 110 GB of diskspace. Is there any way I can download files on demand? That way I can extract information, convert images and store on my disk? The ten available images are of healthy heads, if I'm correct.", "644305": "yep, update the kaggle-ver to the latest one.\nregards,\nA", "639977": "**Download Problems**\nIf you are at the other side of the globe, the bandwidth is a bit reduced. So it can be happen that the download takes more than 24 hours for this bunch of data and some provider swap their IP for private connections every 24 hours. In this case the \"kaggle\" downloader is hanging and the the connection is broken. I have not found a way to restart the process at the failed position.\n\nPlease try to arrange the data in 2 or better 3 zips if possible. This will increase the opportunity to get the data. Thanks a lot. ", "644611": "Is there a way to download into a google cloud bucket?", "630154": "Its working now with kaggle-1.5.6 version. Thanks @philculliton \n\nTo check your kaggle version from command line\n```\n$kaggle --version\nKaggle API 1.5.6\n```", "646382": "Its downloading now. Google Colab users please upgrade pip, uninstall kaggle and then reinstall it with --upgrade option. If you upgrade directly it doesn't upgrades to 1.5.6, it just uses the 1.5.4 version.", "638541": "Just a report. Today, I tried to download the dataset through API but failed 2 times in a row.\n\n> $ kaggle competitions download -c rsna-intracranial-hemorrhage-detection\n\n1st try stopped downloading at 62.0G/156G \n2nd try stopped downloading at 68.7G/156G\n\n> $ kaggle --version\nKaggle API 1.5.6\n\nThis is similar to what was reported in the past.\nNow I think I will just use web API to download as it looks like a way.\n", "1221565": "I am unable to load this file to my Google Drive using Colab. I am getting a recurring error that says that 146.68 GB out of 147.15 GB have been used then the download stops at 64% of the data set.\n\nI have 2 TB in my Google Drive and have Colabs Pro.\n\nI used this command before trying to install it and confirmed my version is kaggle-1.5.10: \n!pip install --upgrade --force-reinstall --no-deps kaggle\n\nAny suggestions on how to fix this?", "848112": "I have installed kaggle API to download the dataset, though I now see that the dataset is no longer available for download on Kaggle. Does anyone have any leads on another source where the original DICOM dataset may be downloaded?", "832652": "I'm working on google colab and I've been through the same problem. but i did two tings .\n\nFirst you have to register your mobile number along with your country code.\nSecond you have to click on last submission on the kaggle dataset page\nThen download kaggle.json file from kaggle.\nAfter that on google colab\n```\n!pip install -q kaggle\n!mkdir -p ~/.kaggle\n!cp kaggle.json ~/.kaggle/\n!chmod 600 ~/.kaggle/kaggle.json\n!kaggle competitions download -c web-traffic-time-series-forecasting\n```", "665448": "`kaggle competitions download -c rsna-intracranial-hemorrhage-detection -f stage_2_test_images` returns 404. There is an open thread for this: https://www.kaggle.com/c/rsna-intracranial-hemorrhage-detection/discussion/115726#latest-665432\n", "665022": "even if we can download we can't unzip it because it's too big to be unzipped in colab", "645096": "Hello everyone!\nI'm trying to download the API from colab and not enough files are just a few samples, is anyone getting it?\nNote: I have already done the pip install --upgrade kaggle and all the settings and yet through the api link only some images are downloaded.", "644836": "![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3825518%2F70f3e72cfc2675976c95924ecde6b0ee%2Fupgrading.png?generation=1570624403202481&alt=media)\nPlease provide a fix for the google colab environment. While upgrading the kaggle api it shows that it has been upgraded to 1.5.6 but while downloading the dataset it still uses the earlier version. Is there any way to fix this. Please forgive me if I sound dumb\n![](https://www.googleapis.com/download/storage/v1/b/kaggle-user-content/o/inbox%2F3825518%2F8aa07a5a20f5471a8781b1072635e8a0%2Fdownloading.png?generation=1570624536629115&alt=media)\n", "634498": "Hi Phil,\nIs there a way to download **only** test data. The API is downloading both the (train + test) data. I have already downloaded train data. So don't want to do that again", "1123133": "", "659919": "", "647348": "Can anyone tell how to save the downloaded zip dataset to google drive from google colaboratory?", "647228": "The complete dataset is not downloading for me. Google colaboratory showing this warning: \n\n**Warning: Looks like you're using an outdated API Version, please consider updating (server 1.5.6 / client 1.5.4)**\n\nHow to update this?\n\nBy the way, the kaggle version is 1.5.6 which I have installed", "641366": "I tried again with the given API. It's still downloading only 20 train images and 20 test images. And by the way, I am working with Google Colaboratory.", "639833": "Only 20 train images and 20 test images are downloading. Can anyone tell how to get the whole thing with API ?", "630133": "", "1086068": "Thanks for kaggle API warning and fix", "646817": "Ahhh! :-) Thank you!" } }