{"cells":[{"metadata":{},"cell_type":"markdown","source":"The datasets are too big in many kaggle competitions. For example, the RSNA intracranial hemorrhage detection dataset is 180G. Sometimes I just want to download a very small subset of the original dataset so I can play with it in my local computer. Below is an example of how I achieve this goal. The idea is to create a Kaggle kernel to copy some data files into an output folder, then zip the folder and download the generated .zip file from Kaggle kernel interface."},{"metadata":{"_uuid":"8f2839f25d086af736a60e9eeb907d3b93b6e0e5","_cell_guid":"b1076dfc-b9ad-4769-8c92-a6c4dae69d19","trusted":true},"cell_type":"code","source":"import os\nimport random\nfrom shutil import copy, make_archive\n\n\ndata_root = '/kaggle/input/rsna-intracranial-hemorrhage-detection/rsna-intracranial-hemorrhage-detection'\nk = 100 # randomly select 100 images from both train and test data folder\nos.makedirs('./dataset', exist_ok=True) # create a dataset folder to hold all the files that I wanted to download\ncopy(os.path.join(data_root, 'stage_2_sample_submission.csv'), 'dataset/stage_2_sample_submission.csv')\ncopy(os.path.join(data_root, 'stage_2_train.csv'), 'dataset/stage_2_train.csv')\nfor d in ['stage_2_train', 'stage_2_test']:\n    # list all images in train/test folder\n    dir_path = os.path.join(data_root, d)\n    files = os.listdir(dir_path)\n    \n    # copy images to target folder\n    target_dir = os.path.join('dataset', d)\n    os.makedirs(target_dir, exist_ok=True) \n    for f in random.choices(files, k=k): # randomly select k images and copy them to the target folder\n        src_file = os.path.join(dir_path, f)\n        copy(src_file, target_dir)\n        \n# zip generated files\nmake_archive(base_name='download_dataset', format='zip', root_dir='dataset')\n","execution_count":null,"outputs":[]},{"metadata":{},"cell_type":"markdown","source":"Now you can download the `download_dataset.zip` from the kernel interface.\n\n![](https://user-images.githubusercontent.com/1262709/69744216-c2e11780-110d-11ea-82b4-88006cc6d0aa.png)"},{"metadata":{"trusted":true},"cell_type":"code","source":"","execution_count":null,"outputs":[]}],"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat":4,"nbformat_minor":1}