{
  "id": 445786,
  "title": "How to download all the data in a reasonable amount of time?",
  "url": "/competitions/UBC-OCEAN/discussion/445786",
  "author_name": "hbhutta",
  "post_date": "2023-10-08T23:42:49.326000",
  "votes": 4,
  "comment_count": 6,
  "views": 0,
  "content": "<p>It took me 6 hours just to fully download the zip file. And when i'm unzipping it I get a disk full error. Should I just use a Kaggle notebook?</p>",
  "messages": [
    {
      "id": 2474086,
      "postDate": "2023-10-08T23:42:49.327Z",
      "content": "<p>It took me 6 hours just to fully download the zip file. And when i'm unzipping it I get a disk full error. Should I just use a Kaggle notebook?</p>",
      "rawMarkdown": "It took me 6 hours just to fully download the zip file. And when i'm unzipping it I get a disk full error. Should I just use a Kaggle notebook?",
      "votes": 4
    },
    {
      "id": 2475174,
      "postDate": "2023-10-09T17:22:24.113Z",
      "content": "<p>use the output of the <a href=\"https://www.kaggle.com/code/khailashsanthakumar/only-small-images-training-dataset-for-download\" target=\"_blank\">notebook</a> and use the way i create filepaths to use it for training</p>",
      "rawMarkdown": "use the output of the [notebook](https://www.kaggle.com/code/khailashsanthakumar/only-small-images-training-dataset-for-download) and use the way i create filepaths to use it for training",
      "votes": 1
    },
    {
      "id": 2476213,
      "postDate": "2023-10-10T12:39:24.067Z",
      "content": "<p>Kaggle API is all you need</p>",
      "rawMarkdown": "Kaggle API is all you need",
      "votes": 2,
      "replies": [
        {
          "id": 2482978,
          "postDate": "2023-10-15T12:01:38.193Z",
          "content": "<p>But downloading the whole Dataset is taking long time <br>\n<strong>! kaggle competitions download -c UBC-OCEAN</strong><br>\nAny other alternative ?</p>",
          "rawMarkdown": "But downloading the whole Dataset is taking long time \n**! kaggle competitions download -c UBC-OCEAN**\nAny other alternative ?"
        }
      ]
    },
    {
      "id": 2565808,
      "postDate": "2023-12-18T09:46:59.683Z",
      "content": "<p>You can use api in terminal</p>",
      "rawMarkdown": "You can use api in terminal"
    },
    {
      "id": 2482983,
      "postDate": "2023-10-15T12:03:17.947Z",
      "content": "<p>Download the whole dataset in Google Colab takes time .</p>\n<ol>\n<li>Either one has to download all the data in google drive and keep referring .</li>\n<li>Each me during training download the data in Google Colab . (This requires better internet connection )</li>\n</ol>",
      "rawMarkdown": "Download the whole dataset in Google Colab takes time .\n1. Either one has to download all the data in google drive and keep referring .\n2. Each me during training download the data in Google Colab . (This requires better internet connection )\n"
    },
    {
      "id": 2482577,
      "postDate": "2023-10-15T06:40:04.533Z",
      "content": "<p>You can do jpeg compression on Kaggle and export it as a dataset.</p>",
      "rawMarkdown": "You can do jpeg compression on Kaggle and export it as a dataset."
    }
  ],
  "comments": [
    {
      "id": 2475174,
      "author_name": "khailash santhakumar",
      "author_url": "",
      "post_date": "2023-10-09T17:22:24.113000",
      "content": "<p>use the output of the <a href=\"https://www.kaggle.com/code/khailashsanthakumar/only-small-images-training-dataset-for-download\" target=\"_blank\">notebook</a> and use the way i create filepaths to use it for training</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 2476213,
      "author_name": "Seeing Times",
      "author_url": "",
      "post_date": "2023-10-10T12:39:24.067000",
      "content": "<p>Kaggle API is all you need</p>",
      "votes": 2,
      "replies": [
        {
          "id": 2482978,
          "author_name": "INDRAJIT SWAIN",
          "author_url": "",
          "post_date": "2023-10-15T12:01:38.193000",
          "content": "<p>But downloading the whole Dataset is taking long time <br>\n<strong>! kaggle competitions download -c UBC-OCEAN</strong><br>\nAny other alternative ?</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 2565808,
      "author_name": "dackyy",
      "author_url": "",
      "post_date": "2023-12-18T09:46:59.683000",
      "content": "<p>You can use api in terminal</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2482983,
      "author_name": "INDRAJIT SWAIN",
      "author_url": "",
      "post_date": "2023-10-15T12:03:17.947000",
      "content": "<p>Download the whole dataset in Google Colab takes time .</p>\n<ol>\n<li>Either one has to download all the data in google drive and keep referring .</li>\n<li>Each me during training download the data in Google Colab . (This requires better internet connection )</li>\n</ol>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2482577,
      "author_name": "Gunes Evitan",
      "author_url": "",
      "post_date": "2023-10-15T06:40:04.533000",
      "content": "<p>You can do jpeg compression on Kaggle and export it as a dataset.</p>",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2474086": "It took me 6 hours just to fully download the zip file. And when i'm unzipping it I get a disk full error. Should I just use a Kaggle notebook?",
    "2475174": "use the output of the [notebook](https://www.kaggle.com/code/khailashsanthakumar/only-small-images-training-dataset-for-download) and use the way i create filepaths to use it for training",
    "2476213": "Kaggle API is all you need",
    "2565808": "You can use api in terminal",
    "2482983": "Download the whole dataset in Google Colab takes time .\n1. Either one has to download all the data in google drive and keep referring .\n2. Each me during training download the data in Google Colab . (This requires better internet connection )\n",
    "2482577": "You can do jpeg compression on Kaggle and export it as a dataset."
  }
}