{
  "id": 456188,
  "title": "What is the storage capacity limit when writing to directories outside /kaggle/working?",
  "url": "/competitions/UBC-OCEAN/discussion/456188",
  "author_name": "Aurora_blue",
  "post_date": "2023-11-18T15:53:10.777000",
  "votes": 3,
  "comment_count": 2,
  "views": 0,
  "content": "<p>In some notebook, tile image data for inference are saved in the temporary directory outside <code>/kaggle/working</code> probably in order to minimize time to read large png. In general, if image is provided in tiff, getting tile function may be in dataloader of pytorch.</p>\n<p>I read the <a href=\"https://www.kaggle.com/docs/notebooks\" target=\"_blank\">technical specifications</a> about kaggle notebook, but could not find the information.</p>\n<blockquote>\n  <p>20 Gigabytes of auto-saved disk space (/kaggle/working)<br>\n  Additional scratchpad disk space (outside /kaggle/working) that will not be saved outside of the current session</p>\n</blockquote>",
  "messages": [
    {
      "id": 2529896,
      "postDate": "2023-11-18T16:22:21.693Z",
      "content": "<p>Usually it's ~70GB total space. We can see it on the resources window <img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F9507963%2F818dfd114d4de26d307322c6493e2589%2FScreenshot_20231118-2149162.png?generation=1700324411987498&amp;alt=media\" alt=\"\"></p>\n<p>When tiling while inferencing, you may tile a batch of 16-32 and then delete the tiles before moving to the next batch. This worked for my team's inference. </p>",
      "rawMarkdown": "Usually it's ~70GB total space. We can see it on the resources window ![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F9507963%2F818dfd114d4de26d307322c6493e2589%2FScreenshot_20231118-2149162.png?generation=1700324411987498&alt=media)\n\nWhen tiling while inferencing, you may tile a batch of 16-32 and then delete the tiles before moving to the next batch. This worked for my team's inference. ",
      "votes": 3,
      "replies": [
        {
          "id": 2529918,
          "postDate": "2023-11-18T16:36:01.450Z",
          "content": "<p>Thanks for helping !!😄</p>",
          "rawMarkdown": "Thanks for helping !!😄"
        }
      ]
    },
    {
      "id": 2529870,
      "postDate": "2023-11-18T15:53:10.777Z",
      "content": "<p>In some notebook, tile image data for inference are saved in the temporary directory outside <code>/kaggle/working</code> probably in order to minimize time to read large png. In general, if image is provided in tiff, getting tile function may be in dataloader of pytorch.</p>\n<p>I read the <a href=\"https://www.kaggle.com/docs/notebooks\" target=\"_blank\">technical specifications</a> about kaggle notebook, but could not find the information.</p>\n<blockquote>\n  <p>20 Gigabytes of auto-saved disk space (/kaggle/working)<br>\n  Additional scratchpad disk space (outside /kaggle/working) that will not be saved outside of the current session</p>\n</blockquote>",
      "rawMarkdown": "In some notebook, tile image data for inference are saved in the temporary directory outside `/kaggle/working` probably in order to minimize time to read large png. In general, if image is provided in tiff, getting tile function may be in dataloader of pytorch.\n\n\nI read the [technical specifications](https://www.kaggle.com/docs/notebooks) about kaggle notebook, but could not find the information.\n\n>20 Gigabytes of auto-saved disk space (/kaggle/working)\nAdditional scratchpad disk space (outside /kaggle/working) that will not be saved outside of the current session",
      "votes": 3
    }
  ],
  "comments": [
    {
      "id": 2529896,
      "author_name": "pjmathematician",
      "author_url": "",
      "post_date": "2023-11-18T16:22:21.693000",
      "content": "<p>Usually it's ~70GB total space. We can see it on the resources window <img src=\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F9507963%2F818dfd114d4de26d307322c6493e2589%2FScreenshot_20231118-2149162.png?generation=1700324411987498&amp;alt=media\" alt=\"\"></p>\n<p>When tiling while inferencing, you may tile a batch of 16-32 and then delete the tiles before moving to the next batch. This worked for my team's inference. </p>",
      "votes": 3,
      "replies": [
        {
          "id": 2529918,
          "author_name": "Aurora_blue",
          "author_url": "",
          "post_date": "2023-11-18T16:36:01.450000",
          "content": "<p>Thanks for helping !!😄</p>",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2529896": "Usually it's ~70GB total space. We can see it on the resources window ![](https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F9507963%2F818dfd114d4de26d307322c6493e2589%2FScreenshot_20231118-2149162.png?generation=1700324411987498&alt=media)\n\nWhen tiling while inferencing, you may tile a batch of 16-32 and then delete the tiles before moving to the next batch. This worked for my team's inference. ",
    "2529870": "In some notebook, tile image data for inference are saved in the temporary directory outside `/kaggle/working` probably in order to minimize time to read large png. In general, if image is provided in tiff, getting tile function may be in dataloader of pytorch.\n\n\nI read the [technical specifications](https://www.kaggle.com/docs/notebooks) about kaggle notebook, but could not find the information.\n\n>20 Gigabytes of auto-saved disk space (/kaggle/working)\nAdditional scratchpad disk space (outside /kaggle/working) that will not be saved outside of the current session"
  }
}