{
  "id": 447319,
  "title": "While submitting Notebook to the compitition ,  I am facing Notebook Out of Memory Error ",
  "url": "/competitions/UBC-OCEAN/discussion/447319",
  "author_name": "sunil thite",
  "post_date": "2023-10-15T10:34:59.388000",
  "votes": 6,
  "comment_count": 6,
  "views": 0,
  "content": "<p>While submitting Notebook to the,  I am facing Notebook Out of Memory Error . Your notebook requested more memory (RAM) than is available. <br>\nPlease provide me the solutions for this problem.</p>",
  "messages": [
    {
      "id": 2482870,
      "postDate": "2023-10-15T10:34:59.390Z",
      "content": "<p>While submitting Notebook to the,  I am facing Notebook Out of Memory Error . Your notebook requested more memory (RAM) than is available. <br>\nPlease provide me the solutions for this problem.</p>",
      "rawMarkdown": "While submitting Notebook to the,  I am facing Notebook Out of Memory Error . Your notebook requested more memory (RAM) than is available. \nPlease provide me the solutions for this problem.",
      "votes": 6
    },
    {
      "id": 2482879,
      "postDate": "2023-10-15T10:44:09.350Z",
      "content": "<p>This is a common problem for such large datasets <a href=\"https://www.kaggle.com/sunilthite\" target=\"_blank\">@sunilthite</a> <br>\nYou may choose the below-</p>\n<ol>\n<li>Import the training data and conduct feature processing separately, preferably on a local PC</li>\n<li>Train your models locally and save them using a .pkl extension. You may use a library like <strong>joblib</strong> for the same</li>\n<li>Use your .pkl extension files (saved models) in an inference process, infer appropriately and submit only relevant components. You may save the use the models as <strong>Kaggle datasets/ Kaggle models</strong></li>\n</ol>\n<p>Best regards!</p>",
      "rawMarkdown": "This is a common problem for such large datasets @sunilthite \nYou may choose the below-\n1. Import the training data and conduct feature processing separately, preferably on a local PC\n2. Train your models locally and save them using a .pkl extension. You may use a library like **joblib** for the same\n3. Use your .pkl extension files (saved models) in an inference process, infer appropriately and submit only relevant components. You may save the use the models as **Kaggle datasets/ Kaggle models**\n\nBest regards!",
      "replies": [
        {
          "id": 2483097,
          "postDate": "2023-10-15T13:25:13.153Z",
          "content": "<p>Thank you so much for this valuable information. I will try this</p>",
          "rawMarkdown": "Thank you so much for this valuable information. I will try this",
          "votes": 2
        },
        {
          "id": 2484574,
          "postDate": "2023-10-16T15:20:30.253Z",
          "content": "<p>I train the model seperately and then save that model . For submissions I created another notebook and use that model and submit the notebook but it also giving ………….Notebook Out of Memory Your notebook requested more memory (RAM) than is available. <br>\nI use batch size = 32 , image_shape = (224x224x3)  , image_data = 54672 images are in patches of 224x224.<br>\nPlease suggest me how to solve this problem ?</p>",
          "rawMarkdown": "I train the model seperately and then save that model . For submissions I created another notebook and use that model and submit the notebook but it also giving .............Notebook Out of Memory Your notebook requested more memory (RAM) than is available. \nI use batch size = 32 , image_shape = (224x224x3)  , image_data = 54672 images are in patches of 224x224.\nPlease suggest me how to solve this problem ?",
          "votes": 3,
          "replies": [
            {
              "id": 2534098,
              "postDate": "2023-11-22T12:03:43.980Z",
              "content": "<p>Hello <a href=\"https://www.kaggle.com/sunilthite\" target=\"_blank\">@sunilthite</a>  did you resolve your probleme</p>",
              "rawMarkdown": "Hello @sunilthite  did you resolve your probleme"
            }
          ]
        },
        {
          "id": 2487388,
          "postDate": "2023-10-18T14:54:22.157Z",
          "content": "<p>I had the same problem. For inference, I used the small images (test_thumbnails). Hoping it will work.</p>",
          "rawMarkdown": "I had the same problem. For inference, I used the small images (test_thumbnails). Hoping it will work.",
          "replies": [
            {
              "id": 2920181,
              "postDate": "2024-07-13T13:40:34.103Z",
              "content": "<p>hello,did you resolve your problem?</p>",
              "rawMarkdown": "hello,did you resolve your problem?"
            }
          ]
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 2482879,
      "author_name": "Ravi Ramakrishnan",
      "author_url": "",
      "post_date": "2023-10-15T10:44:09.350000",
      "content": "<p>This is a common problem for such large datasets <a href=\"https://www.kaggle.com/sunilthite\" target=\"_blank\">@sunilthite</a> <br>\nYou may choose the below-</p>\n<ol>\n<li>Import the training data and conduct feature processing separately, preferably on a local PC</li>\n<li>Train your models locally and save them using a .pkl extension. You may use a library like <strong>joblib</strong> for the same</li>\n<li>Use your .pkl extension files (saved models) in an inference process, infer appropriately and submit only relevant components. You may save the use the models as <strong>Kaggle datasets/ Kaggle models</strong></li>\n</ol>\n<p>Best regards!</p>",
      "votes": 0,
      "replies": [
        {
          "id": 2483097,
          "author_name": "sunil thite",
          "author_url": "",
          "post_date": "2023-10-15T13:25:13.153000",
          "content": "<p>Thank you so much for this valuable information. I will try this</p>",
          "votes": 2,
          "replies": []
        },
        {
          "id": 2484574,
          "author_name": "sunil thite",
          "author_url": "",
          "post_date": "2023-10-16T15:20:30.253000",
          "content": "<p>I train the model seperately and then save that model . For submissions I created another notebook and use that model and submit the notebook but it also giving ………….Notebook Out of Memory Your notebook requested more memory (RAM) than is available. <br>\nI use batch size = 32 , image_shape = (224x224x3)  , image_data = 54672 images are in patches of 224x224.<br>\nPlease suggest me how to solve this problem ?</p>",
          "votes": 3,
          "replies": [
            {
              "id": 2534098,
              "author_name": "DIATTARA Amadou",
              "author_url": "",
              "post_date": "2023-11-22T12:03:43.980000",
              "content": "<p>Hello <a href=\"https://www.kaggle.com/sunilthite\" target=\"_blank\">@sunilthite</a>  did you resolve your probleme</p>",
              "votes": 0,
              "replies": []
            }
          ]
        },
        {
          "id": 2487388,
          "author_name": "Catadanna",
          "author_url": "",
          "post_date": "2023-10-18T14:54:22.157000",
          "content": "<p>I had the same problem. For inference, I used the small images (test_thumbnails). Hoping it will work.</p>",
          "votes": 0,
          "replies": [
            {
              "id": 2920181,
              "author_name": "yangyangyang",
              "author_url": "",
              "post_date": "2024-07-13T13:40:34.103000",
              "content": "<p>hello,did you resolve your problem?</p>",
              "votes": 0,
              "replies": []
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2482870": "While submitting Notebook to the,  I am facing Notebook Out of Memory Error . Your notebook requested more memory (RAM) than is available. \nPlease provide me the solutions for this problem.",
    "2482879": "This is a common problem for such large datasets @sunilthite \nYou may choose the below-\n1. Import the training data and conduct feature processing separately, preferably on a local PC\n2. Train your models locally and save them using a .pkl extension. You may use a library like **joblib** for the same\n3. Use your .pkl extension files (saved models) in an inference process, infer appropriately and submit only relevant components. You may save the use the models as **Kaggle datasets/ Kaggle models**\n\nBest regards!"
  }
}