{
  "id": 411525,
  "title": "error in loading Band file",
  "url": "/competitions/google-research-identify-contrails-reduce-global-warming/discussion/411525",
  "author_name": "Arunodhayan",
  "post_date": "2023-05-19T16:10:29.620000",
  "votes": 1,
  "comment_count": 4,
  "views": 0,
  "content": "<p>band_data = np.load(band_file) <br>\n  File \"/usr/local/lib/python3.8/dist-packages/numpy/lib/npyio.py\", line 440, in load<br>\n    return format.read_array(fid, allow_pickle=allow_pickle,<br>\n  File \"/usr/local/lib/python3.8/dist-packages/numpy/lib/format.py\", line 783, in read_array<br>\n    array.shape = shape<br>\nValueError: cannot reshape array of size 524287 into shape (256,256,8)<br>\nanyone facing such error</p>",
  "messages": [
    {
      "id": 2268636,
      "postDate": "2023-05-21T21:30:38.067Z",
      "content": "<p>Maybe versions issue? I can't see why this would happen unless the file is saved using a differnt numpy / python version..</p>\n<p>What numpy version are you using? [check the environment of the notebook..]</p>\n<p>In general the np arrays are saved as 1D buffers in disc along with the shapes [this is why you see the shape problem… it loads then can not reshape..] </p>\n<p>Maybe try to <code>mmap</code> and not load everything at the same time and see what happends?</p>\n<p>Just to make sure: We are talking about the official data of the comp? which of the files? </p>\n<p>The Devastator.</p>",
      "rawMarkdown": "Maybe versions issue? I can't see why this would happen unless the file is saved using a differnt numpy / python version..\n\nWhat numpy version are you using? [check the environment of the notebook..]\n\nIn general the np arrays are saved as 1D buffers in disc along with the shapes [this is why you see the shape problem... it loads then can not reshape..] \n\nMaybe try to `mmap` and not load everything at the same time and see what happends?\n\n\nJust to make sure: We are talking about the official data of the comp? which of the files? \n\nThe Devastator.\n\n",
      "replies": [
        {
          "id": 2273999,
          "postDate": "2023-05-25T14:42:29.627Z",
          "content": "<p>I redownloaded the files - its working now properly </p>",
          "rawMarkdown": "I redownloaded the files - its working now properly "
        }
      ]
    },
    {
      "id": 2266054,
      "postDate": "2023-05-19T17:11:10.490Z",
      "content": "<p>So basically 256<em>256</em>8 = 524288, so there is one entry missing in your array. Dont know if this helps :D</p>",
      "rawMarkdown": "So basically 256*256*8 = 524288, so there is one entry missing in your array. Dont know if this helps :D",
      "replies": [
        {
          "id": 2266257,
          "postDate": "2023-05-19T21:18:31.647Z",
          "content": "<p>np.load(band_file) - throws this error is it because of file corrupted</p>",
          "rawMarkdown": "np.load(band_file) - throws this error is it because of file corrupted",
          "votes": 1
        }
      ]
    },
    {
      "id": 2265973,
      "postDate": "2023-05-19T16:10:29.620Z",
      "content": "<p>band_data = np.load(band_file) <br>\n  File \"/usr/local/lib/python3.8/dist-packages/numpy/lib/npyio.py\", line 440, in load<br>\n    return format.read_array(fid, allow_pickle=allow_pickle,<br>\n  File \"/usr/local/lib/python3.8/dist-packages/numpy/lib/format.py\", line 783, in read_array<br>\n    array.shape = shape<br>\nValueError: cannot reshape array of size 524287 into shape (256,256,8)<br>\nanyone facing such error</p>",
      "rawMarkdown": "band_data = np.load(band_file) \n  File \"/usr/local/lib/python3.8/dist-packages/numpy/lib/npyio.py\", line 440, in load\n    return format.read_array(fid, allow_pickle=allow_pickle,\n  File \"/usr/local/lib/python3.8/dist-packages/numpy/lib/format.py\", line 783, in read_array\n    array.shape = shape\nValueError: cannot reshape array of size 524287 into shape (256,256,8)\nanyone facing such error\n\n"
    }
  ],
  "comments": [
    {
      "id": 2268636,
      "author_name": "The Devastator",
      "author_url": "",
      "post_date": "2023-05-21T21:30:38.067000",
      "content": "<p>Maybe versions issue? I can't see why this would happen unless the file is saved using a differnt numpy / python version..</p>\n<p>What numpy version are you using? [check the environment of the notebook..]</p>\n<p>In general the np arrays are saved as 1D buffers in disc along with the shapes [this is why you see the shape problem… it loads then can not reshape..] </p>\n<p>Maybe try to <code>mmap</code> and not load everything at the same time and see what happends?</p>\n<p>Just to make sure: We are talking about the official data of the comp? which of the files? </p>\n<p>The Devastator.</p>",
      "votes": 0,
      "replies": [
        {
          "id": 2273999,
          "author_name": "Arunodhayan",
          "author_url": "",
          "post_date": "2023-05-25T14:42:29.627000",
          "content": "<p>I redownloaded the files - its working now properly </p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 2266054,
      "author_name": "Jan H",
      "author_url": "",
      "post_date": "2023-05-19T17:11:10.490000",
      "content": "<p>So basically 256<em>256</em>8 = 524288, so there is one entry missing in your array. Dont know if this helps :D</p>",
      "votes": 0,
      "replies": [
        {
          "id": 2266257,
          "author_name": "Arunodhayan",
          "author_url": "",
          "post_date": "2023-05-19T21:18:31.647000",
          "content": "<p>np.load(band_file) - throws this error is it because of file corrupted</p>",
          "votes": 1,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2268636": "Maybe versions issue? I can't see why this would happen unless the file is saved using a differnt numpy / python version..\n\nWhat numpy version are you using? [check the environment of the notebook..]\n\nIn general the np arrays are saved as 1D buffers in disc along with the shapes [this is why you see the shape problem... it loads then can not reshape..] \n\nMaybe try to `mmap` and not load everything at the same time and see what happends?\n\n\nJust to make sure: We are talking about the official data of the comp? which of the files? \n\nThe Devastator.\n\n",
    "2266054": "So basically 256*256*8 = 524288, so there is one entry missing in your array. Dont know if this helps :D",
    "2265973": "band_data = np.load(band_file) \n  File \"/usr/local/lib/python3.8/dist-packages/numpy/lib/npyio.py\", line 440, in load\n    return format.read_array(fid, allow_pickle=allow_pickle,\n  File \"/usr/local/lib/python3.8/dist-packages/numpy/lib/format.py\", line 783, in read_array\n    array.shape = shape\nValueError: cannot reshape array of size 524287 into shape (256,256,8)\nanyone facing such error\n\n"
  }
}