{
  "id": 496455,
  "title": "Zero Variance Columns",
  "url": "/competitions/leap-atmospheric-physics-ai-climsim/discussion/496455",
  "author_name": "Gunes Evitan",
  "post_date": "2024-04-21T07:22:47.541000",
  "votes": 20,
  "comment_count": 4,
  "views": 0,
  "content": "<h2>Features</h2>\n<p>For features; <code>pbuf_CH4</code>, <code>pbuf_N2O</code> and <code>state_q0002</code> groups have zero variance in some part of their sequence.</p>\n<p><code>['pbuf_CH4_27', 'pbuf_CH4_28', 'pbuf_CH4_29', 'pbuf_CH4_30', 'pbuf_CH4_31', 'pbuf_CH4_32', 'pbuf_CH4_33', 'pbuf_CH4_34', 'pbuf_CH4_35', 'pbuf_CH4_36', 'pbuf_CH4_37', 'pbuf_CH4_38', 'pbuf_CH4_39', 'pbuf_CH4_40', 'pbuf_CH4_41', 'pbuf_CH4_42', 'pbuf_CH4_43', 'pbuf_CH4_44', 'pbuf_CH4_45', 'pbuf_CH4_46', 'pbuf_CH4_47', 'pbuf_CH4_48', 'pbuf_CH4_49', 'pbuf_CH4_50', 'pbuf_CH4_51', 'pbuf_CH4_52', 'pbuf_CH4_53', 'pbuf_CH4_54', 'pbuf_CH4_55', 'pbuf_CH4_56', 'pbuf_CH4_57', 'pbuf_CH4_58', 'pbuf_CH4_59']</code></p>\n<p><code>['pbuf_N2O_27', 'pbuf_N2O_28', 'pbuf_N2O_29', 'pbuf_N2O_30', 'pbuf_N2O_31', 'pbuf_N2O_32', 'pbuf_N2O_33', 'pbuf_N2O_34', 'pbuf_N2O_35', 'pbuf_N2O_36', 'pbuf_N2O_37', 'pbuf_N2O_38', 'pbuf_N2O_39', 'pbuf_N2O_40', 'pbuf_N2O_41', 'pbuf_N2O_42', 'pbuf_N2O_43', 'pbuf_N2O_44', 'pbuf_N2O_45', 'pbuf_N2O_46', 'pbuf_N2O_47', 'pbuf_N2O_48', 'pbuf_N2O_49', 'pbuf_N2O_50', 'pbuf_N2O_51', 'pbuf_N2O_52', 'pbuf_N2O_53', 'pbuf_N2O_54', 'pbuf_N2O_55', 'pbuf_N2O_56', 'pbuf_N2O_57', 'pbuf_N2O_58', 'pbuf_N2O_59']</code></p>\n<p><code>pbuf_N2O</code> and <code>pbuf_CH4</code> have zero variance in the exact part (last 33 columns) so they might have some kind of connection.</p>\n<p><code>['state_q0002_0', 'state_q0002_1', 'state_q0002_2', 'state_q0002_3', 'state_q0002_4', 'state_q0002_5', 'state_q0002_6', 'state_q0002_7', 'state_q0002_8', 'state_q0002_9', 'state_q0002_10', 'state_q0002_11', 'state_q0002_12', 'state_q0002_13', 'state_q0002_14', 'state_q0002_15', 'state_q0002_16', 'state_q0002_17', 'state_q0002_18', 'state_q0002_19', 'state_q0002_20', 'state_q0002_21', 'state_q0002_22', 'state_q0002_23']</code></p>\n<p>For <code>state_q0002</code>, first 24 columns have zero variance.</p>\n<h2>Targets</h2>\n<p>For targets, <code>ptend_q0001</code>, <code>ptend_q0002</code>, <code>ptend_q0003</code>, <code>ptend_u</code> and <code>ptend_v</code> groups have zero variance in some part of their sequence.</p>\n<p><code>['ptend_q0001_4', 'ptend_q0001_5', 'ptend_q0001_6', 'ptend_q0001_7', 'ptend_q0001_8', 'ptend_q0001_9', 'ptend_q0001_10', 'ptend_q0001_11']</code></p>\n<p><code>['ptend_q0002_0', 'ptend_q0002_1', 'ptend_q0002_2', 'ptend_q0002_3', 'ptend_q0002_4', 'ptend_q0002_5', 'ptend_q0002_6', 'ptend_q0002_7', 'ptend_q0002_8', 'ptend_q0002_9', 'ptend_q0002_10', 'ptend_q0002_11', 'ptend_q0002_12', 'ptend_q0002_13', 'ptend_q0002_14', 'ptend_q0002_15', 'ptend_q0002_16', 'ptend_q0002_17', 'ptend_q0002_18', 'ptend_q0002_19', 'ptend_q0002_20', 'ptend_q0002_21', 'ptend_q0002_22', 'ptend_q0002_23']</code></p>\n<p><code>['ptend_q0003_0', 'ptend_q0003_1', 'ptend_q0003_2', 'ptend_q0003_3', 'ptend_q0003_4', 'ptend_q0003_5', 'ptend_q0003_6', 'ptend_q0003_7', 'ptend_q0003_8', 'ptend_q0003_9', 'ptend_q0003_10', 'ptend_q0003_11']</code></p>\n<p><code>['ptend_u_0', 'ptend_u_1', 'ptend_u_2', 'ptend_u_3', 'ptend_u_4', 'ptend_u_5', 'ptend_u_6', 'ptend_u_7', 'ptend_u_8', 'ptend_u_9', 'ptend_u_10', 'ptend_u_11']</code></p>\n<p><code>['ptend_v_0', 'ptend_v_1', 'ptend_v_2', 'ptend_v_3', 'ptend_v_4', 'ptend_v_5', 'ptend_v_6', 'ptend_v_7', 'ptend_v_8', 'ptend_v_9', 'ptend_v_10', 'ptend_v_11']</code></p>\n<p>ptend_q0001 columns between 5th and 12th, ptend_q0002 first 24 columns, ptend_q0003 first 12 columns, ptend_u first 12 columns and ptend_v first 12 columns have zero variance.</p>\n<p>I checked this on randomly sampled 1M samples so this is probably representative for the entire population.</p>",
  "messages": [
    {
      "id": 2765530,
      "postDate": "2024-04-21T07:22:47.540Z",
      "content": "<h2>Features</h2>\n<p>For features; <code>pbuf_CH4</code>, <code>pbuf_N2O</code> and <code>state_q0002</code> groups have zero variance in some part of their sequence.</p>\n<p><code>['pbuf_CH4_27', 'pbuf_CH4_28', 'pbuf_CH4_29', 'pbuf_CH4_30', 'pbuf_CH4_31', 'pbuf_CH4_32', 'pbuf_CH4_33', 'pbuf_CH4_34', 'pbuf_CH4_35', 'pbuf_CH4_36', 'pbuf_CH4_37', 'pbuf_CH4_38', 'pbuf_CH4_39', 'pbuf_CH4_40', 'pbuf_CH4_41', 'pbuf_CH4_42', 'pbuf_CH4_43', 'pbuf_CH4_44', 'pbuf_CH4_45', 'pbuf_CH4_46', 'pbuf_CH4_47', 'pbuf_CH4_48', 'pbuf_CH4_49', 'pbuf_CH4_50', 'pbuf_CH4_51', 'pbuf_CH4_52', 'pbuf_CH4_53', 'pbuf_CH4_54', 'pbuf_CH4_55', 'pbuf_CH4_56', 'pbuf_CH4_57', 'pbuf_CH4_58', 'pbuf_CH4_59']</code></p>\n<p><code>['pbuf_N2O_27', 'pbuf_N2O_28', 'pbuf_N2O_29', 'pbuf_N2O_30', 'pbuf_N2O_31', 'pbuf_N2O_32', 'pbuf_N2O_33', 'pbuf_N2O_34', 'pbuf_N2O_35', 'pbuf_N2O_36', 'pbuf_N2O_37', 'pbuf_N2O_38', 'pbuf_N2O_39', 'pbuf_N2O_40', 'pbuf_N2O_41', 'pbuf_N2O_42', 'pbuf_N2O_43', 'pbuf_N2O_44', 'pbuf_N2O_45', 'pbuf_N2O_46', 'pbuf_N2O_47', 'pbuf_N2O_48', 'pbuf_N2O_49', 'pbuf_N2O_50', 'pbuf_N2O_51', 'pbuf_N2O_52', 'pbuf_N2O_53', 'pbuf_N2O_54', 'pbuf_N2O_55', 'pbuf_N2O_56', 'pbuf_N2O_57', 'pbuf_N2O_58', 'pbuf_N2O_59']</code></p>\n<p><code>pbuf_N2O</code> and <code>pbuf_CH4</code> have zero variance in the exact part (last 33 columns) so they might have some kind of connection.</p>\n<p><code>['state_q0002_0', 'state_q0002_1', 'state_q0002_2', 'state_q0002_3', 'state_q0002_4', 'state_q0002_5', 'state_q0002_6', 'state_q0002_7', 'state_q0002_8', 'state_q0002_9', 'state_q0002_10', 'state_q0002_11', 'state_q0002_12', 'state_q0002_13', 'state_q0002_14', 'state_q0002_15', 'state_q0002_16', 'state_q0002_17', 'state_q0002_18', 'state_q0002_19', 'state_q0002_20', 'state_q0002_21', 'state_q0002_22', 'state_q0002_23']</code></p>\n<p>For <code>state_q0002</code>, first 24 columns have zero variance.</p>\n<h2>Targets</h2>\n<p>For targets, <code>ptend_q0001</code>, <code>ptend_q0002</code>, <code>ptend_q0003</code>, <code>ptend_u</code> and <code>ptend_v</code> groups have zero variance in some part of their sequence.</p>\n<p><code>['ptend_q0001_4', 'ptend_q0001_5', 'ptend_q0001_6', 'ptend_q0001_7', 'ptend_q0001_8', 'ptend_q0001_9', 'ptend_q0001_10', 'ptend_q0001_11']</code></p>\n<p><code>['ptend_q0002_0', 'ptend_q0002_1', 'ptend_q0002_2', 'ptend_q0002_3', 'ptend_q0002_4', 'ptend_q0002_5', 'ptend_q0002_6', 'ptend_q0002_7', 'ptend_q0002_8', 'ptend_q0002_9', 'ptend_q0002_10', 'ptend_q0002_11', 'ptend_q0002_12', 'ptend_q0002_13', 'ptend_q0002_14', 'ptend_q0002_15', 'ptend_q0002_16', 'ptend_q0002_17', 'ptend_q0002_18', 'ptend_q0002_19', 'ptend_q0002_20', 'ptend_q0002_21', 'ptend_q0002_22', 'ptend_q0002_23']</code></p>\n<p><code>['ptend_q0003_0', 'ptend_q0003_1', 'ptend_q0003_2', 'ptend_q0003_3', 'ptend_q0003_4', 'ptend_q0003_5', 'ptend_q0003_6', 'ptend_q0003_7', 'ptend_q0003_8', 'ptend_q0003_9', 'ptend_q0003_10', 'ptend_q0003_11']</code></p>\n<p><code>['ptend_u_0', 'ptend_u_1', 'ptend_u_2', 'ptend_u_3', 'ptend_u_4', 'ptend_u_5', 'ptend_u_6', 'ptend_u_7', 'ptend_u_8', 'ptend_u_9', 'ptend_u_10', 'ptend_u_11']</code></p>\n<p><code>['ptend_v_0', 'ptend_v_1', 'ptend_v_2', 'ptend_v_3', 'ptend_v_4', 'ptend_v_5', 'ptend_v_6', 'ptend_v_7', 'ptend_v_8', 'ptend_v_9', 'ptend_v_10', 'ptend_v_11']</code></p>\n<p>ptend_q0001 columns between 5th and 12th, ptend_q0002 first 24 columns, ptend_q0003 first 12 columns, ptend_u first 12 columns and ptend_v first 12 columns have zero variance.</p>\n<p>I checked this on randomly sampled 1M samples so this is probably representative for the entire population.</p>",
      "rawMarkdown": "## Features\n\nFor features; `pbuf_CH4`, `pbuf_N2O` and `state_q0002` groups have zero variance in some part of their sequence.\n\n`['pbuf_CH4_27', 'pbuf_CH4_28', 'pbuf_CH4_29', 'pbuf_CH4_30', 'pbuf_CH4_31', 'pbuf_CH4_32', 'pbuf_CH4_33', 'pbuf_CH4_34', 'pbuf_CH4_35', 'pbuf_CH4_36', 'pbuf_CH4_37', 'pbuf_CH4_38', 'pbuf_CH4_39', 'pbuf_CH4_40', 'pbuf_CH4_41', 'pbuf_CH4_42', 'pbuf_CH4_43', 'pbuf_CH4_44', 'pbuf_CH4_45', 'pbuf_CH4_46', 'pbuf_CH4_47', 'pbuf_CH4_48', 'pbuf_CH4_49', 'pbuf_CH4_50', 'pbuf_CH4_51', 'pbuf_CH4_52', 'pbuf_CH4_53', 'pbuf_CH4_54', 'pbuf_CH4_55', 'pbuf_CH4_56', 'pbuf_CH4_57', 'pbuf_CH4_58', 'pbuf_CH4_59']`\n\n`['pbuf_N2O_27', 'pbuf_N2O_28', 'pbuf_N2O_29', 'pbuf_N2O_30', 'pbuf_N2O_31', 'pbuf_N2O_32', 'pbuf_N2O_33', 'pbuf_N2O_34', 'pbuf_N2O_35', 'pbuf_N2O_36', 'pbuf_N2O_37', 'pbuf_N2O_38', 'pbuf_N2O_39', 'pbuf_N2O_40', 'pbuf_N2O_41', 'pbuf_N2O_42', 'pbuf_N2O_43', 'pbuf_N2O_44', 'pbuf_N2O_45', 'pbuf_N2O_46', 'pbuf_N2O_47', 'pbuf_N2O_48', 'pbuf_N2O_49', 'pbuf_N2O_50', 'pbuf_N2O_51', 'pbuf_N2O_52', 'pbuf_N2O_53', 'pbuf_N2O_54', 'pbuf_N2O_55', 'pbuf_N2O_56', 'pbuf_N2O_57', 'pbuf_N2O_58', 'pbuf_N2O_59']`\n\n`pbuf_N2O` and `pbuf_CH4` have zero variance in the exact part (last 33 columns) so they might have some kind of connection.\n\n`['state_q0002_0', 'state_q0002_1', 'state_q0002_2', 'state_q0002_3', 'state_q0002_4', 'state_q0002_5', 'state_q0002_6', 'state_q0002_7', 'state_q0002_8', 'state_q0002_9', 'state_q0002_10', 'state_q0002_11', 'state_q0002_12', 'state_q0002_13', 'state_q0002_14', 'state_q0002_15', 'state_q0002_16', 'state_q0002_17', 'state_q0002_18', 'state_q0002_19', 'state_q0002_20', 'state_q0002_21', 'state_q0002_22', 'state_q0002_23']`\n\nFor `state_q0002`, first 24 columns have zero variance.\n\n## Targets\n\nFor targets, `ptend_q0001`, `ptend_q0002`, `ptend_q0003`, `ptend_u` and `ptend_v` groups have zero variance in some part of their sequence.\n\n`['ptend_q0001_4', 'ptend_q0001_5', 'ptend_q0001_6', 'ptend_q0001_7', 'ptend_q0001_8', 'ptend_q0001_9', 'ptend_q0001_10', 'ptend_q0001_11']`\n\n`['ptend_q0002_0', 'ptend_q0002_1', 'ptend_q0002_2', 'ptend_q0002_3', 'ptend_q0002_4', 'ptend_q0002_5', 'ptend_q0002_6', 'ptend_q0002_7', 'ptend_q0002_8', 'ptend_q0002_9', 'ptend_q0002_10', 'ptend_q0002_11', 'ptend_q0002_12', 'ptend_q0002_13', 'ptend_q0002_14', 'ptend_q0002_15', 'ptend_q0002_16', 'ptend_q0002_17', 'ptend_q0002_18', 'ptend_q0002_19', 'ptend_q0002_20', 'ptend_q0002_21', 'ptend_q0002_22', 'ptend_q0002_23']`\n\n`['ptend_q0003_0', 'ptend_q0003_1', 'ptend_q0003_2', 'ptend_q0003_3', 'ptend_q0003_4', 'ptend_q0003_5', 'ptend_q0003_6', 'ptend_q0003_7', 'ptend_q0003_8', 'ptend_q0003_9', 'ptend_q0003_10', 'ptend_q0003_11']`\n\n`['ptend_u_0', 'ptend_u_1', 'ptend_u_2', 'ptend_u_3', 'ptend_u_4', 'ptend_u_5', 'ptend_u_6', 'ptend_u_7', 'ptend_u_8', 'ptend_u_9', 'ptend_u_10', 'ptend_u_11']`\n\n`['ptend_v_0', 'ptend_v_1', 'ptend_v_2', 'ptend_v_3', 'ptend_v_4', 'ptend_v_5', 'ptend_v_6', 'ptend_v_7', 'ptend_v_8', 'ptend_v_9', 'ptend_v_10', 'ptend_v_11']`\n\nptend_q0001 columns between 5th and 12th, ptend_q0002 first 24 columns, ptend_q0003 first 12 columns, ptend_u first 12 columns and ptend_v first 12 columns have zero variance.\n\nI checked this on randomly sampled 1M samples so this is probably representative for the entire population.\n",
      "votes": 20
    },
    {
      "id": 2775660,
      "postDate": "2024-04-25T18:23:45.093Z",
      "content": "<p><a href=\"https://www.kaggle.com/jerrylin96\" target=\"_blank\">@jerrylin96</a> Atleast for the target variables: Could this be a floating point truncation introduced by <a href=\"https://github.com/leap-stc/ClimSim/blob/58cb6a2e4fbc27ffd6acb9050a2f1df44b933a05/climsim_utils/data_utils.py#L379-L385\" target=\"_blank\">https://github.com/leap-stc/ClimSim/blob/58cb6a2e4fbc27ffd6acb9050a2f1df44b933a05/climsim_utils/data_utils.py#L379-L385</a> ? The zero variance columns are all in the upper atmosphere, if I am not mistaken, and could, hence, have very small values.</p>",
      "rawMarkdown": "@jerrylin96 Atleast for the target variables: Could this be a floating point truncation introduced by https://github.com/leap-stc/ClimSim/blob/58cb6a2e4fbc27ffd6acb9050a2f1df44b933a05/climsim_utils/data_utils.py#L379-L385 ? The zero variance columns are all in the upper atmosphere, if I am not mistaken, and could, hence, have very small values.",
      "votes": 3
    },
    {
      "id": 2769549,
      "postDate": "2024-04-23T12:12:59.693Z",
      "content": "<p>Hallo, I already posted it, too, <a href=\"https://www.kaggle.com/code/catadanna/columns-to-delete/notebook\" target=\"_blank\">here</a>.</p>",
      "rawMarkdown": "Hallo, I already posted it, too, [here](https://www.kaggle.com/code/catadanna/columns-to-delete/notebook).",
      "votes": 1
    },
    {
      "id": 2768725,
      "postDate": "2024-04-23T02:35:48.033Z",
      "content": "<p>Amazing discovery!!!</p>",
      "rawMarkdown": "Amazing discovery!!!"
    },
    {
      "id": 2767953,
      "postDate": "2024-04-22T15:32:17.163Z",
      "rawMarkdown": "",
      "isDeleted": true
    }
  ],
  "comments": [
    {
      "id": 2775660,
      "author_name": "Sebastian Hoffmann",
      "author_url": "",
      "post_date": "2024-04-25T18:23:45.093000",
      "content": "<p><a href=\"https://www.kaggle.com/jerrylin96\" target=\"_blank\">@jerrylin96</a> Atleast for the target variables: Could this be a floating point truncation introduced by <a href=\"https://github.com/leap-stc/ClimSim/blob/58cb6a2e4fbc27ffd6acb9050a2f1df44b933a05/climsim_utils/data_utils.py#L379-L385\" target=\"_blank\">https://github.com/leap-stc/ClimSim/blob/58cb6a2e4fbc27ffd6acb9050a2f1df44b933a05/climsim_utils/data_utils.py#L379-L385</a> ? The zero variance columns are all in the upper atmosphere, if I am not mistaken, and could, hence, have very small values.</p>",
      "votes": 3,
      "replies": []
    },
    {
      "id": 2769549,
      "author_name": "Catadanna",
      "author_url": "",
      "post_date": "2024-04-23T12:12:59.693000",
      "content": "<p>Hallo, I already posted it, too, <a href=\"https://www.kaggle.com/code/catadanna/columns-to-delete/notebook\" target=\"_blank\">here</a>.</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 2768725,
      "author_name": "Mingjie Wang",
      "author_url": "",
      "post_date": "2024-04-23T02:35:48.033000",
      "content": "<p>Amazing discovery!!!</p>",
      "votes": 0,
      "replies": []
    },
    {
      "id": 2767953,
      "author_name": "",
      "author_url": "",
      "post_date": "2024-04-22T15:32:17.163000",
      "content": "",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2765530": "## Features\n\nFor features; `pbuf_CH4`, `pbuf_N2O` and `state_q0002` groups have zero variance in some part of their sequence.\n\n`['pbuf_CH4_27', 'pbuf_CH4_28', 'pbuf_CH4_29', 'pbuf_CH4_30', 'pbuf_CH4_31', 'pbuf_CH4_32', 'pbuf_CH4_33', 'pbuf_CH4_34', 'pbuf_CH4_35', 'pbuf_CH4_36', 'pbuf_CH4_37', 'pbuf_CH4_38', 'pbuf_CH4_39', 'pbuf_CH4_40', 'pbuf_CH4_41', 'pbuf_CH4_42', 'pbuf_CH4_43', 'pbuf_CH4_44', 'pbuf_CH4_45', 'pbuf_CH4_46', 'pbuf_CH4_47', 'pbuf_CH4_48', 'pbuf_CH4_49', 'pbuf_CH4_50', 'pbuf_CH4_51', 'pbuf_CH4_52', 'pbuf_CH4_53', 'pbuf_CH4_54', 'pbuf_CH4_55', 'pbuf_CH4_56', 'pbuf_CH4_57', 'pbuf_CH4_58', 'pbuf_CH4_59']`\n\n`['pbuf_N2O_27', 'pbuf_N2O_28', 'pbuf_N2O_29', 'pbuf_N2O_30', 'pbuf_N2O_31', 'pbuf_N2O_32', 'pbuf_N2O_33', 'pbuf_N2O_34', 'pbuf_N2O_35', 'pbuf_N2O_36', 'pbuf_N2O_37', 'pbuf_N2O_38', 'pbuf_N2O_39', 'pbuf_N2O_40', 'pbuf_N2O_41', 'pbuf_N2O_42', 'pbuf_N2O_43', 'pbuf_N2O_44', 'pbuf_N2O_45', 'pbuf_N2O_46', 'pbuf_N2O_47', 'pbuf_N2O_48', 'pbuf_N2O_49', 'pbuf_N2O_50', 'pbuf_N2O_51', 'pbuf_N2O_52', 'pbuf_N2O_53', 'pbuf_N2O_54', 'pbuf_N2O_55', 'pbuf_N2O_56', 'pbuf_N2O_57', 'pbuf_N2O_58', 'pbuf_N2O_59']`\n\n`pbuf_N2O` and `pbuf_CH4` have zero variance in the exact part (last 33 columns) so they might have some kind of connection.\n\n`['state_q0002_0', 'state_q0002_1', 'state_q0002_2', 'state_q0002_3', 'state_q0002_4', 'state_q0002_5', 'state_q0002_6', 'state_q0002_7', 'state_q0002_8', 'state_q0002_9', 'state_q0002_10', 'state_q0002_11', 'state_q0002_12', 'state_q0002_13', 'state_q0002_14', 'state_q0002_15', 'state_q0002_16', 'state_q0002_17', 'state_q0002_18', 'state_q0002_19', 'state_q0002_20', 'state_q0002_21', 'state_q0002_22', 'state_q0002_23']`\n\nFor `state_q0002`, first 24 columns have zero variance.\n\n## Targets\n\nFor targets, `ptend_q0001`, `ptend_q0002`, `ptend_q0003`, `ptend_u` and `ptend_v` groups have zero variance in some part of their sequence.\n\n`['ptend_q0001_4', 'ptend_q0001_5', 'ptend_q0001_6', 'ptend_q0001_7', 'ptend_q0001_8', 'ptend_q0001_9', 'ptend_q0001_10', 'ptend_q0001_11']`\n\n`['ptend_q0002_0', 'ptend_q0002_1', 'ptend_q0002_2', 'ptend_q0002_3', 'ptend_q0002_4', 'ptend_q0002_5', 'ptend_q0002_6', 'ptend_q0002_7', 'ptend_q0002_8', 'ptend_q0002_9', 'ptend_q0002_10', 'ptend_q0002_11', 'ptend_q0002_12', 'ptend_q0002_13', 'ptend_q0002_14', 'ptend_q0002_15', 'ptend_q0002_16', 'ptend_q0002_17', 'ptend_q0002_18', 'ptend_q0002_19', 'ptend_q0002_20', 'ptend_q0002_21', 'ptend_q0002_22', 'ptend_q0002_23']`\n\n`['ptend_q0003_0', 'ptend_q0003_1', 'ptend_q0003_2', 'ptend_q0003_3', 'ptend_q0003_4', 'ptend_q0003_5', 'ptend_q0003_6', 'ptend_q0003_7', 'ptend_q0003_8', 'ptend_q0003_9', 'ptend_q0003_10', 'ptend_q0003_11']`\n\n`['ptend_u_0', 'ptend_u_1', 'ptend_u_2', 'ptend_u_3', 'ptend_u_4', 'ptend_u_5', 'ptend_u_6', 'ptend_u_7', 'ptend_u_8', 'ptend_u_9', 'ptend_u_10', 'ptend_u_11']`\n\n`['ptend_v_0', 'ptend_v_1', 'ptend_v_2', 'ptend_v_3', 'ptend_v_4', 'ptend_v_5', 'ptend_v_6', 'ptend_v_7', 'ptend_v_8', 'ptend_v_9', 'ptend_v_10', 'ptend_v_11']`\n\nptend_q0001 columns between 5th and 12th, ptend_q0002 first 24 columns, ptend_q0003 first 12 columns, ptend_u first 12 columns and ptend_v first 12 columns have zero variance.\n\nI checked this on randomly sampled 1M samples so this is probably representative for the entire population.\n",
    "2775660": "@jerrylin96 Atleast for the target variables: Could this be a floating point truncation introduced by https://github.com/leap-stc/ClimSim/blob/58cb6a2e4fbc27ffd6acb9050a2f1df44b933a05/climsim_utils/data_utils.py#L379-L385 ? The zero variance columns are all in the upper atmosphere, if I am not mistaken, and could, hence, have very small values.",
    "2769549": "Hallo, I already posted it, too, [here](https://www.kaggle.com/code/catadanna/columns-to-delete/notebook).",
    "2768725": "Amazing discovery!!!",
    "2767953": ""
  }
}