{
  "id": 386442,
  "title": "When using TPU in PyTorch, how to distribute samples with undersampling?",
  "url": "/competitions/rsna-breast-cancer-detection/discussion/386442",
  "author_name": "Ju7on9",
  "post_date": "2023-02-13T06:22:30.468000",
  "votes": 0,
  "comment_count": 1,
  "views": 0,
  "content": "<p>Hello, I want to use TPU with PyTorch, but I have a problem.</p>\n<pre><code>train_sampler = torch.utils.data.distributed.DistributedSampler(\n          train_dataset,\n          num_replicas=xm.xrt_world_size(), \n          rank=xm.get_ordinal(), \n          shuffle=)\n\ntrain_data_loader = torch.utils.data.DataLoader(\n        train_dataset,\n        batch_size=BATCH_SIZE,\n        sampler=train_sampler,\n        drop_last=,\n        num_workers=,\n)\n</code></pre>\n<p>The above code is a skeleton of PyTorch TPU code. I would like to add under-sampling in the <code>train_data_loader</code> in order to handle imbalanced dataset, but the sampler of the data loader is DistributedSampler whose role is to distribute sample to multi-cores in the TPU. Is there any way to use under-sampling with DistributedSampler?</p>\n<p>I would appreciate if you answer the question.</p>",
  "messages": [
    {
      "id": 2141899,
      "postDate": "2023-02-13T06:59:11.400Z",
      "content": "<p>I am not well versed on this topic, but I remember seeing a post where people were directed to seek help from <a href=\"https://github.com/pytorch/xla/issues\" target=\"_blank\">https://github.com/pytorch/xla/issues</a> . </p>",
      "rawMarkdown": "I am not well versed on this topic, but I remember seeing a post where people were directed to seek help from https://github.com/pytorch/xla/issues . "
    },
    {
      "id": 2141871,
      "postDate": "2023-02-13T06:22:30.470Z",
      "content": "<p>Hello, I want to use TPU with PyTorch, but I have a problem.</p>\n<pre><code>train_sampler = torch.utils.data.distributed.DistributedSampler(\n          train_dataset,\n          num_replicas=xm.xrt_world_size(), \n          rank=xm.get_ordinal(), \n          shuffle=)\n\ntrain_data_loader = torch.utils.data.DataLoader(\n        train_dataset,\n        batch_size=BATCH_SIZE,\n        sampler=train_sampler,\n        drop_last=,\n        num_workers=,\n)\n</code></pre>\n<p>The above code is a skeleton of PyTorch TPU code. I would like to add under-sampling in the <code>train_data_loader</code> in order to handle imbalanced dataset, but the sampler of the data loader is DistributedSampler whose role is to distribute sample to multi-cores in the TPU. Is there any way to use under-sampling with DistributedSampler?</p>\n<p>I would appreciate if you answer the question.</p>",
      "rawMarkdown": "Hello, I want to use TPU with PyTorch, but I have a problem.\n\n\n```python\ntrain_sampler = torch.utils.data.distributed.DistributedSampler(\n          train_dataset,\n          num_replicas=xm.xrt_world_size(), \n          rank=xm.get_ordinal(), \n          shuffle=True)\n\ntrain_data_loader = torch.utils.data.DataLoader(\n        train_dataset,\n        batch_size=BATCH_SIZE,\n        sampler=train_sampler,\n        drop_last=True,\n        num_workers=0,\n)\n```\n\nThe above code is a skeleton of PyTorch TPU code. I would like to add under-sampling in the `train_data_loader` in order to handle imbalanced dataset, but the sampler of the data loader is DistributedSampler whose role is to distribute sample to multi-cores in the TPU. Is there any way to use under-sampling with DistributedSampler?\n\nI would appreciate if you answer the question."
    }
  ],
  "comments": [
    {
      "id": 2141899,
      "author_name": "Antti Isosalo",
      "author_url": "",
      "post_date": "2023-02-13T06:59:11.400000",
      "content": "<p>I am not well versed on this topic, but I remember seeing a post where people were directed to seek help from <a href=\"https://github.com/pytorch/xla/issues\" target=\"_blank\">https://github.com/pytorch/xla/issues</a> . </p>",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "2141899": "I am not well versed on this topic, but I remember seeing a post where people were directed to seek help from https://github.com/pytorch/xla/issues . ",
    "2141871": "Hello, I want to use TPU with PyTorch, but I have a problem.\n\n\n```python\ntrain_sampler = torch.utils.data.distributed.DistributedSampler(\n          train_dataset,\n          num_replicas=xm.xrt_world_size(), \n          rank=xm.get_ordinal(), \n          shuffle=True)\n\ntrain_data_loader = torch.utils.data.DataLoader(\n        train_dataset,\n        batch_size=BATCH_SIZE,\n        sampler=train_sampler,\n        drop_last=True,\n        num_workers=0,\n)\n```\n\nThe above code is a skeleton of PyTorch TPU code. I would like to add under-sampling in the `train_data_loader` in order to handle imbalanced dataset, but the sampler of the data loader is DistributedSampler whose role is to distribute sample to multi-cores in the TPU. Is there any way to use under-sampling with DistributedSampler?\n\nI would appreciate if you answer the question."
  }
}