{
  "id": 371093,
  "title": "Ray -  PARALLEL processing dicom files and ... do more tasks",
  "url": "/competitions/rsna-breast-cancer-detection/discussion/371093",
  "author_name": "Remek Kinas",
  "post_date": "2022-12-07T22:58:13.590000",
  "votes": 11,
  "comment_count": 0,
  "views": 0,
  "content": "<p>Looking for speed and multiprocessing? I created notebook for inspiration - used Ray for processing dicom files in parallel.</p>\n<p>Since this is demo only I defined two tasks:</p>\n<ul>\n<li>task1 - process dicom files in parallel (dicom2png files)</li>\n<li>task2 - do something with files from task1 (*.png files) in parallel (this is sample so I decided to count files and draw circle in center of the images)</li>\n</ul>\n<p>For communication purposes (between task) I used messages.</p>\n<p>Ray has a lot of possibilities to process data in ML pipeline.</p>\n<p>Notebook is here: <a href=\"https://www.kaggle.com/code/remekkinas/ray-parallel-processing-dicom-files-and/\" target=\"_blank\">https://www.kaggle.com/code/remekkinas/ray-parallel-processing-dicom-files-and/</a></p>\n<p>Hope somebody will use it as inspiration for RSNA solution. Happy kaggling!</p>",
  "messages": [
    {
      "id": 2058404,
      "postDate": "2022-12-07T22:58:13.590Z",
      "content": "<p>Looking for speed and multiprocessing? I created notebook for inspiration - used Ray for processing dicom files in parallel.</p>\n<p>Since this is demo only I defined two tasks:</p>\n<ul>\n<li>task1 - process dicom files in parallel (dicom2png files)</li>\n<li>task2 - do something with files from task1 (*.png files) in parallel (this is sample so I decided to count files and draw circle in center of the images)</li>\n</ul>\n<p>For communication purposes (between task) I used messages.</p>\n<p>Ray has a lot of possibilities to process data in ML pipeline.</p>\n<p>Notebook is here: <a href=\"https://www.kaggle.com/code/remekkinas/ray-parallel-processing-dicom-files-and/\" target=\"_blank\">https://www.kaggle.com/code/remekkinas/ray-parallel-processing-dicom-files-and/</a></p>\n<p>Hope somebody will use it as inspiration for RSNA solution. Happy kaggling!</p>",
      "rawMarkdown": "Looking for speed and multiprocessing? I created notebook for inspiration - used Ray for processing dicom files in parallel.\n\nSince this is demo only I defined two tasks:\n   - task1 - process dicom files in parallel (dicom2png files)\n   - task2 - do something with files from task1 (*.png files) in parallel (this is sample so I decided to count files and draw circle in center of the images)\n\nFor communication purposes (between task) I used messages.\n\nRay has a lot of possibilities to process data in ML pipeline.\n\nNotebook is here: https://www.kaggle.com/code/remekkinas/ray-parallel-processing-dicom-files-and/\n\nHope somebody will use it as inspiration for RSNA solution. Happy kaggling!\n",
      "votes": 11
    }
  ],
  "comments": [],
  "raw_markdown_by_id": {
    "2058404": "Looking for speed and multiprocessing? I created notebook for inspiration - used Ray for processing dicom files in parallel.\n\nSince this is demo only I defined two tasks:\n   - task1 - process dicom files in parallel (dicom2png files)\n   - task2 - do something with files from task1 (*.png files) in parallel (this is sample so I decided to count files and draw circle in center of the images)\n\nFor communication purposes (between task) I used messages.\n\nRay has a lot of possibilities to process data in ML pipeline.\n\nNotebook is here: https://www.kaggle.com/code/remekkinas/ray-parallel-processing-dicom-files-and/\n\nHope somebody will use it as inspiration for RSNA solution. Happy kaggling!\n"
  }
}