{
  "id": 185070,
  "title": "inference Time",
  "url": "/competitions/rsna-str-pulmonary-embolism-detection/discussion/185070",
  "author_name": "qzw",
  "post_date": "2020-09-19T09:10:17.973000",
  "votes": 5,
  "comment_count": 6,
  "views": 0,
  "content": "<p>hi all,</p>\n<p>Have your guys submitted successfully?<br>\nCould you share the inference time?</p>\n<p>thanks</p>",
  "messages": [
    {
      "id": 1017803,
      "postDate": "2020-09-19T09:10:17.973Z",
      "content": "<p>hi all,</p>\n<p>Have your guys submitted successfully?<br>\nCould you share the inference time?</p>\n<p>thanks</p>",
      "rawMarkdown": "hi all,\n\n\nHave your guys submitted successfully?\nCould you share the inference time?\n\nthanks",
      "votes": 5
    },
    {
      "id": 1053398,
      "postDate": "2020-10-19T00:37:36.617Z",
      "content": "<p>The RSNA competition this year is further to test the GPU resource and engineering ability of the players :)</p>",
      "rawMarkdown": "The RSNA competition this year is further to test the GPU resource and engineering ability of the players :)",
      "votes": 1
    },
    {
      "id": 1053275,
      "postDate": "2020-10-18T19:40:10.813Z",
      "content": "<p>Much harder to do blends in this competition due to inability to preprocess most of the test data and to do inference before submitting.</p>\n<p>Not sure if a 4-CPU notebook could do better with parallel models than a 2-CPU/1 GPU notebook.</p>",
      "rawMarkdown": "Much harder to do blends in this competition due to inability to preprocess most of the test data and to do inference before submitting.\n\nNot sure if a 4-CPU notebook could do better with parallel models than a 2-CPU/1 GPU notebook.",
      "votes": 1
    },
    {
      "id": 1053084,
      "postDate": "2020-10-18T15:21:34.923Z",
      "content": "<p>Hey,<br>\nSorry for this late question. I am trying to optimize my inference code. How much time is it taking to infer on a single model? I am using efficient nets and it took me about 2 hours in the public dataset and 7+ hours in private. Can this time be made lesser so as to have different blends? (maybe some simultaneous/parallel processing of CPUs and GPUs… I don't know much but I have seen people using blends in prev competitions). <br>\nThanks </p>",
      "rawMarkdown": "Hey,\nSorry for this late question. I am trying to optimize my inference code. How much time is it taking to infer on a single model? I am using efficient nets and it took me about 2 hours in the public dataset and 7+ hours in private. Can this time be made lesser so as to have different blends? (maybe some simultaneous/parallel processing of CPUs and GPUs... I don't know much but I have seen people using blends in prev competitions). \nThanks ",
      "votes": 1,
      "replies": [
        {
          "id": 1053396,
          "postDate": "2020-10-19T00:26:11.940Z",
          "content": "<p>How are you able to assess the private notebook run time? Do you base it on how long it takes for a LB score to appear?</p>",
          "rawMarkdown": "How are you able to assess the private notebook run time? Do you base it on how long it takes for a LB score to appear?",
          "votes": 1
        },
        {
          "id": 1053436,
          "postDate": "2020-10-19T02:21:52.247Z",
          "content": "<p>Yes, the score appears as soon as the notebook finishes its rerun.</p>",
          "rawMarkdown": "Yes, the score appears as soon as the notebook finishes its rerun.",
          "votes": 1
        }
      ]
    },
    {
      "id": 1053450,
      "postDate": "2020-10-19T02:42:37.637Z",
      "content": "<p>What I am doing so far is pretty naive. <br>\nRead images one by one in CPU --&gt; form batches once we read say 500 images --&gt; feed batch to model in GPU.<br>\nThe first 2 parts take about 10 times more than the GPU time(single batch GPU 2s total time 20s). So the reading part is the bottleneck, and I think parallelizing the tasks in CPUs could save a lot of time. But I have no idea how to do it. Can it be done with joblib and multiprocessing libraries in python? Or are there any other libraries/ resources I could look on.<br>\nThanks</p>",
      "rawMarkdown": "What I am doing so far is pretty naive. \nRead images one by one in CPU --> form batches once we read say 500 images --> feed batch to model in GPU.\nThe first 2 parts take about 10 times more than the GPU time(single batch GPU 2s total time 20s). So the reading part is the bottleneck, and I think parallelizing the tasks in CPUs could save a lot of time. But I have no idea how to do it. Can it be done with joblib and multiprocessing libraries in python? Or are there any other libraries/ resources I could look on.\nThanks"
    }
  ],
  "comments": [
    {
      "id": 1053398,
      "author_name": "Paul Chen",
      "author_url": "",
      "post_date": "2020-10-19T00:37:36.617000",
      "content": "<p>The RSNA competition this year is further to test the GPU resource and engineering ability of the players :)</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1053275,
      "author_name": "quadcore/Richard Epstein",
      "author_url": "",
      "post_date": "2020-10-18T19:40:10.813000",
      "content": "<p>Much harder to do blends in this competition due to inability to preprocess most of the test data and to do inference before submitting.</p>\n<p>Not sure if a 4-CPU notebook could do better with parallel models than a 2-CPU/1 GPU notebook.</p>",
      "votes": 1,
      "replies": []
    },
    {
      "id": 1053084,
      "author_name": "Jose",
      "author_url": "",
      "post_date": "2020-10-18T15:21:34.923000",
      "content": "<p>Hey,<br>\nSorry for this late question. I am trying to optimize my inference code. How much time is it taking to infer on a single model? I am using efficient nets and it took me about 2 hours in the public dataset and 7+ hours in private. Can this time be made lesser so as to have different blends? (maybe some simultaneous/parallel processing of CPUs and GPUs… I don't know much but I have seen people using blends in prev competitions). <br>\nThanks </p>",
      "votes": 1,
      "replies": [
        {
          "id": 1053396,
          "author_name": "Philip Dhingra",
          "author_url": "",
          "post_date": "2020-10-19T00:26:11.940000",
          "content": "<p>How are you able to assess the private notebook run time? Do you base it on how long it takes for a LB score to appear?</p>",
          "votes": 1,
          "replies": []
        },
        {
          "id": 1053436,
          "author_name": "Jose",
          "author_url": "",
          "post_date": "2020-10-19T02:21:52.247000",
          "content": "<p>Yes, the score appears as soon as the notebook finishes its rerun.</p>",
          "votes": 1,
          "replies": []
        }
      ]
    },
    {
      "id": 1053450,
      "author_name": "Jose",
      "author_url": "",
      "post_date": "2020-10-19T02:42:37.637000",
      "content": "<p>What I am doing so far is pretty naive. <br>\nRead images one by one in CPU --&gt; form batches once we read say 500 images --&gt; feed batch to model in GPU.<br>\nThe first 2 parts take about 10 times more than the GPU time(single batch GPU 2s total time 20s). So the reading part is the bottleneck, and I think parallelizing the tasks in CPUs could save a lot of time. But I have no idea how to do it. Can it be done with joblib and multiprocessing libraries in python? Or are there any other libraries/ resources I could look on.<br>\nThanks</p>",
      "votes": 0,
      "replies": []
    }
  ],
  "raw_markdown_by_id": {
    "1017803": "hi all,\n\n\nHave your guys submitted successfully?\nCould you share the inference time?\n\nthanks",
    "1053398": "The RSNA competition this year is further to test the GPU resource and engineering ability of the players :)",
    "1053275": "Much harder to do blends in this competition due to inability to preprocess most of the test data and to do inference before submitting.\n\nNot sure if a 4-CPU notebook could do better with parallel models than a 2-CPU/1 GPU notebook.",
    "1053084": "Hey,\nSorry for this late question. I am trying to optimize my inference code. How much time is it taking to infer on a single model? I am using efficient nets and it took me about 2 hours in the public dataset and 7+ hours in private. Can this time be made lesser so as to have different blends? (maybe some simultaneous/parallel processing of CPUs and GPUs... I don't know much but I have seen people using blends in prev competitions). \nThanks ",
    "1053450": "What I am doing so far is pretty naive. \nRead images one by one in CPU --> form batches once we read say 500 images --> feed batch to model in GPU.\nThe first 2 parts take about 10 times more than the GPU time(single batch GPU 2s total time 20s). So the reading part is the bottleneck, and I think parallelizing the tasks in CPUs could save a lot of time. But I have no idea how to do it. Can it be done with joblib and multiprocessing libraries in python? Or are there any other libraries/ resources I could look on.\nThanks"
  }
}