{
  "id": 465491,
  "title": "Your Approach to the \"Other\" Class?",
  "url": "/competitions/UBC-OCEAN/discussion/465491",
  "author_name": "Belcebub",
  "post_date": "2024-01-04T13:33:15.770000",
  "votes": 1,
  "comment_count": 5,
  "views": 0,
  "content": "<p>Hey everybody,</p>\n<p>first off, thanks to everybody participating and sharing their thoughts &amp; ideas during this competition! It was truly a lot of fun for me :)<br>\nI did not yet read through all of the top rankers explanations, but I wanted to get an general idea from all of you, how you tried to solve the \"Other\" class problem. And if you have any idea what was the most successful technique therefore. </p>\n<p>Myself, I actually tried a lot of things but I don't think any was really successful. <br>\nNext to soft thresholding, my first approach was to extract the latent features from the main subtype classification model and train a One-Class SVM and/or an Isolation Forest model on those latent features in order to detect outliers in the latent space.<br>\nAs this was not improving my results, I tried to train a convolutional autoencoder of the image tiles. The idea behind that was that an unseen class / abnormal tissue would result in a higher reconstruction error than already seen types of cancerous tissue. Anyways, I did not have enough time to finish / optimize that approach. <br>\nEvaluation was a big problem of course… </p>\n<p>What about you? Anyone with some promising results? Would love to hear about some ideas!  </p>",
  "messages": [
    {
      "id": 2587612,
      "postDate": "2024-01-04T21:50:28.803Z",
      "content": "<p>Personally I tried 3 approaches and none worked for me:</p>\n<ol>\n<li>Trained AutoEncoders on all data, in inference time used this model to identify outliers.  (did not worked)</li>\n<li>Optimize my classification model with Softmax head to find a threshold below that assign 'Other' as label.  (did not worked)</li>\n<li>Trained my classification model with Sigmoid to thresholding.  (did not worked)</li>\n</ol>\n<p>based on what I read from posted approaches up to now, thresholding was mostly used. And the most effective approach of outlier detection was by \"10th place solution\", they basically found external data with classes beyond the competition's five categories to improve outlier detection.</p>",
      "rawMarkdown": "Personally I tried 3 approaches and none worked for me:\n1. Trained AutoEncoders on all data, in inference time used this model to identify outliers.  (did not worked)\n2. Optimize my classification model with Softmax head to find a threshold below that assign 'Other' as label.  (did not worked)\n3. Trained my classification model with Sigmoid to thresholding.  (did not worked)\n\nbased on what I read from posted approaches up to now, thresholding was mostly used. And the most effective approach of outlier detection was by \"10th place solution\", they basically found external data with classes beyond the competition's five categories to improve outlier detection.",
      "votes": 1,
      "replies": [
        {
          "id": 2588647,
          "postDate": "2024-01-05T16:18:30.473Z",
          "content": "<p>U have any idea how well our autoencoder approach worked? how much could you gain on the leaderboard by this? </p>",
          "rawMarkdown": "U have any idea how well our autoencoder approach worked? how much could you gain on the leaderboard by this? ",
          "replies": [
            {
              "id": 2589007,
              "postDate": "2024-01-06T00:40:57.997Z",
              "content": "<p>maybe I miss-phrased, but none of my methods helped to identify 'other' category</p>",
              "rawMarkdown": "maybe I miss-phrased, but none of my methods helped to identify 'other' category"
            }
          ]
        }
      ]
    },
    {
      "id": 2587002,
      "postDate": "2024-01-04T14:07:12.480Z",
      "content": "<p>I also tried using traditional OCC strategies such as OneClassSVM, IsolatedForest, and LocalOutlierFactor to the feature obtained from the MIL aggregator…but none of them is useful. My team mate also applied them on patch features, along with some energe-based OOD methods, and all of them are useless. Finally, only combing normal and necrosis tissues to synthesize fake Others shows performance increasement. But I think this approach cannot really classify the rare subtypes.</p>",
      "rawMarkdown": "I also tried using traditional OCC strategies such as OneClassSVM, IsolatedForest, and LocalOutlierFactor to the feature obtained from the MIL aggregator…but none of them is useful. My team mate also applied them on patch features, along with some energe-based OOD methods, and all of them are useless. Finally, only combing normal and necrosis tissues to synthesize fake Others shows performance increasement. But I think this approach cannot really classify the rare subtypes.",
      "votes": 1,
      "replies": [
        {
          "id": 2588642,
          "postDate": "2024-01-05T16:17:22.327Z",
          "content": "<p>Synthesizing fake outliers is a very interesting idea! But hard to realize in a way which helps this specific use case i guess. </p>",
          "rawMarkdown": "Synthesizing fake outliers is a very interesting idea! But hard to realize in a way which helps this specific use case i guess. "
        }
      ]
    },
    {
      "id": 2586922,
      "postDate": "2024-01-04T13:33:15.770Z",
      "content": "<p>Hey everybody,</p>\n<p>first off, thanks to everybody participating and sharing their thoughts &amp; ideas during this competition! It was truly a lot of fun for me :)<br>\nI did not yet read through all of the top rankers explanations, but I wanted to get an general idea from all of you, how you tried to solve the \"Other\" class problem. And if you have any idea what was the most successful technique therefore. </p>\n<p>Myself, I actually tried a lot of things but I don't think any was really successful. <br>\nNext to soft thresholding, my first approach was to extract the latent features from the main subtype classification model and train a One-Class SVM and/or an Isolation Forest model on those latent features in order to detect outliers in the latent space.<br>\nAs this was not improving my results, I tried to train a convolutional autoencoder of the image tiles. The idea behind that was that an unseen class / abnormal tissue would result in a higher reconstruction error than already seen types of cancerous tissue. Anyways, I did not have enough time to finish / optimize that approach. <br>\nEvaluation was a big problem of course… </p>\n<p>What about you? Anyone with some promising results? Would love to hear about some ideas!  </p>",
      "rawMarkdown": "Hey everybody,\n\nfirst off, thanks to everybody participating and sharing their thoughts & ideas during this competition! It was truly a lot of fun for me :)\nI did not yet read through all of the top rankers explanations, but I wanted to get an general idea from all of you, how you tried to solve the \"Other\" class problem. And if you have any idea what was the most successful technique therefore. \n\nMyself, I actually tried a lot of things but I don't think any was really successful. \nNext to soft thresholding, my first approach was to extract the latent features from the main subtype classification model and train a One-Class SVM and/or an Isolation Forest model on those latent features in order to detect outliers in the latent space.\nAs this was not improving my results, I tried to train a convolutional autoencoder of the image tiles. The idea behind that was that an unseen class / abnormal tissue would result in a higher reconstruction error than already seen types of cancerous tissue. Anyways, I did not have enough time to finish / optimize that approach. \nEvaluation was a big problem of course... \n\nWhat about you? Anyone with some promising results? Would love to hear about some ideas!  ",
      "votes": 1
    }
  ],
  "comments": [
    {
      "id": 2587612,
      "author_name": "Ali",
      "author_url": "",
      "post_date": "2024-01-04T21:50:28.803000",
      "content": "<p>Personally I tried 3 approaches and none worked for me:</p>\n<ol>\n<li>Trained AutoEncoders on all data, in inference time used this model to identify outliers.  (did not worked)</li>\n<li>Optimize my classification model with Softmax head to find a threshold below that assign 'Other' as label.  (did not worked)</li>\n<li>Trained my classification model with Sigmoid to thresholding.  (did not worked)</li>\n</ol>\n<p>based on what I read from posted approaches up to now, thresholding was mostly used. And the most effective approach of outlier detection was by \"10th place solution\", they basically found external data with classes beyond the competition's five categories to improve outlier detection.</p>",
      "votes": 1,
      "replies": [
        {
          "id": 2588647,
          "author_name": "Belcebub",
          "author_url": "",
          "post_date": "2024-01-05T16:18:30.473000",
          "content": "<p>U have any idea how well our autoencoder approach worked? how much could you gain on the leaderboard by this? </p>",
          "votes": 0,
          "replies": [
            {
              "id": 2589007,
              "author_name": "Ali",
              "author_url": "",
              "post_date": "2024-01-06T00:40:57.997000",
              "content": "<p>maybe I miss-phrased, but none of my methods helped to identify 'other' category</p>",
              "votes": 0,
              "replies": []
            }
          ]
        }
      ]
    },
    {
      "id": 2587002,
      "author_name": "Zijie Fang",
      "author_url": "",
      "post_date": "2024-01-04T14:07:12.480000",
      "content": "<p>I also tried using traditional OCC strategies such as OneClassSVM, IsolatedForest, and LocalOutlierFactor to the feature obtained from the MIL aggregator…but none of them is useful. My team mate also applied them on patch features, along with some energe-based OOD methods, and all of them are useless. Finally, only combing normal and necrosis tissues to synthesize fake Others shows performance increasement. But I think this approach cannot really classify the rare subtypes.</p>",
      "votes": 1,
      "replies": [
        {
          "id": 2588642,
          "author_name": "Belcebub",
          "author_url": "",
          "post_date": "2024-01-05T16:17:22.327000",
          "content": "<p>Synthesizing fake outliers is a very interesting idea! But hard to realize in a way which helps this specific use case i guess. </p>",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2587612": "Personally I tried 3 approaches and none worked for me:\n1. Trained AutoEncoders on all data, in inference time used this model to identify outliers.  (did not worked)\n2. Optimize my classification model with Softmax head to find a threshold below that assign 'Other' as label.  (did not worked)\n3. Trained my classification model with Sigmoid to thresholding.  (did not worked)\n\nbased on what I read from posted approaches up to now, thresholding was mostly used. And the most effective approach of outlier detection was by \"10th place solution\", they basically found external data with classes beyond the competition's five categories to improve outlier detection.",
    "2587002": "I also tried using traditional OCC strategies such as OneClassSVM, IsolatedForest, and LocalOutlierFactor to the feature obtained from the MIL aggregator…but none of them is useful. My team mate also applied them on patch features, along with some energe-based OOD methods, and all of them are useless. Finally, only combing normal and necrosis tissues to synthesize fake Others shows performance increasement. But I think this approach cannot really classify the rare subtypes.",
    "2586922": "Hey everybody,\n\nfirst off, thanks to everybody participating and sharing their thoughts & ideas during this competition! It was truly a lot of fun for me :)\nI did not yet read through all of the top rankers explanations, but I wanted to get an general idea from all of you, how you tried to solve the \"Other\" class problem. And if you have any idea what was the most successful technique therefore. \n\nMyself, I actually tried a lot of things but I don't think any was really successful. \nNext to soft thresholding, my first approach was to extract the latent features from the main subtype classification model and train a One-Class SVM and/or an Isolation Forest model on those latent features in order to detect outliers in the latent space.\nAs this was not improving my results, I tried to train a convolutional autoencoder of the image tiles. The idea behind that was that an unseen class / abnormal tissue would result in a higher reconstruction error than already seen types of cancerous tissue. Anyways, I did not have enough time to finish / optimize that approach. \nEvaluation was a big problem of course... \n\nWhat about you? Anyone with some promising results? Would love to hear about some ideas!  "
  }
}