{
  "id": 425580,
  "title": "Does your transformer encoders work well for this competition?",
  "url": "/competitions/google-research-identify-contrails-reduce-global-warming/discussion/425580",
  "author_name": "william.wu",
  "post_date": "2023-07-19T12:25:07.897000",
  "votes": 6,
  "comment_count": 8,
  "views": 0,
  "content": "<p>I tried some transformer encoders like <code>maxvit_large_tf_384.in21k_ft_in1k</code>, but none of them beat <code>resnet</code> and <code>efficientnet</code>. How about yours?</p>",
  "messages": [
    {
      "id": 2350679,
      "postDate": "2023-07-19T12:25:07.897Z",
      "content": "<p>I tried some transformer encoders like <code>maxvit_large_tf_384.in21k_ft_in1k</code>, but none of them beat <code>resnet</code> and <code>efficientnet</code>. How about yours?</p>",
      "rawMarkdown": "I tried some transformer encoders like `maxvit_large_tf_384.in21k_ft_in1k`, but none of them beat `resnet` and `efficientnet`. How about yours?",
      "votes": 5
    },
    {
      "id": 2353093,
      "postDate": "2023-07-21T12:45:03.880Z",
      "content": "<p>Coat and Maxvit got good results in my experiment.</p>",
      "rawMarkdown": "Coat and Maxvit got good results in my experiment.",
      "replies": [
        {
          "id": 2364532,
          "postDate": "2023-07-29T12:26:02.750Z",
          "content": "<p>Thanks <a href=\"https://www.kaggle.com/ynhuhu\" target=\"_blank\">@ynhuhu</a> , only efficient net and resnet work for me.</p>",
          "rawMarkdown": "Thanks @ynhuhu , only efficient net and resnet work for me."
        }
      ]
    },
    {
      "id": 2350958,
      "postDate": "2023-07-19T17:03:34.247Z",
      "content": "<p>timm-resnest26d is the best in my experiment</p>",
      "rawMarkdown": " timm-resnest26d is the best in my experiment",
      "replies": [
        {
          "id": 2351712,
          "postDate": "2023-07-20T10:44:30.433Z",
          "content": "<p>Thanks for sharing. For me it's efficientnet-7b</p>",
          "rawMarkdown": "Thanks for sharing. For me it's efficientnet-7b"
        },
        {
          "id": 2351978,
          "postDate": "2023-07-20T14:29:01.230Z",
          "content": "<p>may i ask if you used a larger resolution for the input image or you used the original one for the resnest-26<br>\n?</p>",
          "rawMarkdown": "may i ask if you used a larger resolution for the input image or you used the original one for the resnest-26\n?",
          "replies": [
            {
              "id": 2352746,
              "postDate": "2023-07-21T08:25:48.583Z",
              "content": "<p>yup resnest and efficientnet series . Not sure anyone tried segformer here ?</p>",
              "rawMarkdown": "yup resnest and efficientnet series . Not sure anyone tried segformer here ?",
              "votes": 1
            },
            {
              "id": 2352825,
              "postDate": "2023-07-21T09:35:43.207Z",
              "content": "<p>I have tried segformer m5 for some initial experiments and it has shown relatively good performance, but I gave it up because in my understanding it is not allowed to use code or pre-trained weights from nvidia for this models due to non-commercial <a href=\"https://github.com/NVlabs/SegFormer/blob/master/LICENSE\" target=\"_blank\">license</a>. And it is required by competition rules to open-source the solution under MIT license. </p>\n<p>But as I see we could use some other code open-source implementation e. g. <a href=\"https://huggingface.co/docs/transformers/main/model_doc/segformer\" target=\"_blank\">HF</a> and train from scratch, I have not tried this though.</p>\n<p>Maybe my understanding is not correct here, <a href=\"https://www.kaggle.com/maggiemd\" target=\"_blank\">@maggiemd</a> <a href=\"https://www.kaggle.com/inversion\" target=\"_blank\">@inversion</a> <a href=\"https://www.kaggle.com/joeyhng\" target=\"_blank\">@joeyhng</a> could you please clarify if segformer is allowed?</p>",
              "rawMarkdown": "I have tried segformer m5 for some initial experiments and it has shown relatively good performance, but I gave it up because in my understanding it is not allowed to use code or pre-trained weights from nvidia for this models due to non-commercial [license](https://github.com/NVlabs/SegFormer/blob/master/LICENSE). And it is required by competition rules to open-source the solution under MIT license. \n\nBut as I see we could use some other code open-source implementation e. g. [HF](https://huggingface.co/docs/transformers/main/model_doc/segformer) and train from scratch, I have not tried this though.\n\nMaybe my understanding is not correct here, @maggiemd @inversion @joeyhng could you please clarify if segformer is allowed?",
              "votes": 1
            },
            {
              "id": 2352898,
              "postDate": "2023-07-21T10:29:53.667Z",
              "content": "<p>Didn’t people use it in the Vesuvius comp ?</p>",
              "rawMarkdown": "Didn’t people use it in the Vesuvius comp ?"
            }
          ]
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 2353093,
      "author_name": "ynhuhu",
      "author_url": "",
      "post_date": "2023-07-21T12:45:03.880000",
      "content": "<p>Coat and Maxvit got good results in my experiment.</p>",
      "votes": 0,
      "replies": [
        {
          "id": 2364532,
          "author_name": "william.wu",
          "author_url": "",
          "post_date": "2023-07-29T12:26:02.750000",
          "content": "<p>Thanks <a href=\"https://www.kaggle.com/ynhuhu\" target=\"_blank\">@ynhuhu</a> , only efficient net and resnet work for me.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 2350958,
      "author_name": "E-Max AI",
      "author_url": "",
      "post_date": "2023-07-19T17:03:34.247000",
      "content": "<p>timm-resnest26d is the best in my experiment</p>",
      "votes": 0,
      "replies": [
        {
          "id": 2351712,
          "author_name": "william.wu",
          "author_url": "",
          "post_date": "2023-07-20T10:44:30.433000",
          "content": "<p>Thanks for sharing. For me it's efficientnet-7b</p>",
          "votes": 0,
          "replies": []
        },
        {
          "id": 2351978,
          "author_name": "Merkvp",
          "author_url": "",
          "post_date": "2023-07-20T14:29:01.230000",
          "content": "<p>may i ask if you used a larger resolution for the input image or you used the original one for the resnest-26<br>\n?</p>",
          "votes": 0,
          "replies": [
            {
              "id": 2352746,
              "author_name": "Gaurav Rawat",
              "author_url": "",
              "post_date": "2023-07-21T08:25:48.583000",
              "content": "<p>yup resnest and efficientnet series . Not sure anyone tried segformer here ?</p>",
              "votes": 1,
              "replies": []
            },
            {
              "id": 2352825,
              "author_name": "Mikhail Kotyushev",
              "author_url": "",
              "post_date": "2023-07-21T09:35:43.207000",
              "content": "<p>I have tried segformer m5 for some initial experiments and it has shown relatively good performance, but I gave it up because in my understanding it is not allowed to use code or pre-trained weights from nvidia for this models due to non-commercial <a href=\"https://github.com/NVlabs/SegFormer/blob/master/LICENSE\" target=\"_blank\">license</a>. And it is required by competition rules to open-source the solution under MIT license. </p>\n<p>But as I see we could use some other code open-source implementation e. g. <a href=\"https://huggingface.co/docs/transformers/main/model_doc/segformer\" target=\"_blank\">HF</a> and train from scratch, I have not tried this though.</p>\n<p>Maybe my understanding is not correct here, <a href=\"https://www.kaggle.com/maggiemd\" target=\"_blank\">@maggiemd</a> <a href=\"https://www.kaggle.com/inversion\" target=\"_blank\">@inversion</a> <a href=\"https://www.kaggle.com/joeyhng\" target=\"_blank\">@joeyhng</a> could you please clarify if segformer is allowed?</p>",
              "votes": 1,
              "replies": []
            },
            {
              "id": 2352898,
              "author_name": "Gaurav Rawat",
              "author_url": "",
              "post_date": "2023-07-21T10:29:53.667000",
              "content": "<p>Didn’t people use it in the Vesuvius comp ?</p>",
              "votes": 0,
              "replies": []
            }
          ]
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2350679": "I tried some transformer encoders like `maxvit_large_tf_384.in21k_ft_in1k`, but none of them beat `resnet` and `efficientnet`. How about yours?",
    "2353093": "Coat and Maxvit got good results in my experiment.",
    "2350958": " timm-resnest26d is the best in my experiment"
  }
}