![\"CLAM](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F408221%2Ff3052ba2667ca9ef1ae1459c52026861%2FScreenshot%202024-01-04%20at%2018.46.17.png?generation=1704390874024392&alt=media\")
Business context: UBC Ovarian Cancer Subtype Classification and Outlier Detection
\nData context: [The] challenge in this competition is to classify the type of ovarian cancer from microscopy scans of biopsy samples. Link to data description
\nFinding more public external data was key for me. Overfitting was a big problem due to the small number of examples. Initially I hoped the CLAM or a multiple instance learning (MIL) approach could remedy that, because many images are so large they can be split in tens of thousands of tiles. But my models were still overfitting a lot. I assume the tiles of the same patient are similar in important ways and the models can use that as shortcuts that don't generalize well. Or there is just so much heterogeniety between cancers, so that the number of samples are just not enough to capture all different variants of the subtypes.
I used the segmentation data provided to create synthetic tumor micro array (TMA) images, basically jsut cropping tiny images from the segmentation of the large image. I did this for the cancer tissue and generated some \"Other\" synthetic images by cropping small tiles that were marked as healthy or as stroma.
Following the paper \"A Good Feature Extractor Is All You Need for Weakly Supervised Learning in Histopathology\" I used the pretrained model Lunit-DINO to extract smaller size features to handle the massive image size. I ran the feature extractino in 16-bit so that it runs faster. I didn't see much negative impact on feature quality.
I filtered the tiles containing tissue using the thumbnails and then cropped the tissue tiles using PyVips. I lost so much time on this one, because I first tried to re-write the feature extraction code in CLAM, but couldn't make it work with the resource limits on Kaggle. Then I tried using the large_image library that is made for histopatholy images, but could not make it work with the Kaggle resource limits. I alternated between out of memory, out of disk and timeouts. Finally I made it work using PyVips and asynchronous data loading in PyTorch. This part of the competition was frustrating, as I wanted to spend my time on deep learning and spend weeks on image processing and guessing Kaggle errors.
On the extracted features I trained the CLAM model, which is simiar to MIL, but calculates an attention matrix to weight the tiles. I made some changes to the instance level loss function for the \"Other\" label, as it's sematically different from the other labels. A tile of a slide containing a cancer subtype could still be have the label \"Other\", if the tile would only show healthy tissue.
The following diagram shows the CLAM model from Mahmood Lab @ Harvard/BWH & MGH [1]. This model takes as input concatenated features that have been extracted from all tiles of a Whole Slide Image containing tissue.
\nThe top part of the diagram calculates the attention scores A, a vector with one entry per tile. The bottom part creates an A-weighted sum of the transformed input features h and feeds it into a multi-class classification head.
\n
Diagram from Paul Pham [2]
\nPyTorch Code for my adapted CLAM model:
\n (nn.Module):\n ():\n (Attn_Net_Gated, self).__init__()\n self.attention_a = [\n nn.Linear(L, D),\n nn.Tanh()\n ]\n self.attention_b = [\n nn.Linear(L, D),\n nn.Sigmoid()\n ]\n dropout > :\n self.attention_a.append(nn.Dropout(dropout))\n self.attention_b.append(nn.Dropout(dropout))\n\n self.attention_a = nn.Sequential(*self.attention_a)\n self.attention_b = nn.Sequential(*self.attention_b)\n\n self.attention_c = nn.Linear(D, n_classes)\n\n ():\n a = self.attention_a(x)\n b = self.attention_b(x)\n A = a.mul(b)\n A = self.attention_c(A) \n A, x\n\n\n (nn.Module):\n ():\n ().__init__()\n self.size_dict = {\n : [feature_dim, , ],\n : [feature_dim, , ],\n : [feature_dim, , ],\n : [feature_dim, , ]\n }\n size = self.size_dict[size_arg]\n fc = [nn.Linear(size[], size[]), nn.ReLU()]\n dropout > :\n fc.append(nn.Dropout(dropout))\n gate:\n attention_net = Attn_Net_Gated(L = size[], D = size[], dropout = dropout, n_classes = )\n :\n attention_net = Attn_Net(L = size[], D = size[], dropout = dropout, n_classes = )\n fc.append(attention_net)\n self.attention_net = nn.Sequential(*fc)\n self.classifiers = nn.Linear(size[], n_classes)\n instance_classifiers = [] \n class_idx (n_classes):\n layers = []\n depth_idx (inst_class_depth-):\n divisor = ** depth_idx \n layers.append(nn.Linear(size[] // divisor, size[] // (divisor * )))\n layers.append(nn.ReLU())\n inst_dropout :\n layers.append(nn.Dropout(inst_dropout))\n layers.append(nn.Linear(size[] // **(inst_class_depth-), ))\n instance_classifiers.append(nn.Sequential(*layers)) \n self.instance_classifiers = nn.ModuleList(instance_classifiers)\n self.k_sample = k_sample\n self.instance_loss_fn = instance_loss_fn\n self.n_classes = n_classes\n self.subtyping = subtyping\n self.use_inst_predictions = use_inst_predictions\n self.other_idx = label_mapping[]\n self.class_weights = class_weights\n initialize_weights(self)\n self.to()\n\n\n ():\n torch.full((length, ), , device=device).()\n\n ():\n torch.full((length, ), , device=device).()\n\n \n (): \n device=h.device\n (A.shape) == :\n A = A.view(, -)\n\n is_tma:\n k_sample = self.k_sample // \n :\n k_sample = self.k_sample\n\n k_sample <= math.ceil(A.shape[] / ):\n top_p_ids = torch.topk(A, k_sample)[][-] \n :\n top_p_ids = torch.topk(A, math.ceil(A.shape[] / ))[][-]\n top_p_ids = top_p_ids.repeat(k_sample)[:k_sample]\n top_p = torch.index_select(h, dim=, index=top_p_ids) \n k_sample <= math.ceil(A.shape[] / ):\n top_n_ids = torch.topk(-A, k_sample, dim=)[][-]\n :\n top_n_ids = torch.topk(-A, math.ceil(A.shape[] / ))[][-]\n top_n_ids = top_n_ids.repeat(k_sample)[:k_sample]\n top_n = torch.index_select(h, dim=, index=top_n_ids)\n p_targets = self.create_positive_targets(k_sample, device)\n n_targets = self.create_negative_targets(k_sample, device)\n\n \n \n p_logits = classifier(top_p) \n n_logits = classifier(top_n)\n inst_preds = (p_logits.squeeze() > ).long()\n \n p_loss = self.instance_loss_fn(p_logits.squeeze(), p_targets) * (self.n_classes -)\n n_loss = self.instance_loss_fn(n_logits.squeeze(), n_targets)\n is_tma is_other_class:\n loss = p_loss + n_loss\n : loss = p_loss\n loss, inst_preds, p_targets, p_logits\n\n \n ():\n device=h.device\n (A.shape) == :\n A = A.view(, -)\n\n is_tma:\n k_sample = self.k_sample // \n :\n k_sample = self.k_sample\n\n \n \n k_sample <= math.ceil(A.shape[] / ):\n top_ids = torch.topk(A, k_sample)[][-]\n :\n top_ids = torch.topk(A, math.ceil(A.shape[] / ))[][-]\n top_ids = top_ids.repeat(k_sample)[:k_sample]\n top_inst = torch.index_select(h, dim=, index=top_ids)\n top_targets = self.create_negative_targets(k_sample, device)\n\n\n logits = classifier(top_inst)\n inst_preds = (logits.squeeze() > ).long()\n instance_loss = self.instance_loss_fn(logits.squeeze(), top_targets)\n instance_loss, inst_preds, top_targets, logits\n\n\n ():\n A, h = self.attention_net(h) \n A = torch.transpose(A, , ) \n attention_only:\n A\n A_raw = A\n A = F.softmax(A, dim=) \n M = torch.mm(A, h) \n logits = self.classifiers(M)\n bag_Y_prob = F.softmax(logits.squeeze(), dim=)\n\n is_tma:\n k_sample = self.k_sample // \n :\n k_sample = self.k_sample\n\n all_inst_logits = []\n top_p_ids = \n bag_pred_weight < label : \n total_inst_loss = \n all_inst_preds = []\n all_targets = []\n i ((self.instance_classifiers)):\n classifier = self.instance_classifiers[i]\n i == label.item(): \n is_other_class = (label.item() == self.other_idx)\n instance_loss, inst_preds, targets, inst_logits = self.inst_eval(A, h, classifier, is_tma, is_other_class)\n all_inst_preds.extend(inst_preds.cpu().numpy()) \n all_targets.extend(targets.cpu().numpy())\n \n all_inst_logits.append(inst_logits)\n self.class_weights :\n instance_loss *= self.class_weights[i]\n : \n self.subtyping:\n instance_loss, inst_preds, targets, inst_logits = self.inst_eval_out(A, h, classifier, is_tma)\n all_inst_preds.extend(inst_preds.cpu().numpy())\n all_targets.extend(targets.cpu().numpy()) \n all_inst_logits.append(inst_logits)\n :\n \n\n total_inst_loss += instance_loss \n\n self.subtyping:\n \n total_inst_loss /= * (self.instance_classifiers)\n :\n self.k_sample <= math.ceil(A.shape[] / ):\n top_p_ids = torch.topk(A, k_sample)[][-] \n :\n top_p_ids = torch.topk(A, math.ceil(A.shape[] / ))[][-]\n top_p_ids = top_p_ids.repeat(k_sample)[:k_sample]\n top_p = torch.index_select(h, dim=, index=top_p_ids)\n classifier self.instance_classifiers:\n class_logits = classifier(top_p)\n all_inst_logits.append(class_logits)\n\n\n self.use_inst_predictions: \n all_inst_logits = torch.concatenate(all_inst_logits, axis=) \n \n \n self.k_sample <= math.ceil(A.shape[] / ):\n top_p_ids = torch.topk(A, k_sample)[][-] \n :\n top_p_ids = torch.topk(A, math.ceil(A.shape[] / ))[][-]\n top_p_ids = top_p_ids.repeat(k_sample)[:k_sample]\n\n all_inst_logits =A_raw[, top_p_ids].reshape(-, ) * all_inst_logits\n softmax_inst_probs = torch.softmax(all_inst_logits, dim=)\n agg_inst_probs = softmax_inst_probs \n agg_inst_probs = torch.mean(agg_inst_probs, dim=) \n Y_probs = bag_Y_prob * bag_pred_weight + agg_inst_probs * ( - bag_pred_weight)\n Y_hat = torch.topk(Y_probs, , dim=)[]\n\n results_dict = {}\n bag_pred_weight < :\n results_dict.update({\n : all_inst_logits.detach().cpu().numpy(),\n : agg_inst_probs.detach().cpu().numpy()\n })\n self.use_inst_predictions: \n results_dict.update({\n : softmax_inst_probs.detach().cpu().numpy()\n })\n label :\n results_dict.update({\n : np.array(all_targets),\n : np.array(all_inst_preds).flatten(),\n : total_inst_loss\n })\n\n logits, Y_probs, Y_hat, A_raw, results_dict\nFrom the Cancer Imaging Archive I used the Ovarian Bevacizumab Response and the CPTAC-OV data. The labels didn't map perfectly to the labels of the competition. For example a label would be Papillary Serous Carcinoma, which I assumed means either HGSC or LGSC. I just used a model trained on the my other data to decide which of the two to select.
\nI used the Ovarian Carcinoma Histopathology Dataset by the Hamarneh Lab.
\nI also used data from the Stanford Tissue Microarray Database, most of which didn't have H&E staining, but at least the cell shapes looked similar and I wanted to also have some TMA data.
\nI also got in touch with Cooperative Human Tissue network, who where so nice to allow me to use the data that was publicly available on the website. I thank them for the time they took to discuss this topic, even though I didn't use their data in the end, as they took the data down from their website after my outreach. As the data wasn't available for other participants anymore, I assume I'm not allowed to use it to train my model.
\nFor a long time I pooled all my data and used 5-fold cross validation, making sure multiple images from the same patient would all be in the same fold. Still this lead to inflated validation scores. I seems like my models where overfitting by using some shortcuts from my datasets that do not generalize. Later I excluded the data from the Harmanreh lab completely for validation which lead to much more reliable cross-validation scores.
\nAs the data size of the competition was so large I trained my models locally on my desktop computer with a GTX 4090 card. Feature extraction would take around 6 hours for all my datasets and training my model would take another hour.
\nA big thank you to everyone who made their pathological image data publicly available. This is a tremendous help for anyone who wants to build deep learning models to improve digital pathology, but especially for individuals without access to close source clinical data.
\nBevacizumab: Wang et al. Weakly Supervised Deep Learning for Prediction of Treatment Effectiveness on Ovarian Cancer from Histopathology Images. Computerized Medical Imaging and Graphics. https://doi.org/10.1016/j.compmedimag.2022.102093
\nCPTAC-OV: National Cancer Institute Clinical Proteomic Tumor Analysis Consortium (CPTAC). (2020). The Clinical Proteomic Tumor Analysis Consortium Ovarian Serous Cystadenocarcinoma Collection (CPTAC-OV) (Version 3) [Data set]. The Cancer Imaging Archive. https://doi.org/10.7937/TCIA.ZS4A-JD58
\nHarmanreh Data: Köbel, Martin; Kalloger, Steve E.; Baker, Patricia M.; Ewanowich, Carol A.; Arseneau, Jocelyne; Zherebitskiy, Viktor; Abdulkarim, Soran; Leung, Samuel; Duggan, Máire A.; Fontaine, Dan; et al. (2010). \"Diagnosis of ovarian carcinoma cell type is highly reproducible: a transcanadian study\". The American Journal of Surgical Pathology, 34(7), 984–993. LWW.
\nI used some screenshots for healthy tissue from The Human Protein Atlas and the University of Michigan Histology and Cirtual Microscopy page. I zoomed into their full section image of a healthy ovary, zoomed in to the maximum and took screenshots of many differently looking regions.
\nA second thank you to everyone who made their code or their model weights openly available. This greatly improves innovation and allows individual contributors to stand on the shoulders of giants.
\nCLAM: Lu, M.Y., Williamson, D.F.K., Chen, T.Y. et al. Data-efficient and weakly supervised computational pathology on whole-slide images. Nat Biomed Eng 5, 555–570 (2021). https://doi.org/10.1038/s41551-020-00682-w
\n\n[HistomicsTK(https://github.com/DigitalSlideArchive/HistomicsTK)
\n[large_iamge]https://github.com/girder/large_image
\nWölflein, Georg; Ferber, Dyke; Meneghetti, Asier Rabasco; El Nahhas, Omar S. M.; Truhn, Daniel; Carrero, Zunamys I.; Harrison, David J.; Arandjelović, Ognjen; Kather, Jakob N. (2023). \"A Good Feature Extractor Is All You Need for Weakly Supervised Learning in Histopathology\". arXiv:2311.11772.
\nThanks for you interest in my solution and you can fine me on Twitter here.
", "messages": [ { "id": 2587077, "postDate": "2024-01-04T14:52:01.963Z", "content": "Business context: UBC Ovarian Cancer Subtype Classification and Outlier Detection
\nData context: [The] challenge in this competition is to classify the type of ovarian cancer from microscopy scans of biopsy samples. Link to data description
\nFinding more public external data was key for me. Overfitting was a big problem due to the small number of examples. Initially I hoped the CLAM or a multiple instance learning (MIL) approach could remedy that, because many images are so large they can be split in tens of thousands of tiles. But my models were still overfitting a lot. I assume the tiles of the same patient are similar in important ways and the models can use that as shortcuts that don't generalize well. Or there is just so much heterogeniety between cancers, so that the number of samples are just not enough to capture all different variants of the subtypes.
I used the segmentation data provided to create synthetic tumor micro array (TMA) images, basically jsut cropping tiny images from the segmentation of the large image. I did this for the cancer tissue and generated some \"Other\" synthetic images by cropping small tiles that were marked as healthy or as stroma.
Following the paper \"A Good Feature Extractor Is All You Need for Weakly Supervised Learning in Histopathology\" I used the pretrained model Lunit-DINO to extract smaller size features to handle the massive image size. I ran the feature extractino in 16-bit so that it runs faster. I didn't see much negative impact on feature quality.
I filtered the tiles containing tissue using the thumbnails and then cropped the tissue tiles using PyVips. I lost so much time on this one, because I first tried to re-write the feature extraction code in CLAM, but couldn't make it work with the resource limits on Kaggle. Then I tried using the large_image library that is made for histopatholy images, but could not make it work with the Kaggle resource limits. I alternated between out of memory, out of disk and timeouts. Finally I made it work using PyVips and asynchronous data loading in PyTorch. This part of the competition was frustrating, as I wanted to spend my time on deep learning and spend weeks on image processing and guessing Kaggle errors.
On the extracted features I trained the CLAM model, which is simiar to MIL, but calculates an attention matrix to weight the tiles. I made some changes to the instance level loss function for the \"Other\" label, as it's sematically different from the other labels. A tile of a slide containing a cancer subtype could still be have the label \"Other\", if the tile would only show healthy tissue.
The following diagram shows the CLAM model from Mahmood Lab @ Harvard/BWH & MGH [1]. This model takes as input concatenated features that have been extracted from all tiles of a Whole Slide Image containing tissue.
\nThe top part of the diagram calculates the attention scores A, a vector with one entry per tile. The bottom part creates an A-weighted sum of the transformed input features h and feeds it into a multi-class classification head.
\n
Diagram from Paul Pham [2]
\nPyTorch Code for my adapted CLAM model:
\n (nn.Module):\n ():\n (Attn_Net_Gated, self).__init__()\n self.attention_a = [\n nn.Linear(L, D),\n nn.Tanh()\n ]\n self.attention_b = [\n nn.Linear(L, D),\n nn.Sigmoid()\n ]\n dropout > :\n self.attention_a.append(nn.Dropout(dropout))\n self.attention_b.append(nn.Dropout(dropout))\n\n self.attention_a = nn.Sequential(*self.attention_a)\n self.attention_b = nn.Sequential(*self.attention_b)\n\n self.attention_c = nn.Linear(D, n_classes)\n\n ():\n a = self.attention_a(x)\n b = self.attention_b(x)\n A = a.mul(b)\n A = self.attention_c(A) \n A, x\n\n\n (nn.Module):\n ():\n ().__init__()\n self.size_dict = {\n : [feature_dim, , ],\n : [feature_dim, , ],\n : [feature_dim, , ],\n : [feature_dim, , ]\n }\n size = self.size_dict[size_arg]\n fc = [nn.Linear(size[], size[]), nn.ReLU()]\n dropout > :\n fc.append(nn.Dropout(dropout))\n gate:\n attention_net = Attn_Net_Gated(L = size[], D = size[], dropout = dropout, n_classes = )\n :\n attention_net = Attn_Net(L = size[], D = size[], dropout = dropout, n_classes = )\n fc.append(attention_net)\n self.attention_net = nn.Sequential(*fc)\n self.classifiers = nn.Linear(size[], n_classes)\n instance_classifiers = [] \n class_idx (n_classes):\n layers = []\n depth_idx (inst_class_depth-):\n divisor = ** depth_idx \n layers.append(nn.Linear(size[] // divisor, size[] // (divisor * )))\n layers.append(nn.ReLU())\n inst_dropout :\n layers.append(nn.Dropout(inst_dropout))\n layers.append(nn.Linear(size[] // **(inst_class_depth-), ))\n instance_classifiers.append(nn.Sequential(*layers)) \n self.instance_classifiers = nn.ModuleList(instance_classifiers)\n self.k_sample = k_sample\n self.instance_loss_fn = instance_loss_fn\n self.n_classes = n_classes\n self.subtyping = subtyping\n self.use_inst_predictions = use_inst_predictions\n self.other_idx = label_mapping[]\n self.class_weights = class_weights\n initialize_weights(self)\n self.to()\n\n\n ():\n torch.full((length, ), , device=device).()\n\n ():\n torch.full((length, ), , device=device).()\n\n \n (): \n device=h.device\n (A.shape) == :\n A = A.view(, -)\n\n is_tma:\n k_sample = self.k_sample // \n :\n k_sample = self.k_sample\n\n k_sample <= math.ceil(A.shape[] / ):\n top_p_ids = torch.topk(A, k_sample)[][-] \n :\n top_p_ids = torch.topk(A, math.ceil(A.shape[] / ))[][-]\n top_p_ids = top_p_ids.repeat(k_sample)[:k_sample]\n top_p = torch.index_select(h, dim=, index=top_p_ids) \n k_sample <= math.ceil(A.shape[] / ):\n top_n_ids = torch.topk(-A, k_sample, dim=)[][-]\n :\n top_n_ids = torch.topk(-A, math.ceil(A.shape[] / ))[][-]\n top_n_ids = top_n_ids.repeat(k_sample)[:k_sample]\n top_n = torch.index_select(h, dim=, index=top_n_ids)\n p_targets = self.create_positive_targets(k_sample, device)\n n_targets = self.create_negative_targets(k_sample, device)\n\n \n \n p_logits = classifier(top_p) \n n_logits = classifier(top_n)\n inst_preds = (p_logits.squeeze() > ).long()\n \n p_loss = self.instance_loss_fn(p_logits.squeeze(), p_targets) * (self.n_classes -)\n n_loss = self.instance_loss_fn(n_logits.squeeze(), n_targets)\n is_tma is_other_class:\n loss = p_loss + n_loss\n : loss = p_loss\n loss, inst_preds, p_targets, p_logits\n\n \n ():\n device=h.device\n (A.shape) == :\n A = A.view(, -)\n\n is_tma:\n k_sample = self.k_sample // \n :\n k_sample = self.k_sample\n\n \n \n k_sample <= math.ceil(A.shape[] / ):\n top_ids = torch.topk(A, k_sample)[][-]\n :\n top_ids = torch.topk(A, math.ceil(A.shape[] / ))[][-]\n top_ids = top_ids.repeat(k_sample)[:k_sample]\n top_inst = torch.index_select(h, dim=, index=top_ids)\n top_targets = self.create_negative_targets(k_sample, device)\n\n\n logits = classifier(top_inst)\n inst_preds = (logits.squeeze() > ).long()\n instance_loss = self.instance_loss_fn(logits.squeeze(), top_targets)\n instance_loss, inst_preds, top_targets, logits\n\n\n ():\n A, h = self.attention_net(h) \n A = torch.transpose(A, , ) \n attention_only:\n A\n A_raw = A\n A = F.softmax(A, dim=) \n M = torch.mm(A, h) \n logits = self.classifiers(M)\n bag_Y_prob = F.softmax(logits.squeeze(), dim=)\n\n is_tma:\n k_sample = self.k_sample // \n :\n k_sample = self.k_sample\n\n all_inst_logits = []\n top_p_ids = \n bag_pred_weight < label : \n total_inst_loss = \n all_inst_preds = []\n all_targets = []\n i ((self.instance_classifiers)):\n classifier = self.instance_classifiers[i]\n i == label.item(): \n is_other_class = (label.item() == self.other_idx)\n instance_loss, inst_preds, targets, inst_logits = self.inst_eval(A, h, classifier, is_tma, is_other_class)\n all_inst_preds.extend(inst_preds.cpu().numpy()) \n all_targets.extend(targets.cpu().numpy())\n \n all_inst_logits.append(inst_logits)\n self.class_weights :\n instance_loss *= self.class_weights[i]\n : \n self.subtyping:\n instance_loss, inst_preds, targets, inst_logits = self.inst_eval_out(A, h, classifier, is_tma)\n all_inst_preds.extend(inst_preds.cpu().numpy())\n all_targets.extend(targets.cpu().numpy()) \n all_inst_logits.append(inst_logits)\n :\n \n\n total_inst_loss += instance_loss \n\n self.subtyping:\n \n total_inst_loss /= * (self.instance_classifiers)\n :\n self.k_sample <= math.ceil(A.shape[] / ):\n top_p_ids = torch.topk(A, k_sample)[][-] \n :\n top_p_ids = torch.topk(A, math.ceil(A.shape[] / ))[][-]\n top_p_ids = top_p_ids.repeat(k_sample)[:k_sample]\n top_p = torch.index_select(h, dim=, index=top_p_ids)\n classifier self.instance_classifiers:\n class_logits = classifier(top_p)\n all_inst_logits.append(class_logits)\n\n\n self.use_inst_predictions: \n all_inst_logits = torch.concatenate(all_inst_logits, axis=) \n \n \n self.k_sample <= math.ceil(A.shape[] / ):\n top_p_ids = torch.topk(A, k_sample)[][-] \n :\n top_p_ids = torch.topk(A, math.ceil(A.shape[] / ))[][-]\n top_p_ids = top_p_ids.repeat(k_sample)[:k_sample]\n\n all_inst_logits =A_raw[, top_p_ids].reshape(-, ) * all_inst_logits\n softmax_inst_probs = torch.softmax(all_inst_logits, dim=)\n agg_inst_probs = softmax_inst_probs \n agg_inst_probs = torch.mean(agg_inst_probs, dim=) \n Y_probs = bag_Y_prob * bag_pred_weight + agg_inst_probs * ( - bag_pred_weight)\n Y_hat = torch.topk(Y_probs, , dim=)[]\n\n results_dict = {}\n bag_pred_weight < :\n results_dict.update({\n : all_inst_logits.detach().cpu().numpy(),\n : agg_inst_probs.detach().cpu().numpy()\n })\n self.use_inst_predictions: \n results_dict.update({\n : softmax_inst_probs.detach().cpu().numpy()\n })\n label :\n results_dict.update({\n : np.array(all_targets),\n : np.array(all_inst_preds).flatten(),\n : total_inst_loss\n })\n\n logits, Y_probs, Y_hat, A_raw, results_dict\nFrom the Cancer Imaging Archive I used the Ovarian Bevacizumab Response and the CPTAC-OV data. The labels didn't map perfectly to the labels of the competition. For example a label would be Papillary Serous Carcinoma, which I assumed means either HGSC or LGSC. I just used a model trained on the my other data to decide which of the two to select.
\nI used the Ovarian Carcinoma Histopathology Dataset by the Hamarneh Lab.
\nI also used data from the Stanford Tissue Microarray Database, most of which didn't have H&E staining, but at least the cell shapes looked similar and I wanted to also have some TMA data.
\nI also got in touch with Cooperative Human Tissue network, who where so nice to allow me to use the data that was publicly available on the website. I thank them for the time they took to discuss this topic, even though I didn't use their data in the end, as they took the data down from their website after my outreach. As the data wasn't available for other participants anymore, I assume I'm not allowed to use it to train my model.
\nFor a long time I pooled all my data and used 5-fold cross validation, making sure multiple images from the same patient would all be in the same fold. Still this lead to inflated validation scores. I seems like my models where overfitting by using some shortcuts from my datasets that do not generalize. Later I excluded the data from the Harmanreh lab completely for validation which lead to much more reliable cross-validation scores.
\nAs the data size of the competition was so large I trained my models locally on my desktop computer with a GTX 4090 card. Feature extraction would take around 6 hours for all my datasets and training my model would take another hour.
\nA big thank you to everyone who made their pathological image data publicly available. This is a tremendous help for anyone who wants to build deep learning models to improve digital pathology, but especially for individuals without access to close source clinical data.
\nBevacizumab: Wang et al. Weakly Supervised Deep Learning for Prediction of Treatment Effectiveness on Ovarian Cancer from Histopathology Images. Computerized Medical Imaging and Graphics. https://doi.org/10.1016/j.compmedimag.2022.102093
\nCPTAC-OV: National Cancer Institute Clinical Proteomic Tumor Analysis Consortium (CPTAC). (2020). The Clinical Proteomic Tumor Analysis Consortium Ovarian Serous Cystadenocarcinoma Collection (CPTAC-OV) (Version 3) [Data set]. The Cancer Imaging Archive. https://doi.org/10.7937/TCIA.ZS4A-JD58
\nHarmanreh Data: Köbel, Martin; Kalloger, Steve E.; Baker, Patricia M.; Ewanowich, Carol A.; Arseneau, Jocelyne; Zherebitskiy, Viktor; Abdulkarim, Soran; Leung, Samuel; Duggan, Máire A.; Fontaine, Dan; et al. (2010). \"Diagnosis of ovarian carcinoma cell type is highly reproducible: a transcanadian study\". The American Journal of Surgical Pathology, 34(7), 984–993. LWW.
\nI used some screenshots for healthy tissue from The Human Protein Atlas and the University of Michigan Histology and Cirtual Microscopy page. I zoomed into their full section image of a healthy ovary, zoomed in to the maximum and took screenshots of many differently looking regions.
\nA second thank you to everyone who made their code or their model weights openly available. This greatly improves innovation and allows individual contributors to stand on the shoulders of giants.
\nCLAM: Lu, M.Y., Williamson, D.F.K., Chen, T.Y. et al. Data-efficient and weakly supervised computational pathology on whole-slide images. Nat Biomed Eng 5, 555–570 (2021). https://doi.org/10.1038/s41551-020-00682-w
\n\n[HistomicsTK(https://github.com/DigitalSlideArchive/HistomicsTK)
\n[large_iamge]https://github.com/girder/large_image
\nWölflein, Georg; Ferber, Dyke; Meneghetti, Asier Rabasco; El Nahhas, Omar S. M.; Truhn, Daniel; Carrero, Zunamys I.; Harrison, David J.; Arandjelović, Ognjen; Kather, Jakob N. (2023). \"A Good Feature Extractor Is All You Need for Weakly Supervised Learning in Histopathology\". arXiv:2311.11772.
\nThanks for you interest in my solution and you can fine me on Twitter here.
", "rawMarkdown": "## Context\n\nBusiness context: [UBC Ovarian Cancer Subtype Classification and Outlier Detection](https://www.kaggle.com/competitions/UBC-OCEAN)\n\nData context: [The] challenge in this competition is to classify the type of ovarian cancer from microscopy scans of biopsy samples. [Link to data description](https://www.kaggle.com/competitions/UBC-OCEAN/data)\n\n## Overview of the Approach\n\n- Finding **more public external data was key** for me. Overfitting was a big problem due to the small number of examples. Initially I hoped the [CLAM](https://github.com/mahmoodlab/CLAM) or a multiple instance learning (MIL) approach could remedy that, because many images are so large they can be split in tens of thousands of tiles. But my models were still overfitting a lot. I assume the tiles of the same patient are similar in important ways and the models can use that as shortcuts that don't generalize well. Or there is just so much heterogeniety between cancers, so that the number of samples are just not enough to capture all different variants of the subtypes.\n- I used the **segmentation data provided to create synthetic tumor micro array (TMA) images**, basically jsut cropping tiny images from the segmentation of the large image. I did this for the cancer tissue and generated some \"Other\" synthetic images by cropping small tiles that were marked as healthy or as stroma.\n- Following the paper \"[A Good Feature Extractor Is All You Need for Weakly Supervised Learning in Histopathology](https://arxiv.org/pdf/2311.11772.pdf)\" I used the **pretrained model Lunit-DINO to extract smaller size features** to handle the massive image size. I ran the feature extractino in 16-bit so that it runs faster. I didn't see much negative impact on feature quality.\n\n- **I filtered the tiles containing tissue using the thumbnails and then cropped the tissue tiles using PyVips**. I lost so much time on this one, because I first tried to re-write the feature extraction code in CLAM, but couldn't make it work with the resource limits on Kaggle. Then I tried using the [large_image](https://github.com/girder/large_image) library that is made for histopatholy images, but could not make it work with the Kaggle resource limits. I alternated between out of memory, out of disk and timeouts. Finally I made it work using PyVips and asynchronous data loading in PyTorch. This part of the competition was frustrating, as I wanted to spend my time on deep learning and spend weeks on image processing and guessing Kaggle errors.\n- **On the extracted features I trained the [CLAM](https://github.com/mahmoodlab/CLAM) model**, which is simiar to MIL, but calculates an attention matrix to weight the tiles. I made some changes to the instance level loss function for the \"Other\" label, as it's sematically different from the other labels. A tile of a slide containing a cancer subtype could still be have the label \"Other\", if the tile would only show healthy tissue.\n\n## **Details of the submission**\n### CLAM Model\n\nThe following diagram shows the CLAM model from [Mahmood Lab @ Harvard/BWH & MGH ](https://faisal.ai/)[1]. This model takes as input concatenated features that have been extracted from all tiles of a Whole Slide Image containing tissue. \n\nThe top part of the diagram calculates the attention scores A, a vector with one entry per tile. The bottom part creates an A-weighted sum of the transformed input features h and feeds it into a multi-class classification head.\n\n\n\nDiagram from Paul Pham [2]\n\n\nPyTorch Code for my adapted CLAM model:\n\n```Python\nclass Attn_Net_Gated(nn.Module):\n def __init__(self, L = 1024, D = 256, dropout = 0, n_classes = 1):\n super(Attn_Net_Gated, self).__init__()\n self.attention_a = [\n nn.Linear(L, D),\n nn.Tanh()\n ]\n self.attention_b = [\n nn.Linear(L, D),\n nn.Sigmoid()\n ]\n if dropout > 0:\n self.attention_a.append(nn.Dropout(dropout))\n self.attention_b.append(nn.Dropout(dropout))\n\n self.attention_a = nn.Sequential(*self.attention_a)\n self.attention_b = nn.Sequential(*self.attention_b)\n \n self.attention_c = nn.Linear(D, n_classes)\n\n def forward(self, x):\n a = self.attention_a(x)\n b = self.attention_b(x)\n A = a.mul(b)\n A = self.attention_c(A) # N x n_classes\n return A, x\n\n\nclass CLAM_SB(nn.Module):\n def __init__(self, gate = True, size_arg = \"small\", n_classes=2, dropout = 0, k_sample=8,\n instance_loss_fn=None, subtyping=False, feature_dim=1024, use_inst_predictions=True,\n label_mapping=None, class_weights=None, inst_class_depth=None, inst_dropout=None):\n super().__init__()\n self.size_dict = {\n \"very small\": [feature_dim, 256, 128],\n \"small\": [feature_dim, 512, 256],\n \"big\": [feature_dim, 1024, 512],\n \"xl\": [feature_dim, 2048, 1024]\n }\n size = self.size_dict[size_arg]\n fc = [nn.Linear(size[0], size[1]), nn.ReLU()]\n if dropout > 0:\n fc.append(nn.Dropout(dropout))\n if gate:\n attention_net = Attn_Net_Gated(L = size[1], D = size[2], dropout = dropout, n_classes = 1)\n else:\n attention_net = Attn_Net(L = size[1], D = size[2], dropout = dropout, n_classes = 1)\n fc.append(attention_net)\n self.attention_net = nn.Sequential(*fc)\n self.classifiers = nn.Linear(size[1], n_classes)\n instance_classifiers = [] \n for class_idx in range(n_classes):\n layers = []\n for depth_idx in range(inst_class_depth-1):\n divisor = 2 ** depth_idx \n layers.append(nn.Linear(size[1] // divisor, size[1] // (divisor * 2)))\n layers.append(nn.ReLU())\n if inst_dropout is not None:\n layers.append(nn.Dropout(inst_dropout))\n layers.append(nn.Linear(size[1] // 2**(inst_class_depth-1), 1))\n instance_classifiers.append(nn.Sequential(*layers)) \n self.instance_classifiers = nn.ModuleList(instance_classifiers)\n self.k_sample = k_sample\n self.instance_loss_fn = instance_loss_fn\n self.n_classes = n_classes\n self.subtyping = subtyping\n self.use_inst_predictions = use_inst_predictions\n self.other_idx = label_mapping['Other']\n self.class_weights = class_weights\n initialize_weights(self)\n self.to('cuda')\n\n @staticmethod\n def create_positive_targets(length, device):\n return torch.full((length, ), 1, device=device).float()\n @staticmethod\n def create_negative_targets(length, device):\n return torch.full((length, ), 0, device=device).float()\n \n #instance-level evaluation for in-the-class attention branch\n def inst_eval(self, A, h, classifier, is_tma, is_other_class): \n device=h.device\n if len(A.shape) == 1:\n A = A.view(1, -1)\n \n if is_tma:\n k_sample = self.k_sample // 2\n else:\n k_sample = self.k_sample\n\n if k_sample <= math.ceil(A.shape[1] / 2):\n top_p_ids = torch.topk(A, k_sample)[1][-1] # [1][-1] selects the last index\n else:\n top_p_ids = torch.topk(A, math.ceil(A.shape[1] / 2))[1][-1]\n top_p_ids = top_p_ids.repeat(k_sample)[:k_sample]\n top_p = torch.index_select(h, dim=0, index=top_p_ids) # dim = k_sample x self.size_dict[1]\n if k_sample <= math.ceil(A.shape[1] / 2):\n top_n_ids = torch.topk(-A, k_sample, dim=1)[1][-1]\n else:\n top_n_ids = torch.topk(-A, math.ceil(A.shape[1] / 2))[1][-1]\n top_n_ids = top_n_ids.repeat(k_sample)[:k_sample]\n top_n = torch.index_select(h, dim=0, index=top_n_ids)\n p_targets = self.create_positive_targets(k_sample, device)\n n_targets = self.create_negative_targets(k_sample, device)\n\n # errors get evaluated on positive and negative labels, to also constain A for low attention\n # on negative tiles, but for prediction we only care about the top tiles.\n p_logits = classifier(top_p) # dim = k_sample\n n_logits = classifier(top_n)\n inst_preds = (p_logits.squeeze() > 0).long()\n # we give more weight to the positive targets as we have more negative targets.\n p_loss = self.instance_loss_fn(p_logits.squeeze(), p_targets) * (self.n_classes -1)\n n_loss = self.instance_loss_fn(n_logits.squeeze(), n_targets)\n if not is_tma and not is_other_class:\n loss = p_loss + n_loss\n else: loss = p_loss\n return loss, inst_preds, p_targets, p_logits\n \n #instance-level evaluation for out-of-the-class attention branch\n def inst_eval_out(self, A, h, classifier, is_tma):\n device=h.device\n if len(A.shape) == 1:\n A = A.view(1, -1)\n\n if is_tma:\n k_sample = self.k_sample // 2\n else:\n k_sample = self.k_sample\n\n # we allow at max the top half of the bag to be selected, \n # otherwise we repeat the top half.\n if k_sample <= math.ceil(A.shape[1] / 2):\n top_ids = torch.topk(A, k_sample)[1][-1]\n else:\n top_ids = torch.topk(A, math.ceil(A.shape[1] / 2))[1][-1]\n top_ids = top_ids.repeat(k_sample)[:k_sample]\n top_inst = torch.index_select(h, dim=0, index=top_ids)\n top_targets = self.create_negative_targets(k_sample, device)\n \n\n logits = classifier(top_inst)\n inst_preds = (logits.squeeze() > 0).long()\n instance_loss = self.instance_loss_fn(logits.squeeze(), top_targets)\n return instance_loss, inst_preds, top_targets, logits\n \n\n def forward(self, h, bag_pred_weight:float, is_tma:bool, label=None, attention_only=False):\n A, h = self.attention_net(h) # NxK \n A = torch.transpose(A, 1, 0) # KxN\n if attention_only:\n return A\n A_raw = A\n A = F.softmax(A, dim=1) # softmax over N\n M = torch.mm(A, h) # shape 1 x self.size_dict[1]\n logits = self.classifiers(M)\n bag_Y_prob = F.softmax(logits.squeeze(), dim=0)\n\n if is_tma:\n k_sample = self.k_sample // 2\n else:\n k_sample = self.k_sample\n\n all_inst_logits = []\n top_p_ids = None\n if bag_pred_weight < 1 and label is not None: \n total_inst_loss = 0.0\n all_inst_preds = []\n all_targets = []\n for i in range(len(self.instance_classifiers)):\n classifier = self.instance_classifiers[i]\n if i == label.item(): #in-the-class:\n is_other_class = (label.item() == self.other_idx)\n instance_loss, inst_preds, targets, inst_logits = self.inst_eval(A, h, classifier, is_tma, is_other_class)\n all_inst_preds.extend(inst_preds.cpu().numpy()) \n all_targets.extend(targets.cpu().numpy())\n #class_probs = F.softmax(inst_logits, dim=1)\n all_inst_logits.append(inst_logits)\n if self.class_weights is not None:\n instance_loss *= self.class_weights[i]\n else: #out-of-the-class\n if self.subtyping:\n instance_loss, inst_preds, targets, inst_logits = self.inst_eval_out(A, h, classifier, is_tma)\n all_inst_preds.extend(inst_preds.cpu().numpy())\n all_targets.extend(targets.cpu().numpy()) \n all_inst_logits.append(inst_logits)\n else:\n continue\n \n total_inst_loss += instance_loss \n\n if self.subtyping:\n # the 2 corrects for the upscaling of the positive instance loss\n total_inst_loss /= 2 * len(self.instance_classifiers)\n else:\n if self.k_sample <= math.ceil(A.shape[1] / 2):\n top_p_ids = torch.topk(A, k_sample)[1][-1] # [1][-1] selects the last index\n else:\n top_p_ids = torch.topk(A, math.ceil(A.shape[1] / 2))[1][-1]\n top_p_ids = top_p_ids.repeat(k_sample)[:k_sample]\n top_p = torch.index_select(h, dim=0, index=top_p_ids)\n for classifier in self.instance_classifiers:\n class_logits = classifier(top_p)\n all_inst_logits.append(class_logits)\n\n \n if self.use_inst_predictions: \n all_inst_logits = torch.concatenate(all_inst_logits, axis=1) # dim k_sample x n_classes\n # take probs of all k_sample classifiers and turn them into multi-class probabilities\n # weight tile instance predictions by A\n if self.k_sample <= math.ceil(A.shape[1] / 2):\n top_p_ids = torch.topk(A, k_sample)[1][-1] # [1][-1] selects the last index\n else:\n top_p_ids = torch.topk(A, math.ceil(A.shape[1] / 2))[1][-1]\n top_p_ids = top_p_ids.repeat(k_sample)[:k_sample]\n\n all_inst_logits =A_raw[0, top_p_ids].reshape(-1, 1) * all_inst_logits\n softmax_inst_probs = torch.softmax(all_inst_logits, dim=1)\n agg_inst_probs = softmax_inst_probs \n agg_inst_probs = torch.mean(agg_inst_probs, dim=0) # result has dim k_sample\n Y_probs = bag_Y_prob * bag_pred_weight + agg_inst_probs * (1 - bag_pred_weight)\n Y_hat = torch.topk(Y_probs, 1, dim=0)[1]\n \n results_dict = {}\n if bag_pred_weight < 1:\n results_dict.update({\n 'all_inst_logits': all_inst_logits.detach().cpu().numpy(),\n 'agg_inst_probs': agg_inst_probs.detach().cpu().numpy()\n })\n if self.use_inst_predictions: \n results_dict.update({\n 'softmax_inst_probs': softmax_inst_probs.detach().cpu().numpy()\n })\n if label is not None:\n results_dict.update({\n 'inst_labels': np.array(all_targets),\n 'inst_preds': np.array(all_inst_preds).flatten(),\n 'instance_loss': total_inst_loss\n })\n\n return logits, Y_probs, Y_hat, A_raw, results_dict\n```\n\n\n\n### Description of the Data Used\n\nFrom the [Cancer Imaging Archive](https://www.cancerimagingarchive.net/) I used the [Ovarian Bevacizumab Response](https://doi.org/10.7937/TCIA.985G-EY35) and the [CPTAC-OV](https://doi.org/10.7937/TCIA.ZS4A-JD58) data. The labels didn't map perfectly to the labels of the competition. For example a label would be Papillary Serous Carcinoma, which I assumed means either HGSC or LGSC. I just used a model trained on the my other data to decide which of the two to select.\n\nI used the Ovarian [Carcinoma Histopathology Dataset](https://www.medicalimageanalysis.com/data/ovarian-carcinomas-histopathology-dataset) by the [Hamarneh Lab](https://www.medicalimageanalysis.com/home).\n\nI also used data from the [Stanford Tissue Microarray Database](https://tma.im/cgi-bin/home.pl), most of which didn't have H&E staining, but at least the cell shapes looked similar and I wanted to also have some TMA data.\n\nI also got in touch with [Cooperative Human Tissue network](https://chtn.cancer.gov), who where so nice to allow me to use the data that was publicly available on the website. I thank them for the time they took to discuss this topic, even though I didn't use their data in the end, as they took the data down from their website after my outreach. As the data wasn't available for other participants anymore, I assume I'm not allowed to use it to train my model.\n\n### Validation Setup\n\nFor a long time I pooled all my data and used 5-fold cross validation, making sure multiple images from the same patient would all be in the same fold. Still this lead to inflated validation scores. I seems like my models where overfitting by using some shortcuts from my datasets that do not generalize. Later I excluded the data from the Harmanreh lab completely for validation which lead to much more reliable cross-validation scores.\n\n### Technical Setup\n\nAs the data size of the competition was so large I trained my models locally on my desktop computer with a GTX 4090 card. Feature extraction would take around 6 hours for all my datasets and training my model would take another hour.\n\n\n### Data sources\n\nA big thank you to everyone who made their pathological image data publicly available. This is a tremendous help for anyone who wants to build deep learning models to improve digital pathology, but especially for individuals without access to close source clinical data.\n\nBevacizumab: Wang et al. *Weakly Supervised Deep Learning for Prediction of Treatment Effectiveness on Ovarian Cancer from Histopathology Images.* Computerized Medical Imaging and Graphics. [https://doi.org/10.1016/j.compmedimag.2022.102093](https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.1016%2Fj.compmedimag.2022.102093&data=05|01|kirbyju@mail.nih.gov|a1a4f263214846a156f908da58bd5e3c|14b77578977342d58507251ca2dc2b06|0|0|637919868422135484|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=424GtHH8SDUEjvHtaXZvOwt9hcHaZgl36YHL1tzH6T4%3D&reserved=0)\n\nCPTAC-OV: National Cancer Institute Clinical Proteomic Tumor Analysis Consortium (CPTAC). (2020). **The Clinical Proteomic Tumor Analysis Consortium Ovarian Serous Cystadenocarcinoma Collection (CPTAC-OV)** (Version 3) [Data set]. The Cancer Imaging Archive. https://doi.org/10.7937/TCIA.ZS4A-JD58\n\nHarmanreh Data: Köbel, Martin; Kalloger, Steve E.; Baker, Patricia M.; Ewanowich, Carol A.; Arseneau, Jocelyne; Zherebitskiy, Viktor; Abdulkarim, Soran; Leung, Samuel; Duggan, Máire A.; Fontaine, Dan; et al. (2010). \"Diagnosis of ovarian carcinoma cell type is highly reproducible: a transcanadian study\". *The American Journal of Surgical Pathology*, 34(7), 984–993. LWW.\n\nI used some screenshots for healthy tissue from [The Human Protein Atlas](https://www.proteinatlas.org/learn/dictionary/normal/ovary) and the [University of Michigan Histology and Cirtual Microscopy](https://histology.medicine.umich.edu/resources/female-reproductive-system#ovary-oviduct-suggested-readings) page. I zoomed into their full section image of a healthy ovary, zoomed in to the maximum and took screenshots of many differently looking regions.\n\n\n### Open Source Code\n\nA second thank you to everyone who made their code or their model weights openly available. This greatly improves innovation and allows individual contributors to stand on the shoulders of giants.\n\n[CLAM](https://github.com/mahmoodlab/CLAM): Lu, M.Y., Williamson, D.F.K., Chen, T.Y. et al. Data-efficient and weakly supervised computational pathology on whole-slide images. Nat Biomed Eng 5, 555–570 (2021). https://doi.org/10.1038/s41551-020-00682-w\n\n[PyVips](https://libvips.github.io/pyvips)\n\n[HistomicsTK(https://github.com/DigitalSlideArchive/HistomicsTK)\n\n[large_iamge]https://github.com/girder/large_image\n\n\n### Research Papers\n\nWölflein, Georg; Ferber, Dyke; Meneghetti, Asier Rabasco; El Nahhas, Omar S. M.; Truhn, Daniel; Carrero, Zunamys I.; Harrison, David J.; Arandjelović, Ognjen; Kather, Jakob N. (2023). \"A Good Feature Extractor Is All You Need for Weakly Supervised Learning in Histopathology\". arXiv:2311.11772.\n\n\n**Thanks for you interest in my solution and you can fine me on [Twitter here](https://twitter.com/swanint).**\n", "votes": 26 }, { "id": 2587482, "postDate": "2024-01-04T19:21:23.343Z", "content": "Congratulations! You have grown a lot over the past seven years!
", "rawMarkdown": "Congratulations! You have grown a lot over the past seven years!", "votes": 1 }, { "id": 3029690, "postDate": "2024-10-27T15:59:09.333Z", "content": "There is an IEEE paper that is academic fraud. The title is \"OCEAN - Ovarian Cancer subtypE clAssification and outlier detectionioN using DenseNet121\". It claims that only convolutional networks were used without mentioning multi-instance learning. Then, 2,000 WSI images from the OECEN-UBC competition that our contestants could not get were used for training to obtain a classification accuracy of 99.7%. However, our first place winner only had a test accuracy of 0.6 and a 5-fold cross-validation accuracy of less than 90%![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16846649%2Fff519f283467f5bcf2e5416645a08cfa%2F_20241027234938.png?generation=1730044736614051&alt=media\")
![\"\"](\"https://www.googleapis.com/download/storage/v1/b/kaggle-forum-message-attachments/o/inbox%2F16846649%2F2692818a6d51b91285ace54d23d8b383%2Ffake_acc.png?generation=1730044746592544&alt=media\")
@dantee I have a query regarding the Ovarian Carcinoma Histopathology Dataset by the Hamarneh Lab you used. I'm experiencing bandwidth issues while downloading the dataset, with frequent crashes and speed limitations. Did you encounter similar issues, and if so, do you know of any alternative sources to download the data?
", "rawMarkdown": "@dantee I have a query regarding the Ovarian Carcinoma Histopathology Dataset by the Hamarneh Lab you used. I'm experiencing bandwidth issues while downloading the dataset, with frequent crashes and speed limitations. Did you encounter similar issues, and if so, do you know of any alternative sources to download the data?", "replies": [ { "id": 2956181, "postDate": "2024-08-11T20:59:12.027Z", "content": "Unfortunately I don't have another source. I also had very slow downloads and had to run them for a long time.
", "rawMarkdown": "Unfortunately I don't have another source. I also had very slow downloads and had to run them for a long time." }, { "id": 3144673, "postDate": "2025-03-08T17:16:26.443Z", "content": "I have downloaded this data using \"Free Download Manager\". It is a so pretty app for this task.
\nYou can find it here: https://www.freedownloadmanager.org/
May I ask how you dealt with the compute/time limit in prediction step? I hit the 12-hour timeout when I sampled a mere 20 2560px tiles from each of the full test image using pyvips. [(y, y + h, x, x + w) for y in range(0, image.height, h) for x in range(0, image.width, w)], random.choice() (with repeat), image.crop(), and timeout. Didn't even predict anything, just wrote a dummy submission.csv without passing the tiles through a model.
", "rawMarkdown": "May I ask how you dealt with the compute/time limit in prediction step? I hit the 12-hour timeout when I sampled a mere 20 2560px tiles from each of the full test image using pyvips. [(y, y + h, x, x + w) for y in range(0, image.height, h) for x in range(0, image.width, w)], random.choice() (with repeat), image.crop(), and timeout. Didn't even predict anything, just wrote a dummy submission.csv without passing the tiles through a model.", "replies": [ { "id": 2587622, "postDate": "2024-01-04T22:12:05.257Z", "content": "Yes, the resource limits where quite a struggle for me too. For the feature extraction step I'm using PyTorch Dataloader with num_workers = 4 and do the image loading and cropping of tiles with PyVips inside the Dataloader. This way the tiles get prepared asynchronously on multiply threads while the the feature extractor is doing its work. This approach is more memory hungry, but PyVips is very resourceful if you load the images in the \"sequential\" mode. Sequential mode allows you to read the image only top to bottom, so I always load all tiles in one row at once.
", "rawMarkdown": "Yes, the resource limits where quite a struggle for me too. For the feature extraction step I'm using PyTorch Dataloader with num_workers = 4 and do the image loading and cropping of tiles with PyVips inside the Dataloader. This way the tiles get prepared asynchronously on multiply threads while the the feature extractor is doing its work. This approach is more memory hungry, but PyVips is very resourceful if you load the images in the \"sequential\" mode. Sequential mode allows you to read the image only top to bottom, so I always load all tiles in one row at once.", "votes": 4, "replies": [ { "id": 2591365, "postDate": "2024-01-07T22:29:59.137Z", "content": "Neat trick! Thanks for sharing!
", "rawMarkdown": "Neat trick! Thanks for sharing!" } ] } ] } ], "comments": [ { "id": 2587482, "author_name": "Zaakcii Ru", "author_url": "", "post_date": "2024-01-04T19:21:23.343000", "content": "Congratulations! You have grown a lot over the past seven years!
", "votes": 1, "replies": [] }, { "id": 3029690, "author_name": "Metavers", "author_url": "", "post_date": "2024-10-27T15:59:09.333000", "content": "There is an IEEE paper that is academic fraud. The title is \"OCEAN - Ovarian Cancer subtypE clAssification and outlier detectionioN using DenseNet121\". It claims that only convolutional networks were used without mentioning multi-instance learning. Then, 2,000 WSI images from the OECEN-UBC competition that our contestants could not get were used for training to obtain a classification accuracy of 99.7%. However, our first place winner only had a test accuracy of 0.6 and a 5-fold cross-validation accuracy of less than 90%
@dantee I have a query regarding the Ovarian Carcinoma Histopathology Dataset by the Hamarneh Lab you used. I'm experiencing bandwidth issues while downloading the dataset, with frequent crashes and speed limitations. Did you encounter similar issues, and if so, do you know of any alternative sources to download the data?
", "votes": 0, "replies": [ { "id": 2956181, "author_name": "DanielT", "author_url": "", "post_date": "2024-08-11T20:59:12.027000", "content": "Unfortunately I don't have another source. I also had very slow downloads and had to run them for a long time.
", "votes": 0, "replies": [] }, { "id": 3144673, "author_name": "Ahmed Ehab Egela", "author_url": "", "post_date": "2025-03-08T17:16:26.443000", "content": "I have downloaded this data using \"Free Download Manager\". It is a so pretty app for this task.
\nYou can find it here: https://www.freedownloadmanager.org/
May I ask how you dealt with the compute/time limit in prediction step? I hit the 12-hour timeout when I sampled a mere 20 2560px tiles from each of the full test image using pyvips. [(y, y + h, x, x + w) for y in range(0, image.height, h) for x in range(0, image.width, w)], random.choice() (with repeat), image.crop(), and timeout. Didn't even predict anything, just wrote a dummy submission.csv without passing the tiles through a model.
", "votes": 0, "replies": [ { "id": 2587622, "author_name": "DanielT", "author_url": "", "post_date": "2024-01-04T22:12:05.257000", "content": "Yes, the resource limits where quite a struggle for me too. For the feature extraction step I'm using PyTorch Dataloader with num_workers = 4 and do the image loading and cropping of tiles with PyVips inside the Dataloader. This way the tiles get prepared asynchronously on multiply threads while the the feature extractor is doing its work. This approach is more memory hungry, but PyVips is very resourceful if you load the images in the \"sequential\" mode. Sequential mode allows you to read the image only top to bottom, so I always load all tiles in one row at once.
", "votes": 4, "replies": [ { "id": 2591365, "author_name": "tinkei", "author_url": "", "post_date": "2024-01-07T22:29:59.137000", "content": "Neat trick! Thanks for sharing!
", "votes": 0, "replies": [] } ] } ] } ], "raw_markdown_by_id": { "2587077": "## Context\n\nBusiness context: [UBC Ovarian Cancer Subtype Classification and Outlier Detection](https://www.kaggle.com/competitions/UBC-OCEAN)\n\nData context: [The] challenge in this competition is to classify the type of ovarian cancer from microscopy scans of biopsy samples. [Link to data description](https://www.kaggle.com/competitions/UBC-OCEAN/data)\n\n## Overview of the Approach\n\n- Finding **more public external data was key** for me. Overfitting was a big problem due to the small number of examples. Initially I hoped the [CLAM](https://github.com/mahmoodlab/CLAM) or a multiple instance learning (MIL) approach could remedy that, because many images are so large they can be split in tens of thousands of tiles. But my models were still overfitting a lot. I assume the tiles of the same patient are similar in important ways and the models can use that as shortcuts that don't generalize well. Or there is just so much heterogeniety between cancers, so that the number of samples are just not enough to capture all different variants of the subtypes.\n- I used the **segmentation data provided to create synthetic tumor micro array (TMA) images**, basically jsut cropping tiny images from the segmentation of the large image. I did this for the cancer tissue and generated some \"Other\" synthetic images by cropping small tiles that were marked as healthy or as stroma.\n- Following the paper \"[A Good Feature Extractor Is All You Need for Weakly Supervised Learning in Histopathology](https://arxiv.org/pdf/2311.11772.pdf)\" I used the **pretrained model Lunit-DINO to extract smaller size features** to handle the massive image size. I ran the feature extractino in 16-bit so that it runs faster. I didn't see much negative impact on feature quality.\n\n- **I filtered the tiles containing tissue using the thumbnails and then cropped the tissue tiles using PyVips**. I lost so much time on this one, because I first tried to re-write the feature extraction code in CLAM, but couldn't make it work with the resource limits on Kaggle. Then I tried using the [large_image](https://github.com/girder/large_image) library that is made for histopatholy images, but could not make it work with the Kaggle resource limits. I alternated between out of memory, out of disk and timeouts. Finally I made it work using PyVips and asynchronous data loading in PyTorch. This part of the competition was frustrating, as I wanted to spend my time on deep learning and spend weeks on image processing and guessing Kaggle errors.\n- **On the extracted features I trained the [CLAM](https://github.com/mahmoodlab/CLAM) model**, which is simiar to MIL, but calculates an attention matrix to weight the tiles. I made some changes to the instance level loss function for the \"Other\" label, as it's sematically different from the other labels. A tile of a slide containing a cancer subtype could still be have the label \"Other\", if the tile would only show healthy tissue.\n\n## **Details of the submission**\n### CLAM Model\n\nThe following diagram shows the CLAM model from [Mahmood Lab @ Harvard/BWH & MGH ](https://faisal.ai/)[1]. This model takes as input concatenated features that have been extracted from all tiles of a Whole Slide Image containing tissue. \n\nThe top part of the diagram calculates the attention scores A, a vector with one entry per tile. The bottom part creates an A-weighted sum of the transformed input features h and feeds it into a multi-class classification head.\n\n\n\nDiagram from Paul Pham [2]\n\n\nPyTorch Code for my adapted CLAM model:\n\n```Python\nclass Attn_Net_Gated(nn.Module):\n def __init__(self, L = 1024, D = 256, dropout = 0, n_classes = 1):\n super(Attn_Net_Gated, self).__init__()\n self.attention_a = [\n nn.Linear(L, D),\n nn.Tanh()\n ]\n self.attention_b = [\n nn.Linear(L, D),\n nn.Sigmoid()\n ]\n if dropout > 0:\n self.attention_a.append(nn.Dropout(dropout))\n self.attention_b.append(nn.Dropout(dropout))\n\n self.attention_a = nn.Sequential(*self.attention_a)\n self.attention_b = nn.Sequential(*self.attention_b)\n \n self.attention_c = nn.Linear(D, n_classes)\n\n def forward(self, x):\n a = self.attention_a(x)\n b = self.attention_b(x)\n A = a.mul(b)\n A = self.attention_c(A) # N x n_classes\n return A, x\n\n\nclass CLAM_SB(nn.Module):\n def __init__(self, gate = True, size_arg = \"small\", n_classes=2, dropout = 0, k_sample=8,\n instance_loss_fn=None, subtyping=False, feature_dim=1024, use_inst_predictions=True,\n label_mapping=None, class_weights=None, inst_class_depth=None, inst_dropout=None):\n super().__init__()\n self.size_dict = {\n \"very small\": [feature_dim, 256, 128],\n \"small\": [feature_dim, 512, 256],\n \"big\": [feature_dim, 1024, 512],\n \"xl\": [feature_dim, 2048, 1024]\n }\n size = self.size_dict[size_arg]\n fc = [nn.Linear(size[0], size[1]), nn.ReLU()]\n if dropout > 0:\n fc.append(nn.Dropout(dropout))\n if gate:\n attention_net = Attn_Net_Gated(L = size[1], D = size[2], dropout = dropout, n_classes = 1)\n else:\n attention_net = Attn_Net(L = size[1], D = size[2], dropout = dropout, n_classes = 1)\n fc.append(attention_net)\n self.attention_net = nn.Sequential(*fc)\n self.classifiers = nn.Linear(size[1], n_classes)\n instance_classifiers = [] \n for class_idx in range(n_classes):\n layers = []\n for depth_idx in range(inst_class_depth-1):\n divisor = 2 ** depth_idx \n layers.append(nn.Linear(size[1] // divisor, size[1] // (divisor * 2)))\n layers.append(nn.ReLU())\n if inst_dropout is not None:\n layers.append(nn.Dropout(inst_dropout))\n layers.append(nn.Linear(size[1] // 2**(inst_class_depth-1), 1))\n instance_classifiers.append(nn.Sequential(*layers)) \n self.instance_classifiers = nn.ModuleList(instance_classifiers)\n self.k_sample = k_sample\n self.instance_loss_fn = instance_loss_fn\n self.n_classes = n_classes\n self.subtyping = subtyping\n self.use_inst_predictions = use_inst_predictions\n self.other_idx = label_mapping['Other']\n self.class_weights = class_weights\n initialize_weights(self)\n self.to('cuda')\n\n @staticmethod\n def create_positive_targets(length, device):\n return torch.full((length, ), 1, device=device).float()\n @staticmethod\n def create_negative_targets(length, device):\n return torch.full((length, ), 0, device=device).float()\n \n #instance-level evaluation for in-the-class attention branch\n def inst_eval(self, A, h, classifier, is_tma, is_other_class): \n device=h.device\n if len(A.shape) == 1:\n A = A.view(1, -1)\n \n if is_tma:\n k_sample = self.k_sample // 2\n else:\n k_sample = self.k_sample\n\n if k_sample <= math.ceil(A.shape[1] / 2):\n top_p_ids = torch.topk(A, k_sample)[1][-1] # [1][-1] selects the last index\n else:\n top_p_ids = torch.topk(A, math.ceil(A.shape[1] / 2))[1][-1]\n top_p_ids = top_p_ids.repeat(k_sample)[:k_sample]\n top_p = torch.index_select(h, dim=0, index=top_p_ids) # dim = k_sample x self.size_dict[1]\n if k_sample <= math.ceil(A.shape[1] / 2):\n top_n_ids = torch.topk(-A, k_sample, dim=1)[1][-1]\n else:\n top_n_ids = torch.topk(-A, math.ceil(A.shape[1] / 2))[1][-1]\n top_n_ids = top_n_ids.repeat(k_sample)[:k_sample]\n top_n = torch.index_select(h, dim=0, index=top_n_ids)\n p_targets = self.create_positive_targets(k_sample, device)\n n_targets = self.create_negative_targets(k_sample, device)\n\n # errors get evaluated on positive and negative labels, to also constain A for low attention\n # on negative tiles, but for prediction we only care about the top tiles.\n p_logits = classifier(top_p) # dim = k_sample\n n_logits = classifier(top_n)\n inst_preds = (p_logits.squeeze() > 0).long()\n # we give more weight to the positive targets as we have more negative targets.\n p_loss = self.instance_loss_fn(p_logits.squeeze(), p_targets) * (self.n_classes -1)\n n_loss = self.instance_loss_fn(n_logits.squeeze(), n_targets)\n if not is_tma and not is_other_class:\n loss = p_loss + n_loss\n else: loss = p_loss\n return loss, inst_preds, p_targets, p_logits\n \n #instance-level evaluation for out-of-the-class attention branch\n def inst_eval_out(self, A, h, classifier, is_tma):\n device=h.device\n if len(A.shape) == 1:\n A = A.view(1, -1)\n\n if is_tma:\n k_sample = self.k_sample // 2\n else:\n k_sample = self.k_sample\n\n # we allow at max the top half of the bag to be selected, \n # otherwise we repeat the top half.\n if k_sample <= math.ceil(A.shape[1] / 2):\n top_ids = torch.topk(A, k_sample)[1][-1]\n else:\n top_ids = torch.topk(A, math.ceil(A.shape[1] / 2))[1][-1]\n top_ids = top_ids.repeat(k_sample)[:k_sample]\n top_inst = torch.index_select(h, dim=0, index=top_ids)\n top_targets = self.create_negative_targets(k_sample, device)\n \n\n logits = classifier(top_inst)\n inst_preds = (logits.squeeze() > 0).long()\n instance_loss = self.instance_loss_fn(logits.squeeze(), top_targets)\n return instance_loss, inst_preds, top_targets, logits\n \n\n def forward(self, h, bag_pred_weight:float, is_tma:bool, label=None, attention_only=False):\n A, h = self.attention_net(h) # NxK \n A = torch.transpose(A, 1, 0) # KxN\n if attention_only:\n return A\n A_raw = A\n A = F.softmax(A, dim=1) # softmax over N\n M = torch.mm(A, h) # shape 1 x self.size_dict[1]\n logits = self.classifiers(M)\n bag_Y_prob = F.softmax(logits.squeeze(), dim=0)\n\n if is_tma:\n k_sample = self.k_sample // 2\n else:\n k_sample = self.k_sample\n\n all_inst_logits = []\n top_p_ids = None\n if bag_pred_weight < 1 and label is not None: \n total_inst_loss = 0.0\n all_inst_preds = []\n all_targets = []\n for i in range(len(self.instance_classifiers)):\n classifier = self.instance_classifiers[i]\n if i == label.item(): #in-the-class:\n is_other_class = (label.item() == self.other_idx)\n instance_loss, inst_preds, targets, inst_logits = self.inst_eval(A, h, classifier, is_tma, is_other_class)\n all_inst_preds.extend(inst_preds.cpu().numpy()) \n all_targets.extend(targets.cpu().numpy())\n #class_probs = F.softmax(inst_logits, dim=1)\n all_inst_logits.append(inst_logits)\n if self.class_weights is not None:\n instance_loss *= self.class_weights[i]\n else: #out-of-the-class\n if self.subtyping:\n instance_loss, inst_preds, targets, inst_logits = self.inst_eval_out(A, h, classifier, is_tma)\n all_inst_preds.extend(inst_preds.cpu().numpy())\n all_targets.extend(targets.cpu().numpy()) \n all_inst_logits.append(inst_logits)\n else:\n continue\n \n total_inst_loss += instance_loss \n\n if self.subtyping:\n # the 2 corrects for the upscaling of the positive instance loss\n total_inst_loss /= 2 * len(self.instance_classifiers)\n else:\n if self.k_sample <= math.ceil(A.shape[1] / 2):\n top_p_ids = torch.topk(A, k_sample)[1][-1] # [1][-1] selects the last index\n else:\n top_p_ids = torch.topk(A, math.ceil(A.shape[1] / 2))[1][-1]\n top_p_ids = top_p_ids.repeat(k_sample)[:k_sample]\n top_p = torch.index_select(h, dim=0, index=top_p_ids)\n for classifier in self.instance_classifiers:\n class_logits = classifier(top_p)\n all_inst_logits.append(class_logits)\n\n \n if self.use_inst_predictions: \n all_inst_logits = torch.concatenate(all_inst_logits, axis=1) # dim k_sample x n_classes\n # take probs of all k_sample classifiers and turn them into multi-class probabilities\n # weight tile instance predictions by A\n if self.k_sample <= math.ceil(A.shape[1] / 2):\n top_p_ids = torch.topk(A, k_sample)[1][-1] # [1][-1] selects the last index\n else:\n top_p_ids = torch.topk(A, math.ceil(A.shape[1] / 2))[1][-1]\n top_p_ids = top_p_ids.repeat(k_sample)[:k_sample]\n\n all_inst_logits =A_raw[0, top_p_ids].reshape(-1, 1) * all_inst_logits\n softmax_inst_probs = torch.softmax(all_inst_logits, dim=1)\n agg_inst_probs = softmax_inst_probs \n agg_inst_probs = torch.mean(agg_inst_probs, dim=0) # result has dim k_sample\n Y_probs = bag_Y_prob * bag_pred_weight + agg_inst_probs * (1 - bag_pred_weight)\n Y_hat = torch.topk(Y_probs, 1, dim=0)[1]\n \n results_dict = {}\n if bag_pred_weight < 1:\n results_dict.update({\n 'all_inst_logits': all_inst_logits.detach().cpu().numpy(),\n 'agg_inst_probs': agg_inst_probs.detach().cpu().numpy()\n })\n if self.use_inst_predictions: \n results_dict.update({\n 'softmax_inst_probs': softmax_inst_probs.detach().cpu().numpy()\n })\n if label is not None:\n results_dict.update({\n 'inst_labels': np.array(all_targets),\n 'inst_preds': np.array(all_inst_preds).flatten(),\n 'instance_loss': total_inst_loss\n })\n\n return logits, Y_probs, Y_hat, A_raw, results_dict\n```\n\n\n\n### Description of the Data Used\n\nFrom the [Cancer Imaging Archive](https://www.cancerimagingarchive.net/) I used the [Ovarian Bevacizumab Response](https://doi.org/10.7937/TCIA.985G-EY35) and the [CPTAC-OV](https://doi.org/10.7937/TCIA.ZS4A-JD58) data. The labels didn't map perfectly to the labels of the competition. For example a label would be Papillary Serous Carcinoma, which I assumed means either HGSC or LGSC. I just used a model trained on the my other data to decide which of the two to select.\n\nI used the Ovarian [Carcinoma Histopathology Dataset](https://www.medicalimageanalysis.com/data/ovarian-carcinomas-histopathology-dataset) by the [Hamarneh Lab](https://www.medicalimageanalysis.com/home).\n\nI also used data from the [Stanford Tissue Microarray Database](https://tma.im/cgi-bin/home.pl), most of which didn't have H&E staining, but at least the cell shapes looked similar and I wanted to also have some TMA data.\n\nI also got in touch with [Cooperative Human Tissue network](https://chtn.cancer.gov), who where so nice to allow me to use the data that was publicly available on the website. I thank them for the time they took to discuss this topic, even though I didn't use their data in the end, as they took the data down from their website after my outreach. As the data wasn't available for other participants anymore, I assume I'm not allowed to use it to train my model.\n\n### Validation Setup\n\nFor a long time I pooled all my data and used 5-fold cross validation, making sure multiple images from the same patient would all be in the same fold. Still this lead to inflated validation scores. I seems like my models where overfitting by using some shortcuts from my datasets that do not generalize. Later I excluded the data from the Harmanreh lab completely for validation which lead to much more reliable cross-validation scores.\n\n### Technical Setup\n\nAs the data size of the competition was so large I trained my models locally on my desktop computer with a GTX 4090 card. Feature extraction would take around 6 hours for all my datasets and training my model would take another hour.\n\n\n### Data sources\n\nA big thank you to everyone who made their pathological image data publicly available. This is a tremendous help for anyone who wants to build deep learning models to improve digital pathology, but especially for individuals without access to close source clinical data.\n\nBevacizumab: Wang et al. *Weakly Supervised Deep Learning for Prediction of Treatment Effectiveness on Ovarian Cancer from Histopathology Images.* Computerized Medical Imaging and Graphics. [https://doi.org/10.1016/j.compmedimag.2022.102093](https://gcc02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdoi.org%2F10.1016%2Fj.compmedimag.2022.102093&data=05|01|kirbyju@mail.nih.gov|a1a4f263214846a156f908da58bd5e3c|14b77578977342d58507251ca2dc2b06|0|0|637919868422135484|Unknown|TWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D|3000|||&sdata=424GtHH8SDUEjvHtaXZvOwt9hcHaZgl36YHL1tzH6T4%3D&reserved=0)\n\nCPTAC-OV: National Cancer Institute Clinical Proteomic Tumor Analysis Consortium (CPTAC). (2020). **The Clinical Proteomic Tumor Analysis Consortium Ovarian Serous Cystadenocarcinoma Collection (CPTAC-OV)** (Version 3) [Data set]. The Cancer Imaging Archive. https://doi.org/10.7937/TCIA.ZS4A-JD58\n\nHarmanreh Data: Köbel, Martin; Kalloger, Steve E.; Baker, Patricia M.; Ewanowich, Carol A.; Arseneau, Jocelyne; Zherebitskiy, Viktor; Abdulkarim, Soran; Leung, Samuel; Duggan, Máire A.; Fontaine, Dan; et al. (2010). \"Diagnosis of ovarian carcinoma cell type is highly reproducible: a transcanadian study\". *The American Journal of Surgical Pathology*, 34(7), 984–993. LWW.\n\nI used some screenshots for healthy tissue from [The Human Protein Atlas](https://www.proteinatlas.org/learn/dictionary/normal/ovary) and the [University of Michigan Histology and Cirtual Microscopy](https://histology.medicine.umich.edu/resources/female-reproductive-system#ovary-oviduct-suggested-readings) page. I zoomed into their full section image of a healthy ovary, zoomed in to the maximum and took screenshots of many differently looking regions.\n\n\n### Open Source Code\n\nA second thank you to everyone who made their code or their model weights openly available. This greatly improves innovation and allows individual contributors to stand on the shoulders of giants.\n\n[CLAM](https://github.com/mahmoodlab/CLAM): Lu, M.Y., Williamson, D.F.K., Chen, T.Y. et al. Data-efficient and weakly supervised computational pathology on whole-slide images. Nat Biomed Eng 5, 555–570 (2021). https://doi.org/10.1038/s41551-020-00682-w\n\n[PyVips](https://libvips.github.io/pyvips)\n\n[HistomicsTK(https://github.com/DigitalSlideArchive/HistomicsTK)\n\n[large_iamge]https://github.com/girder/large_image\n\n\n### Research Papers\n\nWölflein, Georg; Ferber, Dyke; Meneghetti, Asier Rabasco; El Nahhas, Omar S. M.; Truhn, Daniel; Carrero, Zunamys I.; Harrison, David J.; Arandjelović, Ognjen; Kather, Jakob N. (2023). \"A Good Feature Extractor Is All You Need for Weakly Supervised Learning in Histopathology\". arXiv:2311.11772.\n\n\n**Thanks for you interest in my solution and you can fine me on [Twitter here](https://twitter.com/swanint).**\n", "2587482": "Congratulations! You have grown a lot over the past seven years!", "3029690": "There is an IEEE paper that is academic fraud. The title is \"OCEAN - Ovarian Cancer subtypE clAssification and outlier detectionioN using DenseNet121\". It claims that only convolutional networks were used without mentioning multi-instance learning. Then, 2,000 WSI images from the OECEN-UBC competition that our contestants could not get were used for training to obtain a classification accuracy of 99.7%. However, our first place winner only had a test accuracy of 0.6 and a 5-fold cross-validation accuracy of less than 90%", "2827514": "@dantee I have a query regarding the Ovarian Carcinoma Histopathology Dataset by the Hamarneh Lab you used. I'm experiencing bandwidth issues while downloading the dataset, with frequent crashes and speed limitations. Did you encounter similar issues, and if so, do you know of any alternative sources to download the data?", "2587556": "May I ask how you dealt with the compute/time limit in prediction step? I hit the 12-hour timeout when I sampled a mere 20 2560px tiles from each of the full test image using pyvips. [(y, y + h, x, x + w) for y in range(0, image.height, h) for x in range(0, image.width, w)], random.choice() (with repeat), image.crop(), and timeout. Didn't even predict anything, just wrote a dummy submission.csv without passing the tiles through a model." } }