{
  "id": 445449,
  "title": "Dealing with Large Image Size of Carcinomas Subtypes (HGSC, LGSC, MC, EC) and Outliers.",
  "url": "/competitions/UBC-OCEAN/discussion/445449",
  "author_name": "Marília Prata",
  "post_date": "2023-10-07T02:45:40.492000",
  "votes": 44,
  "comment_count": 6,
  "views": 0,
  "content": "<h1>For those facing issues with Large image size.</h1>\n<ul>\n<li><p>By Darien Schettler:<br>\n\"How Darien Deal With WSI (Whole Slide Images aka. RAM Crashingly Large Images)\" :<br>\nOpenSlide<br>\npyvips and libvips<br>\n<a href=\"https://www.kaggle.com/code/dschettler8845/mcsai-how-to-interact-with-large-tif-files\" target=\"_blank\">https://www.kaggle.com/code/dschettler8845/mcsai-how-to-interact-with-large-tif-files</a><br>\n<a href=\"https://www.kaggle.com/competitions/mayo-clinic-strip-ai/discussion/335976\" target=\"_blank\">https://www.kaggle.com/competitions/mayo-clinic-strip-ai/discussion/335976</a></p></li>\n<li><p>By Jirka Borovec:<br>\nYou could read it with Pillow.<br>\nJust need to set the higher max image size, see Jirka's Borovec notebook with resize and crop<br>\n<a href=\"https://www.kaggle.com/code/jirkaborovec/bloodclots-classif-eda-load-crop-images\" target=\"_blank\">https://www.kaggle.com/code/jirkaborovec/bloodclots-classif-eda-load-crop-images</a></p></li>\n</ul>\n<p>Cancer Subtype: EDA &amp; load WSI + prune BG<br>\n<a href=\"https://www.kaggle.com/code/jirkaborovec/cancer-subtype-eda-load-wsi-prune-bg\" target=\"_blank\">https://www.kaggle.com/code/jirkaborovec/cancer-subtype-eda-load-wsi-prune-bg</a></p>\n<p>Check also all Jirca's Borovec Public code on this UBC-OCEAN Competition.</p>\n<ul>\n<li>By NickUzmenkov:  You can use rasterio to handle heavy images, as it doesn't load the whole image into memory at once.<br>\n<a href=\"https://www.kaggle.com/competitions/mayo-clinic-strip-ai/discussion/335739\" target=\"_blank\">https://www.kaggle.com/competitions/mayo-clinic-strip-ai/discussion/335739</a></li>\n</ul>\n<h1>Carcinomas: HGSC, LGSC, MC, EC</h1>\n<p>\"High-grade serous carcinoma (HGSC) is a type of tumour that arises from the serous epithelial layer in the abdominopelvic cavity and is mainly found in the ovary. HGSCs make up the majority of ovarian cancer cases and have the lowest survival rates.\"</p>\n<p><a href=\"https://en.wikipedia.org/wiki/High-grade_serous_carcinoma\" target=\"_blank\">https://en.wikipedia.org/wiki/High-grade_serous_carcinoma</a></p>\n<p>\"Low-grade serous carcinoma (LGSC) of the ovary is a rare histological subtype of epithelial ovarian carcinoma. It has distinct clinical behavior and a specific molecular profile. Compared with high-grade serous carcinoma, this tumor presents at a younger age, has an indolent course, and is associated with prolonged survival.\"</p>\n<p><a href=\"https://pubmed.ncbi.nlm.nih.gov/35204549/\" target=\"_blank\">https://pubmed.ncbi.nlm.nih.gov/35204549/</a></p>\n<p>\"The incidence of mucinous carcinoma (MC) ranges from 3% to 11% in Western countries and over 10% in most Asian countries, with its frequency varying among each region. Furthermore, MC has been detected in early stages of cancer, has a lower response to chemotherapy, and has been associated with a worse prognosis, particularly in advanced stages than in high-grade serous carcinomas. Due to the relatively low incidence of MC, appropriate individual treatment has not been established.\"</p>\n<p><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8360460/\" target=\"_blank\">https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8360460/</a></p>\n<p>\"Endometrioid carcinoma comprises 10% to 20% of ovarian carcinomas and is bilateral in 28% of cases. Up to 42% are associated with endometriosis, and 15% to 20% with a coexisting adenocarcinoma of the endometrium. Endometrioid carcinomas are usually cystic and solid tumors with foci of necrosis and hemorrhage.\"</p>\n<p><a href=\"https://www.sciencedirect.com/topics/medicine-and-dentistry/endometrioid-carcinoma#:~:text=Endometrioid%20carcinoma%20comprises%2010%25%20to,bilateral%20in%2028%25%20of%20cases.&amp;text=Up%20to%2042%25%20are%20associated,coexisting%20adenocarcinoma%20of%20the%20endometrium.&amp;text=Endometrioid%20carcinomas%20are%20usually%20cystic,foci%20of%20necrosis%20and%20hemorrhage\" target=\"_blank\">https://www.sciencedirect.com/topics/medicine-and-dentistry/endometrioid-carcinoma#:~:text=Endometrioid%20carcinoma%20comprises%2010%25%20to,bilateral%20in%2028%25%20of%20cases.&amp;text=Up%20to%2042%25%20are%20associated,coexisting%20adenocarcinoma%20of%20the%20endometrium.&amp;text=Endometrioid%20carcinomas%20are%20usually%20cystic,foci%20of%20necrosis%20and%20hemorrhage</a>.</p>\n<h1>Investigating Outliers</h1>\n<p>\"What can we gain from looking at the outliers?: An investigation into long and short-term ovarian cancer survivors\"</p>\n<p>\"Some patients with high-grade serous ovarian cancer (HGSOC) respond exceptionally well to therapy, while others experience rapid disease relapse. The mechanisms behind these disparate outcomes are poorly understood.\"</p>\n<p>\"Researchers investigated the differences between HGSOC tumour samples from exceptionally long-term survivors – those who have survived for over 10 years after treatment – and tumour samples from exceptionally short-term survivors – those whose succumb to the disease in less than two years after therapy.\"</p>\n<p><a href=\"https://oicr.on.ca/what-can-we-gain-from-looking-at-the-outliers-an-investigation-into-long-and-short-term-ovarian-cancer-survivors/\" target=\"_blank\">https://oicr.on.ca/what-can-we-gain-from-looking-at-the-outliers-an-investigation-into-long-and-short-term-ovarian-cancer-survivors/</a></p>\n<h1>Outliers (Sidechains, Ramachandran, RSR)</h1>\n<p>RAMACHANDRAN OUTLIERS<br>\n\"Ramachandran outliers are those amino acids with non-favorable dihedral angles, and the Ramachandran plot is a powerful tool for making those evident. Most of the time, Ramachandran outliers are a consequence of mistakes during the data processing.\"However, sometimes Ramachandran outliers might play a special role in function.\"</p>\n<p><a href=\"https://proteopedia.org/wiki/index.php/Ramachandran_outlier\" target=\"_blank\">https://proteopedia.org/wiki/index.php/Ramachandran_outlier</a></p>\n<p>SIDECHAINS OUTLIERS<br>\n\"Protein sidechains mostly adopt certain (combinations of) preferred torsion angle values (called rotamers or rotameric conformers), much like their backbone torsion angles (as assessed in the Ramachandran analysis). MolProbity considers the sidechain conformation of a residue to be an outlier if its set of torsion angles is not similar to any preferred combination. The sidechain outlier score is calculated as the percentage of residues with an unusual sidechain conformation with respect to the total number of residues for which the assessment is available.\"</p>\n<p>RSRZ OUTLIERS<br>\n\"The real-space R-value (RSR) is a measure of the quality of fit between a part of an atomic model (in this case, one residue) and the data in real space . The RSR Z-score (RSRZ) is a normalisation of RSR specific to a residue type and a resolution bin . RSRZ is calculated only for standard amino acids and nucleotides in protein, DNA and RNA chains. A residue is considered an RSRZ outlier if its RSRZ value is greater than 2. The RSRZ outlier score as shown in the slider graph is calculated as the percentage RSRZ outliers with respect to the total number of residues for which RSRZ was computed. This is calculated by the EDS (Electron-Density Server) component of the validation pipeline which is a re-implementation of the software used by the Uppsala EDS server.\"</p>\n<p><a href=\"https://www.wwpdb.org/validation/XrayValidationReportHelp#:~:text=The%20sidechain%20outlier%20score%20is,which%20the%20assessment%20is%20available\" target=\"_blank\">https://www.wwpdb.org/validation/XrayValidationReportHelp#:~:text=The%20sidechain%20outlier%20score%20is,which%20the%20assessment%20is%20available</a>.</p>\n<h1>That's my beginner effort trying to bring some insights for this UBC-OCEAN Competition.</h1>\n<p>Contributions (e.g. comments)  by experienced Kagglers will be highly appreciated.</p>",
  "messages": [
    {
      "id": 2472113,
      "postDate": "2023-10-07T02:45:40.493Z",
      "content": "<h1>For those facing issues with Large image size.</h1>\n<ul>\n<li><p>By Darien Schettler:<br>\n\"How Darien Deal With WSI (Whole Slide Images aka. RAM Crashingly Large Images)\" :<br>\nOpenSlide<br>\npyvips and libvips<br>\n<a href=\"https://www.kaggle.com/code/dschettler8845/mcsai-how-to-interact-with-large-tif-files\" target=\"_blank\">https://www.kaggle.com/code/dschettler8845/mcsai-how-to-interact-with-large-tif-files</a><br>\n<a href=\"https://www.kaggle.com/competitions/mayo-clinic-strip-ai/discussion/335976\" target=\"_blank\">https://www.kaggle.com/competitions/mayo-clinic-strip-ai/discussion/335976</a></p></li>\n<li><p>By Jirka Borovec:<br>\nYou could read it with Pillow.<br>\nJust need to set the higher max image size, see Jirka's Borovec notebook with resize and crop<br>\n<a href=\"https://www.kaggle.com/code/jirkaborovec/bloodclots-classif-eda-load-crop-images\" target=\"_blank\">https://www.kaggle.com/code/jirkaborovec/bloodclots-classif-eda-load-crop-images</a></p></li>\n</ul>\n<p>Cancer Subtype: EDA &amp; load WSI + prune BG<br>\n<a href=\"https://www.kaggle.com/code/jirkaborovec/cancer-subtype-eda-load-wsi-prune-bg\" target=\"_blank\">https://www.kaggle.com/code/jirkaborovec/cancer-subtype-eda-load-wsi-prune-bg</a></p>\n<p>Check also all Jirca's Borovec Public code on this UBC-OCEAN Competition.</p>\n<ul>\n<li>By NickUzmenkov:  You can use rasterio to handle heavy images, as it doesn't load the whole image into memory at once.<br>\n<a href=\"https://www.kaggle.com/competitions/mayo-clinic-strip-ai/discussion/335739\" target=\"_blank\">https://www.kaggle.com/competitions/mayo-clinic-strip-ai/discussion/335739</a></li>\n</ul>\n<h1>Carcinomas: HGSC, LGSC, MC, EC</h1>\n<p>\"High-grade serous carcinoma (HGSC) is a type of tumour that arises from the serous epithelial layer in the abdominopelvic cavity and is mainly found in the ovary. HGSCs make up the majority of ovarian cancer cases and have the lowest survival rates.\"</p>\n<p><a href=\"https://en.wikipedia.org/wiki/High-grade_serous_carcinoma\" target=\"_blank\">https://en.wikipedia.org/wiki/High-grade_serous_carcinoma</a></p>\n<p>\"Low-grade serous carcinoma (LGSC) of the ovary is a rare histological subtype of epithelial ovarian carcinoma. It has distinct clinical behavior and a specific molecular profile. Compared with high-grade serous carcinoma, this tumor presents at a younger age, has an indolent course, and is associated with prolonged survival.\"</p>\n<p><a href=\"https://pubmed.ncbi.nlm.nih.gov/35204549/\" target=\"_blank\">https://pubmed.ncbi.nlm.nih.gov/35204549/</a></p>\n<p>\"The incidence of mucinous carcinoma (MC) ranges from 3% to 11% in Western countries and over 10% in most Asian countries, with its frequency varying among each region. Furthermore, MC has been detected in early stages of cancer, has a lower response to chemotherapy, and has been associated with a worse prognosis, particularly in advanced stages than in high-grade serous carcinomas. Due to the relatively low incidence of MC, appropriate individual treatment has not been established.\"</p>\n<p><a href=\"https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8360460/\" target=\"_blank\">https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8360460/</a></p>\n<p>\"Endometrioid carcinoma comprises 10% to 20% of ovarian carcinomas and is bilateral in 28% of cases. Up to 42% are associated with endometriosis, and 15% to 20% with a coexisting adenocarcinoma of the endometrium. Endometrioid carcinomas are usually cystic and solid tumors with foci of necrosis and hemorrhage.\"</p>\n<p><a href=\"https://www.sciencedirect.com/topics/medicine-and-dentistry/endometrioid-carcinoma#:~:text=Endometrioid%20carcinoma%20comprises%2010%25%20to,bilateral%20in%2028%25%20of%20cases.&amp;text=Up%20to%2042%25%20are%20associated,coexisting%20adenocarcinoma%20of%20the%20endometrium.&amp;text=Endometrioid%20carcinomas%20are%20usually%20cystic,foci%20of%20necrosis%20and%20hemorrhage\" target=\"_blank\">https://www.sciencedirect.com/topics/medicine-and-dentistry/endometrioid-carcinoma#:~:text=Endometrioid%20carcinoma%20comprises%2010%25%20to,bilateral%20in%2028%25%20of%20cases.&amp;text=Up%20to%2042%25%20are%20associated,coexisting%20adenocarcinoma%20of%20the%20endometrium.&amp;text=Endometrioid%20carcinomas%20are%20usually%20cystic,foci%20of%20necrosis%20and%20hemorrhage</a>.</p>\n<h1>Investigating Outliers</h1>\n<p>\"What can we gain from looking at the outliers?: An investigation into long and short-term ovarian cancer survivors\"</p>\n<p>\"Some patients with high-grade serous ovarian cancer (HGSOC) respond exceptionally well to therapy, while others experience rapid disease relapse. The mechanisms behind these disparate outcomes are poorly understood.\"</p>\n<p>\"Researchers investigated the differences between HGSOC tumour samples from exceptionally long-term survivors – those who have survived for over 10 years after treatment – and tumour samples from exceptionally short-term survivors – those whose succumb to the disease in less than two years after therapy.\"</p>\n<p><a href=\"https://oicr.on.ca/what-can-we-gain-from-looking-at-the-outliers-an-investigation-into-long-and-short-term-ovarian-cancer-survivors/\" target=\"_blank\">https://oicr.on.ca/what-can-we-gain-from-looking-at-the-outliers-an-investigation-into-long-and-short-term-ovarian-cancer-survivors/</a></p>\n<h1>Outliers (Sidechains, Ramachandran, RSR)</h1>\n<p>RAMACHANDRAN OUTLIERS<br>\n\"Ramachandran outliers are those amino acids with non-favorable dihedral angles, and the Ramachandran plot is a powerful tool for making those evident. Most of the time, Ramachandran outliers are a consequence of mistakes during the data processing.\"However, sometimes Ramachandran outliers might play a special role in function.\"</p>\n<p><a href=\"https://proteopedia.org/wiki/index.php/Ramachandran_outlier\" target=\"_blank\">https://proteopedia.org/wiki/index.php/Ramachandran_outlier</a></p>\n<p>SIDECHAINS OUTLIERS<br>\n\"Protein sidechains mostly adopt certain (combinations of) preferred torsion angle values (called rotamers or rotameric conformers), much like their backbone torsion angles (as assessed in the Ramachandran analysis). MolProbity considers the sidechain conformation of a residue to be an outlier if its set of torsion angles is not similar to any preferred combination. The sidechain outlier score is calculated as the percentage of residues with an unusual sidechain conformation with respect to the total number of residues for which the assessment is available.\"</p>\n<p>RSRZ OUTLIERS<br>\n\"The real-space R-value (RSR) is a measure of the quality of fit between a part of an atomic model (in this case, one residue) and the data in real space . The RSR Z-score (RSRZ) is a normalisation of RSR specific to a residue type and a resolution bin . RSRZ is calculated only for standard amino acids and nucleotides in protein, DNA and RNA chains. A residue is considered an RSRZ outlier if its RSRZ value is greater than 2. The RSRZ outlier score as shown in the slider graph is calculated as the percentage RSRZ outliers with respect to the total number of residues for which RSRZ was computed. This is calculated by the EDS (Electron-Density Server) component of the validation pipeline which is a re-implementation of the software used by the Uppsala EDS server.\"</p>\n<p><a href=\"https://www.wwpdb.org/validation/XrayValidationReportHelp#:~:text=The%20sidechain%20outlier%20score%20is,which%20the%20assessment%20is%20available\" target=\"_blank\">https://www.wwpdb.org/validation/XrayValidationReportHelp#:~:text=The%20sidechain%20outlier%20score%20is,which%20the%20assessment%20is%20available</a>.</p>\n<h1>That's my beginner effort trying to bring some insights for this UBC-OCEAN Competition.</h1>\n<p>Contributions (e.g. comments)  by experienced Kagglers will be highly appreciated.</p>",
      "rawMarkdown": "#For those facing issues with Large image size.\n\n- By Darien Schettler:\n\"How Darien Deal With WSI (Whole Slide Images aka. RAM Crashingly Large Images)\" :\n OpenSlide\npyvips and libvips\nhttps://www.kaggle.com/code/dschettler8845/mcsai-how-to-interact-with-large-tif-files\nhttps://www.kaggle.com/competitions/mayo-clinic-strip-ai/discussion/335976\n\n- By Jirka Borovec:\nYou could read it with Pillow.\nJust need to set the higher max image size, see Jirka's Borovec notebook with resize and crop\nhttps://www.kaggle.com/code/jirkaborovec/bloodclots-classif-eda-load-crop-images\n\nCancer Subtype: EDA & load WSI + prune BG\nhttps://www.kaggle.com/code/jirkaborovec/cancer-subtype-eda-load-wsi-prune-bg\n\nCheck also all Jirca's Borovec Public code on this UBC-OCEAN Competition.\n\n- By NickUzmenkov:  You can use rasterio to handle heavy images, as it doesn't load the whole image into memory at once.\nhttps://www.kaggle.com/competitions/mayo-clinic-strip-ai/discussion/335739\n\n#Carcinomas: HGSC, LGSC, MC, EC\n\n\"High-grade serous carcinoma (HGSC) is a type of tumour that arises from the serous epithelial layer in the abdominopelvic cavity and is mainly found in the ovary. HGSCs make up the majority of ovarian cancer cases and have the lowest survival rates.\"\n\nhttps://en.wikipedia.org/wiki/High-grade_serous_carcinoma\n\n\"Low-grade serous carcinoma (LGSC) of the ovary is a rare histological subtype of epithelial ovarian carcinoma. It has distinct clinical behavior and a specific molecular profile. Compared with high-grade serous carcinoma, this tumor presents at a younger age, has an indolent course, and is associated with prolonged survival.\"\n\nhttps://pubmed.ncbi.nlm.nih.gov/35204549/\n\n\"The incidence of mucinous carcinoma (MC) ranges from 3% to 11% in Western countries and over 10% in most Asian countries, with its frequency varying among each region. Furthermore, MC has been detected in early stages of cancer, has a lower response to chemotherapy, and has been associated with a worse prognosis, particularly in advanced stages than in high-grade serous carcinomas. Due to the relatively low incidence of MC, appropriate individual treatment has not been established.\"\n\nhttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC8360460/\n\n\"Endometrioid carcinoma comprises 10% to 20% of ovarian carcinomas and is bilateral in 28% of cases. Up to 42% are associated with endometriosis, and 15% to 20% with a coexisting adenocarcinoma of the endometrium. Endometrioid carcinomas are usually cystic and solid tumors with foci of necrosis and hemorrhage.\"\n\nhttps://www.sciencedirect.com/topics/medicine-and-dentistry/endometrioid-carcinoma#:~:text=Endometrioid%20carcinoma%20comprises%2010%25%20to,bilateral%20in%2028%25%20of%20cases.&text=Up%20to%2042%25%20are%20associated,coexisting%20adenocarcinoma%20of%20the%20endometrium.&text=Endometrioid%20carcinomas%20are%20usually%20cystic,foci%20of%20necrosis%20and%20hemorrhage.\n\n#Investigating Outliers\n\n\"What can we gain from looking at the outliers?: An investigation into long and short-term ovarian cancer survivors\"\n\n\"Some patients with high-grade serous ovarian cancer (HGSOC) respond exceptionally well to therapy, while others experience rapid disease relapse. The mechanisms behind these disparate outcomes are poorly understood.\"\n\n\"Researchers investigated the differences between HGSOC tumour samples from exceptionally long-term survivors – those who have survived for over 10 years after treatment – and tumour samples from exceptionally short-term survivors – those whose succumb to the disease in less than two years after therapy.\"\n\nhttps://oicr.on.ca/what-can-we-gain-from-looking-at-the-outliers-an-investigation-into-long-and-short-term-ovarian-cancer-survivors/\n\n#Outliers (Sidechains, Ramachandran, RSR)\n\nRAMACHANDRAN OUTLIERS\n\"Ramachandran outliers are those amino acids with non-favorable dihedral angles, and the Ramachandran plot is a powerful tool for making those evident. Most of the time, Ramachandran outliers are a consequence of mistakes during the data processing.\"However, sometimes Ramachandran outliers might play a special role in function.\"\n\nhttps://proteopedia.org/wiki/index.php/Ramachandran_outlier\n\nSIDECHAINS OUTLIERS\n\"Protein sidechains mostly adopt certain (combinations of) preferred torsion angle values (called rotamers or rotameric conformers), much like their backbone torsion angles (as assessed in the Ramachandran analysis). MolProbity considers the sidechain conformation of a residue to be an outlier if its set of torsion angles is not similar to any preferred combination. The sidechain outlier score is calculated as the percentage of residues with an unusual sidechain conformation with respect to the total number of residues for which the assessment is available.\"\n\nRSRZ OUTLIERS\n\"The real-space R-value (RSR) is a measure of the quality of fit between a part of an atomic model (in this case, one residue) and the data in real space . The RSR Z-score (RSRZ) is a normalisation of RSR specific to a residue type and a resolution bin . RSRZ is calculated only for standard amino acids and nucleotides in protein, DNA and RNA chains. A residue is considered an RSRZ outlier if its RSRZ value is greater than 2. The RSRZ outlier score as shown in the slider graph is calculated as the percentage RSRZ outliers with respect to the total number of residues for which RSRZ was computed. This is calculated by the EDS (Electron-Density Server) component of the validation pipeline which is a re-implementation of the software used by the Uppsala EDS server.\"\n\nhttps://www.wwpdb.org/validation/XrayValidationReportHelp#:~:text=The%20sidechain%20outlier%20score%20is,which%20the%20assessment%20is%20available.\n\n#That's my beginner effort trying to bring some insights for this UBC-OCEAN Competition.\n\nContributions (e.g. comments)  by experienced Kagglers will be highly appreciated.\n",
      "votes": 43
    },
    {
      "id": 2484838,
      "postDate": "2023-10-16T17:51:35.077Z",
      "content": "<p>BTW, I have also added loading WSI with <code>pyvips</code> installation for offline use (can be handy for inference)<br>\n<a href=\"https://www.kaggle.com/code/jirkaborovec/cancer-subtype-eda-load-wsi-prune-bg\" target=\"_blank\">https://www.kaggle.com/code/jirkaborovec/cancer-subtype-eda-load-wsi-prune-bg</a></p>",
      "rawMarkdown": "BTW, I have also added loading WSI with `pyvips` installation for offline use (can be handy for inference)\nhttps://www.kaggle.com/code/jirkaborovec/cancer-subtype-eda-load-wsi-prune-bg",
      "votes": 3,
      "replies": [
        {
          "id": 2485072,
          "postDate": "2023-10-16T22:57:51.160Z",
          "content": "<p>Hi Jirca,</p>\n<p>I added your link inside the topic. Additionally, I mentioned your remarkable contributions (Six Notebooks till now) to this UBC-OCEAN Competition.</p>",
          "rawMarkdown": "Hi Jirca,\n\nI added your link inside the topic. Additionally, I mentioned your remarkable contributions (Six Notebooks till now) to this UBC-OCEAN Competition."
        }
      ]
    },
    {
      "id": 2473134,
      "postDate": "2023-10-08T01:08:55.130Z",
      "content": "<p>Thanks! Do you have any advice on dealing with out-of-memory problems? As I loop over the test image to create the predictions, I quickly run out of memory after a few predictions. </p>\n<p>My last try was to include garbage collection 'collect()' after every prediction.</p>",
      "rawMarkdown": "Thanks! Do you have any advice on dealing with out-of-memory problems? As I loop over the test image to create the predictions, I quickly run out of memory after a few predictions. \n\nMy last try was to include garbage collection 'collect()' after every prediction.",
      "votes": 1,
      "replies": [
        {
          "id": 2473138,
          "postDate": "2023-10-08T01:24:16.337Z",
          "content": "<p>Hi Delgado,</p>\n<p>Save Memory. Tip by Kirderf:</p>\n<p>\"How it works: He saved the codes for every solution in an own python file which he run separately in isolated memory. He also take advantages of the new Kaggle env. feature that checks if we are in the hidden test running phase or only in the submitting phase, and use this by saving a submission sample in submitting phase and when in the phase of running the hidden private test phase, using the real code. This use only seconds of the GPU quota.\"</p>\n<p><a href=\"https://www.kaggle.com/competitions/mayo-clinic-strip-ai/discussion/349200\" target=\"_blank\">https://www.kaggle.com/competitions/mayo-clinic-strip-ai/discussion/349200</a></p>\n<p>And Kirderf's code: Mayo inference, memory and GPU quota efficient<br>\n<a href=\"https://www.kaggle.com/code/kirderf/mayo-inference-memory-and-gpu-quota-efficient\" target=\"_blank\">https://www.kaggle.com/code/kirderf/mayo-inference-memory-and-gpu-quota-efficient</a></p>",
          "rawMarkdown": "Hi Delgado,\n\nSave Memory. Tip by Kirderf:\n\n\"How it works: He saved the codes for every solution in an own python file which he run separately in isolated memory. He also take advantages of the new Kaggle env. feature that checks if we are in the hidden test running phase or only in the submitting phase, and use this by saving a submission sample in submitting phase and when in the phase of running the hidden private test phase, using the real code. This use only seconds of the GPU quota.\"\n\nhttps://www.kaggle.com/competitions/mayo-clinic-strip-ai/discussion/349200\n\nAnd Kirderf's code: Mayo inference, memory and GPU quota efficient\nhttps://www.kaggle.com/code/kirderf/mayo-inference-memory-and-gpu-quota-efficient",
          "votes": 4
        }
      ]
    },
    {
      "id": 2472249,
      "postDate": "2023-10-07T06:43:23.493Z",
      "content": "<p>Great summary as usual <a href=\"https://www.kaggle.com/mpwolke\" target=\"_blank\">@mpwolke</a> </p>",
      "rawMarkdown": "Great summary as usual @mpwolke ",
      "votes": 1,
      "replies": [
        {
          "id": 2473136,
          "postDate": "2023-10-08T01:15:21.293Z",
          "content": "<p>Thank you Rahman for your appreciation.</p>",
          "rawMarkdown": "Thank you Rahman for your appreciation."
        }
      ]
    }
  ],
  "comments": [
    {
      "id": 2484838,
      "author_name": "Jirka",
      "author_url": "",
      "post_date": "2023-10-16T17:51:35.077000",
      "content": "<p>BTW, I have also added loading WSI with <code>pyvips</code> installation for offline use (can be handy for inference)<br>\n<a href=\"https://www.kaggle.com/code/jirkaborovec/cancer-subtype-eda-load-wsi-prune-bg\" target=\"_blank\">https://www.kaggle.com/code/jirkaborovec/cancer-subtype-eda-load-wsi-prune-bg</a></p>",
      "votes": 3,
      "replies": [
        {
          "id": 2485072,
          "author_name": "Marília Prata",
          "author_url": "",
          "post_date": "2023-10-16T22:57:51.160000",
          "content": "<p>Hi Jirca,</p>\n<p>I added your link inside the topic. Additionally, I mentioned your remarkable contributions (Six Notebooks till now) to this UBC-OCEAN Competition.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    },
    {
      "id": 2473134,
      "author_name": "Jose Mauricio Delgado",
      "author_url": "",
      "post_date": "2023-10-08T01:08:55.130000",
      "content": "<p>Thanks! Do you have any advice on dealing with out-of-memory problems? As I loop over the test image to create the predictions, I quickly run out of memory after a few predictions. </p>\n<p>My last try was to include garbage collection 'collect()' after every prediction.</p>",
      "votes": 1,
      "replies": [
        {
          "id": 2473138,
          "author_name": "Marília Prata",
          "author_url": "",
          "post_date": "2023-10-08T01:24:16.337000",
          "content": "<p>Hi Delgado,</p>\n<p>Save Memory. Tip by Kirderf:</p>\n<p>\"How it works: He saved the codes for every solution in an own python file which he run separately in isolated memory. He also take advantages of the new Kaggle env. feature that checks if we are in the hidden test running phase or only in the submitting phase, and use this by saving a submission sample in submitting phase and when in the phase of running the hidden private test phase, using the real code. This use only seconds of the GPU quota.\"</p>\n<p><a href=\"https://www.kaggle.com/competitions/mayo-clinic-strip-ai/discussion/349200\" target=\"_blank\">https://www.kaggle.com/competitions/mayo-clinic-strip-ai/discussion/349200</a></p>\n<p>And Kirderf's code: Mayo inference, memory and GPU quota efficient<br>\n<a href=\"https://www.kaggle.com/code/kirderf/mayo-inference-memory-and-gpu-quota-efficient\" target=\"_blank\">https://www.kaggle.com/code/kirderf/mayo-inference-memory-and-gpu-quota-efficient</a></p>",
          "votes": 4,
          "replies": []
        }
      ]
    },
    {
      "id": 2472249,
      "author_name": "Kalilur Rahman",
      "author_url": "",
      "post_date": "2023-10-07T06:43:23.493000",
      "content": "<p>Great summary as usual <a href=\"https://www.kaggle.com/mpwolke\" target=\"_blank\">@mpwolke</a> </p>",
      "votes": 1,
      "replies": [
        {
          "id": 2473136,
          "author_name": "Marília Prata",
          "author_url": "",
          "post_date": "2023-10-08T01:15:21.293000",
          "content": "<p>Thank you Rahman for your appreciation.</p>",
          "votes": 0,
          "replies": []
        }
      ]
    }
  ],
  "raw_markdown_by_id": {
    "2472113": "#For those facing issues with Large image size.\n\n- By Darien Schettler:\n\"How Darien Deal With WSI (Whole Slide Images aka. RAM Crashingly Large Images)\" :\n OpenSlide\npyvips and libvips\nhttps://www.kaggle.com/code/dschettler8845/mcsai-how-to-interact-with-large-tif-files\nhttps://www.kaggle.com/competitions/mayo-clinic-strip-ai/discussion/335976\n\n- By Jirka Borovec:\nYou could read it with Pillow.\nJust need to set the higher max image size, see Jirka's Borovec notebook with resize and crop\nhttps://www.kaggle.com/code/jirkaborovec/bloodclots-classif-eda-load-crop-images\n\nCancer Subtype: EDA & load WSI + prune BG\nhttps://www.kaggle.com/code/jirkaborovec/cancer-subtype-eda-load-wsi-prune-bg\n\nCheck also all Jirca's Borovec Public code on this UBC-OCEAN Competition.\n\n- By NickUzmenkov:  You can use rasterio to handle heavy images, as it doesn't load the whole image into memory at once.\nhttps://www.kaggle.com/competitions/mayo-clinic-strip-ai/discussion/335739\n\n#Carcinomas: HGSC, LGSC, MC, EC\n\n\"High-grade serous carcinoma (HGSC) is a type of tumour that arises from the serous epithelial layer in the abdominopelvic cavity and is mainly found in the ovary. HGSCs make up the majority of ovarian cancer cases and have the lowest survival rates.\"\n\nhttps://en.wikipedia.org/wiki/High-grade_serous_carcinoma\n\n\"Low-grade serous carcinoma (LGSC) of the ovary is a rare histological subtype of epithelial ovarian carcinoma. It has distinct clinical behavior and a specific molecular profile. Compared with high-grade serous carcinoma, this tumor presents at a younger age, has an indolent course, and is associated with prolonged survival.\"\n\nhttps://pubmed.ncbi.nlm.nih.gov/35204549/\n\n\"The incidence of mucinous carcinoma (MC) ranges from 3% to 11% in Western countries and over 10% in most Asian countries, with its frequency varying among each region. Furthermore, MC has been detected in early stages of cancer, has a lower response to chemotherapy, and has been associated with a worse prognosis, particularly in advanced stages than in high-grade serous carcinomas. Due to the relatively low incidence of MC, appropriate individual treatment has not been established.\"\n\nhttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC8360460/\n\n\"Endometrioid carcinoma comprises 10% to 20% of ovarian carcinomas and is bilateral in 28% of cases. Up to 42% are associated with endometriosis, and 15% to 20% with a coexisting adenocarcinoma of the endometrium. Endometrioid carcinomas are usually cystic and solid tumors with foci of necrosis and hemorrhage.\"\n\nhttps://www.sciencedirect.com/topics/medicine-and-dentistry/endometrioid-carcinoma#:~:text=Endometrioid%20carcinoma%20comprises%2010%25%20to,bilateral%20in%2028%25%20of%20cases.&text=Up%20to%2042%25%20are%20associated,coexisting%20adenocarcinoma%20of%20the%20endometrium.&text=Endometrioid%20carcinomas%20are%20usually%20cystic,foci%20of%20necrosis%20and%20hemorrhage.\n\n#Investigating Outliers\n\n\"What can we gain from looking at the outliers?: An investigation into long and short-term ovarian cancer survivors\"\n\n\"Some patients with high-grade serous ovarian cancer (HGSOC) respond exceptionally well to therapy, while others experience rapid disease relapse. The mechanisms behind these disparate outcomes are poorly understood.\"\n\n\"Researchers investigated the differences between HGSOC tumour samples from exceptionally long-term survivors – those who have survived for over 10 years after treatment – and tumour samples from exceptionally short-term survivors – those whose succumb to the disease in less than two years after therapy.\"\n\nhttps://oicr.on.ca/what-can-we-gain-from-looking-at-the-outliers-an-investigation-into-long-and-short-term-ovarian-cancer-survivors/\n\n#Outliers (Sidechains, Ramachandran, RSR)\n\nRAMACHANDRAN OUTLIERS\n\"Ramachandran outliers are those amino acids with non-favorable dihedral angles, and the Ramachandran plot is a powerful tool for making those evident. Most of the time, Ramachandran outliers are a consequence of mistakes during the data processing.\"However, sometimes Ramachandran outliers might play a special role in function.\"\n\nhttps://proteopedia.org/wiki/index.php/Ramachandran_outlier\n\nSIDECHAINS OUTLIERS\n\"Protein sidechains mostly adopt certain (combinations of) preferred torsion angle values (called rotamers or rotameric conformers), much like their backbone torsion angles (as assessed in the Ramachandran analysis). MolProbity considers the sidechain conformation of a residue to be an outlier if its set of torsion angles is not similar to any preferred combination. The sidechain outlier score is calculated as the percentage of residues with an unusual sidechain conformation with respect to the total number of residues for which the assessment is available.\"\n\nRSRZ OUTLIERS\n\"The real-space R-value (RSR) is a measure of the quality of fit between a part of an atomic model (in this case, one residue) and the data in real space . The RSR Z-score (RSRZ) is a normalisation of RSR specific to a residue type and a resolution bin . RSRZ is calculated only for standard amino acids and nucleotides in protein, DNA and RNA chains. A residue is considered an RSRZ outlier if its RSRZ value is greater than 2. The RSRZ outlier score as shown in the slider graph is calculated as the percentage RSRZ outliers with respect to the total number of residues for which RSRZ was computed. This is calculated by the EDS (Electron-Density Server) component of the validation pipeline which is a re-implementation of the software used by the Uppsala EDS server.\"\n\nhttps://www.wwpdb.org/validation/XrayValidationReportHelp#:~:text=The%20sidechain%20outlier%20score%20is,which%20the%20assessment%20is%20available.\n\n#That's my beginner effort trying to bring some insights for this UBC-OCEAN Competition.\n\nContributions (e.g. comments)  by experienced Kagglers will be highly appreciated.\n",
    "2484838": "BTW, I have also added loading WSI with `pyvips` installation for offline use (can be handy for inference)\nhttps://www.kaggle.com/code/jirkaborovec/cancer-subtype-eda-load-wsi-prune-bg",
    "2473134": "Thanks! Do you have any advice on dealing with out-of-memory problems? As I loop over the test image to create the predictions, I quickly run out of memory after a few predictions. \n\nMy last try was to include garbage collection 'collect()' after every prediction.",
    "2472249": "Great summary as usual @mpwolke "
  }
}