{"metadata":{"kernelspec":{"language":"python","display_name":"Python 3","name":"python3"},"language_info":{"pygments_lexer":"ipython3","nbconvert_exporter":"python","version":"3.6.4","file_extension":".py","codemirror_mode":{"name":"ipython","version":3},"name":"python","mimetype":"text/x-python"}},"nbformat_minor":4,"nbformat":4,"cells":[{"cell_type":"markdown","source":"# **Background**","metadata":{}},{"cell_type":"markdown","source":"According to the WHO, breast cancer is the most commonly occurring cancer worldwide. In 2020 alone, there were 2.3 million new breast cancer diagnoses and 685,000 deaths. Regular mammograms (mammogram is the low-dose x-rays of the breast) is the process which can helps to detect breast cancer at its early stage, followed by other methodologies like ultrasounds, other imaging tests and biopsies. \nDepending on the patient's condition, the mammography test price ranges from Rs. 800 and could go up to Rs. 3000 in India. Currently, screening mammography systems are expensive to operate since early diagnosis of breast cancer necessitates the expertise of highly trained human observers. This issue will probably get worse due to an impending scarcity of radiologists in various countries. A high prevalence of false positive results is another side effect of mammography screening. This may lead to unwarranted concern, difficult follow-up care, further imaging tests, and occasionally the requirement for tissue sampling (often a needle biopsy). ","metadata":{}},{"cell_type":"markdown","source":"# Goal of the Competition","metadata":{}},{"cell_type":"markdown","source":"The objective of this challenge is to create a model to identify breast cancer from screening mammograms received from routine screening.","metadata":{}},{"cell_type":"markdown","source":"## Required Libraries","metadata":{}},{"cell_type":"code","source":"import numpy as np\nimport pandas as pd\nimport seaborn as sns\nimport matplotlib.pyplot as plt\n%matplotlib inline\nimport missingno as msno\nimport plotly_express as px\nimport pydicom as dicom\nfrom pydicom import dcmread\n","metadata":{"execution":{"iopub.status.busy":"2022-12-01T20:12:40.465696Z","iopub.execute_input":"2022-12-01T20:12:40.46647Z","iopub.status.idle":"2022-12-01T20:12:41.696585Z","shell.execute_reply.started":"2022-12-01T20:12:40.466361Z","shell.execute_reply":"2022-12-01T20:12:41.695204Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Loading the Dataset","metadata":{}},{"cell_type":"code","source":"Train_data = pd.read_csv('/kaggle/input/rsna-breast-cancer-detection/train.csv')\nTest_data = pd.read_csv('/kaggle/input/rsna-breast-cancer-detection/test.csv')","metadata":{"execution":{"iopub.status.busy":"2022-12-01T20:12:41.697909Z","iopub.execute_input":"2022-12-01T20:12:41.698234Z","iopub.status.idle":"2022-12-01T20:12:41.785268Z","shell.execute_reply.started":"2022-12-01T20:12:41.698206Z","shell.execute_reply":"2022-12-01T20:12:41.784Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Train_data.head()","metadata":{"execution":{"iopub.status.busy":"2022-12-01T20:12:41.786589Z","iopub.execute_input":"2022-12-01T20:12:41.78693Z","iopub.status.idle":"2022-12-01T20:12:41.809747Z","shell.execute_reply.started":"2022-12-01T20:12:41.786899Z","shell.execute_reply":"2022-12-01T20:12:41.808579Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"## Data Description","metadata":{}},{"cell_type":"markdown","source":"| Attributes | Details | \n| --- | --- |\n| site_id | ID code for the source hospital. | \n| patient_id | ID code for the patient. | \n| image_id | ID code for the image. | \n| laterality | Whether the image is of the left or right breast. | \n| view | The orientation of the image. The default for a screening exam is to capture two views per breast. | \n| age | The patient's age in years. | \n| implant | Whether or not the patient had breast implants. Site 1 only provides breast implant information at the patient level, not at the breast level. | \n| density | A rating for how dense the breast tissue is, with A being the least dense and D being the most dense. Extremely dense tissue can make diagnosis more difficult. | \n| machine_id | An ID code for the imaging device. | \n| cancer | Whether or not the breast was positive for cancer. The target value. Only provided for train. |\n| biopsy | Whether or not a follow-up biopsy was performed on the breast. Only provided for train. |\n| invasive | If the breast is positive for cancer, whether or not the cancer proved to be invasive. Only provided for train. |\n| BIRADS | Breast Imaging Reporting & Data System output. 0 if the breast required follow-up, 1 if the breast was rated as negative for cancer, and 2 if the breast was rated as normal. Only provided for train. |\n| prediction_id | The ID for the matching submission row. Multiple images will share the same prediction ID. Test only. |\n| difficult_negative_case | True if the case was unusually difficult. Only provided for train. | ","metadata":{}},{"cell_type":"markdown","source":"## Data Inspection","metadata":{}},{"cell_type":"code","source":"print('Train Dataset')\nprint('Number of ROWs:', Train_data.shape[0], '\\nNumber of COLOUMNs:', Train_data.shape[1])\nprint('---------------------------------------------------------------------------')\nprint('Test Dataset')\nprint('Number of ROWs:', Test_data.shape[0], '\\nNumber of COLOUMNs:', Test_data.shape[1])","metadata":{"execution":{"iopub.status.busy":"2022-12-01T20:12:41.812515Z","iopub.execute_input":"2022-12-01T20:12:41.812899Z","iopub.status.idle":"2022-12-01T20:12:41.820436Z","shell.execute_reply.started":"2022-12-01T20:12:41.812842Z","shell.execute_reply":"2022-12-01T20:12:41.819234Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"print('Number of unique patients:',Train_data.patient_id.nunique())\nprint('Number of unique machine_id:',Train_data.machine_id.nunique())\n# print('Average mammogram for each patient:',Train_data.groupby(\"patient_id\")[\"image_id\"].agg(\"count\").reset_index()[\"image_id\"].mean())","metadata":{"execution":{"iopub.status.busy":"2022-12-01T20:12:41.822447Z","iopub.execute_input":"2022-12-01T20:12:41.82302Z","iopub.status.idle":"2022-12-01T20:12:41.837334Z","shell.execute_reply.started":"2022-12-01T20:12:41.822976Z","shell.execute_reply":"2022-12-01T20:12:41.835803Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Train_data.info()","metadata":{"execution":{"iopub.status.busy":"2022-12-01T20:12:41.839346Z","iopub.execute_input":"2022-12-01T20:12:41.839707Z","iopub.status.idle":"2022-12-01T20:12:41.867468Z","shell.execute_reply.started":"2022-12-01T20:12:41.839675Z","shell.execute_reply":"2022-12-01T20:12:41.866545Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Train_data.describe().T","metadata":{"execution":{"iopub.status.busy":"2022-12-01T20:12:41.868738Z","iopub.execute_input":"2022-12-01T20:12:41.869128Z","iopub.status.idle":"2022-12-01T20:12:41.932459Z","shell.execute_reply.started":"2022-12-01T20:12:41.869094Z","shell.execute_reply":"2022-12-01T20:12:41.931102Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"msno.bar(Train_data,figsize=(20, 5),fontsize=12)\nplt.grid()","metadata":{"execution":{"iopub.status.busy":"2022-12-01T20:12:41.934339Z","iopub.execute_input":"2022-12-01T20:12:41.934737Z","iopub.status.idle":"2022-12-01T20:12:43.122198Z","shell.execute_reply.started":"2022-12-01T20:12:41.934705Z","shell.execute_reply":"2022-12-01T20:12:43.121116Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"dups = Train_data.duplicated()\nprint('Total no of duplicate values in dataset = %d' % (dups.sum()))\n\nTrain_data[dups]","metadata":{"execution":{"iopub.status.busy":"2022-12-01T20:12:43.123661Z","iopub.execute_input":"2022-12-01T20:12:43.124275Z","iopub.status.idle":"2022-12-01T20:12:43.161655Z","shell.execute_reply.started":"2022-12-01T20:12:43.12424Z","shell.execute_reply":"2022-12-01T20:12:43.160554Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize = (22,7))\nplt.subplot(1,2,1)\nsns.countplot(x = Train_data.BIRADS)\nplt.xticks(np.arange(3), ['Required follow-up', 'Negative for cancer','Rated as normal'], fontsize=12)\nplt.xlabel('Breast Imaging Reporting & Data System Output', fontsize=12)\nplt.ylabel('Count', fontsize=12)\n\nplt.subplot(1,2,2)\nsns.boxplot(x = Train_data.cancer, y = Train_data.age, data = Train_data) \nplt.xticks(np.arange(2), ['Negative', 'Positive'], fontsize=12)\nplt.xlabel('Breast Cancer', fontsize=12)\nplt.ylabel('Age', fontsize=12)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-12-01T20:12:43.163169Z","iopub.execute_input":"2022-12-01T20:12:43.1636Z","iopub.status.idle":"2022-12-01T20:12:43.550182Z","shell.execute_reply.started":"2022-12-01T20:12:43.163548Z","shell.execute_reply":"2022-12-01T20:12:43.549126Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"plt.figure(figsize = (24,5))\nplt.subplot(1,3,1)\nsns.countplot(x = Train_data.cancer)\nplt.xticks(np.arange(2), ['Negative', 'Positive'], fontsize=12)\nplt.xlabel('Breast Cancer', fontsize=12)\nplt.ylabel('Count', fontsize=12)\nplt.subplot(1,3,2)\nsns.countplot(x = Train_data.biopsy)\nplt.xticks(np.arange(2), ['Performed', 'Not Performed'], fontsize=12)\nplt.xlabel('Follow-up Biopsy', fontsize=12)\nplt.ylabel('Count', fontsize=12)\nplt.subplot(1,3,3)\nsns.countplot(x = Train_data.invasive)\nplt.xticks(np.arange(2), ['Yes', 'No'], fontsize=12)\nplt.xlabel('Invesive Breast Cancer', fontsize=12)\nplt.ylabel('Count', fontsize=12)\nplt.show()","metadata":{"execution":{"iopub.status.busy":"2022-12-01T20:12:43.551397Z","iopub.execute_input":"2022-12-01T20:12:43.551695Z","iopub.status.idle":"2022-12-01T20:12:44.066608Z","shell.execute_reply.started":"2022-12-01T20:12:43.551668Z","shell.execute_reply":"2022-12-01T20:12:44.065514Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"Train_data.groupby(['cancer','implant','invasive'])['cancer'].count()","metadata":{"execution":{"iopub.status.busy":"2022-12-01T20:12:44.067955Z","iopub.execute_input":"2022-12-01T20:12:44.068326Z","iopub.status.idle":"2022-12-01T20:12:44.082824Z","shell.execute_reply.started":"2022-12-01T20:12:44.068269Z","shell.execute_reply":"2022-12-01T20:12:44.081595Z"},"trusted":true},"execution_count":null,"outputs":[]},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]},{"cell_type":"markdown","source":"Next will explore on image data","metadata":{}},{"cell_type":"code","source":"","metadata":{},"execution_count":null,"outputs":[]}]}