{ "id": 372190, "title": "Notebook stops during fit: Successful run", "url": "/competitions/rsna-breast-cancer-detection/discussion/372190", "author_name": "Andrew Schleiss", "post_date": "2022-12-14T16:54:48.450000", "votes": 0, "comment_count": 2, "views": 0, "content": "

Hi all

\n

I hope somone can help, I cant seem to figure out why the notebook stops during training on the 1st epoch

\n

The notebook runs for a few minutes, gets to epoch 1 and stops
\nThe kernel run shows as succesful but the rest of the notebook hasnt executed
\n\"Capture.jpg\"

\n

\"Capture2.jpg\"

\n

Any help is appreciated
\nNotebook is here

", "messages": [ { "id": 2065562, "postDate": "2022-12-14T20:15:07.757Z", "content": "

Some pointer

\n\n

\"image\"

\n\n
 strategy.scope():\n    ResNet = ResNet50(\n                  include_top=,\n                  input_shape=(RESIZE,RESIZE,),\n                  pooling = \n     )\n    ResNet.trainable = \n\n    model = Sequential(\n        [\n            layers.InputLayer(input_shape=(RESIZE,RESIZE,)),\n            ResNet,   \n            layers.Dense() \n        ]\n    )\n
\n

Also note, you set no activation to last layer, so you'll get logit (not prob score); adjust your metrics accordingly.

", "rawMarkdown": "Some pointer\n\n- ~~Don't use augmentaiotn layer inside the model build while using TPU.~~ Check: https://keras.io/guides/preprocessing_layers/\n\n![image](https://user-images.githubusercontent.com/17668390/208264848-df826565-fcb6-425b-94d5-3cf07af66d72.png)\n\n\n\n- Rescaling, Resing can be added.\n- If you use `.repeat()`, you might need to provde `step_per_epoch`.\n- Removing `.cache()` and recheck.\n\n\n```python\nwith strategy.scope():\n ResNet = ResNet50(\n include_top=False,\n input_shape=(RESIZE,RESIZE,3),\n pooling = 'avg'\n )\n ResNet.trainable = False\n \n model = Sequential(\n [\n layers.InputLayer(input_shape=(RESIZE,RESIZE,3)),\n ResNet, \n layers.Dense(1) \n ]\n )\n```\n\nAlso note, you set no activation to last layer, so you'll get logit (not prob score); adjust your metrics accordingly.\n \n", "replies": [ { "id": 2065575, "postDate": "2022-12-14T20:45:06.373Z", "content": "

Thank you @ipythonx the point on repeat() was the issue. ill also have a look at the other points.
\nMuch appreciated

", "rawMarkdown": "Thank you @ipythonx the point on repeat() was the issue. ill also have a look at the other points.\nMuch appreciated" } ] }, { "id": 2065442, "postDate": "2022-12-14T16:54:48.450Z", "content": "

Hi all

\n

I hope somone can help, I cant seem to figure out why the notebook stops during training on the 1st epoch

\n

The notebook runs for a few minutes, gets to epoch 1 and stops
\nThe kernel run shows as succesful but the rest of the notebook hasnt executed
\n\"Capture.jpg\"

\n

\"Capture2.jpg\"

\n

Any help is appreciated
\nNotebook is here

", "rawMarkdown": "Hi all\n\nI hope somone can help, I cant seem to figure out why the notebook stops during training on the 1st epoch\n\nThe notebook runs for a few minutes, gets to epoch 1 and stops \nThe kernel run shows as succesful but the rest of the notebook hasnt executed\n[![Capture.jpg](https://i.postimg.cc/Bb8RqDh4/Capture.jpg)](https://postimg.cc/YLHdR4Ks)\n\n[![Capture2.jpg](https://i.postimg.cc/bNZqJSGG/Capture2.jpg)](https://postimg.cc/2VNs967m)\n\nAny help is appreciated\n[Notebook is here](https://www.kaggle.com/slythe/rsna-keras-resnet-modelling)" } ], "comments": [ { "id": 2065562, "author_name": "Innat", "author_url": "", "post_date": "2022-12-14T20:15:07.757000", "content": "

Some pointer

\n\n

\"image\"

\n\n
 strategy.scope():\n    ResNet = ResNet50(\n                  include_top=,\n                  input_shape=(RESIZE,RESIZE,),\n                  pooling = \n     )\n    ResNet.trainable = \n\n    model = Sequential(\n        [\n            layers.InputLayer(input_shape=(RESIZE,RESIZE,)),\n            ResNet,   \n            layers.Dense() \n        ]\n    )\n
\n

Also note, you set no activation to last layer, so you'll get logit (not prob score); adjust your metrics accordingly.

", "votes": 0, "replies": [ { "id": 2065575, "author_name": "Andrew Schleiss", "author_url": "", "post_date": "2022-12-14T20:45:06.373000", "content": "

Thank you @ipythonx the point on repeat() was the issue. ill also have a look at the other points.
\nMuch appreciated

", "votes": 0, "replies": [] } ] } ], "raw_markdown_by_id": { "2065562": "Some pointer\n\n- ~~Don't use augmentaiotn layer inside the model build while using TPU.~~ Check: https://keras.io/guides/preprocessing_layers/\n\n![image](https://user-images.githubusercontent.com/17668390/208264848-df826565-fcb6-425b-94d5-3cf07af66d72.png)\n\n\n\n- Rescaling, Resing can be added.\n- If you use `.repeat()`, you might need to provde `step_per_epoch`.\n- Removing `.cache()` and recheck.\n\n\n```python\nwith strategy.scope():\n ResNet = ResNet50(\n include_top=False,\n input_shape=(RESIZE,RESIZE,3),\n pooling = 'avg'\n )\n ResNet.trainable = False\n \n model = Sequential(\n [\n layers.InputLayer(input_shape=(RESIZE,RESIZE,3)),\n ResNet, \n layers.Dense(1) \n ]\n )\n```\n\nAlso note, you set no activation to last layer, so you'll get logit (not prob score); adjust your metrics accordingly.\n \n", "2065442": "Hi all\n\nI hope somone can help, I cant seem to figure out why the notebook stops during training on the 1st epoch\n\nThe notebook runs for a few minutes, gets to epoch 1 and stops \nThe kernel run shows as succesful but the rest of the notebook hasnt executed\n[![Capture.jpg](https://i.postimg.cc/Bb8RqDh4/Capture.jpg)](https://postimg.cc/YLHdR4Ks)\n\n[![Capture2.jpg](https://i.postimg.cc/bNZqJSGG/Capture2.jpg)](https://postimg.cc/2VNs967m)\n\nAny help is appreciated\n[Notebook is here](https://www.kaggle.com/slythe/rsna-keras-resnet-modelling)" } }