unable to train my model for an epoch of more 26 count

[Tchouanga__Franck]

please I am unable to train my model to the epoch I really want for the exact results I want to occur. please help how can I optimize the jobs the better train my model for accuracy and lose values I want to reach

[AthiraM_Intel]

Hi,

Thanks for reaching out to us. Could you please share the following details:

1. More details about your workload. 

2. Which model are you using?

3. Are you getting any error while training?  If so please share the screenshot 

[Tchouanga__Franck]

The following shows out how the model hangout why processing in the job

I actually submitted a job with a wall time of 24:00:00

I wish to increase or may have help in other to optimize my model for it to fit within that time constraint

[JananiC_Intel]

Hi,

Thanks for the update.

As we know 24 hours is the maximum wall time ,after that it will be killed ,you can use the checkpoint .Checkpoint will save your model after every epoch so that you can resume your training.

[Tchouanga__Franck]

please how do we continue training a model from a checkpoint

[JananiC_Intel]

Hi,

From the screenshot attached ,we understand that you are using keras model. In order to save the model and resume training you can refer the below link.

https://keras.rstudio.com/articles/tutorial_save_and_restore.html

Also you can refer the below case.

https://software.intel.com/en-us/forums/intel-devcloud/topic/849464#

Hope this answers your question.

[JananiC_Intel]

Hi,

Could you please give us an update whether the issue got resolved?

[JananiC_Intel]

Hi,

We are closing this case assuming your issue is resolved. Please feel free to raise a new thread in case of further issues.