I have tried the steps from the below link for retraining the Person Vehicle Bike Detector Model with my own datasets(Taken 370 Images) by developing Jupyter Notebook File in Dev Cloud and able to do set up successfully in virtual environment on Dev Cloud.
I have changed the num_steps=50000 in config file and written a job script which includes the retraining command. I have submitted the job to the queue by using the below command for Retraining 50000 steps.
job_id_core = !qsub bicycle_retrain_job.sh -l nodes=1:idc001skl:tank-870:i5-6500te -N bicycle_train_core
After creating 1850 checkpoint, job file execution stopped automatically by throwing an error “PBS: job killed: walltime 3639 exceeded limit 3600 “
I have attached Log File for your reference.
Please help us in resolving the issue.
Thanks for reaching out. Intel Devcloud for the Edge is not designed for DNN training. It has compute nodes to edge inference. The better choice for training would be Intel DevCloud for Data-centric Workloads — it has compute nodes more suitable for training workloads.