Community
cancel
Showing results for 
Search instead for 
Did you mean: 
idata
Community Manager
482 Views

Inception-resnet-v2 NaN

I'm trying to get an inception-resnet-v2 network running on the movidius stick, but it gives me a NaN result.

 

I'm using caffe for this one, not tensorflow. Gotten from https://github.com/soeaver/caffe-model/blob/master/cls/inception/deploy_inception-resnet-v2.prototxt

 

I'm using it pretrained, and replaced the final fully connected layer to train it on our own data. All other weights are not changed during training.

 

I tried pinpointing where it goes wrong and the Eltwise layer 'inception_resnet_v2_a1_residual_eltwise' seems to be the first where the result of caffe and the result of ncs seem to differ significantly (180%, compared to 0.5% the layers before). Which eventually leads to consistent NaN outputs at the end of the network.

 

This is my prototxt:

 

https://gist.github.com/anonymous/1308671b77212b5397f5319c074336c6

 

This is the pretrained network:

 

https://drive.google.com/open?id=0B9mkjlmP0d7zNFY1b2s4NXVsRVU
Tags (3)
0 Kudos
2 Replies
idata
Community Manager
180 Views

@brainshift, is Caffe a hard requirement, or can you use TensorFlow instead? If you are open to using TensorFlow, use https://github.com/movidius/ncappzoo/blob/master/tensorflow/inception/Makefile as a template to run inception-resnet-v2. Pretrained checkpoint files are available here - https://github.com/tensorflow/models/tree/master/research/slim#Pretrained. You will have to extract the output_node_name from tensorboard.

idata
Community Manager
180 Views

I have the same problem with Eltwise layer in SFD face detector https://github.com/sfzhang15/SFD .

 

Results of mvNCCheck for full network:

 

Blob generated

 

USB: Transferring Data…

 

USB: Myriad Execution Finished

 

USB: Myriad Connection Closing.

 

USB: Myriad Connection Closed.

 

Result: (1, 750, 7)

 

1) 51 nan

 

2) 72 nan

 

3) 79 nan

 

4) 23 nan

 

5) 44 nan

 

Expected: (1, 1, 7)

 

1) 0 0.0

 

2) 6 -1.0

 

3) 5 -1.0

 

4) 4 -1.0

 

5) 3 -1.0

 

/usr/local/bin/ncsdk/Controllers/Metrics.py:75: RuntimeWarning: invalid value encountered in greater

 

------------------------------------------------------------

 

Obtained values

 

------------------------------------------------------------

 

Obtained Min Pixel Accuracy: nan% (max allowed=2%), Fail

 

Obtained Average Pixel Accuracy: nan% (max allowed=1%), Fail

 

Obtained Percentage of wrong values: 64100.0% (max allowed=0%), Fail

 

Obtained Pixel-wise L2 error: nan% (max allowed=1%), Fail

 

Obtained Global Sum Difference: nan

 

------------------------------------------------------------

 

When I remove branch contained Eltwise layer from network the Obtained values are Pass.

Reply