How to debug sdk infer wrong result?

[idata]

Hi, I have a pre-trained inceptionv3.pb file, and convert to graph file successfully without error.

But the infer result is all NaN in python.

Debug log didn't help anything.

Start download to NCS... D: [ 0] ncFifoWriteElem:2592 No layout conversion is needed 0 D: [ 0] convertDataTypeAndLayout:170 src data type 1 dst data type 0 D: [ 0] convertDataTypeAndLayout:172 SRC: w 299 h 299 c 3 w_s 12 h_s 3588 c_s 4 D: [ 0] convertDataTypeAndLayout:174 DST: w 299 h 299 c 3 w_s 6 h_s 1794 c_s 2 D: [ 0] ncFifoWriteElem:2617 write count 0 num_elements 2 userparam 0x7fb0fc143cd0 I: [ 0] ncGraphQueueInference:3048 trigger start I: [ 0] ncGraphQueueInference:3145 trigger end D: [ 0] ncFifoReadElem:2686 No layout conversion is needed 1 D: [ 0] convertDataTypeAndLayout:170 src data type 0 dst data type 1 D: [ 0] convertDataTypeAndLayout:172 SRC: w 1 h 1 c 4 w_s 8 h_s 8 c_s 2 D: [ 0] convertDataTypeAndLayout:174 DST: w 1 h 1 c 4 w_s 16 h_s 16 c_s 4 D: [ 0] ncFifoReadElem:2714 num_elements 2 userparam 0x7fb0fc143ef0 output length 16 Cost: 0.35880494117736816 output data is: [ nan nan nan nan] ******************************************************************************* inception-v3 on NCS ******************************************************************************* [3 2 1 0] 3 3:yumi nan 2 2:tudou nan 1 1:shulin nan 0 0:huangdi nan

I don't know where to start with, someone any suggestion?

[idata]

I meet the same problem too

[idata]

@chenxiaoyu3 Can you provide a link to your python code?

[idata]

Hi,

I get almost the same error, using the same model, but with different tensor dimensions once 32 x32 in the second version 299 x 299:

• When I use the 32 x 32 model the prediction succeeds perfectly without errors.
• When I use the 299 x 299 model instead I can not continue getting the following debug:

D: [ 0] ncDeviceCreate:307 ncDeviceCreate index 0 D: [ 0] ncDeviceCreate:307 ncDeviceCreate index 1 1 neural compute devices found! D: [ 0] ncDeviceCreate:307 ncDeviceCreate index 0 D: [ 0] ncDeviceCreate:307 ncDeviceCreate index 1 D: [ 0] ncDeviceOpen:523 File path /usr/local/lib/mvnc/MvNCAPI-ma2450.mvcmd I: [ 0] ncDeviceOpen:529 ncDeviceOpen() XLinkBootRemote returned success 0 I: [ 0] ncDeviceOpen:567 XLinkConnect done - link Id 0 D: [ 0] ncDeviceOpen:581 done I: [ 0] ncDeviceOpen:583 Booted 2-ma2450 -> VSC I: [ 0] getDevAttributes:382 Device attributes I: [ 0] getDevAttributes:385 Device FW version: 2.8.2450.16e I: [ 0] getDevAttributes:387 mvTensorVersion 2.8 I: [ 0] getDevAttributes:388 Maximum graphs: 10 I: [ 0] getDevAttributes:389 Maximum fifos: 20 I: [ 0] getDevAttributes:391 Maximum graph option class: 1 I: [ 0] getDevAttributes:393 Maximum device option class: 1 I: [ 0] getDevAttributes:394 Device memory capacity: 522059056 Hello NCS! Device opened normally. I: [ 0] ncGraphAllocate:959 Starting Graph allocation sequence I: [ 0] ncGraphAllocate:1025 Sent graph D: [ 0] ncGraphAllocate:1050 Graph Status 0 rc 0 D: [ 0] ncGraphAllocate:1096 Input tensor w 299 h 299 c 3 n 1 totalSize 536406 wstide 6 hstride 1794 cstride 2 layout 0 D: [ 0] ncGraphAllocate:1109 output tensor w 1 h 1 c 2 n 1 totalSize 4 wstide 4 hstride 4 cstride 2 layout 1 I: [ 0] ncGraphAllocate:1163 Graph allocation completed successfully I: [ 0] ncFifoCreate:2163 Init fifo I: [ 0] ncFifoAllocate:2358 Creating fifo I: [ 0] ncFifoCreate:2163 Init fifo I: [ 0] ncFifoAllocate:2358 Creating fifo D: [ 0] ncFifoWriteElem:2629 No layout conversion is needed 0 D: [ 0] convertDataTypeAndLayout:169 src data type 1 dst data type 0 D: [ 0] convertDataTypeAndLayout:171 SRC: w 299 h 299 c 3 w_s 12 h_s 3588 c_s 4 D: [ 0] convertDataTypeAndLayout:173 DST: w 299 h 299 c 3 w_s 6 h_s 1794 c_s 2 D: [ 0] ncFifoWriteElem:2654 write count 0 num_elements 2 userparam 0x7f4fe3040d58 I: [ 0] ncGraphQueueInference:3089 trigger start I: [ 0] ncGraphQueueInference:3186 trigger end E: [ 0] dispatcherEventReceive:235 dispatcherEventReceive() Read failed -1 E: [ 0] eventReader:253 Failed to receive event, the device may have reset E: [ 0] ncFifoReadElem:2735 Failed to read fifo element