What's the reason showing error message from firmware as: Softmax axis parameter = NULL?

[idata]

Hi,

When we use C API to load tensorflow Inception -V3 graph under example code, after calling mvncGetResult, we got error number '-11', with additional debugging, this error is indicating error as: Softmax axis parameter = NULL.

Could anyone shed some lights on this? It is meaning there are something wrong with load graph and/or firmware?

sample codes section:

retCode = mvncGetResult(graphHandle, &resultData16, &lenResultData, &userParam); if (retCode == MVNC_OK) { …… } else { printf("Error ret: %d\n", retCode); char *debuginfo; unsigned debuginfolen; int rc = mvncGetGraphOption(graphHandle, MVNC_DEBUG_INFO, (void **)&debuginfo, &debuginfolen); if(rc == 0) { printf("GetResult failed, myriad error: %s\n", debuginfo); } }Then you got result as:Error ret: -11GetResult failed, myriad error: Softmax axis parameter = NULL

[idata]

From Mvnc.h:

MVNC_MYRIAD_ERROR = -11, // An error has been reported by the device, use MVNC_DEBUG_INFO

So is there any way to know what happen to device or what 'Softmax axis parameter = NULL' is meaning?

P.S, the SDK version is 1.11, and the environment is 32bit and a derived linux.

[idata]

@Austin Try recompiling the graph file with the same NCSDK version you are using. A graph file that is created with a specific version of the NCSDK is exclusive to that version of the NCSDK and you may be using a graph file that was created with a previous version of the NCSDK.

Also I recommend updating to NCSDK 1.12 as we have added additional support for networks like FaceNet. Version 1.12 can be found @ https://github.com/movidius/ncsdk/releases/tag/v1.12.00.01.

[idata]

Thank you, Tome!

Actually, I am sure we are using the same graph with the same NCSDK version. The same C app with the same graph file works well on host (32bit X86), but when we copy them to target (32bit X86), it failed.Anyway, I will try if we could be lucky with 1.12.

And could you also share some thoughts on the error code '-11' or decoded message: Softmax axis parameter = NULL? Does this mean the content of loaded graph file is a mess or it is meaning something else?

Thanks for your time.

[idata]

@Austin In regards to the Softmax axis parameter = NULL error, it is most likely receiving null on the axis which the Softmax is being performed. I'm not exactly sure what would cause this on the target system while it runs with no problems on the host system except maybe the target system is running an older version of the SDK. Please double check your target system and let me know. Thanks.

[idata]

Hi Tome,

We had additional tests and:

1) double confirmed we use the same SDK version for both host and target.2) 1.12 has the same issues.3) The error happened on both 32bit/64bit environment. 4) The execution on host has 'issue' as well, even though it gave result, but the 'probability' is only 0.021.The C test app was made by changing run.cpp under examples/caffe/GoogLeNet/cpp, with a very little changes in dimension and mean/std to comply with run.py under examples/tensorflow/Inception_v3.The low 'probability' may be from some color convert process which is in run.py but not in the changed C.

But there is still weird thing:

With the same app and graph, why can host app run (but with wrong result) but target app ran with Softmax error? At lease the target should output the same result even though it is wrong with obvious low probability.

I also attached the changes here.

--- run.cpp 2018-01-26 06:23:45.509622288 -0800 +++ err-run.cpp 2018-01-26 06:25:50.445622024 -0800 @@ -37,13 +37,15 @@ #define IMAGE_FILE_NAME "../../../data/images/nps_electric_guitar.png" -// 16 bits. will use this to store half precision floats since C++ has no +// 16 bits. will use this to store half precision floats since C++ has no // built in support for it. typedef unsigned short half; // GoogleNet image dimensions, network mean values for each channel in BGR order. -const int networkDim = 224; -float networkMean[] = {0.40787054*255.0, 0.45752458*255.0, 0.48109378*255.0}; +const int networkDim = 299; +//float networkMean[] = {0.40787054*255.0, 0.45752458*255.0, 0.48109378*255.0}; +float networkMean[] = {128, 128, 128}; +float networkStd = 1/128; // Load a graph file // caller must free the buffer returned. @@ -120,9 +122,9 @@ half *LoadImage(const char *path, int re green = imgfp32[3*i+1]; red = imgfp32[3*i+0]; - imgfp32[3*i+0] = blue-mean[0]; - imgfp32[3*i+1] = green-mean[1]; - imgfp32[3*i+2] = red-mean[2]; + imgfp32[3*i+0] = (blue-mean[0]) * networkStd; + imgfp32[3*i+1] = (green-mean[1]) * networkStd; + imgfp32[3*i+2] = (red-mean[2]) * networkStd; // uncomment to see what values are getting passed to mvncLoadTensor() before conversion to half float //printf("Blue: %f, Grean: %f, Red: %f \n", imgfp32[3*i+0], imgfp32[3*i+1], imgfp32[3*i+2]); @@ -138,22 +140,24 @@ int main(int argc, char** argv) mvncStatus retCode; void *deviceHandle; char devName[NAME_SIZE]; + int loglevel = 2; + mvncSetGlobalOption(MVNC_LOG_LEVEL, &loglevel, sizeof(loglevel)); retCode = mvncGetDeviceName(0, devName, NAME_SIZE); if (retCode != MVNC_OK) { // failed to get device name, maybe none plugged in. printf("No NCS devices found\n"); exit(-1); } - + // Try to open the NCS device via the device name retCode = mvncOpenDevice(devName, &deviceHandle); if (retCode != MVNC_OK) - { // failed to open the device. + { // failed to open the device. printf("Could not open NCS device\n"); exit(-1); } - - // deviceHandle is ready to use now. + + // deviceHandle is ready to use now. // Pass it to other NC API calls as needed and close it when finished. printf("Successfully opened NCS device!\n"); @@ -225,14 +229,23 @@ int main(int argc, char** argv) } printf("Index of top result is: %d\n", maxIndex); printf("Probability of top result is: %f\n", resultData32[maxIndex]); - } + } else { + printf("Error ret: %d\n", retCode); + + char *debuginfo; + unsigned debuginfolen; + int rc = mvncGetGraphOption(graphHandle, MVNC_DEBUG_INFO, (void **)&debuginfo, &debuginfolen); + if(rc == 0) { + printf("GetResult failed, myriad error: %s\n", debuginfo); + } + } } retCode = mvncDeallocateGraph(graphHandle); graphHandle = NULL; } - free(graphFileBuf); + free(graphFileBuf); retCode = mvncCloseDevice(deviceHandle); deviceHandle = NULL; }

Thanks

[idata]

@Austin Just want to confirm that you are using the InceptionV3 model with the above code.

[idata]

Tome, Yes, we used Caffe GoogLeNet run.cpp as template, and had a little changes on mean/std according to sample in Inception V3 python script, then after got executable file, we use Inception V3 graph file.

[idata]

@Austin Try changing the line "float networkStd = 1/128;" to "float networkStd = 1.0/128;". Please let me know if this works for you. Thanks.

[idata]

Yes, it works! With this correction, both host and target work now with right probability is 0.94 now. Thank you, Tome!

[idata]

@Austin Glad to hear that!