compile_tool removed from 2021.1 ? (and myriad_compile confusion)

[Hawkes__Rycharde]

Hi,

I've installed 2021.1 on both Windows and Ubuntu.  The compile_tool binary is no longer present but the online documentation for it still exists.  Has it been deprecated??

myriad_compile is present but its behaviour seems to have changed.  Previously (2020.1) I could compile a model with 12 shaves/cmx slices without error so:

myriad_compile -ip U8 -VPU_NUMBER_OF_SHAVES 12 -VPU_NUMBER_OF_CMX_SLICES 12 -m mymodel.xml

But now the output contains both a warning of a deprecated option (which I have not explicitly used) and that I've specified too many shaves:

Inference Engine: 
API version ............ 2.1
Build .................. 2021.1.0-1237-bece22ac675-releases/2021/1
Description ....... API
[Warning][VPU][Config] Deprecated option was used : VPU_MYRIAD_PLATFORM
Cannot allocate "24" shaves: only 16 is available

If I specify 12 shaves, why is it trying to allocate 24?

[Iffa_Intel]

Greetings,

Fyi, the compile _tool is available in the <openvinoFolder>/deployment_tools/tools/compile_tool

The myriad_compile commands depend on the type of your model. By default the -ip would specifies the FP16 precision.

Make sure your model is similar to what you put into command.

This might be the culprit tothe message  that you got

Cannot allocate "24" shaves: only 16 is available

You could ignore the warning as long as it's not an error.

Sincerely,

Iffa

[Hawkes__Rycharde]

Thanks, I hadn't spotted that it had been moved to a different location than myriad_compile.

I convert the same model with different numbers of shaves.  Only when I specify more than 8 shaves do I get the "error" message about allocating too many shaves.  Again, I don't understand why specifying 12 shaves and cmx slices would then be interpreted as specifying 24 shaves.  I will check if I have problems using the output model.

[Iffa_Intel]

As I mentioned previously, I believe that depends on your model's precision type (FP16,FP32,INT8).

Make sure your -ip input command is correct according to your model.

If you don't have any other inquiries shall I close this thread?

Sincerely,

Iffa

[Hawkes__Rycharde]

I've tried adjusting the -ip option but this also produces the message:

compile_tool -m face-detection-retail-0004.xml -d MYRIAD -ip FP16 -VPU_NUMBER_OF_SHAVES 12 -VPU_NUMBER_OF_CMX_SLICES 12
Inference Engine: 
	API version ............ 2.1
	Build .................. 2021.1.0-1237-bece22ac675-releases/2021/1
	Description ....... API
[Warning][VPU][Config] Deprecated option was used : VPU_MYRIAD_PLATFORM
Cannot allocate "24" shaves: only 16 is available

Have I misunderstood something?

[Iffa_Intel]

Fyi,

MYRIAD/NCS2/VPU only supports the precision of FP16. Hence, the input precision of FP16 is the only one valid for NCS2. The maximum allocation for this is 16, that is why you cannot allocate 24 (VPU_NUMBER_OF_SHAVES 12 + VPU_NUMBER_OF_CMX_SLICES 12 = 24). Max allocation for both is 8.

Note: NUMBER_OF_SHAVES must be less than NUMBER_OF_CMX_SLICES.

Sincerely,

Iffa

[Hawkes__Rycharde]

The input precision should be set as per the model which may be FP16 but could also be U8 - internally it will be converted to FP16 but that is not the same as specifying -ip FP16 on every compile.  I've been using this for the past year.

I've always set the NUMBER_OF_SHAVES equal to the NUMBER_OF_CMX_SLICES.  Why do you say SHAVES must be less than SLICES?

I've discovered that if you specify a config file rather than use the command-line arguments that I can still compile for more than 8 shaves/slices without error, e.g. compile.txt contains:

MYRIAD_NUMBER_OF_SHAVES 12	
MYRIAD_NUMBER_OF_CMX_SLICES 12
MYRIAD_THROUGHPUT_STREAMS 1	

Execute using:

compile_tool -m face-detection-retail-0004.xml -d MYRIAD -ip U8 -c compile.txt
Inference Engine: 
	API version ............ 2.1
	Build .................. 2021.1.0-1237-bece22ac675-releases/2021/1
	Description ....... API
[Warning][VPU][Config] Deprecated option was used : VPU_MYRIAD_PLATFORM
Done. LoadNetwork time elapsed: 2724 ms

I don't know why the command-line args prevent specifying the same.  In fact, I would be happy if you could point me to some documentation on the usage of compile_tool because the manual page is minimal and none of this appears to be documented.

[Hawkes__Rycharde · Accepted solution]

As a follow-up to my last post, this github pull request for OpenVINO describes the cause of the message and the confusion: Add command line input option to configure VPU_THROUGHPUT_STREAMS 

[Iffa_Intel]

we don't have any other documentation for the compile_tool.

The code here could help to answer your question previously- https://github.com/openvinotoolkit/openvino/blob/master/inference-engine/src/vpu/graph_transformer/src/graph_transformer.cpp

VPU_THROW_UNLESS(g_compileEnv->config.numSHAVEs <= g_compileEnv->config.numCMXSlices,

R"(Value of configuration option ("{}") must be not greater than value of configuration option ("{}"), but {} > {} are provided)",

ie::MYRIAD_NUMBER_OF_SHAVES, ie::MYRIAD_NUMBER_OF_CMX_SLICES, config.numSHAVEs, config.numCMXSlices);

That is:

"MYRIAD_NUMBER_OF_SHAVES" must be not greater than value of configuration option "MYRIAD_NUMBER_OF_CMX_SLICES"

Glad to know that github thread helps.

If you don't have any other inquiries I shall close this thread.

Sincerely,

Iffa

[Iffa_Intel]

Greetings,

Intel will no longer monitor this thread since we have provided a solution. If you need any additional information from Intel, please submit a new question

Sincerely,

Iffa