DL models on FPGA with OpenVINO

[vinmor]

Good morning everyone,

I have I have the "Developer Kit for OpenVINO toolkit" board. The host PC has Ubuntu 16.04 operating system with OpenVINO 2019 R1.
I have two questions:

1)
I was wondering if there is a list of all layers supported by my FPGA for my particular version of OpenVINO.
Also, I wanted to know if there are DL models that can be run directly on FPGA and not in a HETERO FPGA-CPU way.
If so, what are they?

2)
I wanted to know if there is a way to do "transfer learning" with my version of OpenVINO.
I knew about the existence of "OpenVINO training extensions" but I can't find the old version. I only find new versions that are not compatible with my system.
How can I do? Is there another way?

Thanks in advance,
Best regards.

[JohnT_Intel]

Hi,

You will need to contact Terasic on the supported model as the bitstream is provided by them.

You can also get the development guide from https://www.terasic.com.tw/cgi-bin/page/archive_download.pl?Language=English&No=1216&FID=d07426f4fc59ae6a0182cb8b5f599885.

[vinmor]

Hi JohnT_Intel,

Yes thank you. 

I already had the guide but I couldn't find the answer to my two questions.

[JohnT_Intel]

Hi,

Have you contacted Terasic? The reason is that the bitstream is provided by them.

[vinmor]

Hi JohnT_Intel,

yes I just contacted Terasic about that question, thanks so much for the advice.

However, I don't know if you can help me with a more general question.

Do the levels supported by the FPGA plug-in depend on the particular FPGA or are they the same for all supported FPGAs?

(e.g. Intel Arria 10 GX FPGA Development Kit, Intel Programmable Acceleration Card with Intel Arria 10 GX FPGA, "Starter Kit for OpenVINO toolkit" with Cyclone V, "DE5a-Net-DDR4" with Arria 10, etc.).

I ask this because I found this link:

 https://docs.openvino.ai/archive/2019_R1/_docs_IE_DG_supported_plugins_Supported_Devices.html

which shows in the section "Supported Layers" the levels supported by the FPGA plug-in: What does it refer to?

In addition, in the Intel distribution of OpenVINO toolkit 2019 R1 with FPGA support, I found bitstream files with DL model names, for example "2019R1_RC_FP16_MobileNet_Clamp.aocx" or "2019R1_RC_FP16_ResNet_SqueezeNet_VGG.aocx", in "/opt/intel/2019_r1/openvino/bitsreams/" path.

I was wondering what these files were and how you can use them.

Thanks in advance,

Best regards,
Vincenzo.

[JohnT_Intel]

Hi,

This is pregenerated bitstream for the board that you are using. It will contain the layer to run specific NN layer.

You will need to programmed the board with specific aocx file before you are able to run the OpenVINO application on the board.

[vinmor]

Hi,

thanks for the quick response.

I think I'm starting to understand better, but I still have some doubts.

Please answer my questions in an orderly manner because I need to understand these aspects which are not very clear to me.

1)

I understand that you need to program the board with a specific bitsream file to use OpenVINO. So will the layers supported by the FPGA depend on the bitsream used? Is this correct or am I wrong?

2)

So, if the supported layers depend on the loaded bitsream, I don't understand what the list of layers supported by the FPGA plug-in shown in this link refers to.

https://docs.openvino.ai/archive/2019_R1/_docs_IE_DG_supported_plugins_Supported_Devices.html

Maybe the layers indicated in the list are all the possible layers that the FPGA can support, while the bitstreams select only some of them? Can you confirm or am I wrong?

3)

Also I saw that the Intel distribution of the OpenVINO 2019 toolkit with FPGA support provides bitsream for specific models (like squeezenet, googlenet, etc).

By programming the FPGA with such AOCX files, might I be able to run the particular model directly on the FPGA with the "-d FPGA" command? Or do I necessarily have to use HETERO mode with a fallback device? So, I'm wondering if it's somehow possible to break away from the fallback device and just use the FPGA. If so, with which board or which bitstream? I hope my question is clear.

4)

The bitsream files available from the Intel distribution of OpenVINO 2019 with FPGA support are only valid for the following families, correct me if I'm wrong:

• Intel® Arria® 10 GX FPGA Development Kit,
• Intel® Programmable Acceleration Card with Intel® Arria® 10 GX FPGA,
• Intel® Vision Accelerator Design with an Intel® Arria® 10 FPGA.

Did I get it right?

While Terasic also provided bitstream for the following two kit:

• Starter Platform for OpenVINO™ Toolkit with Cyclone V FPGA
• DE5a-Net-DDR4 with Arria 10 FPGA

Please confirm this or correct me if I'm wrong.

Obviously a bitstream file that is valid for one card is not compatible for another, is this correct?

5)

Is it possible to create custom bitstreams to program the board for OpenVINO? If so in what way?

Thanks in advance

Best regards.

Vincenzo

[JohnT_Intel]

Hi,

Below is the answer.

1) Yes, as different model might have different requirement and features needed. It will more important if you would like to have it fully run in FPGA.

2) You may refer to https://docs.openvino.ai/archive/2019_R1/_docs_IE_DG_supported_plugins_FPGA.html where it will provide the information on what is the supported model for that bitstream. If not you will need to ask Terassic on the bitstream as it is providedby them

3) If all the features is supported on the bitstream then you can run with "-d FPGA". If the features is not supported then it will not break away but will mention that it is not able to run with "-d FPGA".

4) Yes, you are correct. For the Terassic board then you will need to contact them on the bitstream available as it is not part of the OpenVINO installation package.

5) You will need to contact Terassic if you are interested on the building custom bitstream for their board. If not you will need to contact your local sales person for Intel AI Suite(https://www.intel.com/content/www/us/en/software/programmable/fpga-ai-suite/overview.html).

[vinmor]

Hi,

thanks so much for the replies! Now it's much clearer!

1) So, for example, if a particular layer is supported by the FPGA plug-in, in reality this may not be executed on the FPGA if it is not foreseen by the bitstream loaded on the board. Did I get it right?

2) Also I saw that you can specify "-l <CPU_EXTENSION>" to expand the levels supported by the CPU plug-in. Is there something similar for FPGA? Or not?

This is a general discussion, not necessarily regarding terasic kits.

Thanks in advance

Best regards.

Vincenzo

[JohnT_Intel]

Hi,

1) So, for example, if a particular layer is supported by the FPGA plug-in, in reality this may not be executed on the FPGA if it is not foreseen by the bitstream loaded on the board. Did I get it right?

It will flag error if you are nnot using FPGA that is programmed with the correct bitstream.

2) Also I saw that you can specify "-l <CPU_EXTENSION>" to expand the levels supported by the CPU plug-in. Is there something similar for FPGA? Or not?

Currerntly there is no expension supported on FPGA side.

[vinmor]

Hi,
everything is clear, thanks!
One last thing, do you know if the PCIe connection is strictly necessary to use only the FPGA as target device with "-d FPGA"? If I want to use my notebook to program one of the OpenVINO compatible kits, I cannot connect the card via PCIe. So I was wondering if the USB connection alone was sufficient for this purpose.
Thanks in advance,
best regards.

[JohnT_Intel]

Hi,

If you are using the new AI Suite then you can use any PC to program it not to run the OpenVINO. OpenVINO need to be run on PCIe system or ARM that is included part of the FPGA.