Intel FPGA AI Suite Inference Engine

[RubenPadial]

Hello,

I'm using Intel FPGA AI 2023.2 on ubuntu 20.04 host computer and trying to infer a custom CNN in a Intel Arria 10 SoC FPGA.

I have followed Intel FPGA AI Suite SoC Design Example Guide and I'm able to copile the Intel FPGA AI suite IP and run the M2M and S2M examples.

I have also compiled the grpah for my custom NN and I'm trying to run it with the Inter FPGA AI suite IP but I have not clear how to do it. I'm trying to use the dla_benchmark app provided but for example, the input data of my NN (it is trained and graph was compiled in this way) must be float whereas the input data of the IP must be int8 if I'm not wrong.

Another problem I have is regarding the ground truth file. I have a ground truth file for each imput file because each groud truth is a 225 array.

Is there any additional information or guide to run custom model with Intel FPGA AI Suite?

Thank you in advance

[RubenPadial]

Hello @JohnT_Intel ,

What changes are needed?

[JohnT_Intel]

Hi,

May I know if you regenerate the model using the TensorFlow 2 models optimization?

https://docs.openvino.ai/2023.2/openvino_docs_MO_DG_prepare_model_convert_model_Convert_Model_From_TensorFlow.html

[RubenPadial]

Hello @JohnT_Intel,

According to the link you sent only  mo --saved_model_dir <SAVED_MODEL_DIRECTORY> for saved model files. That exacly what I'm doing.

[RubenPadial]

Please, find in the following link the complete TF model, IR model and Compiled Graph https://consigna.ugr.es/download.php?files_ids=63731 or https://consigna.ugr.es/?s=download&token=779d3377-e785-43aa-ab82-6e14c50072fe

[JohnT_Intel]

Hi,

I observed that the file provided does not support certain layer on FPGA side. This will cause the you to pass between FPGA and CPU.

Hi,

I have check on the file you provided, it looks there is multiple layer that is not supported by FPGA.

Layer (Name: StatefulPartitionedCall/model/dense_1/BiasAdd, Type: Eltwise) is not supported: FPGA plugin: all parent nodes are not supported by FPGA.

Layer (Name: Constant_2560, Type: Constant) is not supported:

Layer (Name: StatefulPartitionedCall/model/dense_1/Tensordot, Type: Reshape) is not supported:

FPGA plugin: layer is not executed on FPGA because it does not have preceding layer on FPGA.

Layer (Name: StatefulPartitionedCall/model/dense_1/Tensordot/MatMul, Type: FullyConnected) is not supported:

FPGA plugin: this Fully Connected layer does not have preceding convolutional layer / sequence.

Layer (Name: StatefulPartitionedCall/model/re_lu_7/Relu, Type: Relu) is not supported:

FPGA plugin: layer is not executed on FPGA because it does not have preceding layer on FPGA.

Layer (Name: StatefulPartitionedCall/model/dense/BiasAdd, Type: Eltwise) is not supported:

FPGA plugin: 'Constant' nodes are not supported as input of nodes Eltwise

Layer (Name: Constant_2559, Type: Constant) is not supported:

[RubenPadial]

Hello @JohnT_Intel ,

According to Intel FPGA AI Suite. IP Reference Manual section 2.3. some of that layers should be compatible like the ReLU layer. Could you confirm? In addition, the model is compiled for HETERO plugin (--fplugin option), if the layer cannot be implemented in FPGA, it should be implemented in the CPU. What solution do you suggest?

Plase note that someof that layers are not in the intioan TensorFlow model and they are included by the OpenVINO model optimizer (mo) application.

Add tags

[JohnT_Intel]

You will need to check on A10_Performance.arch on what type of confiuguration setting you are using on the FPGA design.

[JohnT_Intel]

Hi,

Can we consolidate all the AI Suite disucssion into single forum discussion?

[RubenPadial]

Hello @JohnT_Intel ,

I prefer to manage the issues or questions in different threads as they seem to have different root causes.