How can i suppress weight compression?

[DSJung]

Hi. I'm really new to use openvino.

I was looking for the way i can distribute my tensorflow model to only cpu local machine, and I wanted to decrease inference duration, so i found openvino

but when i tried conversion of my tensorflow model, though i significantly decreased inference time, inference accuracy became totally chaotic. is it normal?

So I tried to figure out how conversion affected my model, and i figured out that even first normalization layer or conv2D layer conversion makes difference. ( I cut first few layers of my model and tried conversion again)

Env:

windows11

python 3.9.13

Openvino 24.0.0

tensorflow 2.8.4

test code

I compared output from keras model and converted ir model to see what happens

i cut my model and made for test, just have one normalization and one conv2d layer

kr_model # test model

kr_model.save(test_save_path)
test_ir_path=Path("/test/ir/")
test_ov_model=ov.convert_model(test_save_path, input=[[1,224,224,3]])
ov.save_model(test_ov_model, test_ir_path, compress_to_fp16=False)

test_model=core.read_model(test_ir_path)
test_ov_model = core.compile_model(model=test_model, device_name=device.value)
output_key = test_ov_model.output(0)

test_ir_out=test_ov_model(test_img)[output_key]
test_kr_out=kr_model(test_img)

1. Can I conver model to openvino model without any compression?

2. Can i set weights directly into ir model?

3. I saw solution from web using 'inference_engine' but my openvino library doesn't have it. is it version issue?

4. is there other solution i can convert my keras model without any accuracy loss? or acceptable loss amount, To inference on local machine only with intel cpu 

[Aznie_Intel]

Hi DSJung,

Thanks for reaching out.

The weight compression function can be implemented with Neural Network Compression Framework (NNCF). You may read through the Compressing Models During Training documentation for more details.

You may also convert your model using the command line interface or Model conversion API.

Below is how to use openvino.convert_model for Keras Applications:

import tensorflow as tf

import openvino as ov

tf_model = tf.keras.applications.ResNet50(weights="imagenet")

ov_model = ov.convert_model(tf_model)

###### Option 1: Save to OpenVINO IR:

# save model to OpenVINO IR for later use

ov.save_model(ov_model, 'model.xml')

###### Option 2: Compile and infer with OpenVINO:

# compile model

compiled_model = ov.compile_model(ov_model)

# prepare input_data

import numpy as np

input_data = np.random.rand(1, 224, 224, 3)

# run inference

result = compiled_model(input_data)

You may check more references in Convert a Model documentation.

Regards,

Aznie

[Aznie_Intel]

Hi DSJung,

This thread will no longer be monitored since we have provided a solution. If you need any additional information from Intel, please submit a new question.

Regards,

Aznie