Different segmentation model result by openvino between SkyLake and CascadeLake CPU

[TinyStone]

I encountered the problem that the openvino results using SkyLake CPU and CascadeLake CPU are different.

The environment is:

OpenVINO version: 2020.3

Program Language: Python 3.6

Machine one: Intel(R) Xeon(R) W-2133 CPU

Machine two: Intel(R) Xeon(R) Silver 4214 CPU

I used same code and model file in machine one and machine two, but the segmentation result is different.

The following image is the results on machine one and machine two captured by ITK-SNAP.

I tested other two segmentation models, and each model's result is different.

How should I fix this problem or it's a normal case?

[Zulkifli_Intel]

Hi Tinystone,

Thank you for reaching out.

I noticed that you are using a very old version of OpenVINO. Is there any particular reason you use this version?

I did some tests using Image Segmentation Python Demo with these 2 CPUs, Coffee Lake (9th Gen) and Skylake (6th Gen). I wasn't able to test using Cascade Lake due to some reason.

These are the results from the test:

Coffee Lake (9th Gen)

 Skylake (6th Gen)

Based on the output above, we can see that both CPUs give the same segmentation result. Therefore, I strongly suggest you use our latest OpenVINO version to experience the latest features and improvements.

Regards,

Zulkifli  

[TinyStone]

@Zulkifli_Intel Thank you for your quick test and response.

>>I noticed that you are using a very old version of OpenVINO. Is there any particular reason you use this version?

    The testing segmentation model and code are developed in 2021 by using OpenVino 2020.3 on Sky Lake CPU, and recently I executed the code using OpenVINO 2022.3.0 on Cascade Lake CPU, I found that the segmentation results are different.

I tested the code and model using  OpenVino 2020.3 on Sky Lake CPU and Cascade Lake CPU again, the results is different.

>>Therefore, I strongly suggest you use our latest OpenVINO version to experience the latest features and improvements.

I will test the code and models using  OpenVino 2022.3.0 on Sky Lake CPU and Cascade Lake CPU again, and share the result to you.

[TinyStone]

@Zulkifli_Intel 

I tested my segmentation model with same code and data using 2022.3 and using 2020.3 in different generation CPU again.

In my case, the inferenced segmentation results only contain 0 and 1, and I calculated the similarly of results in different generation CPU with same version OpenVino, the results are as follows:

The result actuary of version 2022.3 on different CPU is better than 2020.3, but the difference is not disappeared.

The data2 result of 2020.3 has big difference on different CPU, and I attached the images

In the image, the white color and red color region are results of different generation CPU by using same OpenVino version.

Compared with version 2020.3, the white region and red region have little difference in version 2022.3

However, results of 2022.3 are not exactly same (little difference) on different generation CPU,

and I want to know whether it is normal behavior of OpenVino.

Thank you very much.

[Zulkifli_Intel]

Hi Tinystone,

Thank you for sharing your findings

Usually, inference results should not differ when using different CPUs. However, there are possibility that the inference results be different due to bugs and errors, floating point, multithreading, and several more factors.

Regards,

Zulkifli 

[TinyStone]

@Zulkifli_Intel 

Thank you for your explanation.

>>there are possibility that the inference results be different due to bugs and errors, floating point, multithreading, and several more factors.

    Could you tell me which stage bugs and errors happened in?   The influenced code or the OpenVino library ?

    Currently, floating point and multithreading could be set in my influenced code and my influenced code is as follows:

        ie = Core()

        ie.set_property("CPU", {"INFERENCE_PRECISION_HINT": "f32", "CPU_THREADS_NUM": "12"})

        ovnet = ie.read_model(model=xmlpath, weights=binpath)

        model = ie.compile_model(model=ovnet, device_name="CPU")

        ovoutput = list(ovexec_out.outputs)[0]

        val_pred = model(inputs={'0': val_images})[ovoutput]

Regards,

TinyStone

---

[Zulkifli_Intel]

Hi Tinystone,

I might not be able to specify the exact area that caused the differences. Since you are using an old version of OpenVINO the big differences in inference results could be a bug from the version. For the performance wise, there will be a few differences when compared with different CPUs.

Regards,

Zulkifli

[Zulkifli_Intel]

Thank you for your question. If you need any additional information from Intel, please submit a new question as this thread is no longer being monitored.