Opevino R3.1 HDDL performance decrease after inference iteration

[hung__barry]

Hello everyone,

I use the IEI Mustang-V100-MX8 (16 Myriad) with newest OpenVINO toolkit R2019 3.1 to run the HDDL inference by benchmark_app on the Windows 10. I found that the performance will decrease after inference iteration

I iterate the 10 times for my test with the following command

benchmark_app.exe -m C:\Users\user\Desktop\temp\barry\OpenVINO_2019R310\test_data\MobileNet-SSD\VINO_2019_R310\FP32\MobileNet-SSD.xml -i C:\Users\user\Desktop\temp\barry\OpenVINO_2019R310\test_data\testSSD.bmp -d HDDL -api sync -nireq 1

Here is the summary from my test. By the way, the detail is in the attached file:

Count:      1742 iterations
Duration:   60010.53 ms
Latency:    34.50 ms
Throughput: 28.99 FPS

Temperature [74.1]  [66.2]  [73.7]  [71.9]  [64.4]  [64.4]  [63.4]  [66.4]  [64.6]  [73.5]  [73.7]  [74.5]  [65.2]  [74.1]  [65.8]  [72.7]

Count:      1530 iterations
Duration:   60120.30 ms
Latency:    34.49 ms
Throughput: 28.99 FPS

Temperature [75.3]  [67.3]  [73.9]  [72.1]  [65.0]  [64.8]  [64.0]  [67.1]  [64.6]  [74.9]  [74.7]  [74.5]  [65.8]  [74.3]  [66.2]

Count:      1021 iterations
Duration:   60226.78 ms
Latency:    34.58 ms
Throughput: 28.92 FPS

Temperature [75.3]  [67.7]  [74.9]  [72.7]  [64.8]  [64.8]  [64.4]  [66.9]  [65.0]  [75.6]  [75.1]  [74.9]  [65.6]  [74.5]  [67.1]  [73.7]

Count:      691 iterations
Duration:   60001.07 ms
Latency:    35.51 ms
Throughput: 28.16 FPS

Temperature [75.8]  [67.5]  [74.9]  [72.7]  [65.2]  [65.2]  [64.8]  [67.7]  [65.2]  [75.1]  [75.3]  [75.3]  [66.0]  [74.9]  [66.9]  [74.5]

Count:      575 iterations
Duration:   60066.63 ms
Latency:    35.48 ms
Throughput: 28.19 FPS

Temperature [76.4]  [67.5]  [75.1]  [73.3]  [65.8]  [66.4]  [64.6]  [67.5]  [65.6]  [76.2]  [75.8]  [75.8]  [67.1]  [75.1]  [67.3]  [74.5]

Count:      533 iterations
Duration:   60064.46 ms
Latency:    35.51 ms
Throughput: 28.16 FPS

Temperature [76.4]  [67.5]  [75.3]  [73.9]  [66.0]  [65.0]  [64.6]  [67.1]  [65.6]  [76.0]  [76.0]  [75.6]  [66.4]  [75.6]  [67.1]  [74.7]

Count:      521 iterations
Duration:   60005.13 ms
Latency:    35.53 ms
Throughput: 28.15 FPS

Temperature [76.0]  [68.3]  [75.8]  [73.5]  [65.4]  [65.2]  [65.6]  [67.9]  [65.4]  [76.0]  [75.6]  [76.2]  [66.9]  [76.0]  [67.7]  [74.3]

Count:      515 iterations
Duration:   60117.44 ms
Latency:    35.50 ms
Throughput: 28.17 FPS

Temperature [77.0]  [68.1]  [75.8]  [74.1]  [66.2]  [65.8]  [65.0]  [67.7]  [66.0]  [76.4]  [76.4]  [76.0]  [67.1]  [75.3]  [67.3]  [75.1]

Count:      509 iterations
Duration:   60147.81 ms
Latency:    35.57 ms
Throughput: 28.11 FPS

Temperature [76.8]  [68.5]  [75.8]  [74.7]  [65.6]  [65.6]  [65.2]  [67.7]  [65.6]  [76.2]  [76.0]  [76.0]  [66.9]  [76.0]  [67.7]  [75.3]

As you can see, the count is decrease after each turn, and HDDL did not seem to enter the throttle state because the temperature is between the 65~76.

I cannot figure out where the problem is. Is there any suggestion? Any help would be appreciated .

Thanks