Intel® Distribution of OpenVINO™ Toolkit
Community support and discussions about the Intel® Distribution of OpenVINO™ toolkit, OpenCV, and all things computer vision-related on Intel® platforms.
6026 Discussions

Demo code not running when external GPU installed

Stratman
Novice
966 Views

Hi,

 

Hope you can help us out please?

We have a system running a Tiger Lake UP3 processor (i7) with built in 96 Execution Units, i.e. Intel(R) Gen12LP HD Graphics (iGPU). Have installed OpenVINO 2021.4 on top of Ubuntu 20.04 and using the benchmark_app demo we get excellent results for the CPU and GPU – double the FPS for the GPU compared to the CPU.

 

We want to demonstrate the benefits of running AI on the GPU of the Tiger Lake visually so want to run the Social Distancing demo. This all works fine when displaying the output using the built-in graphics i.e. iGPU. However, we want to offload all the display graphics onto an external GPU, thus allowing 100% of the Intel GPU to be used for inferencing. So to do this we have installed a separate external Nvidia graphics card into the system to take care of the graphics display.

 

This all works fine when selecting the –d CPU option, but when trying –d GPU it fails due to the Nvidia driver being installed. I can’t see any way of specifying a specific GPU device as part of the command parameters. Here is the output of the lspci specifically looking at the VGA devices:-

 

00:02.0 VGA compatible controller: Intel Corporation Device 9a49 (rev 01)

07:00.0 VGA compatible controller: NVIDIA Corporation GP107GLM [Quadro P2000 Mobile] (rev a1)

 

This is the partial output from the clinfo command when the external card is fitted:-

 

Number of platforms 1
Platform Name NVIDIA CUDA
Platform Vendor NVIDIA Corporation
Platform Version OpenCL 3.0 CUDA 11.4.153
Platform Profile FULL_PROFILE
Platform Extensions cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_fp64 cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_icd cl_khr_gl_sharing cl_nv_compiler_options cl_nv_device_attribute_query cl_nv_pragma_unroll cl_nv_copy_opts cl_khr_gl_event cl_nv_create_buffer cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_nv_kernel_attribute cl_khr_device_uuid cl_khr_pci_bus_info
Platform Host timer resolution 0ns
Platform Extensions function suffix NV

Platform Name NVIDIA CUDA
Number of devices 1
Device Name Quadro P2000
Device Vendor NVIDIA Corporation
Device Vendor ID 0x10de
Device Version OpenCL 3.0 CUDA
Driver Version 470.82.00
Device OpenCL C Version OpenCL C 1.2
Device Type GPU
Device Topology (NV) PCI-E, 07:00.0

...

 

And this is the output with no external graphics card:-

Number of platforms 1
Platform Name Intel(R) OpenCL HD Graphics
Platform Vendor Intel(R) Corporation
Platform Version OpenCL 3.0
Platform Profile FULL_PROFILE
Platform Extensions cl_khr_byte_addressable_store cl_khr_fp16 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_icd cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_intel_subgroups cl_intel_required_subgroup_size cl_intel_subgroups_short cl_khr_spir cl_intel_accelerator cl_intel_driver_diagnostics cl_khr_priority_hints cl_khr_throttle_hints cl_khr_create_command_queue cl_intel_subgroups_char cl_intel_subgroups_long cl_khr_il_program cl_intel_spirv_media_block_io cl_intel_spirv_subgroups cl_khr_spirv_no_integer_wrap_decoration cl_intel_unified_shared_memory_preview cl_khr_mipmap_image cl_khr_mipmap_image_writes cl_intel_planar_yuv cl_intel_packed_yuv cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_image2d_from_buffer cl_khr_depth_images cl_intel_media_block_io cl_khr_3d_image_writes cl_intel_va_api_media_sharing cl_intel_subgroup_local_block_io
Platform Host timer resolution 1ns
Platform Extensions function suffix INTEL

Platform Name Intel(R) OpenCL HD Graphics
Number of devices 1
Device Name Intel(R) Gen12LP HD Graphics NEO
Device Vendor Intel(R) Corporation
Device Vendor ID 0x8086
Device Version OpenCL 3.0 NEO
Driver Version 20.28.17293
Device OpenCL C Version OpenCL C 3.0
Device Type GPU
Device Profile FULL_PROFILE
Device Available Yes
Compiler Available Yes
Linker Available Yes
Max compute units 96

...

 

Any ideas on how we could get this to work please?

Many thanks in advance,

Chris

Labels (2)
0 Kudos
10 Replies
Wan_Intel
Moderator
940 Views

Hi Stratman,

Thank you for reaching out to us.


Have a try with these steps that had been discussed in previous post.


For Linux:

1. Enable Integrated Graphics in the BIOS. In BIOS menu, the setting was under Chipset > Internal Graphics > [Enabled].

2. Follow the required steps to enable the toolkit components to use the Intel® Processor Graphics (GPU) for Linux.


For Windows:

1. Update the graphics driver using Device Manager > Display adapters > Intel(R) HD Graphics > Search automatically for updated driver software.

2. Follow the required steps to enable the toolkit components to use Intel® Processor Graphics (GPU) for Windows.


Regards,

Wan


Stratman
Novice
934 Views

Hi Wan,

Thanks for getting back to us. Yes I did see that previous post and we have tried the same, i.e. set the BIOS to internal or integrated. When we do this we get both devices listed in the lspci scan:-

00:02.0 VGA compatible controller: Intel Corporation Device 9a49 (rev 01)

07:00.0 VGA compatible controller: NVIDIA Corporation GP107GLM [Quadro P2000 Mobile] (rev a1)

 

If we set it to disabled we only get:-

07:00.0 VGA compatible controller: NVIDIA Corporation GP107GLM [Quadro P2000 Mobile] (rev a1)

 

So we know we have the internal GPU enabled. It seems 'clinfo' doesn't see both devices, always defaults/only sees the external card for some reason. We know the Intel driver is installed OK as when the external graphics card is removed 'clinfo' see's the device - see above output.

I would have thought both devices should be listed in 'clinfo' - but if they were, how do we then select which GPU device to use?

In the previous post the solution mentions that both devices were installed and running, is it possible to get an lspci and clinfo output for that setup please?

 

Best regards,

Chris 

Wan_Intel
Moderator
883 Views

Hi Stratman,

Thanks for your information and thanks for your patience.

 

For your information, I can enumerate both Intel and NVIDIA devices on the system with clinfo command-line application.

 

able_to_detect_gpu.png

 

The result of running the Pipeline Demo Script with GPU plugin is shown as follows:

 

result.png

 

The following information is the environment details from my end:

 

·      Version of the OpenVINO™ toolkit = OpenVINO™ toolkit 2021.4.2

 

·      Operating System = Ubuntu 20.04.3 LTS, kernel version 5.11

 

·      Hardware specifications

 

·      Intel® Core™ i5-11400 Processor, driver version 21.37.20939

 

·      NVIDIA GeForce GTX 1650 SUPER, driver version 470.86

 

I noticed that you are using the OpenVINO™ toolkit 2021.4, and your Intel Driver version is 20.28.17293.

 

Please try the following steps and run clinfo command-line application again:

 

1.   Upgrade to the latest version of the OpenVINO™ toolkit which is version 2021.4.2 for new capabilities and performance improvements. You may download the latest version from here.

 

2.   Install the recommended Ubuntu kernel version for your system. You may refer to the Ubuntu kernel release cycle to select the Ubuntu kernel.

 

3.   Install the recommended NVIDIA Driver for your NVIDIA product. You may refer to NVIDIA Driver Downloads to select the NVIDIA Driver. Command Line NVIDIA installation method is available here.

 

4.   Step to install the Driver version via system package manager is available here. Or, you can install the driver version 21.29.20389 or 20.35.17767 for your Intel® Processor Graphics (GPU) via the following steps:

 

a.   Go to the install_dependencies directory:

cd /opt/intel/openvino_2021/install_dependencies/

 

b.   Update the driver version:

sudo nano install_NEO_OCL_driver.sh

 

c.    Replace '19.41.14441' to ‘21.29.20389’ or ‘20.35.17767’ at Line 23

 

d.   Run the script:

sudo -E ./install_NEO_OCL_driver.sh

 

To check available inference engine devices, you can run Hello Query Device Python Sample.

 

Hope this helps.

 

Regards,

Wan

 

Stratman
Novice
836 Views

Hi Wan,

 

Thanks very much for taking the time for putting a system together and coming back to us with a very comprehensive response.

I will try out your recommendations (probably won't be until the New Year now) and come back to you.

 

Best regards,

Chris

Wan_Intel
Moderator
724 Views

Hi Chris,

 

Just wanted to follow up and see if my previous response was helpful.

Has your issue been resolved?

Please let us know if you have any questions.

 

 

Best regards,

Wan

 

Stratman
Novice
703 Views

Hi Wan,

 

We will hopefully be looking at this again this week, will come back to you with the results.

Thanks again.

 

Best regards,

Chris

Stratman
Novice
663 Views

Hi Wan,

Good news in that we have followed your steps and now we can run the demo when we have an external graphics card fitted, so thanks again for your help with this.

However, as mentioned above, the reason for doing this we wanted to offload all the display graphics onto an external GPU, thus allowing 100% of the Intel GPU to be used for inferencing to see if the processing of the display was affecting the performance of the iGPU. By doing this it has shown a small improvement of 2-3 FPS but it still lags the CPU by 7-8 FPS, i.e. 30FPS when inferencing on the CPU and only 23 FPS when running on the iGPU.

As mentioned above, when running the benchmark_app this shows over a 100% improvement when running on the iGPU compared to the CPU.

Do you know of a visual OpenVINO demo, i.e. one that shows FPS that we could run that demonstrates the benefits of running AI on the GPU of the Tiger Lake? We would like to use this at some big exhibitions this year to promote our (and Intel's!) products.

Thanks again for your help with this.

Best regards,

Chris 

Wan_Intel
Moderator
656 Views

Hi Stratman,

Glad to know that my previous response was helpful.

 

For your information, you can use Social Distance C++ Demo, a retail social distance application that detects people, measures the distance between them, and show Frame per second (FPS) during the inference.

 

On another note, I have run the application with person-detection-retail-0013 and person-reidentification-retail-0277 using a CPU and a GPU. You may download the sample videos as input here.

 

The inference results are shown as follows:

CPU Plugin: 28.2FPS for (730 / 1) frames

CPU_social_distance.JPG

 

GPU Plugin: 43.0FPS for (730 / 1) frames

GPU_social_distance.JPG

 

 

Regards,

Wan

 

Stratman
Novice
622 Views

Hi Wan,

 

Thanks again for your help with this, much appreciated.

We don't get quite the same numbers as you and we are running this under Ubuntu, but we think this should be sufficient for our demo.

Please consider this case closed.

 

Best regards,

Chris

Wan_Intel
Moderator
602 Views

Hi Chris,

It’s my pleasure.

 

This thread will no longer be monitored since this issue has been resolved.

If you need any additional information from Intel, please submit a new question.

 

 

Best regards,

Wan


Reply