OpenCL* for CPU
Ask questions and share information on Intel® SDK for OpenCL™ Applications and OpenCL™ implementations for Intel® CPU
Announcements
This forum covers OpenCL* for CPU only. OpenCL* for GPU questions can be asked in the GPU Compute Software forum. Intel® FPGA SDK for OpenCL™ questions can be ask in the FPGA Intel® High Level Design forum.
1663 Discussions

OpenCL on KabyLake (i7-7700k) and ubuntu 16.04

Dutta__Sankha
Beginner
456 Views

Hello

I am new to openCL programming but have fair amount of experience in GPGPU. My goal is to run some benchmark application using OpenCL and then to start writing my own code. I am using a KabyLake processor and want to use the GPU in it. I have followed the instructions mentioned in https://software.intel.com/en-us/articles/opencl-drivers and https://software.intel.com/en-us/intel-opencl/documentation. But I couldn't get it to work. The GPU is not getting detected and only CPU is detected. I was wondering if someone could provide me a step-by-step proper installation methodology. I am using ubuntu 16.04 (4.13.0) and I want to use OpenCL 2.0. I would be highly obliged if someone could provide me a guide to install OpenCL in the mentioned configuration. Please let me know if there are any further details that would be required. Thank you.

 

0 Kudos
5 Replies
Michael_C_Intel1
Moderator
456 Views

Hi SankhaD,

Thanks for the interest and sharing the deployment experience.

Here's the relevant content on the opencl-drivers article as of 20181017:

Linux* OS Ubuntu* 16.04.x (deb)

 

On the graphics runtime...

What feedback did you receive? Can you describe specifically where you were blocked in the deployment instructions...? Was it either the Readme or FAQ... or elsewhere? Can you share the CPU model number in use? It's possible that CPU doesn't have on die graphics.... check ark.intel.com.

edit: SKU mentioned in title... it has Intel® HD Graphics 630... should be OK for NEO runtime.

Precompiled Intel® Graphics Compute Runtime for OpenCL™ Driver packages are downloadable in .deb format underneath releases. In Ubuntu* OS, the apt facilities can deploy that .deb file. After installing the .deb, the runtime reference file should be updated in the regular location for OpenCL™ deployments. (i.e. /etc/OpenCL/vendors). The .so's them selves typically end up in /opt/intel. Users shouldn't usually have to bother with the .icd's or the .so's directly... but it can be reassuring to check what's going on.

Where were you blocked with the SDK/developer tools?

I highly recommend a new person use the updated OpenCL developer tools within Intel® System Studio 2019 over the Intel® SDK for OpenCL™ Applications 2017. The 2019 release has lots of usability, stability, bug fixes, and hopefully installation simplification.

 

-MichaelC

Dutta__Sankha
Beginner
456 Views

Hello Micheal

Thank you for getting back to me. I actually followed the steps mentioned in Intel® Graphics Compute Runtime for OpenCL™ Driver and I was able to build it without any issue. The driver built fine. My actual issue was with the sdk installation. So after the registration I downloaded the ubuntu version of the SDK but couldn't manage to install it after doing the gunzip. After gunzip I was converting the file to be executable using chmod but couldn't get to make it run. I have also tried different ways for sdk installation (after searching from the internet), but none of them actually worked. Every time I was trying to make this work, I using clinfo for verification. I was checking the list of devices but couldn't get to see the GPU device, which lead me to conclude that the issue is not solved.

Then I came across this post (https://software.intel.com/en-us/forums/opencl/topic/796730) in this forum and in the 3rd comment the user have posted that that the sdk file generating from gunzip is actually another tar file (in disguise) and needed to be converted. So I followed his methodology and indeed the sdk got installed. Now I am able to see the GPU device if I do a sudo clinfo (and not just clinfo). So from the beginning I would enumerate the steps that I followed this time.

1. Installing the .deb package of the Intel® Graphics Compute Runtime for OpenCL™ Driver under the release section (https://github.com/intel/compute-runtime/releases). I downloaded the .deb pre-compiled package 18.41.11649 version and installed it first. However, I was also able to build the driver successfully from the source code by following the build instructions in https://github.com/intel/compute-runtime/blob/master/documentation/BUILD_Ubuntu.md.

2. Then I downloaded the opencl sdk package and followed the 3rd comment mentioned in https://software.intel.com/en-us/forums/opencl/topic/796730 and I was able to install the sdk successfully after resolving some required dependencies. 

3. I carried out sudo clinfo and the following is the output 

Number of platforms                               2
  Platform Name                                   Experimental OpenCL 2.1 CPU Only Platform
  Platform Vendor                                 Intel(R) Corporation
  Platform Version                                OpenCL 2.1 LINUX
  Platform Profile                                FULL_PROFILE
  Platform Extensions                             cl_khr_icd cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_byte_addressable_store cl_khr_depth_images cl_khr_3d_image_writes cl_intel_exec_by_local_thread cl_khr_spir cl_khr_fp64 cl_khr_image2d_from_buffer 
  Platform Host timer resolution                  1ns
  Platform Extensions function suffix             INTEL
 
  Platform Name                                   Intel(R) OpenCL HD Graphics
  Platform Vendor                                 Intel(R) Corporation
  Platform Version                                OpenCL 2.1 
  Platform Profile                                FULL_PROFILE
  Platform Extensions                             cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_fp16 cl_khr_depth_images cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_icd cl_khr_image2d_from_buffer cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_intel_subgroups cl_intel_required_subgroup_size cl_intel_subgroups_short cl_khr_spir cl_intel_accelerator cl_intel_media_block_io cl_intel_driver_diagnostics cl_intel_device_side_avc_motion_estimation cl_khr_priority_hints cl_khr_throttle_hints cl_khr_create_command_queue cl_khr_fp64 cl_khr_subgroups cl_khr_il_program cl_khr_mipmap_image cl_khr_mipmap_image_writes cl_intel_planar_yuv cl_intel_packed_yuv cl_intel_motion_estimation cl_intel_advanced_motion_estimation 
  Platform Host timer resolution                  1ns
  Platform Extensions function suffix             INTEL
 
  Platform Name                                   Experimental OpenCL 2.1 CPU Only Platform
Number of devices                                 1
  Device Name                                     Intel(R) Core(TM) i7-7700K CPU @ 4.20GHz
  Device Vendor                                   Intel(R) Corporation
  Device Vendor ID                                0x8086
  Device Version                                  OpenCL 2.1 (Build 10)
  Driver Version                                  1.2.0.10
  Device OpenCL C Version                         OpenCL C 2.0 
  Device Type                                     CPU
  Device Profile                                  FULL_PROFILE
  Max compute units                               8
  Max clock frequency                             4200MHz
  Device Partition                                (core)
    Max number of sub-devices                     8
    Supported partition types                     by counts, equally, by names (Intel)
  Max work item dimensions                        3
  Max work item sizes                             8192x8192x8192
  Max work group size                             8192
  Preferred work group size multiple              128
  Max sub-groups per work group                   1
  Preferred / native vector sizes                 
    char                                                 1 / 32      
    short                                                1 / 16      
    int                                                  1 / 8       
    long                                                 1 / 4       
    half                                                 0 / 0        (n/a)
    float                                                1 / 8       
    double                                               1 / 4        (cl_khr_fp64)
  Half-precision Floating-point support           (n/a)
  Single-precision Floating-point support         (core)
    Denormals                                     Yes
    Infinity and NANs                             Yes
    Round to nearest                              Yes
    Round to zero                                 No
    Round to infinity                             No
    IEEE754-2008 fused multiply-add               No
    Support is emulated in software               No
    Correctly-rounded divide and sqrt operations  No
  Double-precision Floating-point support         (cl_khr_fp64)
    Denormals                                     Yes
    Infinity and NANs                             Yes
    Round to nearest                              Yes
    Round to zero                                 Yes
    Round to infinity                             Yes
    IEEE754-2008 fused multiply-add               Yes
    Support is emulated in software               No
    Correctly-rounded divide and sqrt operations  No
  Address bits                                    64, Little-Endian
  Global memory size                              16670203904 (15.53GiB)
  Error Correction support                        No
  Max memory allocation                           4167550976 (3.881GiB)
  Unified memory for Host and Device              Yes
  Shared Virtual Memory (SVM) capabilities        (core)
    Coarse-grained buffer sharing                 Yes
    Fine-grained buffer sharing                   Yes
    Fine-grained system sharing                   Yes
    Atomics                                       Yes
  Minimum alignment for any data type             128 bytes
  Alignment of base address                       1024 bits (128 bytes)
  Preferred alignment for atomics                 
    SVM                                           64 bytes
    Global                                        64 bytes
    Local                                         0 bytes
  Max size for global variable                    65536 (64KiB)
  Preferred total size of global vars             65536 (64KiB)
  Global Memory cache type                        Read/Write
  Global Memory cache size                        262144
  Global Memory cache line                        64 bytes
  Image support                                   Yes
    Max number of samplers per kernel             480
    Max size for 1D images from buffer            260471936 pixels
    Max 1D or 2D image array size                 2048 images
    Base address alignment for 2D image buffers   64 bytes
    Pitch alignment for 2D image buffers          64 bytes
    Max 2D image size                             16384x16384 pixels
    Max 3D image size                             2048x2048x2048 pixels
    Max number of read image args                 480
    Max number of write image args                480
    Max number of read/write image args           480
  Max number of pipe args                         16
  Max active pipe reservations                    32767
  Max pipe packet size                            1024
  Local memory type                               Global
  Local memory size                               32768 (32KiB)
  Max constant buffer size                        131072 (128KiB)
  Max number of constant args                     480
  Max size of kernel argument                     3840 (3.75KiB)
  Queue properties (on host)                      
    Out-of-order execution                        Yes
    Profiling                                     Yes
    Local thread execution (Intel)                Yes
  Queue properties (on device)                    
    Out-of-order execution                        Yes
    Profiling                                     Yes
    Preferred size                                4294967295 (4GiB)
    Max size                                      4294967295 (4GiB)
  Max queues on device                            4294967295
  Max events on device                            4294967295
  Prefer user sync for interop                    No
  Profiling timer resolution                      1ns
  Execution capabilities                          
    Run OpenCL kernels                            Yes
    Run native kernels                            Yes
    Sub-group independent forward progress        No
    IL version                                    SPIR-V_1.0
    SPIR versions                                 1.2
  printf() buffer size                            1048576 (1024KiB)
  Built-in kernels                                
  Device Available                                Yes
  Compiler Available                              Yes
  Linker Available                                Yes
  Device Extensions                               cl_khr_icd cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_byte_addressable_store cl_khr_depth_images cl_khr_3d_image_writes cl_intel_exec_by_local_thread cl_khr_spir cl_khr_fp64 cl_khr_image2d_from_buffer 
 
  Platform Name                                   Intel(R) OpenCL HD Graphics
Number of devices                                 1
  Device Name                                     Intel(R) Gen9 HD Graphics NEO
  Device Vendor                                   Intel(R) Corporation
  Device Vendor ID                                0x8086
  Device Version                                  OpenCL 2.1 NEO 
  Driver Version                                  18.41.11649
  Device OpenCL C Version                         OpenCL C 2.0 
  Device Type                                     GPU
  Device Profile                                  FULL_PROFILE
  Max compute units                               24
  Max clock frequency                             0MHz
  Device Partition                                (core)
    Max number of sub-devices                     0
    Supported partition types                     None
  Max work item dimensions                        3
  Max work item sizes                             256x256x256
  Max work group size                             256
  Preferred work group size multiple              32
  Max sub-groups per work group                   32
  Preferred / native vector sizes                 
    char                                                16 / 16      
    short                                                8 / 8       
    int                                                  4 / 4       
    long                                                 1 / 1       
    half                                                 8 / 8        (cl_khr_fp16)
    float                                                1 / 1       
    double                                               1 / 1        (cl_khr_fp64)
  Half-precision Floating-point support           (cl_khr_fp16)
    Denormals                                     Yes
    Infinity and NANs                             Yes
    Round to nearest                              Yes
    Round to zero                                 Yes
    Round to infinity                             Yes
    IEEE754-2008 fused multiply-add               Yes
    Support is emulated in software               No
    Correctly-rounded divide and sqrt operations  No
  Single-precision Floating-point support         (core)
    Denormals                                     Yes
    Infinity and NANs                             Yes
    Round to nearest                              Yes
    Round to zero                                 Yes
    Round to infinity                             Yes
    IEEE754-2008 fused multiply-add               Yes
    Support is emulated in software               No
    Correctly-rounded divide and sqrt operations  Yes
  Double-precision Floating-point support         (cl_khr_fp64)
    Denormals                                     Yes
    Infinity and NANs                             Yes
    Round to nearest                              Yes
    Round to zero                                 Yes
    Round to infinity                             Yes
    IEEE754-2008 fused multiply-add               Yes
    Support is emulated in software               No
    Correctly-rounded divide and sqrt operations  No
  Address bits                                    64, Little-Endian
  Global memory size                              13336162304 (12.42GiB)
  Error Correction support                        No
  Max memory allocation                           4294959104 (4GiB)
  Unified memory for Host and Device              Yes
  Shared Virtual Memory (SVM) capabilities        (core)
    Coarse-grained buffer sharing                 Yes
    Fine-grained buffer sharing                   No
    Fine-grained system sharing                   No
    Atomics                                       No
  Minimum alignment for any data type             128 bytes
  Alignment of base address                       1024 bits (128 bytes)
  Preferred alignment for atomics                 
    SVM                                           64 bytes
    Global                                        64 bytes
    Local                                         64 bytes
  Max size for global variable                    65536 (64KiB)
  Preferred total size of global vars             4294959104 (4GiB)
  Global Memory cache type                        Read/Write
  Global Memory cache size                        524288
  Global Memory cache line                        64 bytes
  Image support                                   Yes
    Max number of samplers per kernel             16
    Max size for 1D images from buffer            268434944 pixels
    Max 1D or 2D image array size                 2048 images
    Base address alignment for 2D image buffers   4 bytes
    Pitch alignment for 2D image buffers          4 bytes
    Max 2D image size                             16384x16384 pixels
    Max 3D image size                             16384x16384x2048 pixels
    Max number of read image args                 128
    Max number of write image args                128
    Max number of read/write image args           128
  Max number of pipe args                         16
  Max active pipe reservations                    1
  Max pipe packet size                            1024
  Local memory type                               Local
  Local memory size                               65536 (64KiB)
  Max constant buffer size                        4294959104 (4GiB)
  Max number of constant args                     8
  Max size of kernel argument                     1024
  Queue properties (on host)                      
    Out-of-order execution                        Yes
    Profiling                                     Yes
  Queue properties (on device)                    
    Out-of-order execution                        Yes
    Profiling                                     Yes
    Preferred size                                131072 (128KiB)
    Max size                                      67108864 (64MiB)
  Max queues on device                            1
  Max events on device                            1024
  Prefer user sync for interop                    Yes
  Profiling timer resolution                      83ns
  Execution capabilities                          
    Run OpenCL kernels                            Yes
    Run native kernels                            No
    Sub-group independent forward progress        Yes
    IL version                                    SPIR-V_1.0 
    SPIR versions                                 1.2 
  printf() buffer size                            4194304 (4MiB)
  Built-in kernels                                block_motion_estimate_intel;block_advanced_motion_estimate_check_intel;block_advanced_motion_estimate_bidirectional_check_intel;
  Motion Estimation accelerator version (Intel)   2
  Device Available                                Yes
  Compiler Available                              Yes
  Linker Available                                Yes
  Device Extensions                               cl_khr_3d_image_writes cl_khr_byte_addressable_store cl_khr_fp16 cl_khr_depth_images cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_icd cl_khr_image2d_from_buffer cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_intel_subgroups cl_intel_required_subgroup_size cl_intel_subgroups_short cl_khr_spir cl_intel_accelerator cl_intel_media_block_io cl_intel_driver_diagnostics cl_intel_device_side_avc_motion_estimation cl_khr_priority_hints cl_khr_throttle_hints cl_khr_create_command_queue cl_khr_fp64 cl_khr_subgroups cl_khr_il_program cl_khr_mipmap_image cl_khr_mipmap_image_writes cl_intel_planar_yuv cl_intel_packed_yuv cl_intel_motion_estimation cl_intel_advanced_motion_estimation 
 
NULL platform behavior
  clGetPlatformInfo(NULL, CL_PLATFORM_NAME, ...)  No platform
  clGetDeviceIDs(NULL, CL_DEVICE_TYPE_ALL, ...)   No platform
  clCreateContext(NULL, ...) [default]            No platform
  clCreateContext(NULL, ...) [other]              Success [INTEL]
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_CPU)  No platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_GPU)  No platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_ACCELERATOR)  No platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_CUSTOM)  No platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_ALL)  No platform

 

As you can see the GPU got detected properly.

However, I have some following question and some feedback:

1. In the NULL platform behavior the

clCreateContextFromType(NULL, CL_DEVICE_TYPE_CPU)  No platform
clCreateContextFromType(NULL, CL_DEVICE_TYPE_GPU)  No platform

Why the output is showing as No Platform both for CPU and GPU?

2. You mentioned that using Intel System Studio would be a better choice. I have tried to use that as well. The installation was fine but the device was not getting detected with the studio as well.

a. Can you please provide me with some guide as to how can I program OpenCL using the integrated GPU in Intel System Studio?

b. How can I write OpenCL code after installing the SDK (not system studio)?

3. What would be my next steps as to execute an OpenCL program on iGPU. Can you please provide me with a step by step guidance right after the sdk gets installed.

Now I just want to provide you a feedback regarding my experience. If any hobbyist or a curious person tries to use the OpenCL on integrated GPU of Intel then that person would be very much frustrated and demotivated as the installation guidelines was ill-informed. I wasted 2 days because nowhere it was mentioned that the file after doing the gunzip is another file (and not executable) which needed to be converted to tar.gz file. I am fortunate enough to find the post that addresses that issue. Even now I am not sure if everything is good and I can assess it after running some tests and also after hearing back from you. So considering the OpenCL SDK as a industry level software, a proper documentation regarding successful installation, the issues during installation and how those issues can be mitigated is very much missing. It would be very much helpful if Intel could create a proper documentation regarding the proper installation guidelines then it would be helpful. As far as of now, nothing like that exists. Whatever is mentioned as installation guidelines are not proper installation guidelines, they are kind of working principle of OpenCL on intel products which would nice to look into if only the software gets installed properly and user gets to use it without any issue. Thank you so much again for your reply.

 

Dutta__Sankha
Beginner
456 Views

I would also like to add something after my last comment. There are 2 issues that came up which I am listing below:

1. The first is the code builder plug-in error which is mentioned in the post https://software.intel.com/en-us/forums/opencl/topic/759485. So I would try to solve using the post. But even after solving the issue, I guess, the following issue would come up.

2. I was able to install the system studio but the GPU device doesn't get listed if I launch the system without sudo access. Once I launch without the sudo then the gpu is not showing in the discovered devices same as clinfo. So I was wondering if there is any solution to this problem as to why I need sudo access to get the gpu recognized and how that can be avoided.

Thank you again.

Michael_C_Intel1
Moderator
456 Views

Hi SankhaD,

Thanks for the questions and feedback.

Please make sure to check the release notes for the SDK or developer tools chosen here: https://software.intel.com/en-us/articles/intel-sdk-for-opencl-applications-release-notes.

1. Checking the Khronos specification that behavior appears to be  implementation defined.  https://www.khronos.org/registry/OpenCL/sdk/2.1/docs/man/xhtml/clCreateContextFromType.html see properties. The Khronos standard references are a key resource for any new developer.

2.  the device was not getting detected with the studio as well.

Can you clarify what this means: 'detected with the Studio'?

  • Intel® System Studio 2019 OpenCL™ developer tools is the latest update for the provisions in the Intel® SDK for OpenCL™ Applications 2017.
  • Intel® SDK for OpenCL™ Applications 2017 was not validated for Intel® Graphics Compute Runtime for OpenCL™ Driver (NEO) in Linux* environments. It was also only validated for specific Linux* OS kernels and SRB5.0 implementation, not Intel® Graphics Compute Runtime for OpenCL™ Dirver (NEO). NEO usage with the 2017 SDK may provide some tool functionalities, but NEO usage with the 2017 SDK is not supported.
  • IDE plugins have platform interrogation facilities that do allow for some OpenCL™ platform information viewing... is that what you mean by not getting detected with the studio?

2a. 

Please see the getting started guide for a walkthrough of what the SDK/developer tools offers. 

https://software.intel.com/en-us/articles/sdk-for-opencl-2019-gsg <---- strongly recommended based on feedback, and the system specified.

https://software.intel.com/en-us/articles/sdk-for-opencl-2017-gsg

https://software.intel.com/en-us/articles/sdk-for-opencl-2017-gsg-linux-os

Keep in mind that Linux* OS distributions can vary quite a bit from one another. Even the same distribution may have differing style of deployments. This may alter prerequisite setup.

OpenCL™ programs need to select the Intel® Graphics Technology device... so long as the OpenCL™ version (i.e. 1.2/2.0/2.1) calls and data structures match the implementation and the interrogated device target... Then, contexts, command, queues, NDRange kernel launches, and memory calls to marshall and execute the kernel are good to go.

The 2019 developer tools have an IDE template program for a 'helloworld' and they target Intel® Graphics Technology. Looking at those would be a good start.

For Intel® Graphics Technology specific extensions were added to the standard to get better performance for the graphics u-architecture. These are the best examples: https://github.com/intel/compute-samples. The doc's on github should be pretty good walkthroughs.

2b. There are many options. Strictly speaking, an IDE environment is not needed. OpenCL™ ICD Loader interrogator runtime is needed.... these are described in the GSGs linked above. SDKs contain the runtime, developers can write a c/cpp application and link against that runtime at the command line and without an IDE.

3. Create or obtain a program that uses Intel® Graphics Technology. Please look at GSGs, look at these samples https://github.com/intel/compute-samples, and work from any graphics target templates included with the IDE.

Some of the blocking issues observed are located in the release notes... New users (especially on Linux* OS) are strongly encouraged to review the release notes: https://software.intel.com/en-us/articles/intel-sdk-for-opencl-applications-release-notes. Documentation for a Windows* OS walkthrough for Intel® System Studio 2019 is up here: https://software.intel.com/en-us/articles/sdk-for-opencl-2019-gsg-windows-oThis walkthrough is geared toward new developers. The Linux* OS equivalent is work in progress. 

Thanks,

-MichaelC

 

 

Dutta__Sankha
Beginner
456 Views

Hello Micheal 

Thank you for your reply. As per your suggestion I installed the intel system studio (again) and I was trying to use that. So the system studio is in the  /opt/intel/ folder and the launch script is iss_ide_eclipse-launcher.sh. So when I launch it with sudo ./iss_ide_eclipse-launcher.sh then under the OpenCL Platform Information, I can see the Gen9 HD Graphics is enlisted. Otherwise, the device is not getting listed. I want to know why this is happening and how can I avoid it, I mean launching without sudo and still able to enlist the GPU device.

Actually going over your post I am confused with the driver usage w.r.t to the OpenCL software tool. You mentioned that "Intel® SDK for OpenCL™ Applications 2017 was not validated for Intel® Graphics Compute Runtime for OpenCL™ Driver (NEO) in Linux* environments.I am not sure which driver to use with which software tool. So let's say I am using Intel System Studio to write my OpenCL code, then I have to use the NEO driver? And if I am using Intel OpenCL SDK then I have to use the SRB5.0 driver? Or given my specification I can use SRB5.0 in either case. 

Intel System Studio 2019 ultimate edition--> Either Intel® Graphics Compute Runtime for OpenCL™ Driver (NEO) or SRB5.0 OR Intel® Graphics Compute Runtime for OpenCL™ Driver (NEO) ONLY

Intel SDK 2017 --> SRB5.0 (ONLY)

Also can you provide me a link to download SRB5.0 and also to get some note about the driver. Thank you.

 

 

Reply