Intel Advisor cpu-to-gpu offload

[TestDR]

Hi,

Intel Advisor cpu-to-gpu offload modelling for a specific GPU (--config gen9 for example) provides an estimate of GFLOPS for the code we are analysing.  Is this GFLOPS for the host CPU or for the GPU  we specify? 

[Rahila_T_Intel]

Hi,

Thank you for posting in Intel Communities.

Offload Modeling perspective has two workflows:

With CPU-to-GPU modeling, you can profile an application running on a CPU and model its performance on a target GPU device to determine if you should offload parts of your application to the GPU.

With GPU-to-GPU modeling, you can profile an application running on a GPU and model its performance on a different GPU device to estimate a potential speedup from running your application on the different target.

Offload Modeling can predict the code's performance if run on a GPU and lets you experiment with accelerator configuration parameters.

The Intel Advisor produces upper-bound speedup estimates using a bounds and bottlenecks performance model. 

It takes measured x86 CPU metrics and application characteristics as an input and applies an analytical model to estimate execution time and speedup on a target GPU.

In CPU-to-GPU modeling, GFLOPS (Offload Modeling) is the "Estimated number of giga floating-point operations per second".

It is collected during the Performance Modeling analysis in the Offload Modeling perspective and found in the Accelerated Regions tab > Code Regions pane > Estimated FLOAT Operations column group.

Also, when profiling a GPU application with Light data transfer simulation mode, you will get memory traffic estimation only for CPU code.

For references please check the below links:

https://www.intel.com/content/www/us/en/docs/advisor/user-guide/2023-0/model-offloading-to-a-gpu.html

https://www.intel.com/content/www/us/en/docs/advisor/user-guide/2023-0/work-with-standalone-html-reports.html

https://www.intel.com/content/www/us/en/docs/advisor/user-guide/2023-0/examine-data-transfers-for-modeled-regions.html

https://www.intel.com/content/www/us/en/docs/advisor/user-guide/2023-0/accelerator-metrics.html#GUID-51661E37-03B8-4051-91B1-EB05EDFB4D62

If this resolves your issue, make sure to accept this as a solution. This would help others with similar issue. 

Thanks

[TestDR]

Thanks for the information. I will look into the information provided.

[Rahila_T_Intel]

Hi,

We appreciate your interest in using Intel Advisor. 

Please let us know whether we can close this case if you don't have any other issues.

Thanks

[TestDR]

Hi,

I had a look. The GFLOP estimates provided by the Advisor varies between runs,. Why does it not give the same GFLOPS number each time I run?  What could be the reason for such variation? 

[Rahila_T_Intel]

Hi,

Sorry for the delay in response.

We are checking on this internally and waiting for the updates from the Dev team. We will share the updates with you.

Thanks

[Rahila_T_Intel]

Hi,

We tried to reproduce your issue from our end with a simple example (matmul). But the results for the offload modeling looks correct and were in the range of the "real" implementation. Also 2 tests got the same result. 

Which means GFLOP numbers should be exact. 

Could you please provide the below details to assist you better.

1. Could you please provide which application you are using and more details about that application?
2. Share the sample reproducer to try from our end (sample reproducer is an exact replica of your working sample)
3. Snapshots of the result page showing different GFLOP numbers.

Thanks

[Rahila_T_Intel]

Hi,

We haven't heard back from you. Could you please give an update?

Thanks

[Rahila_T_Intel]

Hi,

We have not heard back from you. This thread will no longer be monitored by Intel. If you need further assistance, please post a new question.

Thanks