Analyzers
Support for Analyzers (Intel VTune™ Profiler, Intel Advisor, Intel Inspector)
Announcements
Welcome to the Intel Community. If you get an answer you like, please mark it as an Accepted Solution to help others. Thank you!
4554 Discussions

Intel Advisor Benchmarks for memory/cache and FP/INT capabilities

drMikeT
New Contributor I
917 Views

Hello, 

Can we get some details on the micro-benchmarks that Intel Advisor (2020.02 or 2021.x) deploys to understand L1, 2, 3 and DRAM level BW and FP/INT capabilities of the underling cores?

Are these available by themselves? Or, do they derive from open-source ones (and which)?

Thank you!

Michael

0 Kudos
15 Replies
ChithraJ_Intel
Moderator
902 Views

Hi Michael,


Thanks for posting in Intel Community Forums.

Based on our understanding, there is no such Benchmarks available with Intel Advisor. We will discuss this with Subject Matter Experts and let you know the updates. As a work around, we would suggest you to try the Memory-Level Roofline Analysis in Intel® Advisor. Memory-Level Roofline analysis shows interactions between different memory levels(L1,L2,L3 & DRAM) available on your system and provides automatic guidance to improve your application performance.

Please refer to the below documentation for more details:

https://software.intel.com/content/www/xl/es/develop/articles/memory-level-roofline-model-with-advis...


Regards,

Chithra


drMikeT
New Contributor I
889 Views

Hi Chithra.

 

The Intel Advisor, uses microbenchmarks measures L1/2/3 and DRAM max BW and max FP/INT ops/sec per core on the system it runs on while it is profiling user codes. It uses these figures to generate the Roofline models/plots to guide the optimization of user codes.

I would like to know and if possible to get these microbenchmarks and run them at the command line so I can automate the profiling and optimization process via scripts. 

 

Are these microbenchmarks available or Open Source? That would be great for my work.

 

Thanls

Michael

ChithraJ_Intel
Moderator
869 Views

Hi Michael,


We are checking your issue with Engineering team and will get back to you soon once we receive an update.


Regards,

Chithra


drMikeT
New Contributor I
856 Views

Thanks Chithra

 

Happy New Year !

Intel thread Advisor reports per core performance limits (L1, L2, L3, DRAM vector/scalar BW and FP/INT vector/scalar add/FMA) limits at the roofs settings in the Roofline tab.

One important question I have is how do I validate that the reported numbers are what the underlying h/w supports? 

Conversely, I would like to generate these figures manually and use them to do my own roofline analysis of codes in an automated fashion as opposed to using the GUI for larger numbers of test cases. We often have to analyze runs with large number of different inputs from parameter ranges. Using the GUI is prohibitive. 

Using the command line advixe-cl is not that much more productive since we cannot easily automate the profiling and evaluation of large number of application runs when we vary its input arguments.

Can we get the codes in Intel Advisor that produce these per core performance limits ? It would assist us to automate performance profiling and tuning.

 

thanks

Michael

drMikeT
New Contributor I
854 Views

Is there any way to dump the per core FP/INT and memory limits into a text file? 

I saw advixe-cl --report=roofs but it does not seem to report everything.

 

thanks --Michael

Ruslan_M_Intel
Employee
832 Views

If you really want to play with benchmark tool you can use 'mem_bench' from bin subdirectory of advisor. On the other hand you could use PythonAPI to get an access to Advisor project. I'm not sure that you will  get all raw data you want (such as unprocessed benchmark results) but you can try to find more information about Advisor PythonAPI here

drMikeT
New Contributor I
823 Views

Thanks Ruslan_M !

 

I have actually noticed the mem_bench and I gave it a few runs guessing the arguments. Does Intel Thread Advisor use mem_bench to obtain the perf statistics for L1, L2, L3 and DRAM BW? 

Is there any documentation or can you provide command lines for running mem_bench for L1, L2 and L3? 

Are mem_bench algorithms related to those in mlc (Intel Memory Latency Checker in https://software.intel.com/content/www/us/en/develop/articles/intelr-memory-latency-checker.html?wap...

Any suggestions with how to run either tool (or any other) at the command line for L1, L2 and L3 BW measurements? That would help me a lot to automate things.

 

Thanks!

Michael

 

Ekaterina_G_Intel
785 Views

Hello Michael!

mem_bench is old utility to run benchmarks and it requires specific parameters. Intel Advisor is not using it now. 

Intel Advisor benchmarks are not open source. I'm sorry but there is no way to run benchmarks without profiling an application. We can suggest you 

To run benchmarks

1) Each time run roofline collection from command line on light application

advixe-cl -c roofline –project-dir<project_path> -- <application> <application parameters>

2) Run advixe-runtrc tool for project-dir with pre-collected survey result (to not repeat survey collection)

advixe-cl -c survey –project-dir<project_path> -- <application> <application parameters>

advixe-runtrc -r <project_path>/e000/trc000 --benchmarks=dram,mcdram,cpu_cache,flop_peak,int -- <application> <application parameters>

You can stop application after benchmarking pressing “ctrl+c” after the message “advixe: Peak bandwidth measurement finished” was printed.

After that add “e000/trc000” subdir to <project_path> specified on survey.

To print results

1) advixe-cl -r roofs –project-dir<project_path>

   Please let us know what exactly is not printed by -r roofs? It should print all measured benchmarks. 

2) Use Python API script roofs.py, which is available in install_dir/pythonapi/examples

   source advixe-vars.sh

  $ advixe-python install_dir/pythonapi/examples/roofs.py <project_path>

To get more information about Python API https://software.seek.intel.com/LP=18275 

drMikeT
New Contributor I
765 Views

Thanks Ekaterina!

I saw 

$ advixe-runtrc --help 

...

-cache-config=<string> Cache Configuration
...
-enable-cache-simulation Model CPU cache behavior on your application

If I specify the -cache-config= parameters while running the benchmarks, will it measure cache BWs as if the cache had that specification (cache size, associativity, etc.)? 

 

Regards

Michael

Ekaterina_G_Intel
748 Views

No, these parameters are for profiling your application and not benchmarks. Benchmarks are not parameterized using advixe-runtrc as they are automatically adjusted based on current hardware. Maybe you can do it based mem-bench, but it is old version of benchmarks.

As about cache-simulation parameters, Intel Advisor uses this functionality to get application's loop/function traffic for all levels of memory subsytem. Intel Advisor Memory Level Roofline can show you this breakdown. https://software.intel.com/content/www/us/en/develop/articles/memory-level-roofline-model-with-advis... 

drMikeT
New Contributor I
733 Views

Ekaterina, thanks!

BTW, Intel Advisor is a great and quite useful tool!  I have been involved with computer architectures and performance analysis and I understand the challenges and all the hard work that goes on behind the scenes.

I assume that the numbers of INT/FP scalar/vector arithmetic  and L1, 2, 3 BWs reported per core have been verified? I am mostly interested in the L1, 2 and 3 figures. I am using low assembly level benchmarks (AVX512 and AVX2 vector instructions) but I have never managed to get L1, 2, 3 and DRAM BW as high as that reported. 

Can U offer some insight how the cache benchmarks work ? Are the tests related to those Intel Memory Latency Checker (https://software.intel.com/content/www/us/en/develop/articles/intelr-memory-latency-checker.html)?

Or U can point me to articles explaining the methodology behind the cache benchmarks

thanks!

Michael

Ekaterina_G_Intel
679 Views

Thank you Michael!

I am glad that Intel Advisor was helpful for you. 

CPU cache benchmarks allocate memory that fit into required cache and use highest ISA available on machine. Intel Advisor also supports scalar cache benchmarks. We validate it by comparing with theoretical peaks supported by hardware.

Unfortunately, there is no any specific article for benchmarks methodology.

Hope this helps,

Ekaterina 

Zakhar_M_Intel1
Employee
510 Views

Hi Mike,

For your questions regarding benchmarking principles, consider sending email to vector_advisor@intel.com, so we could possibly dive deeper on this sophisticated topic.

Zakhar.

ChithraJ_Intel
Moderator
645 Views

Hi Michael,


Could you please confirm whether your query is clarified or not? If so, can we discontinue monitoring this thread?


Regards,

Chithra


ChithraJ_Intel
Moderator
631 Views

Hi Michael,


Thanks for giving kudos to the solution by Engineering team. We hope your issue got clarified. Also, thanks for the valuable feedback about Intel Advisor and great to know that Intel Advisor is helpful tool for you. We would discontinue monitoring this thread. Please feel free to raise a new thread in case of further Issues/queries.


Regards,

Chithra


Reply