Intel Advisor Benchmarks for memory/cache and FP/INT capabilities

[drMikeT]

Hello, 

Can we get some details on the micro-benchmarks that Intel Advisor (2020.02 or 2021.x) deploys to understand L1, 2, 3 and DRAM level BW and FP/INT capabilities of the underling cores?

Are these available by themselves? Or, do they derive from open-source ones (and which)?

Thank you!

Michael

[ChithraJ_Intel]

Hi Michael,

Thanks for posting in Intel Community Forums.

Based on our understanding, there is no such Benchmarks available with Intel Advisor. We will discuss this with Subject Matter Experts and let you know the updates. As a work around, we would suggest you to try the Memory-Level Roofline Analysis in Intel® Advisor. Memory-Level Roofline analysis shows interactions between different memory levels(L1,L2,L3 & DRAM) available on your system and provides automatic guidance to improve your application performance.

Please refer to the below documentation for more details:

https://software.intel.com/content/www/xl/es/develop/articles/memory-level-roofline-model-with-advisor.html

Regards,

Chithra

[drMikeT]

Hi Chithra.

The Intel Advisor, uses microbenchmarks measures L1/2/3 and DRAM max BW and max FP/INT ops/sec per core on the system it runs on while it is profiling user codes. It uses these figures to generate the Roofline models/plots to guide the optimization of user codes.

I would like to know and if possible to get these microbenchmarks and run them at the command line so I can automate the profiling and optimization process via scripts. 

Are these microbenchmarks available or Open Source? That would be great for my work.

Thanls

Michael

[ChithraJ_Intel]

Hi Michael,

We are checking your issue with Engineering team and will get back to you soon once we receive an update.

Regards,

Chithra

[drMikeT]

Thanks Chithra

Happy New Year !

Intel thread Advisor reports per core performance limits (L1, L2, L3, DRAM vector/scalar BW and FP/INT vector/scalar add/FMA) limits at the roofs settings in the Roofline tab.

One important question I have is how do I validate that the reported numbers are what the underlying h/w supports? 

Conversely, I would like to generate these figures manually and use them to do my own roofline analysis of codes in an automated fashion as opposed to using the GUI for larger numbers of test cases. We often have to analyze runs with large number of different inputs from parameter ranges. Using the GUI is prohibitive. 

Using the command line advixe-cl is not that much more productive since we cannot easily automate the profiling and evaluation of large number of application runs when we vary its input arguments.

Can we get the codes in Intel Advisor that produce these per core performance limits ? It would assist us to automate performance profiling and tuning.

thanks

Michael

[drMikeT]

Is there any way to dump the per core FP/INT and memory limits into a text file? 

I saw advixe-cl --report=roofs but it does not seem to report everything.

thanks --Michael

[Ruslan_M_Intel]

If you really want to play with benchmark tool you can use 'mem_bench' from bin subdirectory of advisor. On the other hand you could use PythonAPI to get an access to Advisor project. I'm not sure that you will  get all raw data you want (such as unprocessed benchmark results) but you can try to find more information about Advisor PythonAPI here

[drMikeT]

Thanks Ruslan_M !

I have actually noticed the mem_bench and I gave it a few runs guessing the arguments. Does Intel Thread Advisor use mem_bench to obtain the perf statistics for L1, L2, L3 and DRAM BW? 

Is there any documentation or can you provide command lines for running mem_bench for L1, L2 and L3? 

Are mem_bench algorithms related to those in mlc (Intel Memory Latency Checker in https://software.intel.com/content/www/us/en/develop/articles/intelr-memory-latency-checker.html?wapkw=mlc)? 

Any suggestions with how to run either tool (or any other) at the command line for L1, L2 and L3 BW measurements? That would help me a lot to automate things.

Thanks!

Michael

[Ekaterina_G_Intel]

Hello Michael!

mem_bench is old utility to run benchmarks and it requires specific parameters. Intel Advisor is not using it now. 

Intel Advisor benchmarks are not open source. I'm sorry but there is no way to run benchmarks without profiling an application. We can suggest you 

To run benchmarks

1) Each time run roofline collection from command line on light application

advixe-cl -c roofline –project-dir<project_path> -- <application> <application parameters>

2) Run advixe-runtrc tool for project-dir with pre-collected survey result (to not repeat survey collection)

advixe-cl -c survey –project-dir<project_path> -- <application> <application parameters>

advixe-runtrc -r <project_path>/e000/trc000 --benchmarks=dram,mcdram,cpu_cache,flop_peak,int -- <application> <application parameters>

You can stop application after benchmarking pressing “ctrl+c” after the message “advixe: Peak bandwidth measurement finished” was printed.

After that add “e000/trc000” subdir to <project_path> specified on survey.

To print results

1) advixe-cl -r roofs –project-dir<project_path>

   Please let us know what exactly is not printed by -r roofs? It should print all measured benchmarks. 

2) Use Python API script roofs.py, which is available in install_dir/pythonapi/examples

   source advixe-vars.sh

  $ advixe-python install_dir/pythonapi/examples/roofs.py <project_path>

To get more information about Python API https://software.seek.intel.com/LP=18275 

[drMikeT]

Thanks Ekaterina!

I saw 

$ advixe-runtrc --help 

...

-cache-config=<string> Cache Configuration
...
-enable-cache-simulation Model CPU cache behavior on your application

If I specify the -cache-config= parameters while running the benchmarks, will it measure cache BWs as if the cache had that specification (cache size, associativity, etc.)? 

Regards

Michael

[Ekaterina_G_Intel]

No, these parameters are for profiling your application and not benchmarks. Benchmarks are not parameterized using advixe-runtrc as they are automatically adjusted based on current hardware. Maybe you can do it based mem-bench, but it is old version of benchmarks.

As about cache-simulation parameters, Intel Advisor uses this functionality to get application's loop/function traffic for all levels of memory subsytem. Intel Advisor Memory Level Roofline can show you this breakdown. https://software.intel.com/content/www/us/en/develop/articles/memory-level-roofline-model-with-advisor.html 

[drMikeT]

Ekaterina, thanks!

BTW, Intel Advisor is a great and quite useful tool!  I have been involved with computer architectures and performance analysis and I understand the challenges and all the hard work that goes on behind the scenes.

I assume that the numbers of INT/FP scalar/vector arithmetic  and L1, 2, 3 BWs reported per core have been verified? I am mostly interested in the L1, 2 and 3 figures. I am using low assembly level benchmarks (AVX512 and AVX2 vector instructions) but I have never managed to get L1, 2, 3 and DRAM BW as high as that reported. 

Can U offer some insight how the cache benchmarks work ? Are the tests related to those Intel Memory Latency Checker (https://software.intel.com/content/www/us/en/develop/articles/intelr-memory-latency-checker.html)?

Or U can point me to articles explaining the methodology behind the cache benchmarks

thanks!

Michael

[Ekaterina_G_Intel]

Thank you Michael!

I am glad that Intel Advisor was helpful for you. 

CPU cache benchmarks allocate memory that fit into required cache and use highest ISA available on machine. Intel Advisor also supports scalar cache benchmarks. We validate it by comparing with theoretical peaks supported by hardware.

Unfortunately, there is no any specific article for benchmarks methodology.

Hope this helps,

Ekaterina 

[Zakhar_M_Intel1]

Hi Mike,

For your questions regarding benchmarking principles, consider sending email to vector_advisor@intel.com, so we could possibly dive deeper on this sophisticated topic.

Zakhar.

[ChithraJ_Intel]

Hi Michael,

Could you please confirm whether your query is clarified or not? If so, can we discontinue monitoring this thread?

Regards,

Chithra

[ChithraJ_Intel]

Hi Michael,

Thanks for giving kudos to the solution by Engineering team. We hope your issue got clarified. Also, thanks for the valuable feedback about Intel Advisor and great to know that Intel Advisor is helpful tool for you. We would discontinue monitoring this thread. Please feel free to raise a new thread in case of further Issues/queries.

Regards,

Chithra

[Jay14]

Hello michael,
Had you found any way of getting the peak cache bandwidth in automated CLI ?
Thank you
Jay