Use Intel Advisor on Linux cloud and analyze locally on Windows?

[Jones__Brian]

I use Intel VTune Profiler 2020 on Linux in the cloud using the command line, then download the analysis files for local analysis using the VTune Windows GUI.

I would like to do the same thing with Intel Advisor -- run analyses on Linux in the cloud and analyze locally using the Windows GUI. I have been reading the Advisor documentation but I haven't see any information on doing that with Advisor.

How can I use Advisor on Linux in the cloud and analyze the results locally using the Windows GUI, like I do with VTune? Do I need two separate copies of Advisor, one for Linux and the other for Windows?

Thanks.

[Zakhar_M_Intel1 · Accepted solution]

Dear Jones,

In general, it is exactly like for your VTune usage. You need 2 versions of Advisor installed on Windows (for viewing) and Linux (for data  collection in cloud). Advisor only adds one extra feature called "snapshot" (see links below).

On the other hand, if all you need from Advisor is basic Roofline (CPU or GPU) or e.g. "Offload Advisor" data, then you don't need to install Advisor on Windows. Instead you can generate Roofline or Offload Advisor data as self-containing dynamic HTML file; so you generate it via command line in cloud and then copy HTML file to arbitrary device (even cell phone) with HTML browser and view the roofline data here.

Here is an example of self-containing HTML generation flow

advixe-cl --collect=roofline --project-dir=/user/test/vec_project -- <sample_dir>/vec_sample/vec_samples

advixe-cl --report=roofline --project-dir=/user/test/vec_project --report-output=./roofline.html​

This topic is covered in more details at given Advisor "Cookbooks" and articles: 

1) [this is linux to MAC, but you can do the same for Windows, just use Windows version]  https://software.intel.com/content/www/us/en/develop/documentation/advisor-cookbook/top/analyze-performance-remotely-and-visualize-results-on-macos.html

2) MPI-specific, see first part regarding "remote flow": https://software.intel.com/content/www/us/en/develop/articles/analyzing-intel-mpi-applications-using-intel-advisor.html

[Jones__Brian]

I ran the following on Linux in the cloud and downloaded the result folders to my local WIndows machine:

advixe-cl --collect=survey ./My_Program.exe
advixe-cl --collect=map ./My_Program .exe
advixe-cl --collect=dependencies ./My_Program .exe

Then I opened my Windows GUI for Advisor. I clicked the "Open Result" icon and opened the advixeproj file. It correctly shows sub-folders for survey, map and dependencies. When I open any of them, all I see in each of the window panes is:

     No Data
     To collect data about your application's performance, compile your application in Release Build settings and run Survey analysis.

My application is a C program (My_Program.exe) that calls a shared object written in assembly language (NASM). I assume Advisor can work with assembly language programs and shared objects because VTune does, so I don't think that's the problem.

Next to the Application field I click "Browse" and browse to the My_Program.exe, but Advisor says the file "is not an executable binary." Maybe that's because this is Windows and the binary is for Linux.

In your answer above you said this should work basically like VTune, so my question is: how do I view results from a Linux cloud server downloaded to my local Windows machine for analysis with the Windows GUI?

[Zakhar_M_Intel1]

Hi Jones,

What is the wallclock time of your program (how much time it takes to execute, just very roughly speaking)?  Survey analysis is based on TPSS time sampling, so if your hot loops/functions should take at least ~0.01-0.1 seconds, otherwise the data may be not so statistically representative and can partially be abandoned by that reason

Also, for non-NASM part, do you have any time-consuming functions written in C ? Most importantly, did you compile your program (eg specifically C part) with -g to enable debug info in the application binary?

As for your original question – looks like you did it right  (although strictly speaking - for Advisor - it is better to explicitly specify –project-dir option, making sure it is the same across analysis types). Also to figure out what’s going on – it is enough to start from just doing survey analysis and checking if you can open corresponding data on Windows.

If all is good (-g, expect long wallclock in particular in c-part etc), but No Data issue persists – then please share the project-dir you’ve copied to the windows.

Zakhar.

[Jones__Brian]

Hi Zakhar,

The C part is just a wrapper that reads a data file and passes a pointer to NASM. The bulk of the run time is spent in the NASM shared object. Everything is assembled/compiled with debug symbols (I use Clang for C and it's gfull in Clang).

However, the wall clock time may be the issue. I will try it again with a longer run (10x as much input data) and see if that works. If not I'll post the project-dir data here.

[Jones__Brian]

I ran the survey analysis again with 10x more input data than my previous run:

advixe-cl --collect=survey ./My_Program.exe

I downloaded the collection folder (hs002) into a new project folder, but the Windows GUI would not open it as a new project because it uses the same advixeproj file. So I put it in the same project folder as before and opened it. Unfortunately, same problem (no data).

Here is what I got at the end of the run:

advixe: Collection stopped.
advixe: Warning: The application returned a non-zero exit value.
advixe: Opening result 24 % Resolving information for `libtpsstool.so'
advixe: Warning: Cannot locate debugging information for file `/opt/intel/adr_2020.1.0.605410/lib64/libtpsstool.so'.
advixe: Opening result 100 % done
advixe: Preparing frequently used data 0 % done
advixe: Preparing frequently used data 100 % done

Elapsed Time: 5.13s
Total CPU time: 8.51997

I would like to upload the hs002 folder but your site does not allow me to select an entire folder. Are there specific files in that folder that I should upload?

Also, how can I create a new project with a new advixeproj file? Do I need to delete or rename the old one?

It would help if Intel had some documentation about how to view results from a Linux cloud test using the local Windows GUI. I haven't found anything on the Intel site and very little on Stack Overflow. I think a lot of developers would be using my workflow, so it would help us to have some guidance in this area.

[Zakhar_M_Intel1]

Hi,

From provided output it looks like all is good now, the data is here. So the issue is not with profile, but with how you copy it.

As i noted before, With Advisor you have to specify "--project-dir" and then you have to copy project directory from Linux to Windows. For Advisor, project-dir is essential concept and you should think of it mostly as of "atomic" entry.

Therefore you should not individually copy "result dirs" (hs001, hs002, mp001 and so on) and "recreate advisor projects" on Windows after it. This kind of approach is more like "backdoor" and is not recommended. That's probably the main confusion in this mail thread and the main difference between vtune and advisor which confused you.

So, what you should do (most straightforward approach):

1) [on linux] advixe-cl --collect=survey  --project-dir ./my_project_dir  ./My_Program.exe

2) copy my_project_dir folder from Linux to Windows

3) [on windows] advixe-gui ./my_project_dir

And that should be it.

(so you can ignore more info below if you don't have time and interest to dive deeper)

This is documented in various places, in particular in the links i provided to you before. I admit that those articles may focus on more advanced scenarios (like MPI) and various useful details as opposed to explaining very basic ideas at a first place (will try to improve documentation on that).

Let me additionally provide you with more advanced example from the links I shared before. If you look at  https://software.intel.com/content/www/us/en/develop/documentation/advisor-cookbook/top/analyze-performance-remotely-and-visualize-results-on-macos.html  , you can find following info (again, this is more advanced "snapshot" method compared to what i explained above with just copying project-dir), shortened QUOTE:

While on the remote system:

• Run the Roofline analysis from a command prompt: 

  advixe-cl --collect=roofline --project-dir=/user/test/vec_project  -- <sample_dir>/vec_sample/vec_samples
• Optional : If you only need the Roofline report, you can export it as an HTML file with the following command:
  advixe-cl --report=roofline --project-dir=/user/test/vec_project --report-output=./roofline.html

• On the remote system, pack your analysis results into a snapshot with the my_proj_snapshot name:
  advixe-cl --snapshot --project-dir=/user/test/vec_project --pack --cache-sources --cache-binaries -- /tmp/my_proj_snapshot
• Copy  my_proj_snapshot.advixeexpz to the local host.
• Switch to the local system.
• Open the snapshot in the Intel Advisor GUI to view the results:
• From the command prompt:

  advixe-gui my_proj_snapshot.advixeexpz
• From the GUI: Launch the Intel Advisor, go to 

  File > Open > Result , and navigate to the copied snapshot.

Hope it helps.

P.S. i'm not managing this site, but i expect that if you pack all your files into single zip / some tarball - then it will be possible to attach it to the post, although I feel like there is no issue with your project-  it is just trouble wrt how you copy it back and force.

[Jones__Brian]

Thank you for your reply. I followed your directions and downloaded the project directory to my Windows machine. But I can't get advixe-gui to work on Windows. Whether I run it from the C: prompt or the installation folders I get the same message:

C:>advixe-gui ./C:\Test_Data\WL_SC\opt\_Advisor_Projects

"advixe-gui is not recognized as an internal or external command, operable program or batch file."

Then I tried changing to the installation directories:

cd C:\Program Files (x86)\IntelSWTools\Advisor
OR
cd C:\Program Files (x86)\IntelSWTools\Advisor 2020

but in both cases I still get "advixe-gui is not recognized as an internal or external command, operable program or batch file."

I researched this but I haven't found anything to explain why I get this message.  Can I just op;en the GUI and open the project folder from there without using the command line?

[Zakhar_M_Intel1]

Hi! 

>Can I just open the GUI and open the project folder from there without using the command line?

Yes, of course you can! That's another way. So you can find advisor if pressing your "windows" button and typing "advisor", as usual on windows...

As for your issue with command line advixe-gui on windows - you just didn't run so called "vars" script beforehand. So the standard way is:

cd C:\Program Files (x86)\IntelSWTools\Advisor 2020

advixe-vars.bat

advixe-gui <project-dir-to-open>

<Press "Show My result" in GUI>

In more details it is described at the top of corresponding Advisor "Getting Started" , e.g. here: https://software.intel.com/content/www/us/en/develop/documentation/get-started-with-advisor/top/intel-advisor-workflows/discover-where-vectorization-will-pay-off-the-most.html