Running finalize on different host from profiling

[Gamsa__Ben]

Hi.  I need to run some code to be profiled on a special set of machines, but because they are a limited resource, I would like to do the finalization on a different machine.  All of the machines have essentially the same set up (same OS, same file systems mounted, same libraries).  However, when I try to finalize on a different host vtune detects that the host is different and doesn't automatically find the libraries and debug information, even though all of the information is in exactly the same place.  The application is big and complicated, with hundreds of libraries and code spread over a large directory tree; it doesn't seem like I can just specify a top-level directory and have the tool find everything.  Is there a way I can tell vtune to behave the same as if I was running finalize on the same host?

Thanks,

Ben

[Denis_M_Intel]

Hi Ben,

 If the machine where you finalize VTune result has the same directory tree and binaries, specifying a top-level directory via -search-dir option might help.
And you may want to try the deferred finalization to collect checksums of binary files that will help to validate binary files on another machine.

[Gamsa__Ben]

Thanks.  I tried various options for search-dir, like using just "/" or the top of the project tree where all of the files are, but it didn't seem to help it find things.  I also tried the deferred finalization option instead of none, but with that option although it didn't complain about finalization being run on the wrong host it also didn't seem to find any symbols or other information.  Perhaps I'm using it incorrectly, but there were no error messages.  I ran it on the command line and then started the gui, but although it considered it properly finalized it didn't seem to have information about hotspots.

[Denis_M_Intel]

 If a binary is /xxx/dir1/dir2/dir3/my_binary on the target system and its path on the different machine is /yyy/zzz/dir1/dir2/dir3/my_binary, "/yyy/zzz" (or any of 'yyy/zzz/dir1', 'yyy/zzz/dir1/dir2', 'yyy/zzz/dir1/dir2/dir3') search directory should work: vtune -finalize -r <result_path> -search-dir=/yyy/zzz.

Are there any warnings about your modules of interest in VTune finalization output?

[Gamsa__Ben]

So, it looks like maybe the deferred option actually works. I think the problem I had with my testing is that I was testing a small portion of a small test case, and sometimes I think it didn’t get any samples, and so appeared not to work. I retested on a larger chunk of code and using deferred finalization mode on one machine and then finalizing on a different machine worked, without any search-dir option. It seems that the checksums allow it to trust that the files it finds are the right ones. When I retested with option “none” it kept reporting libraries weren’t found during finalization even though it had the right path. I’m not entirely sure what was going on but it seems like deferred mode is what I’m looking for. 
 

Thanks

[Gamsa__Ben]

Now I'm really confused.  When I tried the test on a larger design, even with the data collection size reduced to 500MB (default seems to be 1GB) and with deferred finalization mode enabled, it still took about 10 hours after the program finished for it to finish processing the data.  I thought in deferred mode it was just computing checksums on libraries and binaries, so I'm not sure what it is doing.  I did double-check and vtune does not think it's been finalized when I used deferred mode and still thinks the finalization step is required, so it doesn't look like it was doing actual finalization.  The run-time seems to be related to the size of the data collected (it even says that it will take a long time to finish profiling once the program terminates when the data collected is the full 500MB) since my small test run and my full run are mostly the same except for the data the program is processing, yet the time to finish up after profiling (in deferred mode, without finalization) goes from a couple of minutes to 10 hours.

[AthiraM_Intel]

Hi,

We are forwarding this case to concerned team.

Thanks

[James_T_Intel]

Is this related to your other thread at https://software.intel.com/en-us/forums/vtune/topic/852506?

[Gamsa__Ben]

Yes, the two topics are related.  Because finalization takes a significant amount of time, I would like to run it on a different host to avoid wasting time on scarce resources.  If finalization were a small fraction of the main run-time, then I guess I wouldn't really care about running it on a different host, but in general both topics are of interest to me.

[JananiC_Intel]

Hi,

Since your issue is addressed in the other thread(https://community.intel.com/t5/Analyzers/Speeding-up-finalize/m-p/1175516#M18299),can we close the case?

Regards,

Janani Chandran

[JananiC_Intel]

Hi,

Since your issue is addressed in the other thread we are closing this case. If you need any additional information, please submit a new question as this thread will no longer be monitored.

Regards,

Janani Chandran