Profiling workloads within toil

[kaplannp]

Hello,

I'd like to use vtune hotspots to profile an application from the command line. The application uses toil internally , and I'd like to get profiling results for all the jobs within the toil pipeline. Is this possible using vtune? When I try the obvious `vtune -collect hotspots ./myapp` the application hangs.

Any leads or pointers to documentation would be greatly appreciated!

Thanks,

Noah

[yuzhang3_intel · Accepted solution]

Try using hardware event-based sampling mode instead.

vtune -collect hotspots -knob sampling-mode=hw ./myapp

[kaplannp]

Thanks for the help!

This works for some inputs to my application, but on other inputs it freezes and produces
"kernel:[ 1948.765550] watchdog: BUG: soft lockup - CPU#30 stuck for 52s! [_toil_worker:8611]"

-----------EDIT---------------

I've fixed the issue. It was related to the -knob enable-stack-collection=true flag that I was passing to vtune, which caused the machine to run out of memory.

Accepting the answer as this solved my first question.

[yuzhang3_intel]

You can specify the the option '-data-limit' to limit the amount of raw data (in MB) collection.

vtune -collect hotspots -data-limit=200 myApp

For details, please refer to the User Guide below:

https://www.intel.com/content/www/us/en/docs/vtune-profiler/user-guide/2024-2/data-limit.html

[kaplannp]

Thanks!