topic Profiling workloads within toil in Analyzers

Profiling workloads within toil

kaplannp — Tue, 13 Aug 2024 22:55:13 GMT

Hello,

I'd like to use vtune hotspots to profile an application from the command line. The application uses toil internally , and I'd like to get profiling results for all the jobs within the toil pipeline. Is this possible using vtune? When I try the obvious `vtune -collect hotspots ./myapp` the application hangs.

Any leads or pointers to documentation would be greatly appreciated!

Thanks,

Noah

Re: Profiling workloads within toil

yuzhang3_intel — Wed, 14 Aug 2024 01:15:08 GMT

Try using hardware event-based sampling mode instead.

vtune -collect hotspots -knob sampling-mode=hw ./myapp

Re: Profiling workloads within toil

kaplannp — Wed, 14 Aug 2024 22:39:07 GMT

Thanks for the help!

This works for some inputs to my application, but on other inputs it freezes and produces
"kernel:[ 1948.765550] watchdog: BUG: soft lockup - CPU#30 stuck for 52s! [_toil_worker:8611]"

-----------EDIT---------------

I've fixed the issue. It was related to the -knob enable-stack-collection=true flag that I was passing to vtune, which caused the machine to run out of memory.

Accepting the answer as this solved my first question.

Re: Profiling workloads within toil

yuzhang3_intel — Thu, 15 Aug 2024 00:37:05 GMT

You can specify the the option '-data-limit' to limit the amount of raw data (in MB) collection.

vtune -collect hotspots -data-limit=200 myApp

For details, please refer to the User Guide below:

https://www.intel.com/content/www/us/en/docs/vtune-profiler/user-guide/2024-2/data-limit.html

Re: Profiling workloads within toil

kaplannp — Thu, 15 Aug 2024 12:43:53 GMT

Thanks!