Can we get callgraph for Linux device drivers (.ko modules)?

[mfcking]

I was told I can only get callgraph for Linux applications. VTune can not generate callgraph for Linux kernel modules, e.g. network card drivers? Is that true?

If ture, has anyone ever used Linux Tool Trace kit (LTT) to get the callgraph for the Linux drivers? (maybe this is not relevant to VTune)

Message Edited by mfcking@yahoo.com on 06-14-2005 05:07 PM

Message Edited by mfcking@yahoo.com on 06-14-2005 05:09 PM

[jeffrey-gallagher]

First, you can get callgraph data on kernel modules depending on when they come in to use, but, you can NEVER get callgraph data on the kernel itself.

You may or may not know that callgraph works by performing a binary instrumentation on the code you want to profile.

That means when you launch a VTune callgraph session, VTune inserts its profiling codes into the method prologs of your app. Then, VTune runs that instrumented binary to obtain the graph of the flow of control for that app.

To illustrate this, imagine that yourun a simple sampling session, such as:

$ vtl activity -c callgraph -app "/bin/ls" -moi "/bin/ls" run

and then snooped around in the VTune working directories during this experiment (which probably won't run too long), you'd find a new binary called ../bin__ls which is the binary that callgraph created, instrumented,and ran to figure out the /bin/ls flow of control and report it back to you.

If your kernel driver can be loaded up after boot, call graph should have no problems examining it. If however your kernel driver MUST come into play before you're in multiuser mode on the server, then callgraph won't be able to work for you.

You should still be able to sample, however: anything that runs on the processor can be seen via a VTune sampling session, since sampling doesn't require any instrumentation to locate CPU events.

Earlier I mentioned you can't run callgraph on the kernel, and if you stop and think about it that's because you can't boot a standard Linux kernel, instrument a second one, and then BOOT that second one at the same time.

On a closing note, there are some VTune pause and resume APIs that come with the VTune product, that you can invoke in your own code to obtain sampling and callgraph data. BUT, VTune has to be running for this to work, so, until you're in multiuser mode, that won't be of much help either.

I hope this helps!

cheers

jdg

[mfcking]

Hi, Thanks a lot for your very detailed reply. I will try it. I do think what you said help me understand VTune a lot. Thanks again. L.Y.

[Boaz_T_Intel]

I'd like to correct Jeff's answer. VTune cal graph does NOT work on the Linux kernel or on loadable kernel modules, no matter when this are loaded.

Boaz.

[mfcking]

Hi,

Yeah, what you said is true.

BTW, I found VTune can not samplethe ksoftirqd, which is a low priority kernel thread. I could not find any sampling data about this ksoftirqd even though i know it is indeed running.

Do you know why?

[David_A_Intel1]

You must remember that the VTune analyzer uses statistical sampling and just because no samples where collected in that module does not mean it wasn't running or that the analyzer can't sample it. What it does mean is that there was not enough activity in that module's code for a sample to be collected while it was executing.

You need to either ensure that there is significant activity within the modules code or increase the sample rate - the caveat being that if you increase the rate too much you will cause the system to be unresponsible (i.e., to "hang") and most likely have to reboot. :-(