Line-by-line profiling with Shark (and alternatives?)

[trobitaille]

Hi,

I'm interested in line-by-line profiling of a program compiled with the intel fortran compiler. One option I know of is to use Shark.app on MacOS X. However, if I compile a program with

ifort -g test.f90

and profile it in Shark.app, I can see how much time was spent in each routine, but if I try and view the line-by-line profile, then I can only see the assembler view, and the source view is grayed out (see attached image). Is there a way to get this to work with ifort? Are there extra compilation flags I should be using?

Otherwise, are there any alternatives for Mac or Linux that allow line-by-line profiling?

Thanks in advance for any help,

Thomas Robitaille

[Kevin_D_Intel]

Try adding -save-temps to the ifort command-line. This saves the temporary object files containing necessary symbol information that should enable references back into the source code.

[trobitaille]

Quoting - Kevin Davis (Intel)

Try adding -save-temps to the ifort command-line. This saves the temporary object files containing necessary symbol information that should enable references back into the source code.

Thanks for the suggestion - I tried

ifort -save-temps -g test.f90

which creates an additional test.o file but I still can't get the source view in Shark.app when I profile a.out. Do you have any other suggestions for how I could get this to work?

Thanks,

Thomas

[Kevin_D_Intel]

Ok. I'll conduct some tests myself.

What version of Xcode and Shark are you using?

As for alternatives, there is gprof or Saturn. There are some previous threads discussing those (here) and (here).

[trobitaille]

Quoting - Kevin Davis (Intel)

Ok. I'll conduct some tests myself.

What version of Xcode and Shark are you using?

As for alternatives, there is gprof or Saturn. There are some previous threads discussing those (here) and (here).

Thanks! I'm using Shark.app 4.7.2 (263) and ifort (IFORT) 11.1 20090624 (058).

Saturn and gprof work fine to see how much time was spent in a given subroutine/function but don't give me any more detailed line-by-line information which is I'm looking for. Shark also works fine in that way (i.e. routine-based performance). But it's the line-by-line performance I'm trying to get, and I don't think gprof or Saturn do this - am I wrong?

Thanks,

Thomas

[Kevin_D_Intel]

Line-by-line profiling works with 11.1 Fortran compilers under Mac OS 10.5.7 using Xcode 3.0 and Shark 4.5.0 (156.8).

Your version of Shark 4.7.2 (263) suggests using Xcode 3.2.1 (1613). Line-by-line profiling does not work with 11.1 Fortran compilers under Mac OS 10.6.2 using Xcode 3.2.1/Shark 4.7.2 (263). It also does not work with 11.1 compilers and Xcode 3.0/Shark 4.5.0 (156.8). So, what works under Leopard (10.5.7) does not work under Snow Leopard (10.6.2).

I don't currently have Xcode 3.2/Shark 4.7.2 (263) available on Leopard to test, but I will once available as another data point.

Its unclear what/where the issue is here. I'll inquire w/Development and update when I have more.

Sorry I don't have better news.

[trobitaille]

Quoting - Kevin Davis (Intel)

Line-by-line profiling works with 11.1 Fortran compilers under Mac OS 10.5.7 using Xcode 3.0 and Shark 4.5.0 (156.8).

Your version of Shark 4.7.2 (263) suggests using Xcode 3.2.1 (1613). Line-by-line profiling does not work with 11.1 Fortran compilers under Mac OS 10.6.2 using Xcode 3.2.1/Shark 4.7.2 (263). It also does not work with 11.1 compilers and Xcode 3.0/Shark 4.5.0 (156.8). So, what works under Leopard (10.5.7) does not work under Snow Leopard (10.6.2).

I don't currently have Xcode 3.2/Shark 4.7.2 (263) available on Leopard to test, but I will once available as another data point.

Its unclear what/where the issue is here. I'll inquire w/Development and update when I have more.

Sorry I don't have better news.

Thanks for the info and for looking into this.

Thomas

[Kevin_D_Intel]

From information I received and after some additional testing, here's what I know. The root of the issue appears to be that Shark 4.6 (and later) uses symbols from the symbol table and not the Dwarf emissions. There is an open Radar (#6535772) requesting that Shark to use the Dwarf instead but there's no time estimate for this.

Routine names in the symbol table are decorated with a trailing underscore by default in Fortran, but not in the Dwarf. Xcode 3.2/Shark 4.7.2 (263) under Snow Leopard works if the symbol names are tweaked slightly.

One quick method you can try is compiling with the ifort option: -assume nounderscore

This option drops the trailing underscore which seems to be enough for Shark to locate the corresponding source lines. One caution is this could lead to unresolved externals.

I also learned about another Apple Developer tool name Instruments. I found this provides line-by-line profiling without tweaking the symbol names, but I have been unable to figure out how to display the corresponding assembly for Fortran source that it shows with timing information.

Please let us know if either of these help.

[trobitaille]

Quoting - Kevin Davis (Intel)

From information I received and after some additional testing, here's what I know. The root of the issue appears to be that Shark 4.6 (and later) uses symbols from the symbol table and not the Dwarf emissions. There is an open Radar (#6535772) requesting that Shark to use the Dwarf instead but there's no time estimate for this.

Routine names in the symbol table are decorated with a trailing underscore by default in Fortran, but not in the Dwarf. Xcode 3.2/Shark 4.7.2 (263) under Snow Leopard works if the symbol names are tweaked slightly.

One quick method you can try is compiling with the ifort option: -assume nounderscore

This option drops the trailing underscore which seems to be enough for Shark to locate the corresponding source lines. One caution is this could lead to unresolved externals.

I also learned about another Apple Developer tool name Instruments. I found this provides line-by-line profiling without tweaking the symbol names, but I have been unable to figure out how to display the corresponding assembly for Fortran source that it shows with timing information.

Please let us know if either of these help.

Thank you very much for looking into this and for your suggestions. I first tried using

ifort -O0 -g -assume nounderscore test.f90

and

ifort -O0 -g -assume nounderscore -save-temps test.f90

but I still can't get the source view in Shark.

I tried using Instruments.app, and I can use the Sampler and Time Profiler instruments to see a call tree of routines, but I can't figure out how to get the source view you mention - how did you do this?

Thanks,

Thomas

[Kevin_D_Intel]

I was engrossed in so much hacking to remove underscores that maybe I fat fingered some of my testing w/Shark. I will double check my testing and offer another hack I know works in another reply if need be.

To view source, under the call tree view, double-click on one of the routines from your test.f90 and the source should load and show timings. You can then toggle between views using the controls on the bottom of the screen if needed.

[Kevin_D_Intel]

I doubled check my testing and verified compiling with -g -save-temps -assume nounderscore enables source-line viewing under Shark. I was using 11.1.058 earlier (which also worked), but I'm now using 11.1080.

$ ifort -V -g -save-temps test2.f90 -assume nounderscore
Intel Fortran Intel 64 Compiler Professional for applications running on Intel 64, Version 11.1 Build 20091130 Package ID: m_cprof_p_11.1.080

Before using this nounderscore option, I noticed the Source button was inactive, but it becomes active when compiling with the option.

I just also noticed from your original post that your hot spot is MAIN__. I'm profiling a sample program with a main, sub, and sub2. Myhot spotis sub. I restructured my program so MAIN__ was the hot spot and source-line viewisnot active. The nounderscore option does not change the name of MAIN__. Try restructuring your program to create a subroutine containing the guts of the program instead and compile with the options I indicated.

[trobitaille]

Quoting - Kevin Davis (Intel)

I doubled check my testing and verified compiling with -g -save-temps -assume nounderscore enables source-line viewing under Shark. I was using 11.1.058 earlier (which also worked), but I'm now using 11.1080.

$ ifort -V -g -save-temps test2.f90 -assume nounderscore
Intel Fortran Intel 64 Compiler Professional for applications running on Intel 64, Version 11.1 Build 20091130 Package ID: m_cprof_p_11.1.080

Before using this nounderscore option, I noticed the Source button was inactive, but it becomes active when compiling with the option.

I just also noticed from your original post that your hot spot is MAIN__. I'm profiling a sample program with a main, sub, and sub2. Myhot spotis sub. I restructured my program so MAIN__ was the hot spot and source-line viewisnot active. The nounderscore option does not change the name of MAIN__. Try restructuring your program to create a subroutine containing the guts of the program instead and compile with the options I indicated.

Hi Kevin,

Thanks very much for your help - the problem before was indeed that my test program only had a main, which does not work, but moving the test code to a subroutine allows me to view the source code.

One final question: if I have source code that is in a directory, Shark.app has trouble finding the source code. For example, if I put the test.f90 code inside a directory called src/, and have a Makefile outside that directory that looks like this:

FC = ifort -O0 -V -g -save-temps -assume nounderscore

OBJECTS = src/test.f90

b.out: $(OBJECTS)
$(FC) $(OBJECTS) -o b.out

Then if I try profiling b.out, Shark.app says:

Unable to read source file "test.f90" from "src".
Referenced from symbol "sub1" in library "b.out".

I can select the file, but I'm wondering if this is an issue with ifort or with Shark.app?

Thanks again for your help,

Thomas

[Kevin_D_Intel]

Seems like an issue with Shark. The path is clearly encoded in the Dwarf, which again Shark doesn't appear to use. Debuggers which use the Dwarf do not have problem finding it. Doing this also seems to trip up Instruments.

I post again if I learn more.