Intel® Fortran Compiler
Build applications that can scale for the future with optimized code designed for Intel® Xeon® and compatible processors.
Announcements
The Intel sign-in experience is changing in February to support enhanced security controls. If you sign in, click here for more information.

Memory instrumentation

nDS
Novice
839 Views

I am just wondering if the fortran compiler+RTL has some associated tool (other than inspector) that would allow one to instrument a debug executable to report memory usage with some user-defined level of specificity.  I am trying to track down some leaks, and none of (totalview, valgrind, inspector) seems to be indicating precisely what is going on . . .  Thanks.

 

0 Kudos
1 Solution
Ron_Green
Moderator
509 Views

This is going to be more difficult than you think.  Fortran uses a memory manager  in the Fortran Runtime Library (FRTL) of which most tools, written for C or C++, are not aware.  Valgrind for example.  These will flag a lot of 'false positives'.  

 

The Intel Inspector tool is free, and does leak analysis.  It's "ok" but in my experience I had to wade through a lot of false positives though it's far better than Valgrind for analyzing Fortran.

Intel Vtune I think also track 'high water mark' in a memory analysis feature so you can see memory usage in a timeline and relate that back to your code.

Look for both of those tools and kick the tires.

 

Linux gets in the way as well.  It uses a lazy algorithm and doesn't allocate physical pages until you touch them.  So you could allocate a huge array and as you step through it, it appears memory usage keeps going up and up.  Or you allocate a bunch of arrays and use them in different places in the code.  And in Fortran you have automatic variables.  Also, Linux can also keep pages around after freeing them, on the chance you may want another allocation of the same size in the future.  Then if it's an MPI code you have comm buffers created and destroyed.  Or IO buffers for any code.  NFS caching, parallel file system caching buffers, etc.  It's overwhelming. 

 

So I don't think you will find what you are looking for - a real time view memory pages that are actually in use by your core application (not it's IO subsystem or MPI buffering).  It's frustrating, about the only thing you do know is when you blow out of VM and the process crashes hard. 

 

IN GENERAL Fortran is far superior to C/C++ wrt memory leaks.  You really have to go out of your way to create a memory leak in Fortran.  The best way is to call C/C++ and do memory management in both languages OR do the memory management in C outside of fortran allocate/deallocate.  My money is that it's not a leak in the Fortran code, though I have seen a handful of cases over 20 years or so.  Far more likely is that your application is simply allocating too much data OR you have a lot of automatics or array temps created.  Have you tried option -check arg-temp-created?   If your code is legacy with EXTERNAL procedures and large arrays, the temp arrays are probably the culprit.  If so, try -heap-arrays option. 

 

 

View solution in original post

9 Replies
IanH
Black Belt
719 Views

Which platform are you on?  On Windows, the debug C runtime library has functions that allow you to self-instrument your program.  These can be called from Fortran.  Generally the various tools are a more efficient approach though.

nDS
Novice
638 Views

@IanH: Can you point me to a reference to said self-instrumentation (assuming that it can work on linux)?  Thanks.

IanH
Black Belt
624 Views

What I have used is specific to the Windows OS and Microsoft C++ runtime on that OS.  I expect Linux libraries have similar capability though.

nDS
Novice
713 Views

I am on linux, using Intel 2022.

jimdempseyatthecove
Black Belt
591 Views

Does your program allocate to pointers? If so, then memory leak detection will be difficult as allocation to an allocated pointer and/or pointer going out of scope has (had) no requirement to deallocate first. Use of allocatable in lieu of pointer handles this.

Jim Dempsey

nDS
Novice
540 Views

@jimdempseyatthecove: For sure, we use pointers liberally throughout (we also use allocatables when reasonable) . . .  I guess what you are implying is that the native intel tools referred to by IanH are most useful within subroutines, but not across them?  If that's true, then yes, they likely wouldn't be too helpful to us.  Please verify, but either way, thanks for the help.

Ron_Green
Moderator
510 Views

This is going to be more difficult than you think.  Fortran uses a memory manager  in the Fortran Runtime Library (FRTL) of which most tools, written for C or C++, are not aware.  Valgrind for example.  These will flag a lot of 'false positives'.  

 

The Intel Inspector tool is free, and does leak analysis.  It's "ok" but in my experience I had to wade through a lot of false positives though it's far better than Valgrind for analyzing Fortran.

Intel Vtune I think also track 'high water mark' in a memory analysis feature so you can see memory usage in a timeline and relate that back to your code.

Look for both of those tools and kick the tires.

 

Linux gets in the way as well.  It uses a lazy algorithm and doesn't allocate physical pages until you touch them.  So you could allocate a huge array and as you step through it, it appears memory usage keeps going up and up.  Or you allocate a bunch of arrays and use them in different places in the code.  And in Fortran you have automatic variables.  Also, Linux can also keep pages around after freeing them, on the chance you may want another allocation of the same size in the future.  Then if it's an MPI code you have comm buffers created and destroyed.  Or IO buffers for any code.  NFS caching, parallel file system caching buffers, etc.  It's overwhelming. 

 

So I don't think you will find what you are looking for - a real time view memory pages that are actually in use by your core application (not it's IO subsystem or MPI buffering).  It's frustrating, about the only thing you do know is when you blow out of VM and the process crashes hard. 

 

IN GENERAL Fortran is far superior to C/C++ wrt memory leaks.  You really have to go out of your way to create a memory leak in Fortran.  The best way is to call C/C++ and do memory management in both languages OR do the memory management in C outside of fortran allocate/deallocate.  My money is that it's not a leak in the Fortran code, though I have seen a handful of cases over 20 years or so.  Far more likely is that your application is simply allocating too much data OR you have a lot of automatics or array temps created.  Have you tried option -check arg-temp-created?   If your code is legacy with EXTERNAL procedures and large arrays, the temp arrays are probably the culprit.  If so, try -heap-arrays option. 

 

 

nDS
Novice
436 Views

 

@Ron_Green: Oh, I know just how hard it will be . . .  I have experienced nearly everything you mentioned.  That being said, we just updated to the latest version of the compiler suite, and all of the tools are working slightly less good than they did with intel 17.  That, in combination with the fact that we are trying to find some subtle leaks in a 450k line code with a fair number of libraries linked in, led me to reach out.

 

Thanks for the suggestions re: compiler options.  We will give those a shot.  <thumbs up>

 

Ron_Green
Moderator
393 Views

Here is the Intel Inspector Getting Started, with links to all over tutorials and User Guide.  To download the tool the easy way GO HERE for the stand-alone component installer.  No license needed, it's free. 

 

I'd recommend finding an input deck/case/simulation that runs with 1 process for less than 10 minutes, preferably under 5 minutes.  Memory and Threading analysis runs are super slow:  Threading Analysis is like 200x slower - not percentage but 200 times slower.  Memory analysis is not as bad, maybe single-digit X slower.  

Reply