IVF 10.0 Static Verifier

[Les_Neilson]

(originally posted by mistake in the Linux forum)

From the post by Ron Green (pp Steve Lionel) announcing IVF 10

"- Static Verifier builds upon the compiler's interprocedural analysis capability to provide whole-program detection of errors including routine mismatches, variable misuse, OpenMP directive errors and more"

One of my colleagues is currently looking at the new IVF 10 release.

The phrase that caught our eye was "whole program detection of errors". As far as we can tell (in our limited use of only started using it today) it is actually limited to checking code within the same directory. We thought from the above statement it would be able tocheck across libraries too - like some sort of pseudo-link to check argument mismatches when code in one directory (library) calls code in another directory (library).

Have I got it wrong ? (it wouldn't be the first time :-) )

Thanks

Les

[Steven_L_Intel1]

It is not limited to "same directory", but any libraries you build with need to also have been compiled with the static verifier enabled so that the intermediate language is in the library.

To effectively use SV, all the code you want to check needs to be compiled with SV enabled. SV cannot check calls to routines in regular libraries. Of course, if you were using modules to define the interfaces to these libraries, a normal compilation would find most errors.

[Les_Neilson]

Hi Steve hope you had a good holiday.

Mmm. We have a small test project :a directory with a main program and a subroutine (console1.f90 and source1.f90) and another directory with one file convlib.f90 (multiple subroutines)

console1 calls one routine from convlib.f90 and calls source1

convlib.f90 compiles with 0 errors and one warning. But no lib file is created. So does a lib only get created if there are 0 errors and warnings?

From the build.log file

------ Build started: Project: Lib1, Configuration: StaticVerifier|Win32 ------

Compiling with Intel Fortran Compiler 10.0.025 [IA-32]...
ifort /nologo /Zi /Od /gen-interfaces /Qdiag-enable:sv3 /Qdiag-enable:sv-include

/warn:interfaces /module:"StaticVerifier" /object:"StaticVerifier" 

/traceback /check:bounds /libs:static /threads /dbglibs /c /Qvc8 

/Qlocation,link,"C:Program FilesMicrosoft Visual Studio 8VCin" 

"C:Documents and SettingsStrucadMy DocumentsVisual Studio 2005

ProjectsLib1convlib.f90"

Performing static verification...

ifort /Qdiag-enable:sv3 /nologo /Qdiag-enable:sv-include "StaticVerifierconvlib.obj"
C:Documents and SettingsStrucadMy DocumentsVisual Studio 2005ProjectsLib1convlib.f90(20)

: warning #12137: [SV] undeclared procedure "emit_diagnostic"


Lib1 - 0 error(s), 1 warning(s)

? "emit_diagnostic" ? It seems to be objecting to the statement
ich = ichar(string(i:i))
in the following code:

subroutine locase(string)
implicit none
! Converts any upper case chars to lower case chars
! M. Sanguineti MAY 1989
! Arguments
character(*) :: STRING
intent (inout) STRING
! Local variables
integer*4 :: i, ich, leng
! Convert each character
leng = len(string)
do i = 1, leng
ich = ichar(string(i:i))
if ((ich>=65).and.(ich<=90)) then
ich = ich + 32
string(i:i) = char(ich)
end if
end do
end subroutine locase
Thanks
Les

[Steven_L_Intel1]

That's a compiler bug. SV isn't supposed to complain about compiler-generated library calls, but it sometimes does. Please submit a report about it so that we can fix it.

[Les_Neilson]

OK Will do.

Les

[Les_Neilson]

Steve,

from the V10 Release Notes :

"In this release, when Static Verifier is enabled the linker is not invoked so an executable or static/dynamic link library is not produced ..."

I understand about the obj filesproduced under SV notbeing "normal" obj files, and soany SVfilesproduced should not be linked with non-SVfiles etc.but the Release Notes imply thata lib is notproduced.

My colleague simplified his project to one directory with one subroutine which did some simple math addition (to avoid any further problems with intrinsicsand such like). The main program and the subroutine have a simple argument mismatch we were hoping the SV wouldfind.The SV build compiled the subroutine with no errors or warnings, produced the "mod" files, but no lib, which meant that the main program SV project had nothing to link to.

Our code having "growed like Topsy" over the years in several offices (now fortunately consolidated into one development office) has a problem of subroutines with the same name but different signatures in different libraries so we are particularly interestedin cross-library argument mismatches. My perl scripts have helpedidentify many of them (but not mismatchesbetween Fortran andC/C++ routines asimplied is available with SV), andso it would be nice to have another tool to help. Moving to modules would also help of coursebut I don't have the authority to do that just yet, as we're too close to the next release of our software to make major code changes. But we are moving in that direction.

Les

[Steven_L_Intel1]

I'll admit that I have not tried using SV with libraries. I can imagine that it might not work, unless you created the .lib manually. I'll have to play with this and see how it might be improved.

[Les_Neilson]

Hi Steve,

Any progress on this?

Les

[jimdempseyatthecove]

Les,

This is a work around suggestion.

Seeing that SV fails to produce functioning .obj/.exe files and your only interest is in performing a once only verification of several libraries, try the following.

1) Copy the source files to a seperate computer maintaining relative placement (same directory names on seperate system).

2) Test compile without SV to verify files and projects copied correctly.

3) In VS IDE, in the project browse window, for each library project, select each source file and drag and drop it into the main program project. I do not know if you can select and drag multiple files.

4) Remove the empty library projects and dependencies.

5) Compile with SV

Wiedersehen

Jim Dempsey

[Les_Neilson]

Jim

If only it were that simple. :-(

Our test (basicallyan evaluation of the Static Verifier inv10) is limited to one simple exe and one lib. However our whole system consists of over 150 exes, 250 dll's and some 300 or morelibs. *whole program error detection* would be really useful. :-)

Les

[Intel_C_Intel]

I wanted to use it too; tried it but ran out of memory checking only critical errors:

Performing static verification...

Fatal compilation error: Out of memory asking for 16392.

ifort: error #10014: problem during multi-file optimization compilation (code 1)

Machine has 4 gig real mem, running vista 64-bit. Task manager shows mem real mem usage climbed to max, page file use climbed to 3330 MB before it failed. Pagefile much larger than that.

I have only one .exe, no DLLs, two modules, and about 70 fortran routines. When asking for all errors, failing requst is over 200000. Configuration set to no debugging, no optimization.

Hopefully this will get better in future release.

[Steven_L_Intel1]

Yes, this is something I run into a lot with medium and larger sized applications. Please report this problem to Intel Premier Support so that the proper attention can be focused.

[jimdempseyatthecove]

Les,

:) it is still that simple (:

Repetative, but simple.

You do not have a whole program problem, rather you have 150 whole program problems. I am a little concerned about the other message regarding memory overflow on a 4GB x64 system with something simpler than your situation.

I do not use SV so I cannot say how it will deal with DLLs.

I think the first prepratory step would be to require interfaces. And make sure an interface declaration exists in only one file. But this may not be possible inside of a 3rd party lib or dll. For those you just have to hope the documented interfaces are correct.

This sounds like a job for a new hire. It will get them familliar with the application.

Jim Dempsey

[jackosullivan]

Steve,

I was able to get a successful (and quite embarassing -- CVF was a LOT more forgiving!) Static Verification done using IVF 10.0.26 a month ago.

But now, with anidentical set of source files, and identical options, all I get after the rebuild compiles is the "Fatal compilation error: Out of memory asking 2097160." It's a single-exe, quickwin, no-DLL program with 39 modules, 66 .for routines, 2 .f90 routines,that occupies only 48MB Virtual with a 9MB Working Set. Using a machine with 1.5GB RAM, and Windows XP SP2.

The only difference between the two runs (except for some minor code changes) seems to be the sequence in which the sources are allrecompiled andthus the sequence in which the verification command lists the various .obj files.

I couldn't locate how to control thesequence of compilation from within VS.NET 2003.

1) Is there a way to do so?

2) Would/should it make any difference?

Thanks and God bless!

Jack

[Steven_L_Intel1]

Jack,

There is not a way to control the compile order. The biggest problem I have seen with SV is its memory demand. You might try increasing the size of your pagefile but no guarantees.

I'm a bit puzzled by your initial comment. It is true that the ifort compiler catches some errors that CVF didn't, but Static Verifier is an entirely different level of analysis that goes far beyond what CVF could do. What sort of things did it pick up on?

[jackosullivan]

Steve,

Thanks so muchfor your usualvirtually instant attention to my question .. dunno how you do it!

Your question about what new things did SV catch means it is time for confession -- maybe with a capital C. And for "Penance", I'll probably have to light a lot more of those votive candles.

If even only a few percent of these are valid "complaints" by SV, you can easily see why I felt it was "quite embarassing". The only consolation I have is that this code has literally been growing now for 40 years, and there are LOTS of non-osullivan "sins" tangled up in it.

The following is a tally of the errors and warnings I got a month ago before I started "running out of memory":

	Error/Warning
Count	Number	Typical message
181	12144	[SV] "B82X27" is possibly uninitialized
113	12178	[SV] this value of "MUNITS_mp_SI" isn't used in the progr am
111	12049	[SV] index of "XTIME" may be outside the bounds
85	12048	[SV] index of "COMM2_mp_HP" is outside the bounds
74	12137	[SV] undeclared procedure "emit_diagnostic"
23	12020	[SV] wrong number of arguments
19	12040	[SV] argument 4 doesn't conform to dummy argument. See (fileC:_99DevelopmentsourcesFind.for line10)
10	12145	[SV] function "UCASE" is called as subroutine
10	12192	[SV] unreachable statement
8	12255	[SV] index may be outside the bounds for actual "YLAB". See (fileC:_99DevelopmentsourcesPplt.for line102)
7	12030	[SV] size of argument 1 is less than size of dummy argument. See (fileC:_99DevelopmentsourcesPlotr.for line6)
6	12148	[SV] actual intrinsic "for_inquire" is called as subroutine
3	12077	[SV] function result is possibly not set
3	12143	[SV] "IFPLOT" is uninitialized
3	12224	[SV] Buffer overflow
2	12007	[SV] "YMAX" is possibly incorrectly modified in "PLOTR" (arguments 3 and 5)
2	12115	[SV] common block "/BLNKC/" declared only in one program unit
2	12253	[SV] wrong number of arguments for actual intrinsic "eof"
1	12019	[SV] argument 6 is possibly modified in "ADDLEGEND". See (fileC:_99Developmentsourcesaddlgnd.for line24)
1	12174	[SV] procedure "ATMREAD" never returns
664

I'm sure the vast majority of these are caused by the lack of interfaces in most of the code -- something that was hardly even thought of back in the "good ol' days".

You can see why I am anxious to get SV working properly.

Back to it n ow. Maybe those votive candles will get a larger pagefile to do the job.

Thanks again, and God bless!

Jack

[Steven_L_Intel1]

Thanks. A couple of those errors, the ones about for_inquire and emit_diagnostic, are not your errors, but the others look plausible. If you discover that the diagnostics are wrong, please let us know at Intel Premier Support and supply a test case.

[jackosullivan]

Steve,

I've redone the "Static Verification" after expanding my pagefile to 4080MB, and it still bombs after a silent 15 minutes or so with the "Fatal compilation error: Out of memory asking for 2097160" -- the same exact"asking for" as when I had "only" a 2GB pagefile with my 1.5GB RAM. Is this something I should be working at with Premier Support?

Or is there some switch to throw that makes the SV keep a log or provide details as it goes?

Thanks and God bless!

Jack

[Steven_L_Intel1]

A 4GB pagefile isn't going to help with this - the maximum virtual address space on 32-bit is 2GB. (And even if you were using x64, the compiler itself is still a 32-bit app.)

Please do report this to Intel Premier Support and attach your project. It will help the SV team trim the memory usage of SV.

[jackosullivan]

Thanks again, Steve. I'll get this over to Premier Support as soon as I can.

I completely missed that 2GB limit thing. This is a lot deeper into programming than this ol' hydraulic engineer ever wanted to get. After spending so many years trying to keep "my" program from exceeding 640KB memory usage so we wouldn't have to pay premium rates on the old Univac mainframe, I had just treated Windows as "infinitely" capable. Live 'n' learn.

Again, thanks so much for your dedication, and attention to and care of the fortran community.

God bless!

Jack