The numbers of blocks in coverage reports are different in different test frameworks

[Yanshuang]

We are using Intel Fortran Code Coverage Tool to make coverage reports with three test frameworks for same Fortran resource. Now we are having a problem. In the coverage reports, the number of the total blocks is varied for each test framework, but the total amount of files and functions are the same in three reports. The same executable binaries were used in all the run.

As an example, here is the overall coverage report for Ruby unit test framework and NUnit test framework 

 Here is an example. A simple Fortran file has different total blocks in the different tests.

We got similar different total blocks result in the third test set.  

The same compilation and spi file are used for the coverage execution. The operating system is Windows 10 and the compiler version is:  

Intel(R) Visual Fortran Intel(R) 64 Compiler for applications running on Intel(R) 64, Version 19.1.3.311 Build 20201010_000000 

We follow the 4 steps from the code coverage tool instruction https://www.intel.com/content/www/us/en/develop/documentation/fortran-compiler-oneapi-dev-guide-and-reference/top/optimization-and-programming/tools/pgo-tools/code-coverage-tool.html

The compilation is done with with /Qcov--gen:srcpos.

The options we used for the coverage is

codecov -prj RubyTestCoverage -dpi c:\coveragefolder\pgopti.dpi -counts -ccolor #90DBEE -pcolor #B690EE -showdirnames

Why are the numbers of blocks different among variant test frameworks? 

[jimdempseyatthecove]

I haven't used the code coverage tool... but if I were to guess, I suspect the inter-procedural optimization performing inlining of functions and subroutines is affecting the code blocks counts.

As a test, in Release mode, turn off ipo (both single file and inter-file). Then rebuild and then re-run the code coverage tool.

The performance of the app may be reduce a bit, but the information could be what you are looking for. 

Jim Dempsey

EDIT: Also, assure that for Release Build (or whatever configuration you are using), to enable the emmission of Debug information, as well as in Link phase to keep debug symbols. 

[Ron_Green]

I will check on support for Code Coverage tool in ifort.  I know we don't have this in IFX or ICX as llvm has it's own tools and methods.  

[Yanshuang]

Hi @Ron_Green, do you have any update for the issue?

[Ron_Green]

@Yanshuang I had to research this as it is a tool not very much used with ifort.  There have been no changes to this technology in probably 8 years.  This is a technology in our Classic compiler that is developed by our C++ team.  And since they have deprecated ICC, there will be no future work on this tool.  We have requests for a similar technology in our new IFX and ICX LLVM-based compilers but that will be done sometime in the coming years, nothing soon.

So, you can use it as it is today.

I would recommend using debug build, full debug symbols, no OpenMP or parallelism as it is not parallel-aware.  Understand that it may not be reliable.

[jimdempseyatthecove]

You alternatively can use VTune with enabling the tracking of the call stack. This may give you the information you want.

Remember to generate (compile) and include (link) the debug information.

Jim Dempsey