Code compiled for PGO/Coverage produces no .dyn files

[m305330]

Is there any way to get information about why a .dyn file has failed to be produced?

Here's why I ask:

I'm trying tomeasure the test coverage ofa set of 64 static libraries and29 applications built on these libraries. I compiled all the libraries and applications for profile generation (/Qprof-genx, I'm using IVF 10). This produced all the .SPI filesI expected. I ran all the tests I had. This produced many .dyn files, but not nearly the number I expected. For some libraries and applications, there were no .dyn files at all.

I first checked whether the tests covered these applications. I looked at one application that produced no .dyn files. I ran it manually. It should have produced something for a .dyn file. It produced nothing. The file did not appear where I expected it, nor anywhere else on the computer.

This application linked in 21 of these libraries. I recompiled all the libraries "as normal" (no /Qprof-genx). I rebuilt the application. I ran it. It produced a .dyn file. I recompiled each library with /Qprof-genx, one at a time, and retested after each addition. For the first 14 libraries, I kept getting .dyn files for the libraries and the application. After adding the 15th library, I stopped getting .dyn files for the application. I still got them for the libraries, though.

Through careful experimentation, I now know:

1.) There's nothing special about the number 15. For this application, I can add more /Qprof-genx libraries if I leave the 15th "as normal." For others that fail to produce .dyn files, I can add fewer before the .dyn's stop.

2.) It is a function of usage. With some command line arguments, the application writes a .dyn file. With others, it doesn't.

3.) It's not a function of the application's exit status. In all cases where it writes or fails to write the .dyn file, it ends with STOP.

4.) It seems unlikely that the entire 15th module is corrupted with /Qprof-genx. Compiling it with /Qprof-genx and linking it into a second application does not prevent the second application from writing its .dyn files.

Has anyone else seen this? As near as I can tell, something that tracks coverage is getting overwhelmed with too many modules to keep track of, and failing partway though execution. The failure seems to depend on the usage during the test.

My current work-around is to compile and test each module separately. This is not a good work-around, as the tests take 180 minutes to run, and I have 93 different libraries and modules to test.

I'd be much obliged to anyone could shed some light here.

[Brandon_H_Intel]

Hi m305330, welcome to the forums.

This sounds like a pretty tricky issue that may involve an issue with the PGO internals. I've brought this to the attention of our lead PGO developer and he's researching potential causes and should get back to me on this. As soon as I have some next steps for you, I'll let you know.

[Brandon_H_Intel]

By the way, even though termination is showing as normal, are you sure that the application isn't using a non-standard exit mechanism in those cases where the .dyn file isn't being generated? See the documentation at http://software.intel.com/sites/products/documentation/hpc/compilerpro/en-us/cpp/lin/compiler_c/optaps/common/optaps_pigs_ovw.htm for more details on this.

[Brandon_H_Intel]

It looks like there aren't that many ways that no .dyn file would be created. In the more common cases of memory corruption we've seen, there is a .dyn file created, just with truncated or corrupted information. As I mentioned in my last post, if the process is exiting in a non-standard way, this is usually the most likely cause of this kind of problem. Also, if the environment varialbe PROF_NO_DYNAMIC is set, then no .dyn file would be generated, but I doubt that since you have some runs that do generate .dyn files. There is also an internal variable in the PGO runtime that indicates if the info dump to a .dyn file is already in progress. If somehow this variable was erroneously set, then that could prevent the .dyn file from being written, but it would be difficult to verify if this is the case. Can you verify that the problematic runs are still exiting through the standard exit() routine?

[m305330]

Hi Brandon, thank you very much for investigating.

Memory corruption is always an option, especially with the code that I'm testing. Maybe I have an uncommon case: no .dyn file at all.

I checked PROF_NO_DYNAMIC. It's not set.

After I got your earlier reply, I looked once more at the way my programs were exiting. Iwas unable to find anything non-standard. Since my initial post, I have been able to get coverage data for all the modules and executables. I did thiswithout changing the code at all. I just used the procedure I described before, compiling the modulesfor profiling one at atime and running the tests repeatedly. If I had a non-standard exit, could I have gotten any coverage data for these cases at all?