Different results in debug and release

[Martin_B_2]

Running my program in ivf release mode returns other results than ivf debug mode, cvf debug mode and cvf release mode (their results are all the same). After a long search I found that the compiler option /check:all makes the difference.

As I read in several other topics the at least only reason for different results is an coding error. Though my program ist realy long, I would like to try to get these errors out. But compiling and linking it gives me no warning and no error message at all. Is there an option to get more feedback?

Is there a possibility to say which results are the right ones?

[Arjen_Markus]

You do not give any details about the differences. Are they small but noticeable differences in the numerical results? There can be all kinds of causes for such differences. One is, for instance, the inadvertent use of uninitialised variables, another relying on variables keeping their value between routine calls whereas they do not have the SAVE attribute.

Without at least some more information regarding the nature of the differences you see, there is no specific advice we can give you.

Regards,

Arjen

[Martin_B_2]

As I wrote my program is very big and I think the one difference is only the peak of the problem.

At this point the program is in a loop and it calculates the variable zw. This variable is then passed to the function dasin. For that it should not be greater than 1.

In release mode it has these values:

 ZW-1= -1.787382367634882E-002
 ZW-1= -0.828064706180674
 ZW-1= -0.828064706162865
 ZW-1= -0.999999999999615
 ZW-1=  -1.00000000000000
 ZW-1=  -1.00000000000000
 ZW-1= -0.999999999994462
 ZW-1= -0.820298874933507
 ZW-1= -0.820298874878552
 ZW-1= -0.637936726897861
 ZW-1= -0.676805781054647
 ZW-1= -2.653433028854124E-014
 
So everything is just fine. But in cvf and in debug mode of ivf two values are different (the first and the last one) and the last one is even greater than 1:
 
 ZW-1= -1.787382367637202E-002
 ZW-1= -0.828064706180674
 ZW-1= -0.828064706162865
 ZW-1= -0.999999999999615
 ZW-1=  -1.00000000000000
 ZW-1=  -1.00000000000000
 ZW-1= -0.999999999994462
 ZW-1= -0.820298874933507
 ZW-1= -0.820298874878552
 ZW-1= -0.637936726897860
 ZW-1= -0.676805781054647
ZW-1=  2.220446049250313E-015

[LRaim]

The theme of variables used by not defined has been been discussed many times.

If I am not wrong, Intel Fortran compiler does not generate messages in this case, so I would be pleased to have messages of the kind "*** Warning, variable xyz is used without any value assignment ***" at compile time. 

This warning would save a lot of time and to get this I sometime use an old Lahey compiler.  

[Arjen_Markus]

I am no expert wrt the problem you are trying to solve, but these differences could very easily be due to different optimisations in the various configurations. You might try by turning off all optimisation.

Also, the value of ZW that seems to be larger than 1 might actually not be - this could be due to extended precision for an intermediate value. The value of ZW-1 looks suspiciously like epsilon(1.0d0). (I have not checked though).

Regards,

Arjen

[Martin_B_2]

I have tried the cvf and it mentioned one uninitialized variable in the whole program. This one is corrected but the problem still exists.

[Martin_B_2]

Can I rely on the debug results and are the release results incorrect?

[LRaim]

The differences in numerical values are below the precision limits in double precision.

This is about 14-15 decimal digits.

No surprise in the difference of the last value. Moreover it has no sense to print more than 14 decimal digits.

In such a case the result is dependent on the computational order so one should no be surprised by the difference between debug and release.

[mecej4]

It is quite possible that the problem is one of unrealistic expectations rather than one of undefined variables used, etc.

In general, floating point calculations -- performed on two different computers, with two different compilers on one computer, with the same compiler and computer but with two sets of options -- can produce slightly different results.

There are also problems that are ill-conditioned, and sometimes you can only calculate four or five digits (for example) reliably. For an eye-opener, read about Wilkinson's perfidious polynomial.

Here is a simple and instructive example. A program to compute the sum of 1/n² from n=1 to infinity can be written in about 15 lines. 

program harm
implicit none
real s,t
integer n,i
!
read(*,*)n
s=0; t=0
do i=1,n
   t=s
   s=s+(1.0/i)**2
   if(s.eq.t)exit
end do
write(*,*)'i, s = ',i,s
end program

Running this program shows that there is no change in the result once n reaches 4097. The computed sum is 1.64473. The sum of the series is π²/6 = 1.64493.. Now, we know that single precision real calculations have about seven digits (24 bits) precision. Why, then, is the fifth significant digit of the result in error?

[Martin_B_2]

Isnt't is strange that in both modes in cvf and in debug mode of ivf are exactly the same? And (only) when I set /check:all in release mode of ivf they are the same as well.

[Steven_L_Intel1]

Not at all.  CVF uses the old (and slow) x87 instructions for floating point arithmetic - this can often cause intermediate expressions to be held in higher-than-declared precision. Intel Fortran uses the modern SSE instructions, and vectorization, and this can lead to numerical differences. In many cases, the expectations of "the same results" are unrealistic for computational floating point.

[Lorri_M_Intel]

/check:all turns off optimizations.

[Scott_L_]

If the differences in numerical results were significant, one should (i think) presume debug would be closer to intent since the release mode allows more compiler "optimizations" which could take shortcuts and change results.  Again, only if the discrepancy between release and debug were significant, there is the strategy of turning on opimizations one at a time from no opt, or turning opimizations from relase off one at a time to find the largest subset of optimzations that match the /debug or /noopt results,

scott

[Steven_L_Intel1]

Not necessarily. You may find the article I attached of interest, specifically page 10 which shows a case where optimized code is actually closer to the correct result due to fewer overall operations.

443546

[Scott_L_]

I have been saying for a while our software testing (regression testing) has been based on faulty thinking that digital computers are deterministic and running the same program with the same input should produce the same results, other than some numerical noise.   Maybe this pdf will help convince others that reprducibility is not assured with binary digital computers.    It seem surprizing (slightly) there aren't some/any/more commercial tools for managing SW testing and filtering out numeric "noise".  Everyone seems to have to write (and support)  their own.

scott

[FortranFan]

Martin B. wrote:

Isnt't is strange that in both modes in cvf and in debug mode of ivf are exactly the same? And (only) when I set /check:all in release mode of ivf they are the same as well.

In our long experience with many computational codes including some rather expensive 3rd party engineering software that were often based on Fortran, C, and C++ and many of which do indeed go back to the days of Digital Fortran/CVF and Microsoft C/C++ 6.0, whenever we have encountered differences in results from Debug mode (i.e., no optimization) compared to Release (i.e., with compiler optimization, usually /O2 or /O3), it has always been due to our own coding error(s).  Many of these errors have been rather subtle due to the use of COMMON blocks and alignment and EQUIVALENCE and the older approaches for mixed-language programming.  It was never easy to locate the errors, but we owned them i.e., the onus lay on us to get them fixed.

Maybe your situation is not like ours, your code is completely sound, and the issue lies elsewhere.  But if I were you, I would look long and hard in the code first and try different ways, perhaps using many. many write statements in places where your ZW-1 array gets touched or using MessageBox function from Windows kernel32.dll to launch many, many pop-ups along the calculation route to investigate what is going on; sophisticated coders may have better ways to find out what goes on with code in Release mode, but this can be a start.

[Martin_B_2]

Thank you all for your kind support.

When I switch optimization of release mode to /O1, the one variable I observe has - or should I say: seems to have - the same value as in debug mode and in cvf compilations. But what about the many many others? I can't say and I can't check them all.

The calculation time of my program isn't to long. So to have the opportunity to check my program, if something unexpected happens, I think it's the best just to use only the debug version and to forget the release version.

[FortranFan]

Martin B. wrote:

.., I think it's the best just to use only the debug version and to forget the release version.

No, it is not.  If the program and its calculations are of any importance to you and other users of it, then such a position - that only the results from the Debug version will be used - is untenable.

[Martin_B_2]

Why? What's wrong with the debug results? Are they worse then the release results?

[mecej4]

Martin B.: You seem rather resistant to changing some incorrectly formed presumptions. Being able to reproduce results is good, but reproducibility is no guarantee that the result is correct. Similarly, when a calculation is sensitive to slight perturbations in input data, irreproducibility is not necessarily an indication that the program has bugs.

In the end, it is the responsibility of the programmer (and coworkers, if any) to understand the purpose and limitations of the calculation, implement the calculation correctly, and do as much as is possible/reasonable to verify the results. I am afraid that there are no magic compiler options or buttons that we can push and resolve all these issues automatically.

Martin B. wrote:
Why? What's wrong with the debug results? Are they worse [sic]then the release results?

We don't know whether there is anything wrong with any of the results, nor do we know that any of them are correct. The debug results may have a higher probability of being correct, but that is as far as we can go given limited information -- so far, you have not said anything about the physics and mathematics underlying your program.

[FortranFan]

For one, see Steve's comment in message #14.

Secondly, your comments, "..For that it should not be greater than 1...in debug mode of ivf two values are different (the first and the last one) and the last one is even greater than 1:" suggests the possibility of some invalid operations occurring, meaning it may not be just a precision issue but potentially a problem in code.  If the latter is true, even the debug results may be in question, if not for this calculation, then some other and it may surface when the user is least aware.