strange unintialized variable array behaviour

[astro_dave]

Hi everyone,

I have some code in which there was a variable array declared but not intialized. I realised this after seeing some strange results.
I've characterized the problem and I know how to get rid of it, but I would like to know why it is the way it is. Here is an example program:

implicit none
integer testvar(nmax), j
do j = 1, nmax
write(*,*) j, "", testvar(j)
end do
end

where nmax is actually some integer.
When nmax is small then the values of testvar are typically garbage, e.g. output with nmax=5:
11073779242
2134513212
3-1077337612
41073854404
51

However, when nmax increases beyond a critical value the numbers of the first few elements begin to be 0; for example, output with nmax=628:
10
21073743302
etc.

and beyond nmax=628 the number of elements at the beginning of the array that have zero value increases linearly with nmax. i.e.

nmaxnelements=0_at_beginning_of_array
6281
6292
6303
6314
etc.
800173
etc.

and the other array elements have garbage values (between high positive and high negative).
My main query is: why do elements at the beginning of the arrays get initialized with 0 when the array is beyond a certain size?
Why does the number of unitialized variables hit a limit (627 on my system) and then not grow beyond that?
Additionally, the pattern of garbage actually seems quite uniform. That is, the distribution of postive, negative and zero values moves along the array staying roughly constant - why is that? If you could explain this supplementary query that would be a bonus, but I am very eager to find the answer to the former question.
I've tested this with these compilers: ifort, g77, gfortran under a Debian (Ubuntu Hardy Heron) enviroment as well as on Windows with cygwin. 'ifort' initializes all values to 0 so does not show this behaviour; gfortran and g77 do not initialize to 0 and so do exhibit this behaviour.
I realise that I can fix the code by simply initializing the variable, but I am not asking for how to f ix the code. I am instead asking why the behaviour is the way it is.

I would really appreciate it if someone could answer this for me.

Cheers,

David

[TimP]

As you don't specify which options you use, I guess you haven't read the docs for each compiler about options which affect this behavior. Without such options, or with options which are compatible with threading, you take your chances on what might be left in memory accidentally from previous use. With such options, e.g. with g77, you could pay ridiculous penalties in link or startup time, and executable size.

[astro_dave]

Hi Tim,

Thanks for your reply. It's true that I haven't read the compiler docs; I don't have time to do so right now. I hope that doesn't sound ignorant, I have sunk quite a lot of time into this already. I'm helping my girlfriend diagnose a problem with a freely available non-profit program. We're going to report back to the author and wanted a complete understanding of what is going on.
I'm hoping that the experts here can give me an answer or find it out far quicker than I could.

Cheers,

David

[TimP]

You want to understand the behavior of a non-standard program which depends on the options supported by several individual compilers, without reading about those options? If you don't have time, don't waste time asking questions, just fix the program.

[Steven_L_Intel1]

You asked, "why do elements at the beginning of the arrays get initialized with 0 when the array is beyond a certain size?"

The answer is: they don't. Just because a value happens to be zero that doesn't mean it was deliberately initialized to be zero. There is NO default initialization of Fortran variables unless you explicitly write it into your code. None. If you want initialization, write it yourself. You can use an initialization clause on the declaration, a DATA statement or run-time initialization. Do not make any assumptions about the initial value of uninitialized variables.

[astro_dave]

Hi Steve et al.,

Many thanks for your reply. It actually was Dr Fortran himself that I had wanted to pose the question to in the first place!
I'm going to try and clarify things a little. A commonly used, freely available, scientific program has an uninitialized variable. It has been available and used in its current form for a number of years but nobody has reported any issues. The reasons for this could be:
(1) some fortran compilers (such as ifort) do initialize declared variables to 0 by default; people who use these compilers see no problem
(2) other compilers (such as g77 and gfortran) initialise every element to 0 at the beginning of the array (around 600-700 elements at the end of the array are filled with very predictable, patterned junk: highly negative or positive values, or 0). As long as the users have set a high enough number of elements then the program will not use those that contain the junk values and so no problems occur.
(3) people get problems but just don't realise (as they manifest quite subtley), which would mean that some refereed journal papers could contain significant errors.

So, in trying to explore option (2) a little more I was trying to understand the behaviour of compilers such as g77 and gfortran - why, when compiling without any flag options (which is how I believe most users of the program do it), do they begin initializing the beginning of the array when it gets beyond a certain size?
Perhaps try the mini-program I gave in the OP with g77 or gfortran, you will see that the output is as I say, predictable, and essentially the same across various executions, no matter if the machine has been rebooted, run something else, etc. (i.e. it is not the case that I am accessing the same memory in the same state each time).

If it is the case that the answer is not known then that's fine. It's true that a fix to the proram is to simply initialize the variable. We were hoping to be able to give a full explanation of the problem and the exceptions as we are preparing a report for the community. If the report is published and if a third party gives a satisfactory answer to the query that is included in the paper, then that person will get due credit.

Best wishes,

David

[Steven_L_Intel1]

I do not know of any compilers that change their initialization strategy (should they have one) based on array size. You have basically two cases:

1. The compiler does not deliberately initialize the array at all. You may or may not see zeroes there, depending on what type of storage is used for the array (stack, static, etc.) and what the previous contents are. If the array is statically allocated, you're more likely to see zeroes there, but it's not guaranteed. This is the way Intel Fortran behaves.

2. The compiler does initialize static variables to zero. The VMS Fortran compilers do this, in conjunction with the VMS linker. Stack-based variables do not get initialized.

Your choice (3) is really what is going on here, and I see this a lot. Programs that worked for years have had bugs that either didn't manifest or weren't noticed. I've seen customers thank us for revealing 20-year-old errors in applications they have been using. And yes, the results people get can have errors. Note that there are many other ways this can happen, often when proper attention is not paid to floating point computations.