A foolproof way to calculate DO LOOP iterations.

[WSinc]

An afterthought ;

Why not use REAL(16) arithmetic to calculate that?

We are always guaranteed to get the correct answer for ANY combination of inputs,

and the most extreme range (-huge to +huge)

You get  a REAL(16) result, which you would round off to get the final number.

NO_STEPS = (real(STOP,16) - real(start,16)+real(step,16))/real(step,16)

Since the compiler supports REAL(16) arithmetic, this should not cause any problems.

Anyway, I wanted to see what the compiler guys think about this, since it completely

avoids the integer overflow curse.

[Steven_L_Intel1]

The compiler guys and gals aren't interested in using calls to a software library to convert integer loop control values to REAL(16), do calculations, then convert back, just in case the computation might overflow. The current behavior is standard-conforming as best as I can tell. 

[TimP]

Many of us would consider lack of optimization a real problem.  If you like to declare everything with real(real128) that's your option. The Fortran committee once gave us the option of real do parameter but took it away again, but integer (int64) covers a lot.

[WSinc]

"it'll probably work MOST of the time?"

How is that conforming?

Why can't they do a thorough professional job?

Is it too much trouble to get it right?

Where I come from, we tried to do a thorough professional job,

and sloppiness was not tolerated.

How would I correspond with the Fortran committee ?

[Steven_L_Intel1]

The behavior when computations overflow is covered in the standard by:

This part of ISO/IEC 1539 does not specify ... the maximum number of images, or the size or complexity of a program and its data that will exceed the capacity of any particular computing system or the capability of a particular processor, ... the physical properties of the representation of quantities and the method of rounding, approximating, or computing numeric values on a particular processor, except by reference to the IEEE International Standard under conditions specified in Clause 14,

It's been this way since... forever. You are welcome to write a paper with your concerns and submit it to the committee - instructions are here.

I don't consider the current behavior "sloppy". It would be nice if compilers could warn you when a DO loop count computation goes out of range, and I know that some do. The same goes for integer overflows in expressions and conversions. Fortran has never been a language with training wheels - programmers are expected to be familiar with the numeric models and ranges for each type and kind.

[WSinc]

Oh sure, I am familiar with the ranges and numeric models - -

But here we are talking about unpredictable behavior, since the parameters can be determined by EXPRESSIONS, and DATA inputs.

so its impossible to determine the outcome with a reasonable degree of certainty.

If the parameters are given as simple integers,  we would be able to tell the outcome then -

Anyway, rather than submitting a FORMAL PAPER, I'll just send them an E-mail and see what they say about this issue.

[FortranFan]

Bill,

See Quote #12 in your other thread: https://software.intel.com/en-us/forums/topic/542783#comment-1816683

Why don't you use floating-point arithmetic yourself to guard against integer overflow?  This is one area where the standard lays down a basic set of rules and gives the compiler writer flexibility with the integer arithmetic and which, in turn, can benefit all users with optimization, etc. Why try to change that and what'll be the value?

Separately, what you're trying to do is difficult to fathom from an algorithmic stand-point - there may be other alternatives that may help you further.

[Steven_L_Intel1]

Bill, if you're intending to send mail to Dan Nagle, that won't get you a response from the committee. If you want the committee to consider your questions, write a paper and you'll get a response, probably at the next meeting. Or if you will form your question here, I will pass it on to the committee email list and summarize the response.

[Cardin_P_]

tem que converter valores de controle de laços inteiros para o Real (16), fazer cálculos, em seguida, converter de volta, apenas no caso de a computação pode transbordar.

[WSinc]

I know someone on the committee, but not very well.

Anyway, the fundamental question is: Whether they are required to properly do that calculation under ALL circumstances,

particularly if the EXCEPTIONS are never described or documented.

My attitude is: If you are going to CODE something, get it right.

Maybe I am being unreasonable - but I always worked in an environment

where we were punished for being sloppy. - - - 

Maybe I will submit a paper anyway just for fun. it'll be interesting to get a response, but it'll be really slow - - 

In the meantime, it's pretty easy to work around the problem by using floating point.

[jimdempseyatthecove]

>>In the meantime, it's pretty easy to work around the problem by using floating point.

That is misleading. While use of floating point will eliminate the sign flipping, the mantissa has fewer bits than the integer format using the same number of bytes. Therefore, this reduces the range of the numbers of iterations. More importantly, depending on the numbers chosen for start, finish, and step, this may produce an incorrect number of iterations. For these reasons it is better to construct the loop using integers (being mindful of corner conditions), or using an indeterminate do loop where you control the exit condition. But be careful when using a termination condition based on the sum of imprecise rational numbers. (products of iteration number * step is best)

Jim Dempsey

[WSinc]

Huh ? ?

Real(16) numbers have over 110 bits in the mantissa.

SO YOU CAN GET A 33 DECIMAL PLACE INTEGER FROM THOSE.

The INTEGER(8) variables contain 63 bits plus sign = 64 bits.

[WSinc]

I corresponded with a memebr of the Fortran comittee, and he confirmed what I have been saying -

anyway, there is another way to avoid INTEGER OVERFLOW without with having to resort to a higher

number of bytes in the arithmetic.

integer(8) function two_int(istart,istop,istep)
integer(8) istart,istop,istep,i1m,i2m,i1d,i2d
  i1m=mod(istart,istep)
  i1d=istart/istep
  i2m=mod(istop,istep)
  i2d=istop/istep
  two_int=i2d-i1d+(i2m-i1m+istep)/istep
end function

This method also works for 4 byte quantities.

Or any number of bytes.

Now it might still fail if ISTEP =1 but then the answer cannot be contained in the same number of bytes.

and the run time would be past the age of the universe, at least for 8 byte results.

RSSTop

[Steven_L_Intel1]

Yes, Van contacted me about this. Can you name any other compilers that do it the way you want? Are there any real-world applications that would benefit from this more complex computation? 

[WSinc]

When I was working at JPL and the Aerospace Corp, both were science/engineering applications,

and the CPU always had integer overflow traps. so if you got a wrong result, you always knew it.

I don't recall any situations where I got a bad result from doing integer computations there.

Now those were IBM and UNIVAC mainframes - - not PC consoles.

As far as applications, the one that I am involved NOW

with is cryptography, which uses large integers,

and some appear in DO LOOPS as well. Which is why the situations I am talking about can occur in practice.

I cannot speak for others, since I dont know their particular applications.

[Steven_L_Intel1]

Just using large integers is not the issue. What kind of applications would have DO loops with loop counts whose computation might cause an overflow?

[WSinc]

Besides cryptography, another such application is in NUMBER THEORY,

where you want to determine if a set of numbers is prime, -

i.e. which are prime, which are composite.

[FortranFan]

billsincl wrote:

Besides cryptography, another such application is in NUMBER THEORY,

where you want to determine if a set of numbers is prime, -

i.e. which are prime, which are composite.

Yeah, why are loop counters required?  As Steve and Jim have suggested, why can't ordinary DO.. END DO be used instead that gives control and flexibility (and places responsibility) to the coder?

[Steven_L_Intel1]

The only prime number factorization programs that might be affected by this are those written by introductory programming students. Anyone really trying to factor primes would be using different approaches that don't just have a counted DO loop to infinity...

All of the proposed "fixes" for this would very likely hurt optimization, especially vectorization of nested loops.

[mecej4]

There is such a thing as being overly cautious, and being overly cautious can lead to poor performance or even disaster. In the famous Battle of Agincourt, "Approximately 8,000 of the heavily armoured French men-at-arms fought on foot, and needed to close the distance to the English army to engage them in hand-to-hand fighting. If they could close the distance, however, they outnumbered the English men-at-arms by more than 5-to-1..." (quoting Wikipedia here). Unfortunately for them, they had to wade through thick mud to close the distance, and the English longbow archers slaughtered them before they could close.

Similarly, it could be disastrous to use the suggested DO loops with slow INTEGER*8 and REAL*16 arithmetic making many memory accesses versus a small but fast set of calculations in the small register files of the IA/32 and X64 architectures.

"Doing it right" sounds virtuous, but if you spend so much time doing it right that you don't get it done, you will not win praises for being virtuous.

A problem with "foolproof" solutions is that you have to test them on a representative sample of "fools" for proof-of-concept, and volunteers can be hard to secure.

[WSinc]

The prime number method I am talking about DOES NOT INVOLVE FACTORIZATION -

It uses the "bit bucket" approach, or the SIEVE of ERATOSTHENES method.

Nothing you would see in grammar school - college maybe (?).

This is something the Greek mathematician came up with more than 2000 years ago - -

Also called "pebbles on a beach."

Anyway, you have to be very careful when doing the tests to end a DO LOOP,

since you can get the INTEGER OVERFLOW without knowing about it.

Actually, the formula (quote #13) I gave would be an easy fix for INTEL, since it does not involve any higher order

arithmetic. And since it only has to be computed ONCE for the entire loop, it should not significantly affect any

optimization processes. They still have to compute no. of passes, anyway, right?

The fundamental problem is that (STOP-START) in some cases does not fit within the 32 (or 64) bit word, so you can get the integer overflow before you do the divide.

Now of course, those limits have to be really large numbers for that to occur, but who is to say that a given application would NEVER run into that situation?

Especially if its determined by a formula, or by data input ?