arithmetic with arrays

[Roman1]

I am trying to understand how arithmetic order is determined when an expression contains both scalars and arrays.  For example:

real:: a, b
real,allocatable:: X(:), Y(:)
integer,parameter:: N=1000

allocate( X(N), Y(N) )
! X, a,b are initialized, and then used to evaluate Y
Y = X * a * b

How many multiplications would be performed when evaluating Y?  If the first multiplication (X*a) is done first, then to total number of multiplications would be 2*N.  However, if the second multiplication (a*b) is done first, the total number of multiplications would be N+1.  Does the compiler analyze the code to reduce the number of multiplications, or is it the user's responsibility to put the parentheses in the correct place?

Roman

[Steven_L_Intel1]

The compiler is free to do either operation first. I doubt it looks at this and prefers one over the other. Parentheses would be a good idea, but you would also need to use /assume:protect-parens.

[mecej4]

In such situations, the optimization choices (explicitly selected, default optimization, or no optimization) will affect the number of operations performed.

[Roman1]

Thanks for the information.   I did some more reading on this in the documentation, and I found the following in:

https://software.intel.com/en-us/node/579862

"Operators with equal precedence are evaluated in left-to-right order. ..."

I might be misunderstanding something, but I think that the about quote is wrong.  From what I know, and what you said earlier, the evaluation can be done in any order.

[Steven_L_Intel1]

I think you're correct about the documentation and I will bring this up with our team. The standard says (7.1.5.2.4):

Once the interpretation of a numeric intrinsic operation is established, the processor may evaluate any mathematically equivalent expression, provided that the integrity of parentheses is not violated.

Two expressions of a numeric type are mathematically equivalent if, for all possible values of their primaries, their mathematical values are equal. However, mathematically equivalent expressions of numeric type may produce different computational results.

[Steven_L_Intel1]

On further reading, I see that the documentation is correct and I was mistaken about the reassociation. Sort of.

The standard says (7.1.3):

The general form of an expression (7.1.2) also establishes a precedence among operators in the same syntactic class.This precedence determines the order in which the operands are to be combined in determining the interpretation of the expression unless the order is changed by the use of parentheses.

NOTE 7.9
In interpreting a level-2-expr containing two or more binary operators + or –, each operand (add-operand) is combined from left to right. Similarly, the same left-to-right interpretation for a mult-operand in addoperand, as well as for other kinds of expressions, is a consequence of the general form. However, for interpreting a mult-operand expression when two or more exponentiation operators ** combine level-1-expr operands, each level-1-expr is combined from right to left.

For example, the expressions

2.1 + 3.4 + 4.9
2.1 * 3.4 * 4.9
2.1 / 3.4 / 4.9
2 ** 3 ** 4
’AB’ // ’CD’ // ’EF’

have the same interpretations as the expressions
(2.1 + 3.4) + 4.9
(2.1 * 3.4) * 4.9
(2.1 / 3.4) / 4.9
2 ** (3 ** 4)
(’AB’ // ’CD’) // ’EF’

However, the text that says any mathematically equivalent expression can be evaluated is still true, so this doesn't really address your question on optimization choices.

[Roman1]

I am doing some more reading on this subject.  Earlier, you told me about the /assume:protect-parens  option, because, by default, the compiler can ignore the parentheses.  In the standard, I found the following in section 7.1.8, Note 7.31:

For example, in evaluating the expression A + (B - C) where A, B, and C are of numeric types, the
difference of B and C shall be evaluated before the addition operation is performed; the processor shall not
evaluate the mathematically equivalent expression (A + B) - C.

It seems like the default compiler behavior violates the standard.

[Steven_L_Intel1]

Yes, you are right. That is one reason why /standard-semantics includes /assume:protect-parens. 

[jimdempseyatthecove]

IM(not so)HO the default behavior of the compiler is: to do as told with regards to parenthesis.

IVF should have taken the approach to honor parenthesis by default, but for those wishing to ignore the instructions of the programmer, have an option such as /assume:weak-parens

I wonder how may support issues were filed relating to parenthesis not being honored?

Jim Dempsey

[Steven_L_Intel1]

Not enough, I guess. Most programs don't care. This is an aspect of the common Intel optimizer.

[TimP]

The optimizer has improved in recent years with respect to avoiding performance loss due to ignoring parentheses.  I no longer have any vectorizable test cases where performance is lost for that reason.  There are some non-vectorizable ones.

When I was supporting Itanium applications, we had a number of customer issues which were first solved by adding options to optimize without disregarding parentheses (so as to pass the customer's QA).  So the question Jim asks has a long history.

Back in the ia32 days, there were cases where parentheses had to be ignored to achieve satisfactory performance.  For single precision cases,  ia32 extra precision often took care of providing sufficient accuracy while disregarding parens.

Martyn Corden fought a battle for several years to get /Qprotect-parens added to both Intel C++ and Fortran.  It works well now for my test cases.  In the past, it was asserted that since /fp:fast (for Intel but not Microsoft compilers) relies on several technical violations of C and C++ standards, it might as well ignore parentheses as well.

[jimdempseyatthecove]

>> we had a number of customer issues which were first solved by adding options to optimize without disregarding parentheses (so as to pass the customer's QA).

Disregarding parenthesis often leads to breaking convergence routines that are data dependent. IOW they may not show up during testing.

I'd rather have had the "inconvenience" of requiring an option (e.g. /assume:weak-parens) that may potentially break code that has been working for 50 years, than to require an option to not violate the stated intensions of the source code. When the programmer can state (in their mind) an expression without (some) parenthesis, and let the compiler choose the order (of ~equivalent operator precedence), then they should omit the parenthesis. The same treatment should be (have been) made with using lesser precise intrinsic functions (sqrt, sin, cos, ...).

As it stands now, the user does not know what they are getting (without looking under the covers of the IVF compiler).

The underlying problem (imho) was this decision was marketing driven, not engineering driven (certainly not customers who perform QA or other certification driven).

Jim Dempsey

[TimP]

We've been over most of the arguments before.  First is that the microsoft and gnu compilers with which Intel compilers intèroperate do follow the language standards on parentheses, unless options are set to change that behavior.

Gfortran observes parentheses, even under -ffast-math, unless -fno-protect-parens is set. Gcc goes to k&r legacy treatment of parentheses under -ffast-math so has no way to get full optimization while observing parentheses.