Error related to absent optional argument

[mecej4]

The following code is a simplified version of another example posted by "Simon" on the Silverfrost Fortran Forum, see http://forums.silverfrost.com/viewtopic.php?p=18780 . My reading of the Fortran 2008 standard, section 12.5.2.12.3(1) indicates that the code is not conforming since the two elements of array j, which is not present, are referenced in Line-10. Gfortran 4.7 and 5.2 stop with a seg-fault on line-21. Sun Fortran 12.4 on Linux says " Attempting to use a missing optional dummy 'J' Location:  line 10 column 14 of 'pres0.f90'". With Intel Fortran 2015.2.164 and 16.0.0.109 on Linux the program completes execution, and the output was actually what "Simon" expected.

PROGRAM absent
! 
 CALL s1 () 
! 
CONTAINS 
! 
 SUBROUTINE s1 (j) 
  INTEGER, INTENT(IN), OPTIONAL :: j(2)
!
 CALL s2 (j1=j(1),j2=j(2)) 
 END SUBROUTINE s1 
! 
 SUBROUTINE s2 (j1,j2) 
  INTEGER, INTENT(IN), OPTIONAL :: j1, j2
  IF (PRESENT(j1)) THEN 
     PRINT *, 'j1 = ',j1 
  ELSE 
     PRINT *, 'j1 N/A' 
  END IF 
  IF (PRESENT(j2)) THEN 
     PRINT *, 'j2 = ',j2 
  ELSE 
     PRINT *, 'j2 N/A' 
  END IF 
 END SUBROUTINE s2 
END PROGRAM absent 

The program output with Intel Fortran, compiled with -C:

 j1 N/A
 j2 N/A

[FortranFan]

A very good question on comp.lang.fortran.

I feel Intel Fortran is correct based on what I see in the J3 10-007r1 document on Fortran 2008 standard in section 12.5.2.12 Argument presence and restrictions on arguments not present.  But I'm no expert on "standard speak" and I often interpret it wrongly.  

[Kevin_D_Intel]

In the Doctor Fortran series, specifically under Doctor Fortran and the Virtues of Omission, our good Doctor is pretty clear writing,

“If an argument has the OPTIONAL attribute, you can test for its presence with the PRESENT intrinsic. The standard prohibits you from accessing an omitted argument, so use PRESENT to test to see if the argument is present before touching it. That part is simple.”

I agree the elements of “j” are touched inside s1; therefore, believe the ifort behavior is incorrect in this case. I submitted this to Development to have fun with.

(Internal tracking id: DPD200376417)

[FortranFan]

Kevin Davis (Intel) wrote:

In the Doctor Fortran series, specifically under Doctor Fortran and the Virtues of Omission, our good Doctor is pretty clear writing,

“If an argument has the OPTIONAL attribute, you can test for its presence with the PRESENT intrinsic. The standard prohibits you from accessing an omitted argument, so use PRESENT to test to see if the argument is present before touching it. That part is simple.”

I agree the elements of “j” are touched inside s1; therefore, believe the ifort behavior is incorrect in this case. I submitted this to Development to have fun with.

(Internal tracking id: DPD200376417)

It'll be very useful if you/your development team can loop back and point to parts of the Fortran standard that indicate prohibition of use of elements of an optional array dummy argument as actual arguments themselves in procedure invocations corresponding to optional dummy arguments.  Is it item 5) on line 26?  Thanks,

[mecej4]

In lines 39-40 of the Fortran Standard extract that you showed, "it" refers to the whole dummy argument as an indivisible entity. In the example, that is J. You can pass the dummy J, whether it is actually present or not. What you cannot do is to "touch", feel, poke or inquire about the properties of a non-present optional argument (according to my reading of the entire section, including 12.5.2.12.3(1) -- the critical portion. In particular numbered line 19 says that J may not be referenced. How do you obtain J(1) and J(2) without referencing J?

I think that numbered line 26 pertains to structures and structure fields and, more generally, type members that may be data or procedures. I suppose you could consider an array to be a simple variety of structure, but in Fortran the tendency seems to be to keep the concepts of array and structure separate.

In the specific example, to pass J(1) and J(2) you would have to know the shape, size and possibly other properties of J -- properties which you may not ask for if J is not present. Think of a non-present J as a radioactive hot potato.

[Kevin_D_Intel]

Thank you FortranFan. Development is reviewing/referencing that section.

Initial review/comments are the ifort behavior is believed to be correct. Paraphrasing the arguments in support of that, they said, while referencing J(1) and J(2) *might* be construed to be "referenced", the declaration of that is: A data object is referenced when its value is required during execution. However, the values are not actually being used during execution so this doesn’t apply. This case is passing an optional argument to a dummy argument that, itself, is also optional, and that is quite legal by the final "Except ..." paragraph that FortranFan cited.

I will share more once I hear it.

[IanH · Accepted solution]

IMHO item 5 of F2008 12.5.2.12p3 listed above kills it dead.  An array element is a subobject of the array, the subscripting that yields the element is described as an array element selector (1.3.59).  The code is non-conforming, no diagnostic is required, any behaviour by the compiler is acceptable, though it would be nice to get a runtime error when debugging checks are enabled.

[mecej4]

As IanH points out, a subscript expression is a designator (please see line numbered '26' in Fortran Fan's extract from the Fortran 2008 Standard). In the standard document, the word designator is linked to the following definition:

2.1.42
designator name followed by zero or more component selectors, complex part selectors, array section selectors, array element selectors, image selectors, and substring selectors

As used in 12.5.2.12.3(5), the "zero or more" in this definition should be further restricted to "one or more" in order to make the phrase agree with "and with one or more subobject selectors" in the text of 12.5.2.12.3(5), i.e., numbered line 26 shown above in #4.

[Kevin_D_Intel]

Development closed this as not a defect. The earlier Developer opinion was reaffirmed by another writing:

In s1, 'j' is optional and a data object, so it can't be referenced or defined.  But since a data object is defined as being referenced when its value is required, and since passing j(1) and j(2) does not require knowing their values thanks to pass-by-reference being the default passing mechanism in Fortran, this restriction does not apply.

[mecej4]

Kevin Davis (Intel) wrote:

Development closed this as not a defect. The earlier Developer opinion was reaffirmed by another writing:

In s1, 'j' is optional and a data object, so it can't be referenced or defined.  But since a data object is defined as being referenced when its value is required, and since passing j(1) and j(2) does not require knowing their values thanks to pass-by-reference being the default passing mechanism in Fortran, this restriction does not apply.

That conclusion is justifiable, as IanH pointed out, because the code is nonconforming and the compiler is not required to diagnose the problem. The same logic in the compiler, however, when used in a slightly different case, produces unpleasant consequences. Simply change the optional argument from an array of two integers to a structure with two scalar integer components. In this case, the generated code thinks that the second member of the structure is actually present even when the structure itself is absent.

PROGRAM absent
type pair
  integer first,second
end type
!          
 CALL s1 () 
! 
CONTAINS 
! 
 SUBROUTINE s1 (j) 
  type(pair), INTENT(IN), OPTIONAL :: j
!
 CALL s2 (j1=j%first,j2=j%second) 
 END SUBROUTINE s1 
! 
 SUBROUTINE s2 (j1,j2) 
  INTEGER, INTENT(IN), OPTIONAL :: j1, j2
  IF (PRESENT(j1)) THEN 
  	  PRINT *, 'j1 seems to be present'
     PRINT *, 'j1 = ',j1 
  ELSE 
     PRINT *, 'j1 is not present' 
  END IF 
  IF (PRESENT(j2)) THEN 
  	  PRINT *, 'j2 seems to be present'
     PRINT *, 'j2 = ',j2 
  ELSE 
     PRINT *, 'j2 is not present' 
  END IF 
 END SUBROUTINE s2 
END PROGRAM absent 

With or without /check:all and /warn:all, the output is as follows:

 j1 is not present
 j2 seems to be present
forrtl: severe (157): Program Exception - access violation
Image              PC        Routine            Line        Source
pres1.exe          009212BC  _ABSENT_ip_S2              26  pres1.f90
pres1.exe          0092107A  _ABSENT_ip_S1              13  pres1.f90
pres1.exe          00921037  _MAIN__                     6  pres1.f90

It is disconcerting that a sub-object of an absent parent object is regarded as present -- this is a quality-of-implementation issue.

Looking at the machine code throws some light on the behavior, For the code in #1, a NULL was passed for each of the arguments j(1) and j(2). For the code here, a NULL is passed for j%first, and 00000004 is passed for j%second. That second address, since it is not NULL, is taken to signify that j%second is present and the code then progresses towards an access violation when it attempts to use 00000004 as an address to dereference.

[IanH]

Kevin Davis (Intel) wrote:

Development closed this as not a defect. The earlier Developer opinion was reaffirmed by another writing:

In s1, 'j' is optional and a data object, so it can't be referenced or defined.  But since a data object is defined as being referenced when its value is required, and since passing j(1) and j(2) does not require knowing their values thanks to pass-by-reference being the default passing mechanism in Fortran, this restriction does not apply.

There are twelve items in the list of restrictions on non-present optional arguments quoted above by FortranFan, not being able to be referenced or defined is just one of those twelve.  I agree that the appearance as a dummy argument isn't a reference in this case, but that isn't particularly relevant - because one of the other twelve restrictions is being violated.

The example in mecej4's post is an explicit shape array, so there's no array descriptor for the argument.  Make it an assumed shape array, and then apply the logic in the response from development. 

PROGRAM absent2
  IMPLICIT NONE
  CALL s1
CONTAINS
  SUBROUTINE s1(array)
    INTEGER, INTENT(IN), OPTIONAL :: array(:)
    CALL s2(array(2))
  END SUBROUTINE s1
  SUBROUTINE s2(scalar)
    INTEGER, INTENT(IN), OPTIONAL :: scalar
  END SUBROUTINE s2
END PROGRAM absent2

No referencing or defining going on here either, but the program explodes.  That's ok - the code (and mecej4's latest example) is non-conforming because item 5 in 12.5.2.12p3 is being violated so the compiler can do "anything", but it would be nice when runtime checks are active that "anything" was to let the programmer know that they have done something silly.

[mecej4]

ianh wrote:

No referencing or defining going on here either, but the program explodes.  That's ok - the code (and mecej4's latest example) is non-conforming because item 5 in 12.5.2.12p3 is being violated so the compiler can do "anything", but it would be nice when runtime checks are active that "anything" was to let the programmer know that they have done something silly.

Nice and short test program, IanH! However, I do not get any hint of anything wrong (the program runs quietly) unless I specify /check or /check:all.

When I do use /check, an access violation occurs and a traceback is shown if it was requested at compile time.

I suspect that errors of this type in large programs will be hard to detect -- one cannot depend on an access violation to closely follow the execution of the erroneous code. Usually, in the stages of debugging a program, one may see an access violation, after which one recompiles with checks turned in order to find out more about what caused the access violation. 

[jimdempseyatthecove]

IanH,

Nice example. Maybe Steve (when he gets back) can comment on what is supposed to happen when you pass by reference an array element of an optional argument that is not present to a subroutine optional argument (what your example does).  As you note, this could be implemented as a runtime check, or an implementation feature. My preference is at least in this situation, the compiler can (should) insert code to determine if the (optional) array is present or not. When present, pass the reference to the element specified, when not present, pass a "not present" (null reference) indicator. I suspect what is happening is it is generating the address of (2) of the "not present" array, which now is (may be) viewed as present.

Jim Dempsey

[Lorri_M_Intel]

I'm not Steve, although people frequently get us confused since we sit next to each other ...and you can see the resemblance in our pictures.

That said - for the first case, for both arguments J(1) and J(2),  we generate code that checks if "J" is present, and if it is not, passes a ZERO through to the optional argument.   If it is present, then J(1) or J(2) is passed of course.

                --Lorri

[mecej4]

I admit to being confused, and the pictures of the two Steves don't help, because you are both so dazzling!

To the example code in #10, if you add the following two lines after the declarations in Subroutine S2 and then run the program, you will see the problem and the inconsistency in the way that the compiler currently treats non-scalar optional arguments.

  write(*,10)loc(j1),loc(j2)
10 format("Subroutine s2, argument addresses: ",4x,Z8.8,4x,Z8.8)

The output of the newly added WRITE statement makes the problem clear, and explains why the program crashes after access violations occur.

Subroutine s2, argument addresses:     00000000    00000004
 j1 is not present
 j2 seems to be present
forrtl: severe (157): Program Exception - access violation
Image              PC        Routine            Line        Source
pres1.exe          013610D5  _MAIN__                     6  pres1.f90
pres1.exe          01362E5F  Unknown               Unknown  Unknown
pres1.exe          0136323F  Unknown               Unknown  Unknown
KERNEL32.DLL       77593744  Unknown               Unknown  Unknown
ntdll.dll          7795A064  Unknown               Unknown  Unknown
ntdll.dll          7795A02F  Unknown               Unknown  Unknown

I suggest that the compiler should treat this non-conforming code in the same way as it does the code in #1. Whether the declared type of the parent object is an array or a structure, all of its components -- array members or structure members -- should be passed as NULL addresses when the parent object is not present. The second argument address, currently shown as 00000004, should also be 00000000, (i) for the treatment of optional arguments to be consistent, and (ii) to be able to detect absence.

At the assembler/machine code level, all components/elements of a non-present optional argument should have the same address as the address of the entire argument, which in the Intel compiler is NULL (Z'00000000' or Z'0000000000000000').

Thanks. 

P.S. Corrected based on the remarks of #16.

[jimdempseyatthecove]

mecej4,

>>The second argument address should also be 00000004, (i) for the treatment of optional arguments to be consistent, and (ii) to be able to detect absence

Don't you mean: The second argument address, shown as 00000004, should also be 00000000, (i) for the treatment of optional arguments to be consistent, and (ii) to be able to detect absence

Jim Dempsey

[Lorri_M_Intel]

Agreed ... not consistent when referencing derived type fields.

I think we should be doing the same thing as with arrays, that is, checking if the argument was present and if so, passing the field's address.  If not present, pass 0.

[mecej4]

Lorri: I believe that Development closed DPD200376417 as "not a defect"; please see #9. What do I need to do to have that case reopened, or should I do something else? Please advise.

[Kevin_D_Intel]

Nothing else needed. Lorri asked me to follow-up with your reply #10.

[jimdempseyatthecove]

This particular topic brings up an "ethical" (or dogmatic) dilemma. Should the compiler:

a) generate code to "do the right thing" (in our opinion in this case to pass "not present").
b) generate code that quietly produces "undefined behavior". (quiet up until the program crashes)
c) generate code that causes a runtime fault, and explaining why.

Option b) is not acceptable. One could make a case for a) or c). While the standards could specify "undefined behavior", each compiler vendor should select a) or c) but would be permitted to do nothing i.e. b). This is generally SOP when the committee cannot reach consensus.

Please note that, one could equally argue for an additional position:

d) Specifying a reference to an array element of an array not present is ostensibly equivalent to array index out of bounds (and undefined without runtime checks, or detectible with runtime checks). IOW to do so is a coding error. But this this presents a counter argument that one might want to pass in an optional array, then pass down the optional scalar of the first element of the array (without tests for present).

 After adding d), it is almost equivalent to b).

Jim Dempsey

[IanH]

Note that I firmly believe that the standard specifies your d (which is equivalent to b, and there are many, many other examples of b, so clearly b has been considered acceptable in the past!).  All the code examples in this thread are non-conforming - they all use a subobject of something that is not present, as an actual argument.  There is no need for the compiler to first do the "if J is present" test that is mentioned in #14, but equally there is nothing that says the compiler cannot do that test (though the presence of that test does have downsides, in that it is additional overhead for conforming code and demonstrably leads to confusion around what the standard actually permits).

Given the compiler is sticking the test in there, the behaviour for -check builds (yes - I overlooked I had that active - thanks mecej4!) should be reviewed - either issue no runtime diagnostic, or, better (in my opinion, given I think it non-conforming code) complain at runtime with a more specific error message.