ONLY : MYSELF to blame

[jimdempseyatthecove]

This isn't a bug as such, but perhaps a feature request.

I have some code that I am maintaining and enhancing, and in this instance I am annotating it for offload programming on the Xeon Phi. Wishing to keep as much of the annotations out of the source code as possible, I place the !DIR$ ATTRIBUTES OFFLOAD : mic :: foobar into an interface module. This works quite well provided you do not do something like the following:

subroutine FooBar(A,B,C)
use MyInterfaces, ONLY: SomeOtherRoutine

Well, this eliminates considering the interface for FooBar itself (assuming it is contained in the module, it was).

This was definitely my programming error, however, on the other hand, the compiler could implicitly (and conditionally) ONLY-fy interfaces to self. Or, note that self interface was inside the module and issue a warning to the potential for error.

I had a significant amount of issues in trying to figure this out because the code was inside a DLL called from a managed C# forms application. Error diagnostics (defining H_TRACE in this case), was not documented in the offload, and the fact that the app had no console, using it placed the error messaged into the bit bucket. This error was not detectible at compile or link time, rather it was a runtime error only inside the MIC offload. IOW the subroutine was not found in the dynamic load library generated.

Jim Dempsey

[FortranFan]

Steve Lionel (Intel) wrote:

I think the code in #18 is conforming. The error message ifort gives is perplexing, to say the least. I will report this to the developers. Issue ID is DPD200379165.

Steve,

Thanks for your feedback and follow-up.

Jim,

What are your thoughts?  It seems your main point is being able to check whether an implementation is consistent with the interface.  And I suppose it essentially comes into play when the implementation is not in a module.  So, from your perspective, what would prevent you from doing something along the following lines i.e., in your "FooBar" class of procedures (per your original post), make use of compiler directives and a preprocesspr definition (say check_interface) introduce some code along the lines of Message #18:

module myinterfaces

   implicit none

   interface

      subroutine sub()
      end subroutine sub

      subroutine OtherSub()
      end subroutine OtherSub

   end interface

   public :: sub

end module myinterfaces

subroutine sub(s)

   !dec$ if defined (check_interface)
   use myinterfaces, only : OtherSub, Isub => sub
   !dec$ else
   use myinterfaces, only : OtherSub
   !dec$ end if

   character(len=*), intent(in) :: s

   !dec$ if defined (check_interface)
   procedure(Isub), pointer :: foo_ptr => sub
   !dec$ end if

   return

end subroutine sub

and turn on the check_interface preprocessor definition whenever  the interface(s) need to checked.  Is this something you would care to utilize once the issue ID of DPD200379165 is resolved in Intel Fortran?  Of course, this won't work with ELEMENTAL procedures, but other than that, it appears an acceptable workaround.

SUBMODULEs still seem to be the "best" way to go: so can you please elaborate the reasons that would prevent you from overhauling your codes (which may include integration with stuff from clients and vendors, etc.) with SUBMODULEs?

• I would think you would only need to focus on stuff that is presently not in MODULEs (like the FooBar from your original post) and hopefully that is not too much code (given it's now over a couple of decades since the MODULE facility has been available in Fortran and there has been widespread knowledge dissemination of its benefits), 
• The larger challenge is developing the INTERFACE block, but you already have that,
• And therefore, can you not get by with just prepending a code snippet such as "submodule(myinterfaces) xx;contains;module" and appending "end submodule", something that could be automated by some scripts?
• So it doesn't seem as big an effort, hence what's missing in this picture?

Your comments will be very helpful in understanding real-life constraints with someone managing large codes.  Thanks,

[jimdempseyatthecove]

While that works, .... look at all the chicken scratching you have to add to all your subroutines and functions with interfaces. How many hours of work will this add to your code? You are essentially saying "I do not want the compiler to tell me I made an error in the interface declaration".

The whole idea behind compilers is to make life easier for the programmer.

Besides, as much as I have looked for the programmers switch panel on my computer, I cannot locate it. One can no longer toggle in code in binary via a switch panel. I remember the days of programming without a compiler.

Jim Dempsey

[FortranFan]

jimdempseyatthecove wrote:

.. You are essentially saying "I do not want the compiler to tell me I made an error in the interface declaration". ..The whole idea behind compilers is to make life easier for the programmer...

No, that's not what I meant.  To me, this appears to be a case of: "since the mountain won't come to you, you must go to the mountain."

I sincerely hope you are able to convince compiler vendors, especially Intel, to extend the compiler checking (say /warn:interfaces) to help with situations like the one you explain in your original post.  But I was simply coming from the point-of-view that the likelihood of that happening seems low, especially since the standard has moved in a different direction.  And even if you do manage to succeed in convincing Intel et al. with your feature request, the wait time may be rather high.  So what do you do until then?  Just live with "significant amount of issues in trying to figure .. out" such interface mismatches?  Or adopt a workaround or a standards-based solution, at least in codes one is immediately working on?  I was trying to understand why you would or would not go for the latter option.    

[jimdempseyatthecove]

The /warn:interfaces will verify that the interface used by the caller, matches the interface as declared in the interface module.

It will not necessarily verify that the interface in the interface module matches the actual interface as declared in the procedure itself. For this to happen, the procedure itself has to USE the interface module containing the interface to self... but according to adopted standards you cannot do this. A Catch-22 situation.

Jim Dempsey

[jimdempseyatthecove]

>>convincing Intel et al. with your feature request, the wait time may be rather high

I am not waiting, I've worked around this, living with a (mostly) detecting inconsistencies.

What I am doing is making a comment, or strongly worded argument, for why the standards committee took the wrong direction on this. I want a (always) detecting inconsistencies solution. Hopefully, someone will use these arguments in the committee meeting to persuade the adoption of this exception. I do not expect anything to be done with it until Version 2022 (no wait, that is the intended Catch-22 version).

Jim Dempsey

[IanH]

Repeat Offender wrote:

OK, I see the section quoted in the standard. I still reserve my opinion on whether it wipes out my code in Quote #11. But I found another passage in the same section:

If an external procedure does not exist in the program, an interface body for it may be used to specify an explicit specific interface but the procedure shall not be used in any other way.

So I modified my program to something which, with the above qualification, I think you might find more agreeable...

If you wanted to use the procedure pointer association approach, perhaps:

MODULE m
  IMPLICIT NONE
  ABSTRACT INTERFACE
    SUBROUTINE T(x)
      IMPLICIT NONE
      REAL :: x
    END SUBROUTINE T
  END INTERFACE
  PROCEDURE(T) :: S
END MODULE m

PROGRAM p
  USE m
  IMPLICIT NONE
  REAL :: y
  CALL S(y)
END PROGRAM p

SUBROUTINE S(x)
  USE m, ONLY: T
  IMPLICIT NONE
  REAL :: x
  PROCEDURE(T), POINTER :: U
  U => S
END SUBROUTINE S

Note though, that the requirement for a match in characteristics between the pointer and its target is in body text.  Because it is not a constraint you cannot count on a diagnostic from a compiler with this style of approach. 

I agree with FortranFan that better solutions to this problem than the above approach or language modifications, are, source wise, to make the external subprograms separate-module-subprograms in a submodule, or, tool wise, for checking of consistency between interface bodies and between interfaces and procedure definitions to be part of the /warn:interfaces style of diagnostic checks by the compiler.  The former option is available in the language and in compilers today, while the latter requires only moderate implementation effort (if that) for existing processors, requires no modification to Fortran source and is generally applicable.

An approach that centres on interface-to-self, which may raise tricky issues at the language concept level and will be tied to the time frames and vagaries associated with language development, will (to be effective) require modifications to Fortran processors not that dissimilar to the enhanced interface checking mentioned above, and will require modifications to any Fortran source that wants to take advantage of those checks. 

(I say "to be effective", because the requirements around interface-to-self would more than likely end up completely in standard body text, so there would be no guarantee that a conforming processor would even issue a diagnostic in the face of a mismatch.  You could wait ten years for the change to the language standard to be made, and then still just end up in a "waiting on feature request" situation, which is exactly where you are today!)

[jimdempseyatthecove]

Riddle me this...

How can /warn:interfaces warn of interface discrepancy between

Ainterface file and Bprocedure file ...

... When there is no compiler given association between A and B.

Conceivably a build with Ccall file that uses Ainterface file, linked together with an object file containing Bprocedure could do this...

... provided that Bprocedure source file was also part of the build.

So possibly doable. But then this also requires on the /warn:interfaces to .NOT. stop evaluation when (should) the Ccall call is verified to match the Ainterface interface. Additionally it will require interface checking between any file containing a procedure, and ALL explicitly USE'd modules in the complete build which may be multi-project (i.e. search all USE'd modules for conflicting interface). This seems so error prone especially in light that one unrelated USE'd module in the build may contain an entry point of the same name as a different USE'd module which are disambiguated using ONLY =>. Thererfore, how is this implicit association of the procedure's interface resolved between Xinterface to FooBar and Yinterface to FooBar... it cannot. This can olnly be done by

Here is a procedure
Here is an interface
Assert the two agree

Certainly you can assure the two match by placing them in the same file, but then you have the A, B, C issue.

Which may or may not be resolvable by sub-modules... which requires a major re-write of your (everyone's experiencing this issue) code.

There is a striking difference between a committee member's view of how a program should be written and how billions of lines of code are currently implemented. I also see no compelling reason as to why interface to self should not be permitted. By permitting this, then the current interface checking procedures will catch any discrepancy.

Jim Dempsey

[IanH]

I envisage that the necessary checking require compilation of the source containing the interface and the source containing the procedure definition (your Ainterface and Bprocedure) to be in the same progressive instance of a build tree, as the case for the current interface checking.  This same practical requirement would also exist for your proposal.

Current interface checking works by the compiler writing a "genmod" file when it encounters the definition of an external procedure - this same mechanism could be extended (perhaps even using the same file, though there may be benefits to keeping this separate from the current checking) to check for that file when compiling an interface body or procedure declaration statement for an external procedure - if the file exists check the characteristics against it, if it does not exist create the file and populate it with the characteristics specified by the interface.  Similarly, conduct such a check for the existence of that file when compiling the external subprogram that defines the procedure.

This enables checking between explicit interface specifications for the same procedure, and between the definition of the procedure and any explicit interface specification, and these checks would be applicable to all code written against standards from Fortran 90 to today.

Modules (proper) in the source and USE, ONLY: renames, do not influence this process - the name of an external procedure (without a binding label) is a global identifier in the program and must be unique amongst other global identifiers - and that is the key used to identify a particularly external procedure, the existence of other identifiers for the interface in scopes that might USE a module with such an interface is not relevant.  The above checks also do not deal with procedure invocations - the comparison is between explicit interface definition and procedure definition (for invocations through an explicit interface, the existing check of invocation against interface would cover mismatches, invocations through an implicit interface are the territory of the existing interface checking).

I think you overstate the amount of change to source to use the submodule solution suggested by FortranFan.  If you already have a module with the interfaces, it is a change to perhaps six lines of code per current external procedure, and it can be applied progressively.

1. add the MODULE keyword to the opening statement of the interface body in the module.
2. add the MODULE keyword to the opening statement of the external subprogram.
3. insert before the source for the external subprogram with `SUBMODULE (module_name) some_name`.
4. insert immediately after the SUBMODULE statement a CONTAINS statement.
5. append after the source for the former external subprogram `END SUBMODULE some_name`.
6. delete any USE statements referencing module_name from the scope of the now module subprogram (this step may not strictly be required, but I'd advise it for consistency).

For external subprograms in the same file, three of those steps are common.

Changes will also be required to manual build systems, if you are not using an automatic system like the Intel Fortran Visual Studio integration as a build system.

[jimdempseyatthecove]

IanH,

The aggregation of the "genmod" files into one large conglomeration would be fine except for the fact that the author would get all interfaces, not selected interfaces, and would not have the opportunity to annotate the interfaces file. While they can after the generation of this file, there would then be no way to assure further consistency, except with yet another utility program that compared Ainterface interfaces against all genmod generated interface. Again I suggest why write a hack fix utility, when you have the same capability with USE self.

In regard to submodules, as sketched by FortranFan in #10, an issue

The source file Bprocedure must be edited to become a submodule of some project's equivalent to Ainterfaces. This means then it may be difficult to maintain one source file in multiple projects, that may have different requirements for their facsimile of Ainterfaces, or lack thereof. This leads to having multiple copies of the same routine, or falling back on the error prone INCLUDE "MySubroutine.f90".

Jim Dempsey

[IanH]

jimdempseyatthecove wrote:

IanH,

The aggregation of the "genmod" files into one large conglomeration would be fine except for the fact that the author would get all interfaces, not selected interfaces, and would not have the opportunity to annotate the interfaces file. While they can after the generation of this file, there would then be no way to assure further consistency, except with yet another utility program that compared Ainterface interfaces against all genmod generated interface. Again I suggest why write a hack fix utility, when you have the same capability with USE self.

In regard to submodules, as sketched by FortranFan in #10, an issue

The source file Bprocedure must be edited to become a submodule of some project's equivalent to Ainterfaces. This means then it may be difficult to maintain one source file in multiple projects, that may have different requirements for their facsimile of Ainterfaces, or lack thereof. This leads to having multiple copies of the same routine, or falling back on the error prone INCLUDE "MySubroutine.f90".

Jim Dempsey

Control of interface checking would be available on a source file by source file basis, exactly as it is now.  Compiler bugs aside, it seems strange to me that you would only want to check consistency of a select set.  The file or files used to communicate interface specifications from compiler invocation to compiler invocation are not for user consumption.  There would be no reason for a user to annotate anything in these files.  I don't follow you point about further consistency, the checks of consistency would be done by the compiler, at the time that it compiled a particular chunk of source code. 

In your suggested approach, the source file with the external procedure (Bprocedure) is dependent on the module with the interfaces (Ainterfaces), via a USE statement inside the external subprogram.  The coupling if a submodule approach was used would be the same.  From the point of view of client code, the difference would be that the module must be USE'd to access the procedure, rather than just relying on an implicit interface or some other independent specification of interface, but other aspects, such as the ordering of source files for compilation, are the same.

[jimdempseyatthecove]

>> it seems strange to me that you would only want to check consistency of a select set

Owner of code writes library.
Owner of code wishes to distribute binaries of library only.
Owner of code wishes to make a subset of interfaces documented (remainder undocumented).
Owner of code wishes to fully comment (annotate) interfaces as part of end-user documentation.
ergo Owner of code desires to have an interface only module (not unlike IFPORT, etc...)

Now then, subsequent to hand building this interface file, and carefully going through interfaces, manually verifying consistencies, the owner of the code, as part of their maintenance wishes to assert that any and all edits to the source code does not upset the consistency of the distributed interfaces to the actual procedure arguments. Sure, you could re-generate interfaces, but you are unlikely able to run a difference program against the genmod interfaces due to various factors that you as the owner of the code wish to embellish the documented interfaces. Because of this you then are delegated to the lengthy task of manually verifying interfaces (100's to 1000's). This is error prone.

>> There would be no reason for a user to annotate anything in these files.

Why would you not want to document your interfaces??? Isn't somewhat error prone to have two separate files that document the interfaces?

At some point, IVF may integrate to the extent that when entering "CALL FooBar(" the intelesense will pop-up the argument declarations from the "interface file" as annotated by the user. (as you get now with C# and C++).

>>The coupling if a submodule approach was used would be the same.  From the point of view of client code, the difference would be that the module must be USE'd to access the procedure, rather than just relying on an implicit interface

But then this requires everything to be a monolithic repository. This is not case when the application pieces together multiple libraries from different sources.

Other than sticking with dogma, do you have any reasonable argument against self reference (where self and interface to self are same)?

Jim Dempsey

[FortranFan]

Throwing in a couple more cents:

1. Jim is advocating a change to the standard such that a "reference to self" would be permitted via [fortran]subroutine sub(..); use myinterfaces !.. where module myinterfaces includes an interface to sub..[/fortran] so that he can gain the benefit of interface checking with supposedly no change in his code.  At a high level, I totally agree with the idea that Fortran should have a standard "reference to self" facility for consistency checking, beyond SUBMODULEs that is.  I would be very happy if the Fortran standard committee would (again?) review the need for "reference to self" and (re)consider the needs including those of Jim and come up with a solution in the standard that meets those needs as closely as possible.  I don't know how this might happen, perhaps Jim can influence Intel to table a proposal to the committee on his behalf or Jim can join the committee himself!
2. As mentioned earlier, even if the standard were to be revised to include some such facility, it may be a decade before it sees the light of day.  It would appear Intel Fortran compiler already has much of the essential machinery available in the compiler in terms of what it does to offer /warn:interfaces option as well as support SUBMODULEs facility from the Fortran 2008 standard.  One would think a few, good developers on the Intel compiler team can take what's already available and offer some extra warning facility for users (either as part of /warn:interfaces or another option) to help with the situation in the original post.  Note this new thing need not suppress the standard checking explained by Steve in Message #3; it can be simply be an add-on warning message.  Should the committee eventually allow "reference to self" feature, Intel would only need to make some logic adjustments to be consistent with the new standard; not much new development work should ensue then for Intel.  It'll be somewhat along the days of pre-Fortran 90 when some vendor included an extension/feature that finally made it to the 1990 standard!
3. Regardless, it will be good progress in the state-of-computing-using-Fortran if users start making use of SUBMODULEs more and more (assuming Intel compiler team fixes the horrendous compilation cascade issue with Visual Studio soon, so ironical considering SUBMODULEs were also introduced to avoid compilation cascades!).  I can buy the inertia argument as to why those supporting large codebases will be slow to adopt SUBMODULEs, but they should realize they are the ones who will benefit the most.  However, I don't buy Jim's other concern, "it may be difficult to maintain one source file in multiple projects, that may have different requirements for their facsimile of Ainterfaces, or lack thereof. This leads to having multiple copies of the same routine" - it seems to ignore many established good practices on how to package code - think static libraries, DLLs, and so forth.  Any time the implementation is not in the same module as the interface, I think coders should seriously consider using SUBMODULEs.

[Steven_L_Intel1]

The bug from post #18 has been fixed for a future release.