Difference between Fortran 12 and Fortran 15 C++ to Fortran calling?

[Carlos_M_1]

Hi,

I have code that is supposed to support two versions of the development environment (both running on Windows 2012 R2):

1. Visual Studio 2010 Ultimate Version SP1 with Intel(R) Visual Fortran Intel(R) 64 Compiler XE for applications running on Intel(R) 64, Version 12.1.7.371 Build 20120928
2. Visual Studio 2013 Professional 2013 with Intel(R) Visual Fortran Intel(R) 64 Compiler XE for applications running on Intel(R) 64, Version 15.0.4.221 Build 20150407

My C++ code is calling the Fortran subroutines, for example, where the arguments are strings; the length of all the strings (in order) are passed to the Fortran subroutines.  The Fortran subroutine, however, is not receiving the lengths explicitly.

The string length arguments are of type "unsigned int". I was very surprised to see that this seems to work in the first environment, when it fails on the second environment.  Is there a reason for this? Or was I just lucky in the first environment?

I can provide code examples if that would help.

[IanH]

Show the code examples.

Are you using the C interoperability features of the standard Fortran language?  If so, the only interoperable character length is one (C strings map across to Fortran character arrays), and you cannot rely on the lengths being hidden arguments on the Fortran side. 

If you are not using the C interoperability features of the standard language, then you should be!

[Carlos_M_1]

Thank you for your reply. I have attached a zip file with a modification of one example delivered with Intel Fortran.

I am not using the C interoperability features. I will start exploring your suggestion; is there a good reference to read up on the C standard interoperability features?

Carlos

[mecej4]

See https://software.intel.com/en-us/forums/intel-visual-fortran-compiler-for-windows/topic/637855 about passing Fortran string length arguments from C.

One way to fix your code is to use size_t as the type for instrlength, instead of relying on the compiler to zero out the upper 32-bits of the 64-bit argument.

[Carlos_M_1]

Thank you, @mecej, for your consideration of my problem. Some additions to the analysis of my situation:

• I have code in production in the older compiler version, which seems to be working fine. Should I consider backporting the argunent type change in those older versions of code?
• Therein lies the problem: is the code defective even when it has been running like this for several years? Or was the Fortran compiler able to "resolve" this problem before, but is now more exacting in this kind of mismatch? Are we just lucky that it has run well so far?

I definitely feel like I should fix the code, but I just don't know how pervasive this issue is in my installed base.

Some good news is that I can effect the change by requesting the modification of a macro which is used to define the type of the string length argument. This change, however, will affect a large area of our code, and thus the change needs to be inspected for appliccability.

Information that would help me in this analysis is something like: "up to version 12, the mismatch in hidden arguments expressing the lengths of strings was resolved by zeroing out the upper 32 bits." And now that we are in wishful thinkign mode: " the compiler option xxx would ensure the proper handling of this problem."

Another question: the example I sent seems to work well in both platforms. When I paw through the assembly code, I can see no "corruption" in the string length, as it is retrieved from the argument list into the processor registers. When the problem occurs, I can clearly see the length being obtained with some unwanted information in the higher order bits. Is this just because the upper 32 bits are almost always zero by coincidence? I just don't seem to be able to reproduce the problem at will (as in the example I shared with you, where the retrieval of the string length looks clean, but it really is not).

Please forgive me for the "long-winded" response. I have inspected related postings before, but the one you reference is really appropriate.

[jimdempseyatthecove]

>> "up to version 12, the mismatch in hidden arguments expressing the lengths of strings was resolved by zeroing out the upper 32 bits."

If your older code generation targeted a 32-bit executable then size_t is 32-bits. And...
if I were venture to guess, earlier versions of IVF may have limited the size of character variables to that expressed with INTEGER(4), though someone from Intel might be able to answer that. IOW at some point they fixed a bug (and this broke your code).

As for backporting. You either fix it now or later. But later the person who assumes your position may not have a clue as to what is happening with code that used to work.

Jim Dempsey

[mecej4]

Carlos M. wrote:

Information that would help me in this analysis is something like: "up to version 12, the mismatch in hidden arguments expressing the lengths of strings was resolved by zeroing out the upper 32 bits." 

If the hidden length is passed as an argument in a register, note that in the X64 architecture loading a 32-bit data item into the lower 32-bits of a general register causes the upper 32-bits to be zero-filled. For example, mov r9d,... will put a 32-bit value into r9d and fill the (unnamed) upper half of r9 with zero bits. On Windows only four registers: rcx, rdx, r8 and r9 are used to pass integer/pointer arguments. What will happen, then, if the string length argument is passed in memory, instead?

You seem to think that variations in the generated machine code are strongly correlated to compiler version number. How arguments are passed is also affected by optimizations in effect and the nature of the argument list. 

And now that we are in wishful thinking mode: " the compiler option xxx would ensure the proper handling of this problem."

Would that be the "-fix_all_errors_current_and_future" option?

[andrew_4619]

mecej4 wrote:
Would that be the "-fix_all_errors_current_and_future" option?

I like that! But another good option would be "-correctly_guess_what_was_intended_when_compiling_bugs" I think I would pay quite a lot for that option.

[Carlos_M_1]

Absolutely! I thought Fortran compilers were really advanced!

Now, seriously:

mecej4 wrote:

If the hidden length is passed as an argument in a register, note that in the X64 architecture loading a 32-bit data item into the lower 32-bits of a general register causes the upper 32-bits to be zero-filled. For example, mov r9d,... will put a 32-bit value into r9d and fill the (unnamed) upper half of r9 with zero bits. On Windows only four registers: rcx, rdx, r8 and r9 are used to pass integer/pointer arguments. What will happen, then, if the string length argument is passed in memory, instead?

You seem to think that variations in the generated machine code are strongly correlated to compiler version number. How arguments are passed is also affected by optimizations in effect and the nature of the argument list.

This is excellent! The assembly code for the example I included above is behaving well precisely because of this! r9d is used to pass the string length argument! In the examples that fail, the string lengths are being passed in memory. I know that for a fact, since (at the portion of the code that fails) when the string length is brought into rax for the first time in the "Fortran side", it is obtained from a memory location; the memory locations are pretty different, though:

• On the v12 Fortran compiler, the length of interest is at: qword ptr [rbp+#D0h]
• On the v15 Fortran compiler, the length of interest is at: qword ptr [.tmp..T779__V$424]

So it seems quite plausible that the way that C++ is laying out the arguments in memory has something to do with this. The Fortran code must simply be "following suit".

I understand that the argument passing organization would be affected by optimization and the nature of the argument list. I will compare optimization options in the C++ compilers, which was a blind side on my analysis. I was only comparing the Fortran compiler options, and thus asking the Fortran compiler to "do what I think, not what I say".

I will report back with my findings, if that is deemed of use. I would hate to clutter this great resource.

[Carlos_M_1]

Thank you all for your help. My conclusions may seem obvious to you, but I feel obligated to share them due to all the help I got:

• The Fortran side is clean in this area; the problem is in the way that the hidden length items are laid out on memory.
• There is no perceived difference in the way that the v12 and v15 compilers deal with this area; the assembly code looks quite different, but when it comes to the retrieval of the arguments, they both work the same.
• This problem could remain hidden for certain cases:
  • When the argument list is short enough to fit all the length parameters in registers.
  • When the memory location where the length parameters are placed is adjacent to a zeroed-out 32-bit area. I am not sure why this happens to happen more often in one VC compiler (VS2013) than another (VS2010), but that is an issue for a different forum. The compiler options do not look different in any significant way.
• An interesting workaround that exploits the C++ call semantics: if the prototype of the Fortran function has all the string length arguments as "size_t", the rest of the code can remain the same. The C++ compiler takes care of moving all these parameters to a register as 32-bit integers (moving a dword only), then moving them out to memory as a 64-bit number (extracting a qword out of the register).
• The cleanest way, by far, would be to re-define the type I use to be "size_t" for all string length arguments (as a matter of fact, I happen to have a macro defining BINDAID_FLEN for that very purpose).

I would really like to thank the people who gave me such useful information.

I am not sure how a question may be marked as "resolved", but as far as I am concerned, I have received all I needed.

[FortranFan]

Carlos M. wrote:

.. The cleanest way, by far, would be to re-define the type I use to be "size_t" for all string length arguments ..

@Carlos M.,

You asked upthread, "is there a good reference to read up on the C standard interoperability features?" - I didn't check closely but I didn't see your question being addressed.  The standard document for Fortran itself is a good reference:  You can also refer to the books in the Dr Fortran blog for more traditional programming descriptions:

Not sure what kind of flexibility you have to change existing code: if you are planning to update/improve it considerably, especially on the Fortran side, then you may want to follow up closely on the above advice of using the standard C interoperability features and do away with implementation-specific 'hidden' length passing features and make things transparent on both sides of code:

module m

   use, intrinsic :: iso_c_binding, only : c_char, c_int, c_size_t, c_null_char, c_ptr, c_f_pointer, c_loc
   ! Employ named constants and methods in the intrinsic module, as needed

   implicit none

contains

   subroutine FSUB( INT_ARG, P_STR_IN, LEN_STR_IN, P_STR_OUT, LEN_STR_OUT ) bind(C, name="FSUB")
   ! Notice the BIND attribute; that's what does the 'magic' for interoperation with a C companion processor
   ! Preferably 'contain' the Fortran procedure in a module; I find it to be a great help in unit testing
   ! it first with Fortran caller(s) and then invoking the procedure from C++

      !.. Argument list
      integer(c_int), intent(in), value    :: INT_ARG     ! Value attribute to work with value type parameters
      type(c_ptr), intent(in), value       :: P_STR_IN    ! C style pointer in by value
      integer(c_size_t), intent(in), value :: LEN_STR_IN  ! C size_t for explicit passing of length information
      type(c_ptr), intent(in), value       :: P_STR_OUT
      integer(c_size_t), intent(in), value :: LEN_STR_OUT

      !.. Local variables
      character(kind=c_char,len=5) :: INT_STR

      write( INT_STR, fmt="(I5.5)" ) INT_ARG

      blk: block
      ! use the BLOCK facility to set up 'local' Fortran style pointers
      ! utilize C_F_FORTRAN method from ISO_C_BINDING to associate the Fortran style pointers
      ! with the corresponding C parameters

         character(kind=c_char,len=LEN_STR_IN), pointer  :: F_STR_IN => null()
         character(kind=c_char,len=LEN_STR_OUT), pointer :: F_STR_OUT => null()

         call c_f_pointer( cptr=P_STR_IN, fptr=F_STR_IN )
         call c_f_pointer( cptr=P_STR_OUT, fptr=F_STR_OUT )

         ! Operate on the data
         F_STR_OUT = F_STR_IN // INT_STR // c_null_char

         F_STR_IN => null()
         F_STR_OUT => null()

      end block blk

      return

   end subroutine FSUB

end module m

#include <iostream>
using namespace std;

extern "C" {

   // Prototype for the Fortran procedure
   void FSUB( int, const char *, size_t, char *, size_t );

}

int main()
{

   int intarg;
   string instring = "Testing...";
   char outstring [40];

   intarg = 123;

   // Invoke the Fortran procedure to manipulate the char outstring parameter.
   FSUB( intarg, instring.c_str(), instring.length(), outstring, sizeof(outstring) );

   printf("%s\n",outstring);

   return 0;

}

The above code should now be portable across conforming compilers and platforms.  Executing with gfortran,

Testing...00123

and the same with Intel Fortran,

[FortranFan]

@Carlos M.,

Also, note the approach shown in Message #11 above is alright if one is dealing with fixed sized C char arrays.  If the need is to interoperate with character variables in Fortran with ALLOCATABLE attribute, meaning strings of unknown length at compile time but which get defined during execution, you may also to look into ENHANCED interoperability with C features in Fortran 2015, a standard revision in the works, and which is now supported by Intel Fortran.  With Fortran 2015, one uses a 'standard' header ISO_Fortran_binding.h on the C/C++ side and a new CFI_desc_t type from it and related methods (CFI_Establish, etc.) to setup the interoperation:

module m

   use, intrinsic :: iso_c_binding, only : c_char, c_int, c_size_t, c_null_char, c_ptr, c_f_pointer, c_loc
   ! Employ named constants and methods in the intrinsic module, as needed

   implicit none

contains

   subroutine FSUB( P_STR_IN, LEN_STR_IN, STR_OUT, IRC ) bind(C, name="FSUB")
   ! Notice the BIND attribute
   ! Preferably 'contain' the Fortran procedure in a module; I find it to be a great help in unit testing
   ! it first with Fortran caller and they invoking it from C++

      !.. Argument list
      type(c_ptr), intent(in), value       :: P_STR_IN    ! C style pointer in by value
      integer(c_size_t), intent(in), value :: LEN_STR_IN  ! C size_t for explicit passing of length information
      character(kind=c_char,len=1), allocatable, intent(inout), target :: STR_OUT(:)
                                                          ! An allocatable char array
      integer(c_int), intent(inout)        :: IRC         ! A more traditional Fortran style parameter by reference

      !.. Local variables
      character(kind=c_char,len=*), parameter :: Msg = " Check out Fortran 2015 enhanced interoperability with C!"
      integer :: len_str_out

      if ( LEN_STR_IN <= 0 ) then
         IRC = 1
         return
      end if

      !.. Some operation that defines new string length
      len_str_out = LEN_STR_IN + len(Msg) + 1  ! Plus one for null termination

      ! Allocate the out string
      allocate( STR_OUT(len_str_out), stat=IRC )
      if ( IRC /= 0) then
         ! error stop?
         return
      end if

      blk: block

         character(kind=c_char,len=LEN_STR_IN), pointer    :: F_STR_IN => null()
         character(kind=c_char,len=size(STR_OUT)), pointer :: F_STR_OUT => null()

         call c_f_pointer( cptr=P_STR_IN, fptr=F_STR_IN )
         call c_f_pointer( cptr=c_loc(STR_OUT), fptr=F_STR_OUT )

         F_STR_OUT = F_STR_IN // Msg // c_null_char

         F_STR_IN => null()
         F_STR_OUT => null()

      end block blk

      return

   end subroutine FSUB

end module m

#include <iostream>
#include <string>
using namespace std;
#include "ISO_Fortran_binding.h"

extern "C" {

   // Prototype for the Fortran procedure
   void FSUB( const char *, size_t, CFI_cdesc_t *, int * );

}

int main()
{

   int irc = 0;

   string instring = "Hello World!";
   char * outstring;

   // Use macro from ISO_Fortran_binding to set aside an address to "description" of string data
   CFI_CDESC_T(1) str;
   // A pointer to point to above address
   CFI_cdesc_t * desc_outstring;
   CFI_rank_t rank;

   // Set the pointer to C descriptor address
   desc_outstring = (CFI_cdesc_t *)&str;

   // Initialize the C descriptor as a scalar character nonpointer data type
   rank = 1;
   irc = CFI_establish(desc_outstring, NULL, CFI_attribute_allocatable, CFI_type_char, sizeof(char *), rank, NULL);
   if (irc != CFI_SUCCESS) return(irc);

   // Call the Fortran procedure for string manipulation
   FSUB( instring.c_str(), instring.length(), desc_outstring, &irc );
   if (irc != 0) {
      cout << "FSUB failed.\n" << endl;
      return(irc);
   }

   // set outstring to base address of C descriptor
   outstring = (char *)desc_outstring->base_addr;

   cout << outstring << endl;

   // Free the CFI decriptor object for Fortran-C interoperability
   irc = CFI_deallocate( (CFI_cdesc_t *)&str );

   return (irc);

}

Note the above code that make uses of standard Fortran 2015 is not supported by many compilers yet including gfortran, but Intel Fortran does and using compiler 17, upon execution

Hello World! Check out Fortran 2015 enhanced interoperability with C!

[Carlos_M_1]

Great reference, FortranFan; I am all over those books; also, Dr. Fortran's blog looks really good.

There is a large amount of legacy code that needs to be portable across compiler versions and in Windows and Linux, which makes my choices a bit more limited. But this is really good information for us from now on.

I am new to this area of our code, but I am seeing that it has caused a lot of pain across the years. A good way to deal with that would definitely be to use the interoperability features. I think that, when things become difficult in some areas, I may try that approach to replace the code that is in trouble, and really give this code more stability.

[steinberg__elad1]

First of all thank you for a very good example.

I managed to compile and run your code just fine using version 2019_update1 of the Intel compiler, but when I add the check pointers flag, the program crashes with a segmentation fault. Why is this?

FortranFan wrote:

@Carlos M.,

Also, note the approach shown in Message #11 above is alright if one is dealing with fixed sized C char arrays.  If the need is to interoperate with character variables in Fortran with ALLOCATABLE attribute, meaning strings of unknown length at compile time but which get defined during execution, you may also to look into ENHANCED interoperability with C features in Fortran 2015, a standard revision in the works, and which is now supported by Intel Fortran.  With Fortran 2015, one uses a 'standard' header ISO_Fortran_binding.h on the C/C++ side and a new CFI_desc_t type from it and related methods (CFI_Establish, etc.) to setup the interoperation:

module m

   use, intrinsic :: iso_c_binding, only : c_char, c_int, c_size_t, c_null_char, c_ptr, c_f_pointer, c_loc
   ! Employ named constants and methods in the intrinsic module, as needed

   implicit none

contains

   subroutine FSUB( P_STR_IN, LEN_STR_IN, STR_OUT, IRC ) bind(C, name="FSUB")
   ! Notice the BIND attribute
   ! Preferably 'contain' the Fortran procedure in a module; I find it to be a great help in unit testing
   ! it first with Fortran caller and they invoking it from C++

      !.. Argument list
      type(c_ptr), intent(in), value       :: P_STR_IN    ! C style pointer in by value
      integer(c_size_t), intent(in), value :: LEN_STR_IN  ! C size_t for explicit passing of length information
      character(kind=c_char,len=1), allocatable, intent(inout), target :: STR_OUT(:)
                                                          ! An allocatable char array
      integer(c_int), intent(inout)        :: IRC         ! A more traditional Fortran style parameter by reference

      !.. Local variables
      character(kind=c_char,len=*), parameter :: Msg = " Check out Fortran 2015 enhanced interoperability with C!"
      integer :: len_str_out

      if ( LEN_STR_IN <= 0 ) then
         IRC = 1
         return
      end if

      !.. Some operation that defines new string length
      len_str_out = LEN_STR_IN + len(Msg) + 1  ! Plus one for null termination

      ! Allocate the out string
      allocate( STR_OUT(len_str_out), stat=IRC )
      if ( IRC /= 0) then
         ! error stop?
         return
      end if

      blk: block

         character(kind=c_char,len=LEN_STR_IN), pointer    :: F_STR_IN => null()
         character(kind=c_char,len=size(STR_OUT)), pointer :: F_STR_OUT => null()

         call c_f_pointer( cptr=P_STR_IN, fptr=F_STR_IN )
         call c_f_pointer( cptr=c_loc(STR_OUT), fptr=F_STR_OUT )

         F_STR_OUT = F_STR_IN // Msg // c_null_char

         F_STR_IN => null()
         F_STR_OUT => null()

      end block blk

      return

   end subroutine FSUB

end module m

#include <iostream>
#include <string>
using namespace std;
#include "ISO_Fortran_binding.h"

extern "C" {

   // Prototype for the Fortran procedure
   void FSUB( const char *, size_t, CFI_cdesc_t *, int * );

}

int main()
{

   int irc = 0;

   string instring = "Hello World!";
   char * outstring;

   // Use macro from ISO_Fortran_binding to set aside an address to "description" of string data
   CFI_CDESC_T(1) str;
   // A pointer to point to above address
   CFI_cdesc_t * desc_outstring;
   CFI_rank_t rank;

   // Set the pointer to C descriptor address
   desc_outstring = (CFI_cdesc_t *)&str;

   // Initialize the C descriptor as a scalar character nonpointer data type
   rank = 1;
   irc = CFI_establish(desc_outstring, NULL, CFI_attribute_allocatable, CFI_type_char, sizeof(char *), rank, NULL);
   if (irc != CFI_SUCCESS) return(irc);

   // Call the Fortran procedure for string manipulation
   FSUB( instring.c_str(), instring.length(), desc_outstring, &irc );
   if (irc != 0) {
      cout << "FSUB failed.\n" << endl;
      return(irc);
   }

   // set outstring to base address of C descriptor
   outstring = (char *)desc_outstring->base_addr;

   cout << outstring << endl;

   // Free the CFI decriptor object for Fortran-C interoperability
   irc = CFI_deallocate( (CFI_cdesc_t *)&str );

   return (irc);

}

Note the above code that make uses of standard Fortran 2015 is not supported by many compilers yet including gfortran, but Intel Fortran does and using compiler 17, upon execution

Hello World! Check out Fortran 2015 enhanced interoperability with C!

[steinberg__elad1]

bump

[Johnson__Seth]

@steinberg I can see at least one problem:

character(kind=c_char,len=1), allocatable, intent(inout), target

is not a valid dummy argument for C/Fortran interoperability. From the standard, section 15.2.4:

A scalar Fortran variable is interoperable if its type and type parameters are interoperable and it has neither the pointer nor the allocatable attribute.

[Steve_Lionel]

Seth, you're quoting from an older standard. Fortran 2018 allows pointer/allocatable when the corresponding C formal is a "C descriptor", which it is here.