where does a managed code guy go?

[Groos__Geoff]

Hey guys,

I'm a Java developer (occasionally moonlighting as a .net developer) and I am now managing a green-field albeit complex Fortran project. Me and my team are actively developing this code, and we'd like to offer a clean API to our consumers. Internally we're consuming this fortran code from a java application over JNA, which has been going fairly well. We would also like to publish a C and/or C++ interface for clients who want to bind to our project from native environments.

I'm doding specific names like "header files" and "dlls" because dropping back down into Fortran code has made me realize that I've never really understood linkers, they seem like they're doing black magic.

What I would think* I need to do is create a .h file with the reasonably simple contents

//fields matching a fortran TYPE
struct Config{
    public int configInt;
    public int anotherConfigInt;
    //...
}

void startRun(Config config, int otherSettings);
void endRun(Config config, int moreSettings);

with that being backed by something like:

TYPE :: Config
    INTEGER(C_INT) :: configInt
    INTEGER(C_INT) :: anotherConfigInt
END TYPE

SUBROUTINE startRun(config, otherSettings)
    TYPE(Config),VALUE :: config
    INTEGER(C_INT), VALUE :: otherSettings

    !...

END SUBROUTINE

SUBROUTINE endRun(config, otherSettings)
    TYPE(Config),VALUE :: config
    INTEGER(C_INT), VALUE :: otherSettings

    !...

END SUBROUTINE

My issue is with linking: I really don't know what BIND(C) in fortran does, nor !DEC$ ATTRIBUTES, nor on the C side what __stdcall or __declspec or extern "C" do. This is all black magic code installed by the software-plumbers.

Well, maybe its not all that bad, I understand a good deal of this in abstract, but I'm consistently defeated in detail. For example, right now, by adding the appropriate BIND(C, functionName) to this code in addition to !DEC$ ATTRIBUTES DLLEXPORT :: functionName, and an extern "C" on the C side , and by adding the fortranBinary.lib file as an "additional dependency" to the linker, and copying the fortranBinary.dll to the output directory, I do have this code building, compiling, running, and then crashing on the first language-hop. I believe this is because I've configured my project to be a poor mash of static and dynamic linking, but I'm really not sure how to start debugging this (no nuget or maven to rescue me here).

the irony is that JNA, our language to load native code from Java, simply asks you for a dll, and it seems to wire everything up well.

I'm not asking specific questions, and our actual library is of course significantly more complex than what i've mentioned above, but I'd really just like your thoughts and/or links on the best way of immersing myself in the nitty-gritty of cross-native-language (cross-compiler) linking, both statically and dynamically.

Cheers,

-Geoff

[Groos__Geoff]

sorry, and when I mention "cross compiler" I dont mean that I'm compiling for other platforms (read: cross compiling), I mean that I'm mixing the visual studio C++ compiler with ifort.

[mecej4]

There are a number of issues to address when doing mixed-language programming:

• do the data types match (i.e., are there matched pairs such as float in C matching REAL in Fortran) ?
• do the calling conventions match (w.r.t. passing by value, reference, descriptor, etc.) ?
• do the naming conventions match ?

Most of the issues that you discussed in your post are related only to the third category. Do not assume that being able to build an application (without linker errors) implies that the first two categories can be forgotten.

Each language provides escape hatches to allow inter-language linkage. C provides __stdcall attributes in declarations because the STDCALL convention was the default (or only) convention used in some early PC versions of Pascal and Basic. The DEC$ directives serve a similar role in Fortran, and were the only way to mix Fortran with other languages before Fortran 2000 brought us the C-interoperability features.

Java gives you a way to talk to C; so does Fortran. Therefore, C is the go-between language. However, it is not required that you write intermediate glue code in C, only that whatever glue code you do write behaves in the same way as C code.

The Intel Fortran user guide has an entire chapter on mixed-language programming, and Microsoft provides documentation for the linker (LINK.exe) as well as associated tools (DUMPBIN, etc.). You will need to read and understand much of the contents of these manuals. After all, if you were determined to cross a mine-field, you would be willing to take a map of the field with you and give the map a few looks-at, would you not?

There are several examples of using Fortran with C and C# in an Intel Fortran installation.

When you get the code to build and crash, it would help quite a bit if you are able to use an assembly level debugger to catch mismatches between actual and expected function/subprogram arguments.

After you have done all this, you may realize that you have now gone native :) .

[jimdempseyatthecove]

Consider using:

// C++ side
void startRun(Config& config, int otherSettings);
void endRun(Config& config, int moreSettings); 

! Fortran side
TYPE, BIND(C) :: Config
... 

SUBROUTINE startRun(config, otherSettings) 
    TYPE(Config) :: config 
    INTEGER(C_INT), VALUE :: otherSettings 
 
 

Jim Dempsey

[Groos__Geoff]

Thanks for that, I'll update the code I've got above to do that, but I'm reasonably well aware of how C and fortran each fudge pointers (and as somebody whose going from Java back to C I was really happy to see how Fortran does it). We don't have any issues with wrong-dereference levels in our actual code --or at least I dont think we do. We've got so many linkage errors its hard to tell.

... except I cant edit the original post on this forum? really?

[Steven_L_Intel1]

Sorry, post editing is not enabled here (except for moderators.) Just add the new code in a reply.

[jimdempseyatthecove]

I forgot to mention the you will have to add the attributes to the Fortran subroutine that specifies what its name (signature) looks like. Look in the documentation under interoperability. It should have an example or two to get you going.

You must have both C++ and Fortran agree on a name that the Linker can use. Typically you use the C interface.

Jim Dempsey

[JVanB]

Don't change your interface from VALUE to REFERENCE, that's not your problem and will only confuse the issue. Do you have a small example that works with JNA but not C? I tried getting JNA to work, and the docs on that topic are not very illuminating. Installing the java compiler resulted in my computer spitting out all these popups that froze internet explorer; I finally cured that by uninstalling what seemed to be a competing copy of java. The JNA example on the Wikipedia web page errors out at the first line and there is no brief example available to copy to make it go.

You need BIND(C,name= ... on all of your Fortran procedures--you should have needed this for JNA to work with them. Did you use the !DEC$ stuff to interface to JNA?

One last thing: if you do everything in 64 bits there is no difference between C and STDCALL so you can possibly eliminate this as the problem.

[Groos__Geoff]

Ok, so firstly thank you guys for giving my thread some attention, I'm feeling pretty good about my chances at this integration piece when Dr Fortran himself chimes in.

I've created a sandbox to play around with, anybody who wants push-access can have it:

https://github.com/Groostav/Debugging-Fortran

I've got it running but now not how I'd like. It currently generates both a *lib file and a *dll file for the fortran build --I really just want to get everything linking together statically so I can have one nice big blob EXE. What I'm pretty sure I should be able to do is get rid of all the precompiler stuff (both the !DEC$ and the __stdcall) and use purely the extern "C" and BIND(C)... compiler directives(?) That said, I'm fairly certain that what I'm looking for is an option buried somewhere in visual studio... and I don't want to go poking around in that yukey stuff, I want to get to the meat of the issue.

I need to pass fortran callbacks (and I haven't touched C++ since C++03, so I get to write my first C++ lambda :D -- and while I'm on this digression, @Steve Lionel, when is fortran going to support lambdas and closures?)

mecej4 wrote:

Most of the issues that you discussed in your post are related only to the third category. Do not assume that being able to build an application (without linker errors) implies that the first two categories can be forgotten.

Well, one foot in front of the other. But yeah, I got assigned some of this stuff because our main Algorithm-Fortran programmers are saying we're getting "garbage values" passed into some callbacks on our main application, which sounds exactly like what your describing to me --somebody shifted something right an extra word or ordered the parameters backwards.

mecej4 wrote:

The Intel Fortran user guide has an entire chapter on mixed-language programming, and Microsoft provides documentation for the linker (LINK.exe) as well as associated tools (DUMPBIN, etc.). You will need to read and understand much of the contents of these manuals. After all, if you were determined to cross a mine-field, you would be willing to take a map of the field with you and give the map a few looks-at, would you not?

There are several examples of using Fortran with C and C# in an Intel Fortran installation.

When you get the code to build and crash, it would help quite a bit if you are able to use an assembly level debugger to catch mismatches between actual and expected function/subprogram arguments.

I appreciate this, but I'm really not sure where to start reading, and the sources you mentioned are not small, and bear in mind I'm really after this more as an operations guy than as a developer. I guess my point is that those manuals are big and I'm hoping you can point me at interesting entries in the various indexes related to linking.

Specifically the only binary-inspection utility I'm using now is depends. I'd really link to run valgrind or some kind of LLVM-thing on this project so I could maybe peek into the exact loading process, and see the variables as they get passed around, but I'm just not familiar enough with the fortran (or C++ for that matter) tool chain. I pointed dot-peek at both the C++ binary and the Fortran binary and it just started at me blankly.

ifort is generating a pdb file, and the intel compilers integration into visual studio seems so good that I'm really hoping I can just step-into fortran functions, but right now visual studio doesn't want to let me do that. I presume that's in a manual somewhere too. Actually while we're on this topic what I'd really really like is to be able to step into a JNA dynamic proxy and have it load the fortran pdb file inside my java IDE... maybe its worth setting up eclipse to do this...

Anyways, I think what I need to do is start reading, I'm just a little daunted by where to start. If anyone wants to point me at what they think the best thing for me to read is, I'm all ears. I've got lots of snippets from MSDN and from the Intel page, but I'd like something more along the lines of a walk-through or a semantic description, rather than manuals.

But before that I think I want to generate an exhaustive list of the problems that we're having, because I'm hearing a couple different things from a couple different places --it looks like all of our 64bit flows are OK.

[IanH]

extern "C"{
void pauseOptimization(bool);
}

void function takes a bool by value.

..
SUBROUTINE pauseOptimization(meh) BIND(C, name="pauseOptimization")
!DEC$ ATTRIBUTES DLLEXPORT :: pauseOptimization
LOGICAL(C_BOOL),INTENT(IN) :: meh

Subroutine (equivalent of a void function in C) takes a LOGICAL(C_BOOL) (equivalent of a bool in C) by the default argument "passing" method for BIND(C) functions - which is by reference (equivalent of bool* in C).

You need to make those consistent.  Perhaps you need to change the Fortran declaration to `LOGICAL(C_BOOL), INTENT(IN), VALUE :: meh`

Your Fortran project is setup to build a dynamic library, so hence you get the DLL.  If you just want a single exe for your whole project, recreate the project as a static library (this is one of the project template options when you create a fortran project) and then direct the C++ project to link against that static library.  Some of relevant documentation for how to do that is at https://software.intel.com/en-us/node/525352 .