Having problems calling MKL BLAS from VS2005 mixed language project 64bit

[Tony_Garratt]

I am using VS2005, Intel Fortran 10and MKL 10. I have a VS2005 that has C++ routines that call Fortran routines and those Fortran routines call LAPACK and BLAS routines. Everything works fine in 32bit and I am linking statically (I only need to add mkl_s.lib as a additional library).
When switching to win64, I add the following libraries to the build in VS2005

Solvers.lib mkl_core.lib mkl_intel_lp64.lib mkl_intel_thread.lib libguide.lib

The link suceeds. But when the Fortran calls the BLAS routines (e.g. DGEMV), I immediately geta crash.

So, the question is: how do I statically link to the MKL 64bit libraries from VS2005 please? I have tried several different sets of static libs, but either a link error or when I do link successfully, always crash when calling the BLAS.

Thanks!
Tony

[Tony_Garratt]

Quoting - Tony Garratt

Quoting - Gennady Fedorov (Intel)

Really? On my side all works fine. Can you get the testcase for reproducing the problem?

I will see if can extract a portion of the code that will reproduce it as I cant give you the whole project. I reason I am so confident its a bug is because my code works fine when compiled as 32bit; crashes when compiled as 64bit.

I have been communicating with someone else from another company who has been experiencing the same issues. Basically we believecomes down to getting the crash depending on what libraries you pull in at runtime. To get the crash, I believe you have to (1) Have a mixed C++/Fortran VC2005 project (2) be compiling 64 bit (3) linking against the static MKL libraries. If go back to 32bit or single Fortran project, everything is OK.

I have a get-around for now for this problem, so its not so critical to me that it gets fixed.

[sudhir_singh]

Quoting - Gennady Fedorov (Intel)

As an example, I have uploaded to this thread DGEMV.zip file contains
VS2005 based Fortran project is linked with mkl 10.1 with X64 mode.
See more details in Project properties settings.
You can adjust these settings and try to run the project.

I hope it will help you.
--Gennady

Gennady, Tony and Tim,

I have successfully reproduced theDGEMV problem in our solver with a much smaller system.

The smaller system that reproduces the problem is uploaded in a zip file along with this post.

The steps to reproduce the problem are the following:
1. First copy the Input.txt file that has data from our problem in the D: drive as the location is hard coded in the problem code.
2. Then set proper paths for all MKL 7.0 and MKL 10.1 libraries that have been linked in all the 3 projects (dgemv, CppExe and FortranLib).
3. Then add the bin directories for MKL 7.0 (IA32) and MKL 10.1 (EM64T) in path so that the dlls can be found at runtime.
4. Project dgemv is the FORTRAN exe project send by Gennady and modified to use the data from Input.txt which resembles the data from our problem. As stated by Gennady, this executable runs fine on both 32 bit and 64 bit.
5. I created a FORTRAN library based on Gennadys source code in the project FortranLib. This library has a FWRAP function that is called by as c++ exe and that in turn calls a FFCN subroutine that will call DGEMV with the data from same Input.txt file. This sequence of function was chosen to resemble the system that we have in our Solver Project.
6. After building the FortranLib project, build the CppExe project which is the c++ exe that statically links to FortranLib library and calls FWRAP function.
7. On running you would find that while 32 bit CppExe.exe runs fine, the 64 bit crashes with the given error.

The basic Idea is that when a 32 bit or 64 bit FORTRAN executable calls DGEMV, it runs fine. Also when a 32 bit c++ exe calls a 32 bit FORTRAN Library that calls DGEMV it works fine. However when a 64 bit c++ exe calls a 64 bit FORTRAN Library that calls DGEMV it crashes even with the same data as the previous cases. The error for the crash is given below. For my test case based on availability, I used MKL 7.0 for 32 bit linking and MKL 10.1 for 64 bit linking in this example that I have posted.

Error when calling DGEMV on my smaller project with same data:
Unhandled exception at 0x000000014000de34 in CppExe.exe: 0xC0000005: Access violation reading location 0x0000000000000001.

Error when calling DGEMV on our Solver Project:
Unexpected exception at code address 0000000140326834: The thread tried to read from or write to a virtual address for which it does not have the appropriate access. Invalid read access at memory address 0000000000000001.

So this seems more like a bug in MKL 10.1 (EM64T). Even if this is not the bug, since it reproduces our problem if Intel can find out how we can fix this executable, well be able to get our solver project working.

I'll like to thank each of you for your support.

[Tony_Garratt]

Thanks sudhir for reproducing it and sending it to Intel. You are seeing exactly what I have been experiencing too.

[Gennady_F_Intel]

Sudhir,
before reproducing the problem, could you please tell me, what was the reason to use two completely different MKL's versions into one project. Or I missed something?Could you please remove all 7.0 dependencies and try to catch the problem again.
--Gennady

[sudhir_singh]

Quoting - Gennady Fedorov (Intel)

Sudhir,
before reproducing the problem, could you please tell me, what was the reason to use two completely different MKL's versions into one project. Or I missed something?Could you please remove all 7.0 dependencies and try to catch the problem again.
--Gennady

Gennady,

I used 2 different MKL versions because our 32 bit solver project currently uses MKL 7.0 and we are migrating it to 64 bit where we are trying to use MKL 10.1. So these 2 versions will provide the exact case that we have.
However I have changed the same application to use MKL 10.1 for both 32 bit and 64 bit and this still reproduces the crash with 64 bit while running fine with 32 bit. The new application is attached with this post.

Regards,
Sudhir Kumar Singh

[sudhir_singh]

null

Hi Gennady,

I have not heard from you on this problem since I provided the sample code to reproduce it.

We (Invensys)have a support contract with Intel and would like you to state whether we should seek any other channel for getting support on this problem. This crash has become critical to our Project. You can contact me directly at Sudhir.Singh@ips.invensys.com for any further information.

Hope to hear from you soon.

Regards,

Sudhir Kumar Singh

[sudhir_singh]

Finally I myselffixed the DGEMV crash that was blocking us in our 64 bit solver project. The fix was mainly due to a build option regarding External Procedures in the Fortran Library Project.

We had previous been using String Length Argument Passing options as After Individual String Argument. This used to work fine with MKL7.0 and MKL10.1 IA32 (32 bit) libraries. However with MKL 10.1 EM64T (64 bit) libraries, it caused a crash.

Changing this option to After All Arguments (/iface:nomixed_str_len_arg) fixed the crash. No change was needed in C++ project.

This option specifies the type of argument passing convention used for general arguments and for hidden length character arguments.

MKL should be supporting both these conventions for both 32 bit and 64 bit executables. So this crash can be because of something wrong eitherin MKL or in Intel Fortran Compiler. I hope this workaround can help otherswho seethis problem and so have posted it here.

[TimP]

The :nomixed_str_len_arg was well established as the only method in use for 64-bit systems, well before IA 64-bit systems came about. It can hardly be described as a work-around. When you are interested in portable software, you follow the standards, which include now the Fortran iso_c_binding; you don't ask for implementation of an option which has been in disuse by consensus for 8 years. If you don't get a compile time diagnosis when attempting a bad option in 64-bit ifort, that would be a good subject for a formal problem report.

[Tony_Garratt]

Quoting - tim18

The :nomixed_str_len_arg was well established as the only method in use for 64-bit systems, well before IA 64-bit systems came about. It can hardly be described as a work-around. When you are interested in portable software, you follow the standards, which include now the Fortran iso_c_binding; you don't ask for implementation of an option which has been in disuse by consensus for 8 years. If you don't get a compile time diagnosis when attempting a bad option in 64-bit ifort, that would be a good subject for a formal problem report.

Hang on a bit Tim please. You have to remember that many users of MKL may be more mathematics skilled people, may not be expert programmers andmay notfully understand some of the implications of these standards. I was stuck with this issue too. My background is mathematics - Im not an expert programmer, but Im not dumb either and it got me stumped too. It seems both Singh and I were got by the default external procedures option - seems obvious now of course, but the obvious gets past the best of us.

What would have helped us here? I think better documentation of MKL (improving the documentation is a theme I keep banging on about to Intel) could have done it here, or an FAQ of "common build errors". I know it so easy to blame bad documentation, but Intel dont seem to understand that the user base of MKL may not be expert programmers and more maths related folk- they are wanting to use MKL to get better performance on theirnumerical codes, so by definition are likely to be more maths/numerical skilled than programming. Just a little bit more "hand holding" in the documentation for MKLcould go a long way and making sure this hand holding is right up there in a Getting Started Guide for instance. I know it would have helped usavoid several pot-holes with MKL along the way that cost us man hours that could so easily be avoided.

p.s. I see that the 10.1 documentation is much better than 10.0 - thank you Intel. But I would like to venture there is still more to do. Just taking common errors from this forum and posting up an FAQ on the web would be a start, sort of "go here first if you have problems before posting on the forum or submitting a premier ticket"

[Tony_Garratt]

Quoting - Tony Garratt

Hang on a bit Tim please. You have to remember that many users of MKL may be more mathematics skilled people, may not be expert programmers andmay notfully understand some of the implications of these standards. I was stuck with this issue too. My background is mathematics - Im not an expert programmer, but Im not dumb either and it got me stumped too. It seems both Singh and I were got by the default external procedures option - seems obvious now of course, but the obvious gets past the best of us.

What would have helped us here? I think better documentation of MKL (improving the documentation is a theme I keep banging on about to Intel) could have done it here, or an FAQ of "common build errors". I know it so easy to blame bad documentation, but Intel dont seem to understand that the user base of MKL may not be expert programmers and more maths related folk- they are wanting to use MKL to get better performance on theirnumerical codes, so by definition are likely to be more maths/numerical skilled than programming. Just a little bit more "hand holding" in the documentation for MKLcould go a long way and making sure this hand holding is right up there in a Getting Started Guide for instance. I know it would have helped usavoid several pot-holes with MKL along the way that cost us man hours that could so easily be avoided.

p.s. I see that the 10.1 documentation is much better than 10.0 - thank you Intel. But I would like to venture there is still more to do. Just taking common errors from this forum and posting up an FAQ on the web would be a start, sort of "go here first if you have problems before posting on the forum or submitting a premier ticket"

Changing to After All Arguments (/iface:nomixed_str_len_arg) fixed my crashes too. I think the action item for Intel is to consider making this the default for 64bit Fortran projects in dev studio, if the current default is to be discouraged.

[ArturGuzik]

I need to agree with Tim, it's hard to call a workaround. It's just fixing the code/interfaces to external routines. However, my quick check shows, that linker doesn't detect it (can it?) at least with pure BLAS calls (I didn't check 95 interfaces). Am I right?

All that leads to Tony and Sudhir's problems and "hard way" porting to x64 platform. Tony is obviously right that MKL docs need some clearer, better elaboration in many parts (I have myself couple of Premier Support issues on that). The IVF compiler docs state that C interface is provided for x64, ONLY, while MKL says that for C the lenght is passed after (but explains it for IA32 platform). So it's not entirely clear how it goes with x64.

Tony is also right that MANY people using MKL come from more math/numerics that programming "world". If Intel wants to expand the usage of MKL there, need to address issues of "after installation" settings, adjusting etc. I remember it took once 15 posts to help somebody to set up it in VS on Vista.

On the other hand, if somebody uses non-default setting(s) (mixed_str_len_arg in this case) should know what he's doing or accept defaults:-)

A.

[ArturGuzik]

Quoting - Tony Garratt

Changing to After All Arguments (/iface:nomixed_str_len_arg) fixed my crashes too. I think the action item for Intel is to consider making this the default for 64bit Fortran projects in dev studio, if the current default is to be discouraged.

Tony,

nomixed_str_len_arg is default in VS. You probably extracted your project from CVF (IA32 platform), where STDCALL was default and mixed_str_len_arg. Then adding mkl_s.lib made your transition to IVF smooth, but further to x64 a "bit" more difficult.

A.

[Tony_Garratt]

Quoting - ArturGuzik

Tony,

nomixed_str_len_arg is default in VS. You probably extracted your project from CVF (IA32 platform), where STDCALL was default and mixed_str_len_arg. Then adding mkl_s.lib made your transition to IVF smooth, but further to x64 a "bit" more difficult.

A.

Aghhhh! Yes! You are absolutely correct - I started from a win32 project and created an x64 one from that. That explains it! Thank you!

[sudhir_singh]

Quoting - tim18

The :nomixed_str_len_arg was well established as the only method in use for 64-bit systems, well before IA 64-bit systems came about. It can hardly be described as a work-around. When you are interested in portable software, you follow the standards, which include now the Fortran iso_c_binding; you don't ask for implementation of an option which has been in disuse by consensus for 8 years. If you don't get a compile time diagnosis when attempting a bad option in 64-bit ifort, that would be a good subject for a formal problem report.

Well may be it is not a work around. The compiler compiles fine with a wrong configuration and then it crashes at runtime. Should this wrong configuration option be available tointel compiler with 64 bit configuration? Even if it is available, should it not signal errors during compilation. May be thisbehaviour is a bug.
If Mr Tim was so smart to know about all establishedstandards he should have taken a look at the project I posted and could have got back on the way it could be fixed :D
But seems like that was so easy to miss and so difficult to find. So it was there for more than one week, and no one touched it.
In allsince I did not get any support from intel regarding this, and since this entire thread seems to say that its users fault when they get problems,Isuggest removing this thread and let all other users figure out themselves why their ported 32 bit projects won't run when build for x64 machines.
I am not a Fortran developer, I work mainly in C++ and since all Fortran developers in my company couldNOT figure our why this failed, I had to look into it and I did fix it.
Intel needs to do better on supporting its software, may be better involvement in the forums will help.
Now ifanyone isremoving this post for blunt feedback in name of abuse or violation of T&C, please consider removing the entire thread or atleast all my posts in this thread.

[Tony_Garratt]

Quoting - sudhir.singh

Well may be it is not a work around. The compiler compiles fine with a wrong configuration and then it crashes at runtime. Should this wrong configuration option be available tointel compiler with 64 bit configuration? Even if it is available, should it not signal errors during compilation. May be thisbehaviour is a bug.
If Mr Tim was so smart to know about all establishedstandards he should have taken a look at the project I posted and could have got back on the way it could be fixed :D
But seems like that was so easy to miss and so difficult to find. So it was there for more than one week, and no one touched it.
In allsince I did not get any support from intel regarding this, and since this entire thread seems to say that its users fault when they get problems,Isuggest removing this thread and let all other users figure out themselves why their ported 32 bit projects won't run when build for x64 machines.
I am not a Fortran developer, I work mainly in C++ and since all Fortran developers in my company couldNOT figure our why this failed, I had to look into it and I did fix it.
Intel needs to do better on supporting its software, may be better involvement in the forums will help.
Now ifanyone isremoving this post for blunt feedback in name of abuse or violation of T&C, please consider removing the entire thread or atleast all my posts in this thread.

Hi Sudhir.

Tim did come on a bit strong with his point about standards, but pleaselets not delete this thread because it is very valuable to others. Both you and I got hit by this one and it took a lot of time to get to the bottom of it. In the end you solved it for me (thank you!!!!), as I gave up and had to go down a getaround path - all of which costed time and money.

Intel - please get the lessons learned from this one into the documentation someplace as an FAQ. Its a subtle one that caught Sudhir and I out (and at lease one other person who contacted me directly via the forum).

Sudhir brings up a point - should he have gone the premier support issue path for this problem or the forum? Is the purpose of the forum to give support to users of MKL or are they better off submitting a premier support ticket?

[ArturGuzik]

Quoting - Tony Garratt

Sudhir brings up a point - should he have gone the premier support issue path for this problem or the forum? Is the purpose of the forum to give support to users of MKL or are they better off submitting a premier support ticket?

Hi,

I sumbitted support issue on that (clarifying docs and detecting misconfig) and it's not detected by linker. Will see if this helps.

The part of the problem, as Tim said, is that you both used non-standard, and non-default (!!!) in VS setting, instead of setting it up, just copying/extracting from IA32, most probably CVF, STDCALL interface type version (your code was portable, but your project not). I'm not sure it possible to remove the setting in VS (for x64). Please keep in mind that it's integration in VS, and many things cannot be done, as guys at MS change internal interfaces all the time (that's why, as far as I know, there is no source browser etc.). On the other hand I really understand the frustration. One week is a long time. Speed of Premier Support is a different story.

A.

[TimP]

In my initial response, I pointed out that the 64-bit compiler should complain about the use of an argument passing convention which is unlikely to work. As the story came out that automatic conversion from a CVF project, followed by invocation of 64-bit compiler, is a likely way for this to come about, it's more important, IMO, that warnings should be issued at more than one step about attempting to carry over the old CVF convention. So, this exchange should have a useful value.

[TimP]

Quoting - tim18

In my initial response, I pointed out that the 64-bit compiler should complain about the use of an argument passing convention which is unlikely to work. As the story came out that automatic conversion from a CVF project, followed by invocation of 64-bit compiler, is a likely way for this to come about, it's more important, IMO, that warnings should be issued at more than one step about attempting to carry over the old CVF convention. So, this exchange should have a useful value.

I received e-mail this morning about a compiler problem report filed to address this issue. So it is agreed that the compiler should provide better guidance on the option choice.