OMP Parallel for MS Visual Studio with Intel Fortran

[mattsdad]

I have been working on a simulator that simulates multiple objects simultaneously. I use MS Visual Studio with Intel Fortran on Windows and Intel Fortran on Linux. I have made everything within the main simulation loop pure so that we could try making the simulator run faster on multi-processor machines. I added the OMP Parallel Do construct around the main simulation loop. The simulator immediately ran 45% faster on a two processor Linux machine, but the same code is a tiny bit slower on the Windows machine.

Do I need to do something extra in MS Visual Studio to get it to generate code that looks for extra processors? (The Windows machine where the .exe was linked only has one processor.)

[hajek]

Are you sure that the OMP is enabled? IIRC, the env variable (OMP_NUM_THREADS, I think) needst to be set to desired number of processes. Try calling OMP_GET_NUM_THREADS to see if you really have multiple processes.

[mattsdad]

I just tried a pair of runs setting OMP_NUM_THREADS to 8 on one run and to 1 on the other. A four body simulation took about 2% more time when OMP_NUM_THREADS was set to 8. (The machine I did this on has 4 hyper threaded processors that appear to run 8 threads.)

[Steven_L_Intel1]

You have to use /Qopenmp when compiling on Windows, similar as how you use -openmp on Linux. In the Visual Studio environment, this is the Fortran property page Language > Process OpenMP Directives > Enable.

[mattsdad]

When I Enable the Process OpenMP Directives flag, I get a flood of openmp requires C style preprocessor warnings and about two dozen link errors for already defined libraries all generated from (crt0dat.obj). Can these be fixed simply?

[Steven_L_Intel1]

Check the Preprocessor property page to make sure you didn't turn off processing OpenMP condtional compilation. I suggest also making sure that under Libraries you have multi-threaded set and that if you have any C code, it has a compatible setting for libraries under Code Generation.

[mattsdad]

OK, I set the flag under Libraries for multi-threaded. The OpenMP Conditional Compilation flag was already on. I got the same warnings and errors. The first error reads:

error LNK2005: __argc already defined in LIBCMT.lib(crt0dat.obj)

All the other errors are similar with different __function_names and differet LIB_WHATEVER.libs. They all have (crt0dat.obj) at the end.

If it is of any significance, I'm using MS Development Environment 2003 version 7.1.3088 with Intel Fortran Compiler Integeration for MS Visual Studio .NET 2003 version 8.1.2380.2003.


PS. I do have some commercial libraries hdf...lib, libjpeg.lib, zlib.lib and szlib.lib but these are only use outside the parallel section.

[Steven_L_Intel1]

Doesn't matter if the libraries are outside the parallel section - if you link with them, you have to make sure that the library settings match. You'll have to figure out what libraries they want to use. You may be able to do this from a Fortran command prompt window:

dumpbin -directives zlib.lib

etc. and see what default library directives are shown.

Version 8.1, eh?

[mattsdad]

Since the HDF libraries were all associated with output and the parallel section was at the simulator, I simply removed the HDF output capabilities. Doing this allowed a successful compile and link, but when I have the Process OpenMP Directives flag enabled for parallel execution, the process crashes in Intel's code associated with ALLOCATE or DEALLOCATE. (When I eliminated the ALLOCATE code at the first crash site, the problem moved to the DEALLOCATE site of a different data structure.) I dissabled the Process OpenMP Directives flag while continuing to use the multi-threaded libraries. The program works just fine with the multi-threaded libraries. I set Process OpenMP Directives flag to generate sequential code and the program ran fine.

So, how do I get this thing to work with the Process OpenMP Directives flag set to generate parallel code?

[Steven_L_Intel1]

Is there a reason you're not using a current version of the compiler?Debugging this sort of thing in the user forum is not likely to succeed.

[mattsdad]

Yes. We are doing this under a government contract which requires that we produce output in the standard HDF format. The open source, standard HDF4 products are approved for Intel Fortran v8.1 but not for any later versions. We cannot upgrade until either Intel or some volunteer does the approval tests for the later versions.

P.S. Anyone interested can provide their assistance at www.hdfgroup.org

As pointed out earlier, I have reduced the problem. I am not getting the link errors that I was getting originally, because I removed the incompatible libraries. (I will put the output routines back after the OpenMP directives are working.) The problems that appear seem to come from the allocation code, but only when the "Process OpenMP Directives" flag is set to "Generate Parallel Code". My question is what other MS Visual Studio flags are interdependent with the "Process OpenMP Directives" flag, and what should they be set to for the compilation to generate a load module that will use multiple processors in a Windows environment?

[Steven_L_Intel1]

I am not aware of other options that need to be selected.

[jimdempseyatthecove]

The first thing I suggest you try is to write a simple application, one main, that compiles with OpenMP but uses static array (no allocate). Compile and verify that parallel code is indeed working (i.e. you see speed-up as you increase the number of threads from 1 to number of processor cores). If not, then check your option switches, environment variables etc...

Once that is working, try inserting an ALLOCATE in the test code outside the parallel section(s). If that doesn't work then it is likely that you are linking in the wrong library.

Next, place some ALLOCATE and DEALLOCATE in the parallel section so as to test parallel allocation/deallocation. If this does not work (crash??)then there may be a bug in the older 8.1 code.

To attempt a work around, use a critical section around the ALLOCATE and DEALLOCATE. Although this will make the allocations and deallocations run serially, at least they will run. Node, the earlier versions of the heap allocation internally used critical section, so the performance on the older code of using your own critical section won't be affected as much. I am not sure if the newer heap code uses Lock-Free/Wait-Free techniques. If so you will see a tad of a performance hit in using the critical section.

If the critical section works then I suggest using a named critical section as opposed to an unnamed critical section.

Use the experienced gained to address related problems in your application.

Good luck.

Jim Dempsey

[mattsdad]

Although it may create contractual issues with the government, we went ahead and purchased Intel Visual Fortran Compiler Standard Edition, for Windows version 9 and installed the I32 (to work with Visual Studio) and EM64T to generate the executable for the Xeon dual-dual core platform.

The I32 version installed itself into Visual Studio nicely (on a Pentium 4 platform), but it doesn't produce an executable that uses more than one CPU onthe Xeon regardless of the settings of the parameters in Visual Studio described in previous posts. So I have stepped back to nmake to build the executable using EM64T.

First problem, the build environment forEM64T-based applications does not correctly set the path to see the nmake.exe. I was able to get around this problem by editing the ifortvars.bat file to include

C:Program FilesMicrosoft Visual Studio .NET 2003SDKv1.1Bin

in thePATH variable. So, I can ignore this problem, but Intel should consider fixing this in later releases.

Second problem, when the f90 files compile with the /c option, I getthe error message:

ifort: error: could not find 'cl'

Why does the EM64T Fortran compiler require the C/C++ compiler in its path? The Fortran compiler generates the obj file just fine. So, why is this an error? Shouldn't it just be a warning? Better still, the Intel Fortran compiler shouldn't issue this error (or a warning) when the /c flag is set. Regardless, I can ignore this problem as well.

Third problem, the EM64T Fortran compiler cannot link because the LIBC library is not available (or LIBCMT depending on the flags used). Were can I get a copy of these libraries without purchasing yet another compiler?

[TimP]

As Visual Studio doesn't support ifort 64-bit, your choice for Windows support is between 64-bit SDK command line, VS2005 Express (free, no GUI support for Fortran) and VS2005 Standard. You search box should turn up plenty of prior posts on the subject.

[Steven_L_Intel1]

If you're building on an EM64T system and don't have VS2005 (which does support EM64T), then you must first install the Microsoft Platform SDK as described and linked to in the Installation Guide. Please use the specific version indicated. Then reinstall the compiler.

The error about cl.exe is due to the need to figure out which version of Visual C++ you have installed so it can select the proper "buifferoverflow" library. In the next version we'll be doing this a different way.

[mattsdad]

I do not have any version of Visual C++ for an EM64T system. I have MS SDK 1.1 (which does not support EM64T).

There isonly one specific unanswered question in my last post. Where can Iget LIBC and LIBCMT for an EM64T system?

[Steven_L_Intel1]

I don't know what you mean by "MS SDK 1.1" - there is no reference to such a thing in the Installation Guide, and Microsoft has many different SDKs. The Installation Guide lists and links to Microsoft Windows Server 2003 SP1 Platform SDK. This provides what you need - use this version and not any other. Download PSDK-amd64.exe and run it. You need only the "Core SDK". Then reinstall Intel Fortran.

[mattsdad]

Alright, I have given up on the cross compilation attempts. I obtained one of the Xeon dual-dual-core machines, installed Visual Studio 8, downloaded and run PSCK-amd64.exe, installed Intel Fortran 9.1, set all the flags according to the instruction above, compiled and run both a version of the simulator with /QopenMP set and unset. Both of the simulations use only one processor at a time and both take about the same amount of CPU time.

After the installations I described, I still cannot get nmake to generate an .exe. It gets all the way through the compilations and most of the way through link. The flags I used are /O3 /IA64 /Qprec-div /libs:static /module:build /c /object:build/name.obj. And I am getting a warning on the compiles and the link that reads:

ifort: warning: option '/Qvc8' or higher used with '/ML' is not supported

Neither of these flags are set. So why is the warning issued? More importantly, I am getting an error on the link that reads:

Fatal error cannot open "LIBC"
ifort: error: problem during multi-file optimization compilation (code 1)

which forces the link to stop.The %LIB% reads:

C:Program Files (x86)IntelCompilerFortran9.1Ia32Lib;C:Program Files (x86)Microsoft Visual Studio 8VCATLMFCLIB;C:Program Files (x86)Microsoft Visual Studio 8VCLIB;C:Program Files (x86)Microsoft Visual Studio 8VCPlatformSDKlib;C:Program Files (x86)Microsoft Visual Studio 8SDKv2.0lib;C:Program Files (x86)IntelCompilerFortran9.1IA32Lib

Obviously something is missing from this list. Which directory contains the LIBC that is supposed to be included for the flags I've set? LIBC appears in three directories after all these installations.

C:Program FilesMicrosoft Platform SDKLibAMD64

C:Program FilesMicrosoft Platform SDKLibIA64

C:Program Files (x86)MATLAB704syslcclib

If I add the third one to %LIB%, I get floods of link error messages, obviously wrong. If I add either of the first two to %LIB%, I get an error message like:

LIBC.lib(chkstk.obj) : fatal error LNK1112: module machine type 'x64' conflicts with target machine type 'X86'

So, I am back to the question in the earlier post, where do I get the right LIBC?

[mattsdad]

Originally,I was using a CD with a downloaded Fortran 9.1 installation that lacked the on CD instructions. The Xeon dual-dual core machine that I am developing for (and on) has recently had Visual Studio 2005 installed, but the resulting load module being producedis still a 32-bit executable. Given the pieces that I have identified and the Platform SDK which I did install, can I get to a 64-bit executable or do I need still another compiler download? If so, what?

BTW. I did get 32-bit OpenMP code operating on the Xeon with all four processors running at 100% while simulating multiple objects. But, due to a strange combination of conditions, the simulation stops running multiple threads after 40 seconds (about 2100 simulation seconds) and runs single threaded for 60 seconds (about 50 simulation seconds). I am trying to get the VTune Performance Analyzer and the Thread Profiler to work with the simulator to determine the cause, but they can't provide the information I needwhile runninga 32-bit executable.

[Steven_L_Intel1]

I'm uncertain after reading your latest post whether you have 9.1 or 9.0. I would highly recommend 9.1.

If you have 9.0 and want to use that with the PSDK, you'll want to install the compiler for "Intel EM64T". This should find the PSDK and set up command line support for it. You would then use the command line shortcut for "Build Environment for Intel EM64T-based applications" and commands such as "ifort" to build your application.

If you want to use VS2005, you need 9.1 and should download the latest one from the Intel Registration Center. This will recognize VS2005. To build 64-bit applications, go into Build..Configuration Manager and add a "new" x64 configuration. This process is described in the on-disk documentation for using the compiler in Visual Studio.