No OpenMP in VS2005

[stefanborries]

Hello everybody,

it may be a Microsoft problem but since it also involves the Intel Fortran Compiler I will give it a try.
In VS2005 I've told the preprocessor to enable conditional compiling for OpenMP and set Process OpenMP directives in language to 'generate parallel code'.
I do a little dummy loop i.e.

---

include "omp_lib.h"
logical :: ndyn = .true.
integer :: ntasks = 10
integer inth
logical igdyn
dimension itest2(100)
dimension itest3(100)
integer i8
inth=100
igdyn=.false.
i8=1
call omp_set_dynamic(ndyn)
call omp_set_num_threads(ntasks)
!$omp parallel
!$omp master
inth = omp_get_num_threads()
igdyn = omp_get_dynamic()
!$omp end master
!$omp do REDUCTION(+:i8)
do 137 i=0,100
i8=i8+3
!$ itest2(i)=OMP_GET_THREAD_NUM()
itest3(i)=i8
137continue
!$omp end do
!$omp end parallel
print *, 'itest2 = ', itest2, 'num_threads = ', inth, 'i8=',i8,
+'itest3=',itest3
end

---

As far as I know inth should be 10 and in itest2() there should be numbers from 0 to 9.
BUT itest1 equals 1 and in itest2() there are only zeros (which should be the master thread).
When I compile the code with the intel compiler (ifort /Qopenmp test.f) I'm told that the Region has been parallelised and when I start the program everything is fine. itest2() contains the numbers from 0 to 9 and so on.
Any help or hints will be appreciated.
Thanks in advance,
Stefan

[jimdempseyatthecove]

Stefan,

Try the following

On the !$omp parallel add DEFAULT(SHARED)

On the !$omp do add PRIVATE(i)

If that fails check to see if the outer most level of your code that used OpenMP contains

USE OMP_LIB

If not, then you might be using the NO-OP stub routines.

Jim Dempsey

[stefanborries]

Dear Jim,

first of all thanks for the time you spend taking a look at my problem.

I tried the adding of the DEFAULT and the setting of PRIVATE(i) but to no effect.
I have a C-function which calls my Fortran subroutine. Inside of this C-function OpenMP works perfectly well. I can create as many threads as I wish.
I've also tried 'use OMP_LIB' but to no effect.

By the way, aren't variables defined and initialised outside of the parallel region by default shared and the loop-control variable by default private?
'use OMP_LIB' conflicts with '!$ include "omp_lib.h" so I can only use one of the two, but since it makes no difference in the result...

At last I'm not sure with what you mean that I may be using a NO-OP stub routine.
Since I use no optimisation the loop should still be performed.
And I have mentioned in my post, that the loop works perfectly when I compile it on the command line with the command 'ifort /Qopenmp test.f' (where test.f is just a dummy name).

Stefan

[jimdempseyatthecove]

Stefan,

MS C/C++ OpenMP runtime library is different from IVF OpenMP runtime library. This is probably an initialization problem.

Try wrapping your C/C++ main with an IVF Program shellthat has at least one parallel section. This will get IVF through it's OpenMP initialization and then hopefully MS C/C++ OpenMP initialization will succeed as well.

program MyCshell
use omp_lib
integer :: dummy
!$OMP PARALLEL
dummy = OMP_GET_THREAD_NUM()
!$OMP END PARALLEL
call YourCprogram()
end program MyCshell

If that works then try stripping out the unnecessary stuff.

Note, you may encounter problems or at least unexpected results from calls to either runtime system library or some !$OMP constructs.

Examples:

What should IVF OMP_IN_PARALLELreturn if called from a C OpenMP thread that is not it's master thread?

What should!$OMP MASTER do if called from a C OpenMP thread that is not it's master thread?

When the IVF OpenMP state indicates that it is not in a parallel section and is called from a C OpenMP thread that is not the master thread in the C OpenMP threading system how should the IVF OpenMP handle ThreadPrivate data?

There are numerous other control conflicts. The problems observed in IVF may have similar problems observed from the C OpenMP side too.

If you write mixed mode multi-threaded code it is generally recommended to have one of the modes control the threads. Or, be very careful about assumption made.

When I write mixed mode multi-threaded code I do not rely on the OMP_GET_THREAD_NUM() as it is not a thread number but a thread team member number (0:n) of the current thread team. (If you have nested threads in IVF you will also encounter multiple threads with the same OMP_GET_THREAD_NUM() number). Instead I make a Win32 call to get the Windows Thread ID number and use that to disambiguate the threads in either a nested IVF system or in a mixed language multi-threaded system.

Jim Dempsey

[stefanborries]

Jim,

thank for the hints, I'll try to make it work.
What really concerns me is that everything went well when I used the singlecore Notebokk with WindowsXP(32-bit) and now, that I use a dualcore Desktop-PC with WindowsXP(64-bit) it it's not working any longer.
Could the switching between these two systems have made a difference?

Stefan

[jimdempseyatthecove]

Stefan,

I do not think it is a 32-bit to 64-bit issue but rather going from single core to dual core exposed a threading problem that was not encounterable with a single core. In particular look at the code thatmodifiesshared data. Instructions that are atomic with respect to interrupts are not necessarily atomic with respect to multiple processors/cores.

add dword ptr [eax], edx

This is a Read/Modify/Write operation. This may be the final instruction sequenceof "I=I+J" which is atomic with respect to interrupts but is not atomic with respect to multiple processor/core access to the same variable. This instruction is protected in a multi processor/core systemusing the LOCK prefix

lock add dword ptr [eax], edx

and the lock instruction prefix is used in the Interlockedxxxxxxx series of function calls or !$ATOMIC directives (or C OpenMPatomic directives).

Lack of attention to these details will/may produce errors. Or worse the errors do not show up on your development and test systems but show up on the customer production system.

Jim Dempsey

[stefanborries]

Jim,

thanks for the tip.
I know that there is a difference but I must admit for the moment it didn't came to my mind (thanks for the reminder).
What you've said is true but I don't think it has something to do with my actual problem.
I know I should be careful, when I deal with a multicore system when it comes to atomic instructions but my problem as far as I see it isn't a atomic instruction it's the fact that Visual Studio and the Fortran Compiler, together with OpenMP do not create threads. And I don't know where the problem is or what the solution could be.

It seems as if the calls to OpenMP functions for setting OpenMP environment variables do not work correctly and neither does the directive !$OMP PARALLEL NUM_THREADS(anynumber), which should create anynumber of threads or at least more than none.

Shouldn't C/C++ have the same problem?
I've tried the same testcode (or equal as possible) in C/C++ and the number of threads I want were created but not when I try to create the threads in fortran, where I only get the master thread.

The more I think about it, the more I get the impression I should may reinstall Visual Studio and the Fortran Compiler and give it a fresh start.

Any other ideas? Or shall I give the old 'have you tried to reinstall it' a try.

Stefan

EDIT:
Alright,
some news. I've tried a sample code I took from the internet and put it in a new solution in VS2005. And what can I say. It works. Multiple threads were created by the sample fortran code.
So it seems the problem is with my solution. I'll have to take a closer look to it. At the moment I cannot find any differences in the property settings but I'll narrow the problem down. Step by step.
Which does not mean that I would not appreciate any further help or hints. :-)

[jimdempseyatthecove]

Stefan,

When you link a C/C++ program (main) you get its vendors initialization code (put break on 1st statement in main then look at stack).

When you link IVF program you get IVFs initialization code (put break on 1st statement in program then look at stack).

Compare call trees

It sounds like there is a dependency on IVF OpenMP initialization code which is not run when you link with C++ main.

Does a simple IVF (one file)program with OpenMP run multi-threaded?

If so, convert that IVF program to subroutine then call from C++ dummy main (without C++ OpenMP).

If does not run then there is a run time system initialization problem. You might be able to fix this if you can figure out the name of the IVF OpenMP runtime system initialization subroutine name and call that from your C++ initialization code or from your IVF initialization code.

If the 1st test does run, then modify the C++ main to include OpenMP and to start a parallel section _before_ it calls the IVF subroutine. See what happens.

If that fails, move the C++ parallel code to follow the IVF subroutine call. i.e. see if it is a who initialized first situation.

If the two OpenMP systems are independent then it should not matter about calling sequence. But you may have an issue in regards to which C++ thread calls into the IVF OpenMP domain.

If the two OpenMP systems are conflicting (e.g. use same function names) then expect problematic execution.

If the two OpenMP systems are integrated (e.g. use the same libraries) then check to see if either or both support NESTED provided that is what you want.

Jim Dempsey

[stefanborries]

Jim,

thanks for the extensive reply.

• a simple IVF (one file)program with OpenMP runs multi-threaded
• I've tried to turn the one file program into a subroutine(created a static library) and call it from a c++ file but I get unresolved external symbols to (what seems to me) OpenMP (ie ___kmpc_global_thread_num or ___kmpc_ok_to_fork)
  
• Comparing the call trees showed no difference
• I've already discovered that there is a difference between include "omp_lib.h" and USE OMP_LIB though I don't know why (but I think include just inserts the lines while USE does a little bit more
• I've discovered that in my huge project there is a difference between calling on omp_set_num_threads() and !$OMP PARALLEL NUM_THREADS(number) maybe that's why I wasn't able to create threads because the NUM_THREADS() clause creates just the amount of threads I want but when I try a call inside the parallel region (see below) I get the value of 1 and dynamic creation and nesting
  

At the end of this code dyn is 1, nest is 1 and all elements of inum are 0 (master thread)
Any ideas?

---

integer inum
dimension inum(100)
call omp_set_dynamic(.FALSE.)
call omp_set_nested(.FALSE.)
!$OMP PARALLEL NUM_THREADS(100)
!$OMP MASTER
dyn=omp_get_dynamic()
nest=omp_get_nested()
!$OMP END MASTER
!$OMP DO
do 123 i=0,100
inum(i)=omp_get_thread_num()
123 enddo
!$OMP END DO
!$OMP END PARALLEL

---

[Steven_L_Intel1]

I am not an OpenMP expert, but I can tell you a few things.

1. There is no difference between including omp_lib.h and using the module, at least as far as how OpenMP works in your program
2. If you have a C++ main program, you need to add the path to the Fortran LIB folder under Tools > Options > Projects > VC++ Directories > Library Files
3. The Fortran code has to be compiled with /QopenMP
4. The Fortran library project should have the property Libraries > Disable Default Library Search Rules set to No.

[stefanborries]

Thanks for the reply Steve,

1. there seems to be a difference at least in my project (when I include the "omp_lib.h" I'm told that the files are already defined in VCOMP.lib, which is true, since I have switched OpenMP support in my c++-project to 'yes')
   
2. done (but is it really necessary? because I linked them to the c++project)
   
3. done
4. done

points 2 to 4 made no difference (in fact 3 and 4 where already as you suggested)

which leaves me still to the problem that there seems to be a difference between using omp_set_num_threads() and the clause NUM_THREADS(), again at least in my project.
Any further help or hints will be appreciated.

[stefanborries]

Here we go again...

ok, I've figured out why NUM_THREADS() works and omp_set_num_thread not.
There seems to be a conflict between the inclusion of omp.h in c and use omp_lib.

---

c-file (openmp enabled)
#include
//no call to openmp yet
...c-code

---

f-file (openmp enabled)
use omp_lib
!$ call omp_set_num_threads(number)
...f-code
!$ OMP inum=omp_get_num_thread()
...f-code

---

The result is I don't get my threads.
BUT

---

f-file (openmp enabled)
C use omp_lib
external omp_get_num_thread
integer omp_get_num_thread
external omp_set_num_thread
!$ call omp_set_num_threads
...f-code
!$OMP inum=omp_get_num_thread()
...f-code

---

with this I get my number of threads.

Second case:

---

c-file (openmp disabled)
//#include
//no call to openmp yet
...c-code

---

f-file (openmp enabled)
use omp_lib
!$ call omp_set_num_threads(number)
...f-code
!$ OMP inum=omp_get_num_thread()
...f-code

---

works perfectly fine (meaning I get my number of threads).
This is a little embarrassing because I may want to add openMP directives to my c-code in the future so I'll have to enable it.
Any thoughts or hints to a solution will as always be appreciated.
Thanks in advance.
I hope I'm not to blind at the moment to see that it may be an easy to solve problem ;-)

[Steven_L_Intel1]

Are you perhaps using Microsoft's OpenMP library from C and Intel's from Fortran? That could well pose a problem. In a future release we may offer the option of using Microsoft's OpenMP library from Fortran.

[stefanborries]

Hello Steve,

thanks for the reply.
It seems I'm using the Microsoft OpenMP library from c and Intel's from Fortran.

[TimP]

When you mix ifort with non-Intel C, only one of the languages can use OpenMP. In a similar way, if you want to use MKL when you have OpenMP in code compiled by a compiler other than Intel, you can use only the MKL lib_serial at present.

[stefanborries]

I hate to disargee (because I think you are knowing it better than me) but it seems to me I can use OpenMP in Fortran (with the Intel compiler) even if I use OpenMP in c (with the Microsoft compiler).
The thing is I just can't use OpenMP in Fortran by using use omp_lib or include "omp_lib.h".
When I declare the functions by myself. OpenMP works fine.
i.e.
instead of:

---

use omp_lib
!$OMP PARALLEL NUM_THREADS(100)
!$OMP MASTER
tcount=omp_get_num_threads()
!$OMP END MASTER
...
!$OMP END PARALLEL
...

---

I use at the moment:

---

external omp_get_num_threads
integer omp_get_num_threads
!$OMP PARALLEL NUM_THREADS(100)
!$OMP MASTER
tcount=omp_get_num_threads()
!$OMP END MASTER
...
!$OMP END PARALLEL
...

---

At least with this piece of code tcount is 100 as it should be.
With this small program I tried to take a look if different threads were created and worked as intended. (it was just a quick and dirty test to see if everything worked as hoped so don't worry about the messy code)
f-file (in a static lib):
subroutine mysub(ntasks)
c use omp_lib
include "omp_lib.h"
c it doesn't matter if I include the include or use use
logical :: ndyn = .true.
integer ntasks
integer inth
integer itest
logical igdyn
integer itest1
dimension itest2(1000)
dimension itest3(1000)
c if I want the code to be parallelised I have to put these declarations back into the code
c external omp_set_dynamic
c external omp_get_num_threads
c integer omp_get_num_threads
c external omp_set_num_threads
c external omp_get_thread_num
c integer omp_get_thread_num
integer i8
inth=100
igdyn=.false.
i8=1
call OMP_SET_DYNAMIC(ndyn)
call OMP_SET_NUM_THREADS(ntasks)
!$OMP PARALLEL
!$omp master
inth = OMP_GET_NUM_THREADS()
igdyn = OMP_GET_DYNAMIC()
!$omp end master
!$omp do REDUCTION(+:i8)
do 137 i=1,1000
i8=i8+3
!$ itest2(i)=OMP_GET_THREAD_NUM()
itest3(i)=i8
137 continue
!$omp end do
!$omp end parallel
print *, 'itest2 = ', itest2, 'num_threads = ', inth, 'i8=',i8,
+'itest3=',itest3
i8=1
return
end

c-file (in a static lib):
#include "czwischen.h"
void c zwischenSUB(int *value)
{
MYSUB(value);
}
czwischen.h:
#ifndef CZWISCHEN_H
#define CZWISCHEN_H
#include
extern "C" void MYSUB(int *ntasks);
void czwischenSUB(int *value);
#endif

c-file (which calls the hole stuff):
#include "stdafx.h"

int _tmain(int argc, _TCHAR* argv[])
{
int ntasks=100;
czwischenSUB(&ntasks);
return 0;
}

[jimdempseyatthecove]

What it looks like is the "use omp_lib" brings in additional library modules from those your program is referencing. My guess is the omp_lib has a !DEC$ with an ALIAS for a function/subroutine name and when used, it loads the Intel IVF library (with conflicts with MS OpenMP library).

When you use the external I guess only the MS OpenMP library is loaded.

Maybe you can get them both to use one library. But I do not think you will have the same thread pools so you may have some nasty issues.

Jim Dempsey

[Steven_L_Intel1]

For Intel Fortran, there's no difference between including omp_lib.h and USE OMP_LIB. It's the same code, and there's no LIB directives.

[stefanborries]

When I take a look at the names of the threads VS2005 displays there are major differences.
When I use only Fortran they hava the form of '__kmp...' or something like that (I don't recall it at the moment since my workstation isn't fully operational at the moment).
When I do the mix and define my own externals they are called 'WinThread' or something like that.
So it seems you're right Jim.
Thanks for the help, I'll try my best to make it work as intended and will give my experience in it to the board.