Intel® Fortran Compiler
Build applications that can scale for the future with optimized code designed for Intel® Xeon® and compatible processors.
28381 Discussions

COMMON block - Multiple threads issue

Intel_C_Intel
Employee
810 Views
Hello,
I have a fortran dll (source code) that was developed by a third party. This dllhas a (huge) common block.
My managed c++ wrapper class for this dll is used from multiple threads - as a result the dll returns inconsistent results since the common block is overwritten.
Is there a way to avoid this problem ?
What is the standard approach to developed multithreaded dll's in fortran ?
Is there a way to write a function like DLLMain (as in C dll) - so an option would be to write multiple common blocks and assign then ID's in the thread attach part of DLLMain.
thank you.
vikrantca
0 Kudos
14 Replies
Jugoslav_Dujic
Valued Contributor II
810 Views
What is the standard approach to developed multithreaded dll's in fortran ?

Well, same as in C (or any other language) -- don't use global/static data :-).

Is there a way to write a function like DLLMain (as in C dll) - so an option would be to write multiple common blocks and assign then ID's in the thread attach part of DLLMain.

Yes; simply name it DllMain and supply appropriate ALIAS:
integer function DllMain(hModule, fdwReason, lpvReserved)
!DEC$ATTRIBUTES STDCALL, DECORATE, ALIAS:"DllMain":: DllMain
I don't see how you will "write multiple common blocks" though.

One option is to rewrite the common block into a derived TYPE, declare a variable of that TYPE in the Dll's main exported routine, and pass that variable through all argument-lists that require it.

A variation of the theme is to declare a variable of the TYPE to be a POINTER somewhere. Place that POINTER in a module and associate it with thread in DllMain using TLSAlloc, TLSSetValue, TLSGetValue APIs (IVF scalar POINTERs are just void*'s).

Both options require an extensive code rewrite though.

In any case, don't forget to compile with /recursive and /automatic switches.

HTH,
Jugoslav
0 Kudos
Jugoslav_Dujic
Valued Contributor II
810 Views
...ah, I missed the "3rd party dll" part.

In any case, the simple solution is to employ critical sections (InitializeCriticalSection/EnterCriticalSection/LeaveCriticalSection) which will effect in serialization of the calls rather than asynchronous access. It will slow down the access of course, but if the calculation is short and you can live with it...

Jugoslav
0 Kudos
Intel_C_Intel
Employee
810 Views

Thanks for the reply.

As you said - it will need significant code rewrite - thats what Iam afraid of.

Would using the open mp directive THREADPRIVATE help ? I can easily (?) add this attribute to the COMMON block.

thank you.
vikrantca
0 Kudos
Intel_C_Intel
Employee
810 Views

Thanks again for the reply.

The calculation part is not short. The common block is first initialized by reading around 20 different text files (coefficients). These coefficients are then used for some iterative calculations. Each thread needs a different set of coefficients to be initialized (at the beginning).

vikrantca

0 Kudos
Jugoslav_Dujic
Valued Contributor II
810 Views
THREADPRIVATE? Possibly. I missed OpenMP altoghether, but sorry, I'm not familiar with it. Probably someone else will jump in.

In any case, try it on a smaller example first before you attempt to do it on real code.

Jugoslav
0 Kudos
jim_dempsey
Beginner
810 Views
The following works for me:

type TypeThreadContext
SEQUENCE
type(TypeObject), pointer :: pObject
type(TypeTether), pointer :: pTether
type(TypeFSInput), pointer :: pFSInput
integer :: LastObjectLoaded
end type TypeThreadContext
type(TypeThreadContext) :: ThreadContext
COMMON /CONTEXT/ ThreadContext
!$OMP THREADPRIVATE(/CONTEXT/)
Put whatever you want in the user type TypeThreadContext
Then dereference by using ThreadContext.yourVariable here.
To keep from doing a major rewrite I use the preprocessor, FPP, and a set of defines
e.g.
#define FOO ThreadContext.FOO
or if need be...
#define FOO ThreadContext.privateFOO
It is a little bit of work but only in the source file with the common block.
Note, #define must start on column 1.
And you must enable preprocess file.
N.B.
In my application I converted COMMONs to user defined types and made them global context by way of modules. The documentation on how to use COMMON and THREADPRIVATE inside a module wasn't clear. With some experimentation I came up with the above technique.
Jim Dempsey

0 Kudos
jim_dempsey
Beginner
810 Views

If you do not have the source code to the DLL then it will be problematic to make it multi-threaded. You can call the DLL by proxy. e.g. if you have n threads then create n proxy processes. The proxy process opens a Pipe to receive any arguments. And uses another Pipe to return values. In the initialization code of your applicaiton you create process to get thest applications running (which will then read on the private pipe and wait for a "call"). Next during the run of your main multi-threaded application in place of making the call to the DLL you simply write to the thread dependent Pipe. At which point the other process runs and on completion it writes back (in a return pipe).

This might require little change to your code. If need be you can make a DLL wrapper that links staticly with your code. The wrapper will perform the Pipe "calls".

Piping will have more overhead than ThreadPrivate-ing the common but it may be easier to impliment and if the work in the DLL is substantial you may be worried about a fraction of a percent of overhead.

Jim Dempsey

0 Kudos
Intel_C_Intel
Employee
810 Views

Thanks Jim.
I do have the source code for the third pary DLL and I will try our your solution.
I have a question though - your code essentially creates a separate common block for each thread that attaches to the DLL, right ?

Also - as I metioned in the my first post, Iam using a c++ wrapper for the dll. When I create two instances of the cpp class on the same thread, I will still run into the problem with the common block, right ? In which case I need to write some sync code in my cpp wrapper.

thank you.
vikrantca

0 Kudos
jim_dempsey
Beginner
810 Views

The multiple copies of the common block are taken care of by the

!$OMP THREADPRIVATE(/CONTEXT/)

However, you may have additional problems. If your applicaiton is multi-threaded via OpenMP the ThreadPrivate common block (CONTEXT in this case) must be common to both the application and the DLL. Typicaly a DLL is written opaque. Thus you don't have shared data.

So the the ThreadPrivate may not be suitable for the DLL unless you can change the DLL (slightly) to take in as a calling arg a pointer to the context.

Each of your application's threads could allocate this context (to a pointer held in the ThreadPrivate area) and then pass the address of the context area to the DLL. This should work *** as long as the DLL avoids using static temporaries. To avoid this use the options for recursive and reentrant code generation. Also add ", automatic" to any local (small)arrays compiled.

real(8), automatic :: Vector(3)

Without the automatic sometimes the local array gets place into a static area (and gets clobbered by other threads).

Jim Dempsey

0 Kudos
jim_dempsey
Beginner
810 Views

>> Also - as I metioned in the my first post, Iam using a c++ wrapper for the dll. When I create two instances of the cpp class on the same thread, I will still run into the problem with the common block, right ? In which case I need to write some sync code in my cpp wrapper.<<

If the C++ constructor "lives" in global scope then the constructor execuites in your main thread.

If the C++ constructor is in the scope of a function while in the context of an OpenMP thread (or other thread) then in is on the stack (or allocated memory with pointer on the stack) and therefore it "lives" in the context of the thread (unless constructed to global pointer).

From my understanding of your description you will have multiple threads calling your C++ wrapper. Seeing that an application can be written to contain multiple non-OpenMP threads plus multiple OpenMP threads I suggest you restricty the C++ wrapper to be called only from OpenMP threads, or only from Win32 threads, but not both. Your wrapper needs to pass in a context pointer to the DLL (or the OpenMP thread team member number). The DLL will then either use the context pointer in lieu of it's former COMMON data, or the DLL can use the OpenMP thread team member number as an index into an array of what was formerly your COMMON data. Either the context is maintained by your application (when passing in pointer to context) or maintained by the DLL (when passing in the OpenMP thread team member number).

Think of it like a "this" pointer in C++

Jim Dempsey

0 Kudos
Intel_C_Intel
Employee
810 Views
Hello,
My application does not use open mp threads. I just want to use openmp threads for the dll itself. Would that work.
My applicationhas multiple win32/C# threads. Each of these threads creates an instance of the c++ wrapper class. This is the one that wraps the fortran dll. The c++ wrapper then calls LoadLibrary and then uses GetProcAddress to get function pointers. It was implemented this way due to the fact that the dll path is now known before hand.
Thank you for you help.
vikrantca
0 Kudos
Steven_L_Intel1
Employee
810 Views
If the DLL is not designed to be thread-safe, you cannot force it to be so by outside code. You must instead use synchronization code (mutex, etc.), to make sure that no more than one thread at a time is calling the DLL. You also have to worry about C and/or Fortran run-time DLLs referenced by the third-party DLL and whether those are the thread-safe versions.

You can use OpenMP to thread your own code, but again calls to the non-threaded DLL have to be synchronized.
0 Kudos
jim_dempsey
Beginner
810 Views

OpenMP has the requirement that OpenMP will manage it's threads. To do so it maintains context blocks under a requirement that there is only one Master thread. The Master thread is the Process's main thread. The documentation on OpenMP does not indicate the requirement that the Master Thread be the Process (application) main thread but in my limited use of OpenMP I have experienced problems when the OpenMP statments that function in context of OpenMP Master thread was not the Process (application) main thread.

From your description of what you want to do is you have a C# application with multiple threads (not by way of OpenMP). Any one or compination of which could call your DLL by way of the wrapper function you are implimenting. And said DLL itself creates and maintains it's own set of additional threads by way of OpenMP.

The problem with this is the DLL code, with the OpenMP, assumes it is entered in the context of the appliction main thread and that this thread is the only such thread entering the DLL. Once entered (successfuly) the DLL runs multi-threaded via OpenMP.

To overcome this I suggest you reconsider the multiple processes and the Pipe-in of the arguments/return values. Else you will have to abandon OpenMP in the DLL. Note, serialize the use of the DLL will not work when the DLL uses OpenMP as explained earlier there can only be but one Master thread context (not one at a time context).

An alternate way would be for the OpenMP DLL to launch and maintain the multiple C# threads. You still must replace the COMMONs with defined type and then create/maintain multiple instances of these defined types (recall from my earlier post my context was a list of pointers.

Jim Dempsey

0 Kudos
Intel_C_Intel
Employee
810 Views

Thanks Jim. The above said approach seems to be the way to go. I will try out some test code with the above approach.

thanks again.

vikrantca

0 Kudos
Reply