Issue with multiple cores

[João_A_]

Hello everybody!

I've a little problem and I kindly request your help.

I'm running a finite element analysis using abaqus software (a commercial software). Abaqus allows users to create their own subroutines and couple them into abaqus black box solvers to customize your model.

Now I have several subroutines in one fortran file. Some are native to abaqus (that is, subroutines that use abaqus global variables and link my code to abaqus black box) while others are called from these native ones to make some accessory calculations.

To share information between subroutines I used global variables specified in a module which every subroutine uses.

Now, abaqus starts the analysis and calls wach native subroutine in a specific order. If I run it using a single core of my cpus it all goes well but if I run it using multiple cpus I get a problem.

First, I run multiple cpus using a functionality provided by abaqus in which the user only has to specify the number of cores in the DOS command and abaqus will handle the rest for you.

Having this in mind, my analysis consists in a incremental analysis, in which in every increment the subroutines are called until a given convergence criteria is met and the analysis can proceed to the next increment. In may case, as in general, the following increment depends on the results obtained in the last converged increment. Now, my problem is that with multiple cpus abaqus uses each core to solve a specific set of equations, which I'ce came to the conclusion can change between increments (that is in increment i core j can solve equations 1 to 10, but in increment i+1 core j can solve equations 300 to 450 for example), and this may mean that for a specfic core the analysis may already be at increment i+1 without having solved (or at least saved the results) for increment i. This is a big issue because my analysis need information to pass along in a continuous way an dif it is not available at the required time then the results may be wrong....

I post this issue here to see if anyone has has similar experiences or knows a work around or give me an idea to solve this. Basically the problem seems to be that using multiple cores, one core may request information which is yet not available and that can influence the way the rest of the analysis works. Information is not yet available because one other core has not yet passed through the relevant code at the time the one other core passes through another code piece which feeds from the missing information.

Hope I made my problem clear.....

Many thanks,

Joao

[jimdempseyatthecove · Accepted solution]

>>To share information between subroutines I used global variables specified in a module which every subroutine uses

If this information is varying (not static data tables) then the multiple threads may be writing to the same variables, thus corrupting the data from the other threads. You would also have a similar situation if these subroutines use SAVE variables or if they contain local arrays and are not RECURSIVE or compiled with -openmp or other options that force local arrays to be on stack or allocated.

To correct for this, each thread either

a) maintain and pass all pertinent state variables on CALL
b) define a TYPE to contain the state and pass the state type on call
c) Use THREADPRIVATE for the thread global variables

Option c) might be the better choice.

Jim Dempsey

[João_A_]

Many thanks for the valuable information and for your suggestions! I'll give them a try! Thanks once again!