howdy all.
found my ifort/openmp code was yielding widely variable results where run in parallel, but gave consistent (and seemingly correct) results where run on a single processor.
here is the offending section of code:
TwoPiLx = 2.*pi/Lx
csum1 = 0.
ssum1 = 0.
csum2 = 0.
ssum2 = 0.
!$OMP PARALLEL DO DEFAULT(SHARED) PRIVATE(i,DV,DVpart,nny,nnx,rnnx,rnny,K_dot_x,COSkx,SINkx) REDUCTION(+:csum1,ssum1,csum2,ssum2,csum3,ssum3,csum4,ssum4)
DO i=1,n
DV = mass(i)/dens(i)
DVpart(1) = DV*v(1,i)
DVpart(2) = DV*v(2,i)
DO nny = -modes, modes
rnny = REAL(nny)
DO nnx = 0, CEILING(SQRT(REAL(maxmode**2,knd) - rnny**2))
rnnx = REAL(nnx)
K_dot_x = TwoPiLx*(rnnx*x(1,i) + rnny*x(2,i))
COSkx = COS(-K_dot_x)
SINkx = SIN(-K_dot_x)
csum1(nnx,nny) = csum1(nnx,nny) + DVpart(1)*COSkx
ssum1(nnx,nny) = ssum1(nnx,nny) + DVpart(1)*SINkx
csum2(nnx,nny) = csum2(nnx,nny) + DVpart(2)*COSkx
ssum2(nnx,nny) = ssum2(nnx,nny) + DVpart(2)*SINkx
ENDDO
ENDDO
ENDDO
!$OMP END PARALLEL DO
the above gives incorrect and inconsistent results (for csum1, ssum1, csum2, ssum2) when run over more than a single processor, implying perhaps that private variables are not remaining private.
interestingly, it may be fixed by replacing the two component array DVpart(:) by two individual variables, say DVpart1 and DVpart2.
the other solution is to remove the "-ipo" option from the compile options:
-r8 -xT -O3 -static -par-threshold4 -vec-report3 -ipo -parallel -openmp
this seems like a bug to me, but perhaps i'm missing something.
other information that may or may not be relevent:
op: ubuntu gutsy 7.10 2.6.22-12-generic x86_64
ifort version: 10.0.023_intel64
