I am implementing a code (Iterative Matrix Solver : Language is Fortran) which basically works like this :
while (error < tolerance OR iterations < max iterations)
{
Few DGEMV
Few DDOT
Few DNRM
}
The problem is as follows :
1. The time required for solving increases erratically with the increase in number of threads.
I put it in a loop which works like (Not actual code)
export MKL_DYNAMIC=FALSE
for i=1 to 8
export MKL_NUM_THREADS=i
end
The CPU usage is in accordance with the number of threads at all times (For instance, 200% for 2 threads, 800% for 8 threads etc.) but the lowest time (and hence best performance) is observed at 2 Threads which is bothering me.
2. I am linking my program as :
ifort -fp-model source ParaQMR.f90 -xSSE3 -lmkl_blas95 -lmkl_intel -lmkl_intel_thread -lmkl_core -liomp5 -lpthread -m32 && time ./a.out
(I can attach the source code but it will be of no real use since the code reads data from a 2 GB text file which obviously I can't upload)
My hardware is Intel Xeon with 8 cores 8 threads. (32 bit Ubuntu 10.04 < I know 32 bit is a mistake but I doubt it will make a difference)
3. I tried VTune and although I am not a computer major or something, I tried my level best at interpreting the results and what I feel from the graph below is that most of the the time is spent creating and destroying threads each time a new iteration starts. But I am not so sure. Is there any way to circumvent this problem ?
I tried everything but nothing seems to work
I have tried :
1. Using my own Matrix-Vector Multiplication code. It suffers immensely when compared to MKL BLAS.
2. Intel Inspector. Didn't seem to make head or tail of what was happening.
3. Experiment with different flags, compiler (gfortran), gfortran and -fopenmp tag instead etc. Didn't work.
(By didn't work I mean, didn't produce the decrease in time as I expected)
Please help me out and let me know if I should upload more VTune Graphs.
Link Copied
