topic performance issue about pdgesv function in scalapack in Intel® oneAPI Math Kernel Library

performance issue about pdgesv function in scalapack

Hongda_H_ — Wed, 18 Dec 2013 04:23:34 GMT

I am suffering from a problem about pdgesv function in scalapack(C++).I have two computing nodes,each with 16 cores.The time spent solveing the linear Equation with pdgesv in two noodes is almost fifth times larger than running in single node using the same number of processes.

For example,A matrix is 1165*1165,and b matrix is 1165*30000.The job on single node with two processes takes almost 16 seconds while the job on two nodes each with one process takes more than 70 seconds.

I know the reason should be the communication between these two nodes,but why the gap is so big and how to narrow the gap?Could someone give me some suggestions?

This probably the expected

Gennady_F_Intel — Thu, 19 Dec 2013 09:45:00 GMT

This probably the expected gap in the case if 100Mbit Ethernet has been used. Can you give us the test and some additional configurations for checking how it will work on our side?

Quote:Gennady Fedorov (Intel)

Hongda_H_ — Fri, 20 Dec 2013 02:24:10 GMT

Gennady Fedorov (Intel) wrote:

This probably the expected gap in the case if 100Mbit Ethernet has been used. Can you give us the test and some additional configurations for checking how it will work on our side?

Thank you for your reply!! I think I have solved this problem.I just used intelmpi instead of mpich3,then this big gap disappeard.This may be the reason that intel mkl is more suitable to intelmpi.