Does MKL and PARDISO work properly with more than 64 cores?

[Nan_Deng]

This is a continuation of my old question "Does PARDISO have core access limit" with new information. I'll put a quick summary and the new question:

I run a code on Windows based HP DL980 server with 80 cores and 1TB RAM. Originally the OS is Windows Server 2008 SP2 (Which, I was informed, is based on Windows Vista). It seemed work properly.  The IS&T upgraded the OS to Windows Server 2008 R2 SP1 (Based on Windows 7). However, the code now can only access to a small portion of the cores (25%), the rest of the cores are set idle.

I posted my problem here and got help on how to manually set up the MKL access to the cores. After many rounds of trial and error on the machine, we finally were able to run on up to 50% of cores, even for multiple users. But there are still half of the cores set idle.

In one of HP's online documents it is stated: "The processor group concept was introduced in Windows 7 and Windows Server 2008 R2 to support more than 64 logical processors. Any machine with more than 64 logical processors has more than one processor group by necessity. Drivers and applications are all expected to run properly in the multi-group context if coded properly. " 

My question is: Are MKL routines, in particular PARDISO, aware of this "multi-group context" and can work properly with multiple processor groups, each with up to 64 cores? If yes, since which version? (I may need an upgrade then).

[TimP]

If you were able to control 80 cores in a single group, I suspect that may have been an HP-maintained modification of standard Windows Server.  Under a Windows which requires 2 groups to manage all the cores, I suspect you would need to run multiple processes, e.g. under MPI.  I would be interested if an Intel library team expert (MKL or MPI) could confirm this.

The lack of availability of a standard Windows for a group of more than 64 cores has been a disappointment.

[Noah_C_Intel]

Hi, I will research this capability with MKL Engineering and will get back to you ASAP.

[Noah_C_Intel]

MKL Engineering has done several experiments on windows server machines, but they have never tried that large amount of cores before. The consensus opinion is that the MKL Pardiso routine is not aware of this "multi-group context" while working in the Windows Server environment and has been verified to work only when all of the processors are considered via the hardware and/or OS to be on the same board. I believe beyond that context the problem would fall within the realm of Cluster Pardiso.

[Nan_Deng]

Thank you very much for the info.  Can you offer any suggestions on available short-cut or go-around to circumvent the problem? Does Intel Visual Fortran's libraries include Cluster Pardiso?  Do we need a separate upgrade for the intel produce? Or is it commerically available somewhere?  If we can get it from other sources, will this solver be compatible with Visual Fortran and can work under Windows Server? (I believe Win Server 2012 can support 256 cores now, so looks like this is the trend)  Any suggestion in this regard is highly appreciated.

[Noah_C_Intel]

Cluster Pardiso is in a private beta right now. I will put you in contact with my colleague that runs that program for more information.

[Nan_Deng]

That would be great. Thanks.

BTW, I checked with the original PARDISO site, there is a statement that says that the Intel's MKL PARDISO is based on the 2006 version, and not compatible with their latest development. But this statement appears also very old (2 years?). Has MKL PARDISO inplemented all the new refinement for the code since then?

[Gennady_F_Intel]

Deng, I will contact with you via private communication regard to Cluster Pardiso.

[Nan_Deng]

I still have the same core-number-limit problem unsolved and would like to have an update on the current situation:

(1) Some of our servers are updated to Windows Server 2012.  I heard that this new OS may resolve the core number limit problem.  Did Intel do any test run for MKL and PARDISO with this new OS?  Do I need to have any update for the libraries and recompile/modify the code? 

(2) I got one message from Mr. Fedorov regarding the test run of clustered PARDISO beta. I answered his question but haven't heard from him again.  I wonder if the clustered PARDISO is now ready to be released in a new version of the Fortran composer, or I can get it through another software license. It's OK if you want to send me a private message.

[Gennady_F_Intel]

version of Pardiso for distribute computations ( aka CPardiso ) has been released in official version of MKL v.11.2. This version we released 4 weeks ago. Have you tried this version?