topic Greg, actually we have one in Intel® oneAPI Math Kernel Library

P?LAPIV - Issues with MPI I/O

Behm__Greg — Fri, 22 Dec 2017 04:18:48 GMT

Hello,

I have a MPI Fortran code where I am reading in a square Matrix A, a RHS matrix B, and a pivot array IPIV. I currently have been successful at reading in A and B with MPI I/O, solving the linear system, and writing them out with MPI I/O with any number of cores/nodes. Now what I need to do next is pre-pivot (using PZLAPIV) the matrices prior to factoring/solving them. I struggle with using PZLAPIV, there are no examples online as this is usually called internally by MKL or other similar libraries. Note, I do have it working with using PZLAPIV but only when MPI rank = 1 (not a solution), but when I increase the rank to 2+ the pivoting gets messed up some how. The documentation in the PZLAPIV function states:

"The pivot vector may be distributed across a process row or a column. The pivot vector should be aligned with the distributed matrix A."

I think this might be where I am losing critical information. The creation and distribution of IPIV over a p by q grid that aligns with A might be where I made some mistakes. The reason is the subroutine I wrote for MPI I/O to read in a matrix might not be what I need to read in the pivot array. Perhaps when I read in a vector I shouldn't be using MPI_Type_create_darray.

I currently do not have a way to get my code online to aide in debugging it, but I will be incredibly thankful if anyone can work with me. I will try and get pieces of the code that pertains to inquires you may have.

Thank you,

Greg

Greg, actually we have one

Gennady_F_Intel — Fri, 22 Dec 2017 15:49:26 GMT

Greg, actually we have one ScaLapack example ( "pzgetrfx.f ") which uses PZLAPIV. Please take at look at this example. Will that help?

I am not able to find that

Behm__Greg — Fri, 22 Dec 2017 18:47:58 GMT

I am not able to find that example, did you mean to spell it with an "x"? Could you provide a link to the page you're looking at.

I am using http://www.netlib.org/scalapack/explore-html/d9/d92/pzgetrs_8f_source.html for my reference. I do basically everything they do, but I do replace the IA, JA, IB, JB entries with a fixed number "1". Would that be an issue? The issue I believe is my inability to read in the pivot array using MPI-IO darray type. It mentions that reading a vector in MPI IO darray requires the pdims(1) or pdims(2) to be one. That might be why the pivoting is not working.

I need help creating/feeding in a matrix (distribute it) and have MPI-IO read in a binary file Nx1 sized vector. If I don't have to use MPI-IO I don't see this issue. So, I believe what I am seeing is MPI-IO. Are there any examples of MPI-IO using PZLAPIV?

My question is how to read MPI-IO fortran unformatted data into a IPIV structure?

Thanks.