BLACS examples are not working

Yeongha_L_ · ‎04-26-2017

hi, i want to use BLACS so i tested HELLO example code in here.

http://www.netlib.org/blacs/BLACS/Examples.html#HELLO

it seems to easy, but it was not working. i tried to check why, but it was failed.

(actually, i tried C++ version together, but it also has same problem. c++ code is this.

https://andyspiros.wordpress.com/2011/07/08/an-example-of-blacs-with-c/ )

my fortran code is this.

      PROGRAM HELLO 
*     -- BLACS example code --
*     Written by Clint Whaley 7/26/94 
*     Performs a simple check-in type hello world 
*     .. 
*     .. External Functions ..
      INTEGER BLACS_PNUM
      EXTERNAL BLACS_PNUM 
*     .. 
*     .. Variable Declaration ..
      INTEGER CONTXT, IAM, NPROCS, NPROW, NPCOL, MYPROW, MYPCOL
      INTEGER ICALLER, I, J, HISROW, HISCOL 
*     
*     Determine my process number and the number of processes in 
*     machine 
*     
      WRITE(*,*) '!' 
      CALL BLACS_PINFO(IAM, NPROCS) 
*     
*     If in PVM, create virtual machine if it doesn't exist 
*     
      IF (NPROCS .LT. 1) THEN
         IF (IAM .EQ. 0) THEN
            WRITE(*, 1000)
            READ(*, 2000) NPROCS
         END IF
         CALL BLACS_SETUP(IAM, NPROCS)
      END IF 
*     
      WRITE(*,*) '@' 
*     Set up process grid that is as close to square as possible 
*     
      NPROW = INT( SQRT( REAL(NPROCS) ) )
      NPCOL = NPROCS / NPROW 
*     
*     Get default system context, and define grid

*     
      CALL BLACS_GET(0, 0, CONTXT)
      CALL BLACS_GRIDINIT(CONTXT, 'Row', NPROW, NPCOL)
      CALL BLACS_GRIDINFO(CONTXT, NPROW, NPCOL, MYPROW, MYPCOL) 
*     
      WRITE(*,*) '#' 
*     If I'm not in grid, go to end of program 
*     
      IF ( (MYPROW.GE.NPROW) .OR. (MYPCOL.GE.NPCOL) ) GOTO 30

*     
*     Get my process ID from my grid coordinates 
*     
      ICALLER = BLACS_PNUM(CONTXT, MYPROW, MYPCOL) 
*     
*     If I am process {0,0}, receive check-in messages from 
*     all nodes 
*     
      WRITE(*,*) '$' 
      IF ( (MYPROW.EQ.0) .AND. (MYPCOL.EQ.0) ) THEN
            
         WRITE(*,*) ' '

         DO 20 I = 0, NPROW-1
            DO 10 J = 0, NPCOL-1
      
               IF ( (I.NE.0) .OR. (J.NE.0) ) THEN
                  CALL IGERV2D(CONTXT, 1, 1, ICALLER, 1, I, J)
               END IF 
*     
*              Make sure ICALLER is where we think in process grid

*     
              CALL BLACS_PCOORD(CONTXT, ICALLER, HISROW, HISCOL)
              IF ( (HISROW.NE.I) .OR. (HISCOL.NE.J) ) THEN
                 WRITE(*,*) 'Grid error!  Halting . . .'

                 STOP
              END IF
              WRITE(*, 3000) I, J, ICALLER


      
10         CONTINUE 
20      CONTINUE
        WRITE(*,*) ' '
        WRITE(*,*) 'All processes checked in.  Run finished.' 
*     
*     All processes but {0,0} send process ID as a check-in


*     
      ELSE

         CALL IGESD2D(CONTXT, 1, 1, ICALLER, 1, 0, 0)
      END IF

     
30    CONTINUE
              


      CALL BLACS_EXIT(0)

1000  FORMAT('How many processes in machine?') 
2000  FORMAT(I) 
3000  FORMAT('Process {',i2,',',i2,'} (node number =',I,
     $       ') has checked in.')
 
      STOP
      END

compile command is this.

$ mpiifort hello.f -mkl -lmkl_scalapack_lp64 -lmkl_blacs_intelmpi_ilp64

$ mpirun -n 8 ./a.out

error is this

[blacs_example]$ mpirun -n 8 ./a.out
 !
 !
 !
 !
 !
 !
 !
 !
 @
 @
 @
 @
 @
 @
Fatal error in PMPI_Comm_group: Invalid communicator, error stack:
PMPI_Comm_group(179): MPI_Comm_group(comm=0x2e, group=0x7fff39f2d2e0) failed
PMPI_Comm_group(133): Invalid communicator
Fatal error in PMPI_Comm_group: Invalid communicator, error stack:
PMPI_Comm_group(179): MPI_Comm_group(comm=0x88c9f740, group=0x7ffe88c9f3e0) failed
PMPI_Comm_group(133): Invalid communicator
Fatal error in PMPI_Comm_group: Invalid communicator, error stack:
PMPI_Comm_group(179): MPI_Comm_group(comm=0x259c5904, group=0x7fff259c55e0) failed
PMPI_Comm_group(133): Invalid communicator
Fatal error in PMPI_Comm_group: Invalid communicator, error stack:
PMPI_Comm_group(179): MPI_Comm_group(comm=0x3ff, group=0x7ffc216c03e0) failed
PMPI_Comm_group(133): Invalid communicator
Fatal error in PMPI_Comm_group: Invalid communicator, error stack:
PMPI_Comm_group(179): MPI_Comm_group(comm=0x0, group=0x7fffdec349e0) failed
PMPI_Comm_group(133): Invalid communicator
Fatal error in PMPI_Comm_group: Invalid communicator, error stack:
PMPI_Comm_group(179): MPI_Comm_group(comm=0x18bdf740, group=0x7ffe18bdf3e0) failed
PMPI_Comm_group(133): Invalid communicator
 @
 @
Fatal error in PMPI_Comm_group: Invalid communicator, error stack:
PMPI_Comm_group(179): MPI_Comm_group(comm=0x3f, group=0x7ffe09bceae0) failed
PMPI_Comm_group(133): Invalid communicator
Fatal error in PMPI_Comm_group: Invalid communicator, error stack:
PMPI_Comm_group(179): MPI_Comm_group(comm=0x0, group=0x7ffd1e8069e0) failed
PMPI_Comm_group(133): Invalid communicator

please tell me why these are not working...

thank you

Gennady_F_Intel · ‎04-26-2017

you mixed lp64 & ilp64 libs ( -lmkl_scalapack_lp64 -lmkl_blacs_intelmpi_ilp64 ). How to link - pls have a look at the mkl linker adviser - https://software.intel.com/en-us/articles/intel-mkl-link-line-advisor

Yeongha_L_ · ‎04-26-2017

Gennady F. (Intel) wrote:

you mixed lp64 & ilp64 libs ( -lmkl_scalapack_lp64 -lmkl_blacs_intelmpi_ilp64 ). How to link - pls have a look at the mkl linker adviser - https://software.intel.com/en-us/articles/intel-mkl-link-line-advisor

thank you, now it's working, :)