Error: Symbol ‘mpi_gather’ referenced at (1) not found in module ‘mpi’

[JaredFrazier]

When compiling the following dummy program with mpif90,

! @file test_mpi.f90
program test_mpi
  use mpi, only: mpi_gather, mpi_init, mpi_finalize
end program

I get the error:

Error: Symbol ‘mpi_gather’ referenced at (1) not found in module ‘mpi’

Here is how I compile this file and here is information regarding my MPI installation:

 . /opt/intel/oneapi/setvars.sh 

which mpif90
# /opt/intel/oneapi/mpi/2021.14/bin/mpif90

mpif90 --version
# GNU Fortran (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0
# Copyright (C) 2023 Free Software Foundation, Inc.
# This is free software; see the source for copying conditions.  There is NO
# warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

mpif90 test_mpi.f90

Why might this error occur? 

The error can actually be easily resolved but just not including MPI_Gather in the list of routines following the ONLY keyword, but I am interested in the why here since the solution is trivial though a bit puzzling. 

I have also noticed that the ONLY keyword does not seem to affect what is available in the program's scope. The below program uses routines like MPI_Init that have not be explicitly brought into scope by the ONLY keyword. Any insight on why this is would also be useful since I find it a bit odd.

program mpi_send_example
    use mpi, only: MPI_INTEGER, MPI_COMM_WORLD, MPI_STATUS_IGNORE
    implicit none

    integer :: ierr, rank, size
    integer :: tag, source, dest
    integer :: message

    ! Initialize MPI
    call MPI_Init(ierr)
    call MPI_Comm_rank(MPI_COMM_WORLD, rank, ierr)
    call MPI_Comm_size(MPI_COMM_WORLD, size, ierr)

    tag = 0
    source = 0
    dest = 1

    if (size < 2) then
        if (rank == 0) print *, "This program needs at least 2 MPI processes."
        call MPI_Finalize(ierr)
        stop
    end if

    if (rank == source) then
        message = 42
        print *, "Rank", rank, "sending message:", message, "to rank", dest
        call MPI_Send(message, 1, MPI_INTEGER, dest, tag, MPI_COMM_WORLD, ierr)

    else if (rank == dest) then
        call MPI_Recv(message, 1, MPI_INTEGER, source, tag, MPI_COMM_WORLD, MPI_STATUS_IGNORE, ierr)
        print *, "Rank", rank, "received message:", message, "from rank", source
    end if

    call MPI_Finalize(ierr)
end program mpi_send_example

Note that I do not encounter the MPI_GATHER error when using OpenMPI (built either with intel or GNU compilers). E.g.,

# in a different shell so that the intel set vars is no longer true
which mpif90
# /usr/bin/mpif90

mpif90 --version
# GNU Fortran (Ubuntu 13.3.0-6ubuntu2~24.04) 13.3.0
# Copyright (C) 2023 Free Software Foundation, Inc.
# This is free software; see the source for copying conditions.  There is NO
# warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

# compiles without errors
mpif90 test_mpi.f90

For reference, I am using Ubuntu 24.04 LTS.

Thanks!

Jared 

[TobiasK · Accepted solution]

The MPI module is not required to provide explicit interfaces for mpi_gather and other MPI functions:
https://www.mpi-forum.org/docs/mpi-3.1/mpi31-report/node410.htm#Node410

Note: the MPI module is anyway deprecated, please switch to mpi_f08 if you need that type of support.

[JaredFrazier]

For completeness, I did also ask this question on the Fortran language forums to get more details. There are some subtleties that the users over there mention that might be useful to others who run across this intel post.

https://fortran-lang.discourse.group/t/all-mpi-routines-available-even-when-not-explicitly-included-in-the-only-args-list/10443

[TobiasK]

Oh, there are some quite dangerous misunderstandings in that thread.

Of course Intel MPI supports mpi_gather and so on...

The problem is just that you think everything is defined in the MPI module or in mpi.h / mpi.f which is simply not true, the standard does not require that all functions are defined in the module. Please see what the standard defines in the link I provided to you. If you need explicit interfaces, you need to use mpi_f08.

So to be a bit pedantic here, the symbols for mpi_gather, etc are not part of the module but only defined in the library. The module does not provide the functions. Even mpi_f08 does not provide the functions but provides an interface to the functions.

In the end, you are calling an external library,  e.g. the linker must be able to find the symbols but that's all.

What is provided in the module / mpi.f/h is some MPI related variables, like MPI_COMM_WORLD etc.

Since we provide all the source files required to build the modules, you can of course just compile with your favorite GNU compiler the mpi_f08 module and also use that with GNU. (Don't forget to add the appropriate include directories for your module files)

 

[JaredFrazier]

Of course Intel MPI supports mpi_gather and so on...

I think this was the biggest misunderstanding of user rwmsu, otherwise the comments of user RonShepard are quite useful since they provide some information about legacy MPI behavior.

So to be a bit pedantic here, the symbols for mpi_gather, etc are not part of the module but only defined in the library. The module does not provide the functions. Even mpi_f08 does not provide the functions but provides an interface to the functions.

In the end, you are calling an external library, e.g. the linker must be able to find the symbols but that's all.

What is provided in the module / mpi.f/h is some MPI related variables, like MPI_COMM_WORLD etc.

Since we provide all the source files required to build the modules, you can of course just compile with your favorite GNU compiler the mpi_f08 module and also use that with GNU. (Don't forget to add the appropriate include directories for your module files)

This I understand perfectly well now, thank you for being pedantic in any case :))

[JaredFrazier]

Hi Tobias,

Thanks for your response! 

I do find that use mpi_f08 fails with the following output:

f951: Fatal Error: Reading module ‘/opt/intel/oneapi/mpi/2021.14/include/mpi/mpi_f08.mod’ at line 1 column 2: Unexpected EOF
compilation terminated.

An example program producing this error is written below:

! @file mpi_gather_example.f90
program mpi_gather_example
    use mpi_f08
end program mpi_gather_example

Compilation instructions are shown below as well (oneapi/mpi version is 2021.14 which you can see from the error above):

 . /opt/intel/oneapi/setvars.sh 

which mpif90
# /opt/intel/oneapi/mpi/2021.14/bin/mpif90

mpif90 mpi_gather_example.f90

Note, the above program compiles without a problem using OpenMPI v4.1.6 (built with either GNU or Intel Compilers).

For completeness, there should be 112 routines that do not have explicit interfaces implemented per the MPI standard and therefore for which use mpi, only: <routine> will fail. Below is a bash script that writes the names of these routines to a temporary file. The names of these routines output from the below script is also attached as a text file. 

#!/usr/bin/bash 
#
# @description Count choice buffers in man pages of MPI routines since explicit
# interfaces for such routines are NOT provided per the MPI 3.1 and 4.0
# standard (which Intel MPI implements). The names of MPI routines containing
# choice buffers are written to a file in a temporary directory.
#
# @notes
# * Tested on Ubuntu 24.04 LTS with Open MPI 4.1.6 
# * Assumes openmpi docs installed via `sudo apt install openmpi-docs`
# 
# @references 
# * https://community.intel.com/t5/Intel-MPI-Library/Error-Symbol-mpi-gather-referenced-at-1-not-found-in-module-mpi/m-p/1721240/highlight/true#M12229
# * https://www.mpi-forum.org/docs/mpi-3.1/mpi31-report/node410.htm#Node410
# 
tmp_mpi_choice_buffer_routine_out_file="/tmp/mpi_routines_with_choice_buffer.txt"
[[ -f $tmp_mpi_choice_buffer_routine_out_file ]] && rm $tmp_mpi_choice_buffer_routine_out_file

num_mpi_routines_choice=0
num_mpi_man_pages=$(ls /usr/share/man/man3/MPI_* | wc -l)
cnt=0
echo "-----------"
for f in $(ls /usr/share/man/man3/MPI_*)
do
    # log every 10 files 
    if (( $cnt % 10 == 0 )) 
    then
        echo "Checked MPI man pages: $cnt/$num_mpi_man_pages | num choice buffer found: $num_mpi_routines_choice"
    fi

    # check to see if mpi routine man page contains "(choice)" and increment 
    # counter if so 
    mpi_routine=$(basename ${f/\.openmpi*/})
    grep_choice=$(echo "$(man $mpi_routine)" | grep -q "(choice)" 2>/dev/null)
    has_choice=$?
    if [[ $has_choice -eq "0" ]] 
    then
        echo $mpi_routine >> $tmp_mpi_choice_buffer_routine_out_file
        num_mpi_routines_choice=$(( $num_mpi_routines_choice + 1 ))
    fi

    cnt=$(( $cnt + 1 ))
done

echo "-----------"
echo "Counted $num_mpi_routines_choice containing choice buffer"
echo "MPI routine names containing choice buffer written to $tmp_mpi_choice_buffer_routine_out_file"

[TobiasK]

@JaredFrazier 
sorry I missed that you are using Gfortran. For Gfortran you have to compile the f08_module yourself.
You find the source files here:
$I_MPI_ROOT/opt/mpi/binding

Can you try with mpiifx?

[JaredFrazier]

Hi Tobias,

Compiling with mpiifx works without a problem. Thanks!

Based on some reading, mpif90 and mpif77 should not be used at all since they are deprecated, is that also correct? This applies at least for the OpenMPI implementation (see here: https://www.open-mpi.org/faq/?category=mpi-apps#mpifort-vs-mpif77-and-mpif90), is it safe to assume the same applies for the Intel MPI implementation?

[TobiasK]

mpiifx/mpif90/mpif77 are just wrapper that add the necessary linker flags and libraries, basically you don't need them.

As you see OpenMPI 'deprecated' those in 1.7 and now OpenMPI has released 5.0 and the wrappers are still around....

Similarly, there is no such thing as a Fortran77 compiler or Fortran90 compiler it's just a Fortran compiler so mpif77 / mpif90 never made real sense.