Problem enabling 64 bit Fortran compilation on Windows using Intel OneAPI with Cmake on Windows

[vishy]

I am trying to enable 64 bit integer size for a sample hello world kind of Fortran test code, in an MPI setup.

OS: Windows 10

Compiler used: Intel OneAPI 2021.3.0

MPI: Intel MPI

There are 2 scenarios I tried to test,

1. using a single line command to compile an executable directly, mpiifort -o test.exe -i8 test.f90
2. using a CmakeLists file to compile the test with necessary Fortran 64 bit "-i8" option and find_package(MPI). I made sure MPI libraries are picked (impi.lib, libmpi_ilp64.lib) and MPI_Fortran_Compiler points to mpiifort

I use a test of determining the size of MPI_INTEGER using an API MPI_Type_size( ) to check if 64 bit environment is enabled for Fortran/MPI setup.

call MPI_Type_size(MPI_INTEGER, sz, ierrsiz)
print *, 'sizeof(MPI_INTEGER) ', sz

Scenario 1 prints correct size of 8 bytes (64 bit) Scenario 2 prints incorrect size of 4 bytes ( 32 bit)

I do use "-i8" option in Cmake build system to enable 64 bit environment. But MPI still seems to be 32 bit.

Attached files/build logs.

Kindly help.

[VarshaS_Intel]

Hi,

Thanks for reaching out to us.

Thanks for providing all the information. We are working on this internally and will get back to you soon.

Thanks & Regards

Varsha

[Alina_S_Intel]

Firstly, your Cmake uses our NextGen Fortran compiler (ifx.exe) while mpiifort refers to Intel Fortran Compiler Classic (ifort.exe). Please, note that ifx.exe is still in Beta and we recommend using it for openMP offload ONLY. For any other cases, please, continue using ifort. Add -DCMAKE_Fortran_COMPILER="ifort" macro to your cmake command line.

Source files are not identical and this is the reason for different results:

test.f90:

---------------------------------------------------------------

    integer i, size, rank, namelen, ierr, ISIZE

    character (len=MPI_MAX_PROCESSOR_NAME) :: name

    integer stat(MPI_STATUS_SIZE)

    INTEGER  :: sz

    INTEGER  :: ierrsiz  

ISIZE = SIZEOF(rank)

print *, 'sizeof(INTEGER) = ', ISIZE

---------------------------------------------------------------

output:

---------------------------------------------------------------

 sizeof(INTEGER) =           8

 sizeof(MPI_INTEGER)           8

 sizeof(MPI_INTEGER4)           4

 sizeof(MPI_INTEGER8)           8

 Hello world: rank           0 of

---------------------------------------------------------------

sizeof(integer) =   8 is expected for  INTEGER  :: rank

-i8 option changes  sizeof(INTEGER) to 8 and I see 4 without this option. (mpiifort test.f90 VS mpiifort -i8 test.f90)

test_cmakelists_example.f90:

---------------------------------------------------------------

    integer i, ISIZE

    character (len=MPI_MAX_PROCESSOR_NAME) :: name

    integer stat(MPI_STATUS_SIZE)

INTEGER(4)  :: sz

    INTEGER(4)  :: ierr, size, rank, namelen

    INTEGER(4)  :: ierrsiz  

ISIZE = SIZEOF(rank)

print *, 'sizeof(integer) = ', ISIZE

---------------------------------------------------------------

output for test_cmakelists_example.exe:

---------------------------------------------------------------

 sizeof(integer) =           4

 sizeof(MPI_INTEGER)      4

 sizeof(MPI_INTEGER4)      4

 sizeof(MPI_INTEGER8)      8

 Hello world: rank      0 of      1 running on

---------------------------------------------------------------

What I see here when building without Cmake:

>mpiifort test_cmakelists_example.f90

mpifc.bat for the Intel(R) MPI Library 2021.4 for Windows*

Copyright Intel Corporation.

Intel(R) Fortran Intel(R) 64 Compiler Classic for applications running on Intel(R) 64, Version 2021.4.0 Build 20210910_000000

Copyright (C) 1985-2021 Intel Corporation. All rights reserved.

Microsoft (R) Incremental Linker Version 14.29.30040.0

Copyright (C) Microsoft Corporation. All rights reserved.

-out:test_cmakelists_example.exe

-subsystem:console

"/LIBPATH:C:\Program Files (x86)\Intel\oneAPI\mpi\latest\env\..\lib\release"

"/LIBPATH:C:\Program Files (x86)\Intel\oneAPI\mpi\latest\env\..\lib"

impi.lib

test_cmakelists_example.obj

And what I see when adding -i8:

>mpiifort test_cmakelists_example.f90

mpifc.bat for the Intel(R) MPI Library 2021.4 for Windows*

Copyright Intel Corporation.

Intel(R) Fortran Intel(R) 64 Compiler Classic for applications running on Intel(R) 64, Version 2021.4.0 Build 20210910_000000

Copyright (C) 1985-2021 Intel Corporation. All rights reserved.

Microsoft (R) Incremental Linker Version 14.29.30040.0

Copyright (C) Microsoft Corporation. All rights reserved.

-out:test_cmakelists_example.exe

-subsystem:console

"/LIBPATH:C:\Program Files (x86)\Intel\oneAPI\mpi\latest\env\..\lib\release"

"/LIBPATH:C:\Program Files (x86)\Intel\oneAPI\mpi\latest\env\..\lib"

impi.lib

test_cmakelists_example.obj

C:\Users\ashadrin\Downloads\OSC\c05255486_f>mpiifort test_cmakelists_example.f90 -i8

mpifc.bat for the Intel(R) MPI Library 2021.4 for Windows*

Copyright Intel Corporation.

Intel(R) Fortran Intel(R) 64 Compiler Classic for applications running on Intel(R) 64, Version 2021.4.0 Build 20210910_000000

Copyright (C) 1985-2021 Intel Corporation. All rights reserved.

test_cmakelists_example.f90(24): warning #6075: The data type of the actual argument does not match the definition.  [IERR]

    call MPI_INIT (ierr)

-----------------------^

test_cmakelists_example.f90(26): warning #6075: The data type of the actual argument does not match the definition.  [SIZE]

    call MPI_COMM_SIZE (MPI_COMM_WORLD, size, ierr)

--------------------------------------------^

test_cmakelists_example.f90(26): warning #6075: The data type of the actual argument does not match the definition.  [IERR]

    call MPI_COMM_SIZE (MPI_COMM_WORLD, size, ierr)

--------------------------------------------------^

test_cmakelists_example.f90(27): warning #6075: The data type of the actual argument does not match the definition.  [RANK]

    call MPI_COMM_RANK (MPI_COMM_WORLD, rank, ierr)

--------------------------------------------^

test_cmakelists_example.f90(27): warning #6075: The data type of the actual argument does not match the definition.  [IERR]

    call MPI_COMM_RANK (MPI_COMM_WORLD, rank, ierr)

--------------------------------------------------^

test_cmakelists_example.f90(28): warning #6075: The data type of the actual argument does not match the definition.  [NAMELEN]

    call MPI_GET_PROCESSOR_NAME (name, namelen, ierr)

-------------------------------------------^

test_cmakelists_example.f90(28): warning #6075: The data type of the actual argument does not match the definition.  [IERR]

    call MPI_GET_PROCESSOR_NAME (name, namelen, ierr)

----------------------------------------------------^

test_cmakelists_example.f90(30): warning #6075: The data type of the actual argument does not match the definition.  [SZ]

        call MPI_Type_size(MPI_INTEGER, sz, ierrsiz)

------------------------------------------------^

test_cmakelists_example.f90(30): warning #6075: The data type of the actual argument does not match the definition.  [IERRSIZ]

        call MPI_Type_size(MPI_INTEGER, sz, ierrsiz)

----------------------------------------------------^

test_cmakelists_example.f90(32): warning #6075: The data type of the actual argument does not match the definition.  [SZ]

        call MPI_Type_size(MPI_INTEGER4, sz, ierrsiz)

-------------------------------------------------^

test_cmakelists_example.f90(32): warning #6075: The data type of the actual argument does not match the definition.  [IERRSIZ]

        call MPI_Type_size(MPI_INTEGER4, sz, ierrsiz)

-----------------------------------------------------^

test_cmakelists_example.f90(34): warning #6075: The data type of the actual argument does not match the definition.  [SZ]

        call MPI_Type_size(MPI_INTEGER8, sz, ierrsiz)

-------------------------------------------------^

test_cmakelists_example.f90(34): warning #6075: The data type of the actual argument does not match the definition.  [IERRSIZ]

        call MPI_Type_size(MPI_INTEGER8, sz, ierrsiz)

-----------------------------------------------------^

test_cmakelists_example.f90(59): warning #6075: The data type of the actual argument does not match the definition.  [IERR]

    call MPI_FINALIZE (ierr)

---------------------------^

Microsoft (R) Incremental Linker Version 14.29.30040.0

Copyright (C) Microsoft Corporation. All rights reserved.

-out:test_cmakelists_example.exe

-subsystem:console

"/LIBPATH:C:\Program Files (x86)\Intel\oneAPI\mpi\latest\env\..\lib\release"

"/LIBPATH:C:\Program Files (x86)\Intel\oneAPI\mpi\latest\env\..\lib"

libmpi_ilp64.lib

impi.lib

test_cmakelists_example.obj

[vishy]

Thanks for the reply.

I used the same test application for mpiifort and cmake based compilation. And configured "ifort.exe" for CMAKE_Fortran_COMPILER during CMake configuration in command line. The size of MPI_INTEGER when compiled using cmake results in 0 (Zero) instead of 8 bytes.

Following are the outputs for 2 scenarios:

output for mpiifort and i8 compilation on same test.f90:

cmd: mpiifort -o test.exe -i8 test.f90

---------------------------------------------------------------

 sizeof(sz) = 8
sizeof(MPI_INTEGER) 8
sizeof(MPI_INTEGER4) 4
sizeof(MPI_INTEGER8) 8
Hello world: rank 0 of 1 running on 

---------------------------------------------------------------

output for cmake based compilation on same test.f90:

cmd: cmake -S . -B out -G "Ninja" -DCMAKE_Fortran_COMPILER="ifort.exe" & cmake --build out --verbose

<attached cmakelists file and test.f90>

---------------------------------------------------------------

sizeof(sz) = 8
sizeof(MPI_INTEGER) 0
sizeof(MPI_INTEGER4) 0
sizeof(MPI_INTEGER8) 0
Hello world: rank 0 of 0 running on

---------------------------------------------------------------

[Alina_S_Intel]

I've noticed that I recommended you using  -DCMAKE_Fortran_COMPILER="ifort.exe". However, for MPI applications we are using mpiiforr wrapper, so the correct macro here should be -DCMAKE_Fortran_COMPILER="mpiifort.bat". Sorry for the confusion.

It seems that INCLUDE_DIRECTORIES(${MPI_INCLUDE_PATH}) in the CMakeLists.txt is the reason for MPI_INTEGER = 0. I've removed it and got the same result as for 'mpiifort -i8 test.f90':

mpiexec -n 1 out/test.exe

 sizeof(sz) =           8

 sizeof(MPI_INTEGER)           8

 sizeof(MPI_INTEGER4)           4

 sizeof(MPI_INTEGER8)           8

 Hello world: rank           0 of           1 running on

You can check it from cmd using 'mpiifort -I\ -I"C:\Program Files (x86)\Intel\oneAPI\mpi\latest\include" -i8 test.f90'. It should give you 0 instead of 8 \ 4 for MPI_INTEGERs. Then remove -I\ -I"C:\Program Files (x86)\Intel\oneAPI\mpi\latest\include" the result should be correct.

[vishy]

Thanks.

I was able to reproduce the steps above and got 8 byte size for MPI_INTEGER using CMake build system. Instead of using -DCMAKE_Fortran_COMPILER="mpiifort.bat" in command line, I added a set command in CMakeLists.txt : "set(CMAKE_Fortran_COMPILER ${MPI_Fortran_COMPILER})"

I was able to observe 2 weird behaviors using CMake build system:

1. "find_package(MPI COMPONENTS C Fortran REQUIRED)" does not automatically find the required Intel MPI's ILP64 library. By default, only impi.lib is populated under MPI_C_LIBRARIES. For MPI ILP64 (i.e., 8 byte printing of MPI_INTEGER), the libraries need to be linked in correct order as follows: libmpi_ilp64.lib;impi.lib. This, I achieve by manually forcing the variable "MPI_C_LIBRARIES" as used in CMakeLists.txt. Why cant find_package() command do the necessary work?

2. The line in CMakeLists.txt < add_definitions("-DInt=__int64" "-DUInt=unsigned __int64" "-i8" "-i8") >,  has "-i8" twice and is used intentionally. For some unknown reason, if I remove the duplicate "i8" and have a single mention,  size of MPI_INTEGER drops to 0 (zero) from 8

Attaching the CmakeLists.txt and build logs.

[vishy]

Kindly help me wrt above query, where CMake does not populate MPI liubraries automatically using find_package(MPI ..) command and also the behavior I see in using "-i8" flag.

This behavior is especially observed in Windows CMake build.

[Alina_S_Intel]

Apologies for taking so long to respond to your request. The investigation took more time than we expected.

1. The issue with find_package is a known issue and it is on the Cmake side. Please, follow this thread to learn more and track the progress https://gitlab.kitware.com/cmake/cmake/-/issues/19427
2. In your latest CmakeLists, you passed -i8 it to add_definitions("-DInt=__int64" "-DUInt=unsigned __int64" "-i8" "-i8"). However, add_definitions is for preprocessor definitions not for compiler options. Please, use target_compile_options(test PUBLIC ${FORTRAN_FLAG_INT64} "-ilp64" "-i8") instead.