omp simd syntax bug in recent ifx and ifort?

[hakostra1]

Since ifx 2022.2.0 the following code no longer compiles:

SUBROUTINE ompsimdtest(kk, jj, ii, res)
    IMPLICIT NONE

    INTEGER, INTENT(in) :: kk, jj, ii
    REAL, INTENT(inout) :: res(kk, jj, ii)

    INTEGER :: k, j, i
    REAL :: banana

    DO i = 1, ii
        DO j = 1, jj
            !$omp simd &
            !$omp private(banana)
            DO k = 1, kk
                banana = REAL(k)
                res(k, j, i) = res(k, j, i) + banana
            END DO
        END DO
    END DO
END SUBROUTINE ompsimdtest

Please ignore that the code is just to demonstrate a compile error, not to compute something.

This code compiles in all GFortran versions I have tested (up to and including 12.2), ifort from 19.0 to 2021.6.0 and ifx up to and including 2022.1.0.

However, starting with ifx 2022.2.0 and later 2022.2.1, along with ifort 2021.7.0 and 2021.7.1 I get the following error:

/app/example.f90(15): error #5082: Syntax error, found 'DO' when expecting one of: PRIVATE REDUCTION COLLAPSE ORDER LASTPRIVATE SAFELEN EARLY_EXIT ASSERT NONTEMPORAL ...
            DO k = 1, kk
------------^
/app/example.f90(15): error #5082: Syntax error, found ',' when expecting one of: <END-OF-STATEMENT> ;
            DO k = 1, kk
--------------------^
/app/example.f90(13): error #7644: The statement or directive following this OpenMP* directive is incorrect.
            !$omp simd &
------------------^
compilation aborted for /app/example.f90 (code 1)
Compiler returned: 1

The error can be mitigated by truncating the OMP directive to one line (!$omp simd private(banana)). For this silly example that works great, but I have some directives that absolutely does not fit on a line where this problem now occur.

With ifort the problem does not occur at -O0 and -O1. With affected ifx versions it always occur independent on the compile flags.

I tried to read the OMP specification, and could not find anything wrong with the directive. Anyone spotting a mistake in my code or is this a compiler bug?

Link to compiler explorer where this can be tried out with different compilers: https://godbolt.org/z/6dxqzPc97

[jimdempseyatthecove]

See: Conditional Compilation Rules (intel.com)

...
            !$omp simd &
            !$ private(banana)
            DO k = 1, kk
                banana = REAL(k)
                res(k, j, i) = res(k, j, i) + banana
            END DO
...

Note, missing "omp" on continueing line.

Does the above correct the problem?

Jim Dempsey

[hakostra1]

The suggested code without the "omp" on the continuing line does not compile in nether ifx nor ifort, independent on version: https://godbolt.org/z/6eYjefcz9 I tried all version available on the compiler explorer. So that does not seem to work at all. For some reason it works with GFortran.

I also have lots of "ordinary" "omp parallel do" that are continued in the following way:

!$omp parallel do &
!$omp default(none) &
!$omp shared(bp, gsaw, gsae, gsan, gsas, gsat, gsab, gsrap, dp, rhs) &
!$omp shared(omg, rb, kk) &
!$omp private(aw, ae, as, an, ab, at, rap, res, k, kstart, kstop) &
!$omp schedule(static)
DO i = 1, ii
    ! Some code
END DO

this works great with all compilers I have tried. So it seems only to be the "omp simd" that cause problems.

[TheViking]

Hi Jim,

I notice that you seem well versed on Intel OpenMP.  I am mainly versed in Linux but find myself on a windows setup for a project and that is causing me some headache in re OpenMP.  I hope you might have some ideas.

Basic layout is that I have a C# code that call Intel OneMKL via the fortran77 routines in the intel single dynamic library. This all works fine single threaded i.e. sequential. However, when running multithreaded it immediately throws a segfault.

Initially I suspected that it was either a stack issue or a Microsoft created C# issue that causing access problems.  Turns out that it is neither a C# or MKL issue... but a Microsoft environment issue..

I have been running the console app from the VS Studio Dev Command Prompt.. But I was not able to get version, settings, and debug from OpenMP printed at startup.. even with in process env vars set..

Suspecting an OpenMP issue I used the Intel OpenAPI console for window and within the console the code runs flaws and OpenMP prints output at launch and the code correct runs and completes..

My assumption is that VS and Windows prevents OpenMP from launching and running and that results in a segfault preventing MKL from running.

Windows is not my wheelhouse and I was hoping you perhaps have encountered the issue and might have an idea about how to get the executable to work outside the Intel OneAPI console? 

Many thanks in advance..

[Steve_Lionel]

@jimdempseyatthecove , the OpenMP standard and the Intel documentation both say that the full directive prefix is used on continued directives.

[jimdempseyatthecove]

I agree that the example on page 56 of OpenMP standard  shows !$omp on the continuing line.

However, consider that lines beginning with "!$ " are to be expanded when OpenMP is enabled...

... together with "&" on the on the continued line, this should produce a concatenation of the two lines sans "& <CRLF> <WS>!$ "

Or am I mistaken?

Jim Dempsey

[hakostra1]

I believe conditional compilation, as described in section 3.3 of the OpenMP 5.2 standard, is something completely different than what I want to achieve here.

[Ron_Green]

I cannot reproduce the error you are seeing.  No error. What am I missing?

ifx -what -V -qopenmp -c reproomp.f90
Intel(R) Fortran Compiler for applications running on Intel(R) 64, Version 2022.2.0 Build 20220730
Copyright (C) 1985-2022 Intel Corporation. All rights reserved.

 Intel(R) Fortran 22.0-1775.03
$ 
$ more reproomp.f90
SUBROUTINE ompsimdtest(kk, jj, ii, res)
    IMPLICIT NONE

    INTEGER, INTENT(in) :: kk, jj, ii
    REAL, INTENT(inout) :: res(kk, jj, ii)

    INTEGER :: k, j, i
    REAL :: banana

    DO i = 1, ii
        DO j = 1, jj
            !$omp simd &
            !$omp private(banana)
            DO k = 1, kk
                banana = REAL(k)
                res(k, j, i) = res(k, j, i) + banana
            END DO
        END DO
    END DO
END SUBROUTINE ompsimdtest

No  

[Ron_Green]

Ah wait, perhaps you use -qopenmp-simd?  Yes, then I see the issue.  Hadn't thought of using just SIMD parsing.  Looks like a bug in parsing when using -qopenmp-simd.  I'll get a bug report on this.

ron

[hakostra1]

I actually does not need the -qopenmp-simd to trigger the error at all, the error is always there:

$ ifx -what -V -c omptest.F90 
Intel(R) Fortran Compiler for applications running on Intel(R) 64, Version 2022.2.1 Build 20221020
Copyright (C) 1985-2022 Intel Corporation. All rights reserved.

 Intel(R) Fortran 22.0-1775.03
omptest.F90(15): error #5082: Syntax error, found 'DO' when expecting one of: PRIVATE REDUCTION COLLAPSE ORDER LASTPRIVATE SAFELEN EARLY_EXIT ASSERT NONTEMPORAL ...
            DO k = 1, kk
------------^
omptest.F90(15): error #5082: Syntax error, found ',' when expecting one of: <END-OF-STATEMENT> ;
            DO k = 1, kk
--------------------^
omptest.F90(13): error #7644: The statement or directive following this OpenMP* directive is incorrect.
!$omp simd &
------^
compilation aborted for omptest.F90 (code 1)
$
$ more omptest.F90 
SUBROUTINE ompsimdtest(kk, jj, ii, res)
    IMPLICIT NONE

    INTEGER, INTENT(in) :: kk, jj, ii
    REAL, INTENT(inout) :: res(kk, jj, ii)

    INTEGER :: k, j, i
    REAL :: banana


    DO i = 1, ii
        DO j = 1, jj
            !$omp simd &
            !$omp private(banana)
            DO k = 1, kk
                banana = REAL(k)
                res(k, j, i) = res(k, j, i) + banana
            END DO
        END DO
    END DO
END SUBROUTINE ompsimdtest

 however, for practical real-world purposes I use -qopenmp-simd in all my regular builds.

[jimdempseyatthecove]

Also you are 2022.2.1 and Ron is 2022.2.0

Jim Dempsey

[Ron_Green]

Ah, yes, that:  - OpenMP SIMD processing is done BY DEFAULT now with the Intel Fortran compilers.  I objected strongly to this change but was overridden by more senior architects both from the C++ and Fortran sides of the compiler teams.  

You have to explicitly disable SIMD processing with -qno-openmp-simd

[hakostra1]

Ok, nevertheless, the code actually compiles with IFX 2022.1.0 without any errors, even with -qopenmp-simd (it is in my default performance compile flags out of good old habit). So I feel the general conclusion is that there seems to be a parsing error in IFX 2022.2...

[Ron_Green · Accepted solution]

bug ID CMPLRLLVM-42282

[jimdempseyatthecove]

>>Basic layout is that I have a C# code that call Intel OneMKL via the fortran77 routines in the intel single dynamic library. This all works fine single threaded i.e. sequential. However, when running multithreaded it immediately throws a segfault.

It may initially seem counter-intuitive:

   Single threaded apps should link with the MKL threaded library

   Multi-threaded apps should link with the MKL serial (single) threaded library

>>I have been running the console app from the VS Studio Dev Command Prompt.. But I was not able to get version, settings, and debug from OpenMP printed at startup.. even with in process env vars set..

This sounds like an environment and/or PATH issue

1) Does the code run within MS Visual Studio? (I assume yes)

2) "VS Studio Dev Command Prompt". You should run in the "Intel oneAPI command prompt"

.or. run the appropriate batch script in a VS Studio Dev Command Prompt shell prior running your program.

3) Any linker setting for stack size affect the main thread only. Use environment variable OMP_STACKSIZE to specify required stack sizes for OpenMP spawned threads.

4) If you have stack issues withing MKL then also use environment variable MKL_STACKSIZE

*** Do not simply/blindly specify humongous stack sizes to fix a problem ***

Jim Dempsey

[Ron_Green]

This bug is fixed in the latest 2023.1.0 compiler