yet another ifort 18 regression with OpenMP and allocatables (invalid read in scalable_free, frontend.cpp:2868)

[Janus]

Dear Intel team,

it seems something is fundamentally broken when using allocatables in connection with OpenMP in ifort 18.x.

On top of my earlier bug reports #03202444 and #03214078, here is another related problem:

! compile with "-qopenmp -g"
! and run via valgrind

module m

   implicit none

   type :: T
      integer, dimension(:), allocatable :: a
   end type

   integer, save :: N = 1

contains

   subroutine XXX()
      type(T), dimension(1:N) :: alt
      ! empty routine
   end subroutine                        ! valgrind reports: invalid read in scalable_free (frontend.cpp:2868) on exit

   subroutine useless_but_necessary()
      integer :: i
!$omp parallel do default(shared) private(i)
      do i = 1,12
      end do
!$omp end parallel do
   end subroutine

end module


program p

   use OMP_LIB
   use m

   implicit none

!$      print *,omp_get_max_threads()

      call XXX()

end

When compiled with -qopenmp and run through valgrind, this gives a shitload of invalid reads in scalable_free (frontend.cpp:2868):

==25020== Invalid read of size 2
==25020==    at 0x524BFE5: scalable_free (frontend.cpp:2868)
==25020==    by 0x421D29: for_dealloc_allocatable (in /home/janus/fort/ifort_bugs/auto_arr_w_alloc_comp/a.out)
==25020==    by 0x406DE7: for_dealloc_all_nocheck (in /home/janus/fort/ifort_bugs/auto_arr_w_alloc_comp/a.out)
==25020==    by 0x403285: m_mp_xxx_ (p.f90:17)
==25020==    by 0x403521: MAIN__ (p.f90:38)
==25020==    by 0x402E6D: main (in /home/janus/fort/ifort_bugs/auto_arr_w_alloc_comp/a.out)
==25020==  Address 0x1ffeffc07a is on thread 1's stack
==25020==  11206 bytes below stack pointer
==25020==
==25020== Invalid read of size 8
==25020==    at 0x524BFF7: scalable_free (frontend.cpp:2868)
==25020==    by 0x421D29: for_dealloc_allocatable (in /home/janus/fort/ifort_bugs/auto_arr_w_alloc_comp/a.out)
==25020==    by 0x406DE7: for_dealloc_all_nocheck (in /home/janus/fort/ifort_bugs/auto_arr_w_alloc_comp/a.out)
==25020==    by 0x403285: m_mp_xxx_ (p.f90:17)
==25020==    by 0x403521: MAIN__ (p.f90:38)
==25020==    by 0x402E6D: main (in /home/janus/fort/ifort_bugs/auto_arr_w_alloc_comp/a.out)
==25020==  Address 0x1ffeffc060 is on thread 1's stack
==25020==  11232 bytes below stack pointer
==25020==
==25020== Invalid read of size 8
==25020==    at 0x5249711: rml::internal::Block::freePublicObject(rml::internal::FreeObject*) (frontend.cpp:1372)
==25020==    by 0x524C02B: scalable_free (frontend.cpp:2868)
==25020==    by 0x421D29: for_dealloc_allocatable (in /home/janus/fort/ifort_bugs/auto_arr_w_alloc_comp/a.out)
==25020==    by 0x406DE7: for_dealloc_all_nocheck (in /home/janus/fort/ifort_bugs/auto_arr_w_alloc_comp/a.out)
==25020==    by 0x403285: m_mp_xxx_ (p.f90:17)
==25020==    by 0x403521: MAIN__ (p.f90:38)
==25020==    by 0x402E6D: main (in /home/janus/fort/ifort_bugs/auto_arr_w_alloc_comp/a.out)
==25020==  Address 0x1ffeffc010 is on thread 1's stack
==25020==  11264 bytes below stack pointer
==25020==
==25020== Invalid read of size 8
==25020==    at 0x524971B: rml::internal::Block::freePublicObject(rml::internal::FreeObject*) (frontend.cpp:1375)
==25020==    by 0x524C02B: scalable_free (frontend.cpp:2868)
==25020==    by 0x421D29: for_dealloc_allocatable (in /home/janus/fort/ifort_bugs/auto_arr_w_alloc_comp/a.out)
==25020==    by 0x406DE7: for_dealloc_all_nocheck (in /home/janus/fort/ifort_bugs/auto_arr_w_alloc_comp/a.out)
==25020==    by 0x403285: m_mp_xxx_ (p.f90:17)
==25020==    by 0x403521: MAIN__ (p.f90:38)
==25020==    by 0x402E6D: main (in /home/janus/fort/ifort_bugs/auto_arr_w_alloc_comp/a.out)
==25020==  Address 0x1ffeffc010 is on thread 1's stack
==25020==  11264 bytes below stack pointer
==25020==
==25020== Invalid read of size 8
==25020==    at 0x5249732: rml::internal::Block::freePublicObject(rml::internal::FreeObject*) (frontend.cpp:1397)
==25020==    by 0x524C02B: scalable_free (frontend.cpp:2868)
==25020==    by 0x421D29: for_dealloc_allocatable (in /home/janus/fort/ifort_bugs/auto_arr_w_alloc_comp/a.out)
==25020==    by 0x406DE7: for_dealloc_all_nocheck (in /home/janus/fort/ifort_bugs/auto_arr_w_alloc_comp/a.out)
==25020==    by 0x403285: m_mp_xxx_ (p.f90:17)
==25020==    by 0x403521: MAIN__ (p.f90:38)
==25020==    by 0x402E6D: main (in /home/janus/fort/ifort_bugs/auto_arr_w_alloc_comp/a.out)
==25020==  Address 0x1ffeffc018 is on thread 1's stack
==25020==  11256 bytes below stack pointer
==25020==
==25020== Invalid read of size 1
==25020==    at 0x5249740: rml::internal::Block::freePublicObject(rml::internal::FreeObject*) (frontend.cpp:1400)
==25020==    by 0x524C02B: scalable_free (frontend.cpp:2868)
==25020==    by 0x421D29: for_dealloc_allocatable (in /home/janus/fort/ifort_bugs/auto_arr_w_alloc_comp/a.out)
==25020==    by 0x406DE7: for_dealloc_all_nocheck (in /home/janus/fort/ifort_bugs/auto_arr_w_alloc_comp/a.out)
==25020==    by 0x403285: m_mp_xxx_ (p.f90:17)
==25020==    by 0x403521: MAIN__ (p.f90:38)
==25020==    by 0x402E6D: main (in /home/janus/fort/ifort_bugs/auto_arr_w_alloc_comp/a.out)
==25020==  Address 0x10 is not stack'd, malloc'd or (recently) free'd

If you're not so lucky it will segfault on you. I suspect this problem is closely related to #03202444, because it points to the exact same source line in frontend.cpp (2868). See also the longish thread at https://software.intel.com/en-us/forums/intel-fortran-compiler-for-linux-and-mac-os-x/topic/753833

All of these are extremely serious issues for us. Please fix your compiler!

Best regards,

Janus

[jimdempseyatthecove]

I am not setup to test your code. This could be a case of improper dead code elimination.

In XXX add:

print *, size(alt)

in your parallel do loop, insert:

print *,i, omp_get_thread_num()

I know you want a bug fix, and thanks for the reproducer.

Jim Dempsey

[Janus]

jimdempseyatthecove wrote:

I am not setup to test your code. This could be a case of improper dead code elimination.

Um, nope. This has nothing to do with dead code elimination. The reason why the test case looks like a haunted house is because I reduced it to the bare minimum and removed anything that's not necessary to reproduce. I can assure you the problem occurs in very-much-real-world situations as well.

In XXX add:

print *, size(alt)

Doesn't change anything. Size is 1, as expected.

in your parallel do loop, insert:

print *,i, omp_get_thread_num()

No effect. Never even executed. Note the encompassing routine's flowery descriptive name.

I know you want a bug fix, and thanks for the reproducer.

I'd also be very grateful for a practical workaround.

Cheers,

Janus

[Janus · Accepted solution]

This is now support request #03224588, in case it matters to anyone.

Cheers,

Janus

[Devorah_H_Intel]

Janus wrote:

This is now support request #03224588, in case it matters to anyone.

Cheers,

Janus

Thank you for reporting these and other related issues via Online Service Center. Both of these cases are escalated to engineering for a fix.

[Janus]

Devorah H. (Intel) wrote:

Both of these cases are escalated to engineering for a fix.

Thanks for taking care of it!

[Janus]

it seems something is fundamentally broken when using allocatables in connection with OpenMP in ifort 18.x.

On top of my earlier bug reports #03202444 and #03214078, here is another related problem:

[...]

This is now support request #03224588, in case it matters to anyone.

I just tried the freshly released ifort version 18.0.2.199, and I am sorry to report that none of the bugs mentioned above have been fixed (although two of them have already been closed).
 

I would be grateful if someone from Intel's support team could re-open those reports (I can not do it apparently). And I would be even more grateful if some of these bugs could finally be fixed. ifort 18 is (still) seriously broken when it comes to OpenMP and allocation/deallocation.

Cheers,

Janus

[Devorah_H_Intel]

These were not fixed yet. One case was closed because the user had no paid support.  However, you can reopen it by posting an update/reply to the case. 

[Janus]

Devorah H. (Intel) wrote:

These were not fixed yet. One case was closed because the user had no paid support. However, you can reopen it by posting an update/reply to the case.

For #03214078, you replied to me on 01/29/2018: "This issue has been fixed in upcoming Update 2 which I have verified with the internal compiler - no segfault."

That is obviously not true, because the ifort 18.0.2 build, dated 2018-02-10, still exposes the problem. If you would have told me directly that you are closing the issue (although unfixed), just because I reported it from a personal account that is not connected to a support license, I would have had the possibility to report it with another account that has support (we _are_ paying customers indeed).

The only reason why I used my personal account in the first place, is that it is much easier to keep track of replies this way (your bug tracker is quirky enough as it is, and no, I can not reopen any of the closed bugs).

I think it's a rather common situation that the person hitting a bug with your software is not the person managing the licenses (be it on a university cluster or in a company), and in general I think it would not hurt Intel to listen to its users a bit more carefully. Reducing a manageable test case from a large code base can be quite some work. I'm sure not everyone of your users is willing or able to do that, and me personally, I'm increasingly losing any incentive to keep on doing it.

I reported four(!) *runtime* *regressions* (some of which were not easy to track down). None of them was fixed in 18.0.2. If that is not the kind of bug you care about, then what do you care about exactly?

[Devorah_H_Intel]

RE: 03214078 This was escalated to development on 1/29 - The internal bug report was never closed by the development and they were/are planning to fix it (although it is unknown when). I posted the update by mistake (mix up with another ticket) - I apologize for the confusion.

Again, I have asked our development team to expedite this. 

The above is unrelated to the 03214078 case being closed. We take all bug reports very seriously and try to fix as many as we can as soon as possible. I appreciate your patience.