Intel® ISA Extensions
Use hardware-based isolation and memory encryption to provide more code protection in your solutions.
1093 Discussions

Compiler decides to use SSE instead of AVX

alzamos
Beginner
389 Views

Hello!

I'm compiling some code in fortran with the -xAVX option (ifort version 12.1.0 20111011), and depending on whether or not the code is standalone or part of a subroutine, the compiler vectorizes differently and I was wondering why.The code is as follows: 

[fortran]program example

    implicit none

    double precision, dimension(4, 180000, 4) :: qold, q, res

    double precision, dimension(4, 180000) :: adt

    double precision, dimension(4) :: adti, del

    integer :: i, j, diff, acc

    diff = 180000

    do j = 1, diff

        adti(:) = 1.0 / adt(:,j)            # line 10

        do i = 1, 4

            del(:) = adti(:) * res(:,j,i)   # line 12

            q(:,j,i) = qold(:,j,i) - del(:) # line 13

            res(:,j,i) = 0.0                # line 14

        end do

    end do

    acc = sum(q)

    print *, acc

end program [/fortran]

Using -vec-report it says that the loops were indeed vectorized.Furthermore, looking at the assembly, all the relevant instructions were using the ymm registers.However, if I were to encompass the code within a subroutine which takes qold, q, res, adt and diff as arguments and include it in a module, things go differently;This time -vec-report tells me that only lines 10 and 12 could be vectorized, whereas 13 and 14 the vectorization is "possible but seems inefficient".If I decide to force the vectorization to happen using the !DIR$ SIMD directive before lines 13 and 14, it then tells me that the SIMD loop was vectorized.However, looking at the assembly, instead of using AVX instructions it seems to be using SSE instructions.

  • For line 13, 
    • vsubpd    generated_module_mp_update_kernel_caller_$DEL.0.2(%rip), %xmm6, %xmm7 
    • vsubpd    16+generated_module_mp_update_kernel_caller_$DEL.0.2(%rip), %xmm8, %xmm9
  • For line 14,
    • vxorpd      %xmm0, %xmm0, %xmm0  
    • vmovupd   %xmm0, (%rbx,%r9)  
    • vmovupd   %xmm0, 16(%rbx,%r9)   

Because it does two similar instructions for both, with an offset of 16 bytes (128 bits) which is the size of an SSE register, it seems to be doing two SSE vector instructions instead of one AVX instruction.
Does anyone know why this happens and how to change this?

Kind regards,

Alex

0 Kudos
3 Replies
TimP
Honored Contributor III
389 Views
Those are AVX-128 instructions.  With AVX-128, the compiler can take advantage of hardware handling misaligned data well enough that there is no need to peel iterations for alignment.  This strategy may be evaluated as superior if there isn't sufficient information for the compiler to make other than a default guess about loop count or alignment, as a very long loop may be required to establish a preference for AVX-256 with peeling.   If you are interested, you could try loop count and alignment directives to see if they influence a choice between AVX-128 and AVX-256.
If you are interested in further esoterica, you might try to understand why CEAN notation in icc uses AVX-128 more frequently (if that happens in your cases).
Perhaps needless to say, gnu compilers frequently prefer AVX-128 in order to avoid extra code to deal with misalignment.
0 Kudos
Martyn_C_Intel
Employee
389 Views
Hi Alex,
            What the compiler does depends on the information available to it, which in turn depends on how the arguments are passed, whether caller and callee are in the same source file, etc. The compiler has to make conservative choices (wrt performance as well as correctness) if it doesn't have full information.
            You haven't given us all the details, please could you include the source version with the module subroutine that does not vectorize? That will allow us to figure out whether the compiler could do better, or whether there's a more effective way to write the code. I think it will be more useful than me constructing an example that doesn't match yours.
           
Regards,
Martyn
0 Kudos
alzamos
Beginner
389 Views
Sorry for the late reply,

Thank you for both of your answers,

@TimP: Thanks for pointing me to the loop count and alignment directives, I'll take a look at them now and let you know if they work! @Martyn: I'm basically fiddling with some very ugly generated code produced by a source to source compiler to see how to change it in such a way that ifort will vectorize the instructions, below is the subroutine in question and the subroutine which calls the 'problematic' subroutine (both within the same source file): (BS is a macro defined at the beginning of the file, usually set to 4) [fortran] subroutine update_kernel_caller(qold, q, res, adt, rms, diff) implicit none integer :: diff real(kind=8), dimension(BS,diff/BS,4) :: qold, q, res real(kind=8), dimension(BS,*) :: adt real(kind=8), dimension(1) :: rms real(kind=8), dimension(BS) :: adti, del real(kind=8), dimension(BS) :: acc integer :: i,j do j=1,diff/BS adti(:) = 1.0 / adt(:,j) do i=1,4 del(:) = adti(:) * res(:,j,i) q(:,j,i) = qold(:,j,i) - del(:) res(:,j,i) = 0.0 acc(:) = del(:) * del(:) rms(1) = rms(1) + sum(acc) end do end do end subroutine [/fortran] This is called by : (The opDats originally contain data in AoS form, the mkl method turns it into SoA and stores the result into the topDats, the system clocks are just for benchmarking purposes) [fortran] subroutine update_kernel(opDat1,opDat2,opDat3,opDat4,opDat5,sliceStart,sliceEnd) implicit none real(kind=8), dimension(0:*) :: opDat1 real(kind=8), dimension(0:*) :: opDat2 real(kind=8), dimension(0:*) :: opDat3 real(kind=8), dimension(0:*) :: opDat4 real(kind=8), dimension(0:*) :: opDat5 integer(kind=4) :: sliceStart integer(kind=4) :: sliceEnd integer(kind=4) :: i1, diff integer(8) :: time1, time2, count_rate, count_max real(kind=8), allocatable, dimension(:) :: topDat1, topDat2, topDat3 diff = sliceEnd - sliceStart call system_clock(time1, count_rate, count_max) allocate (topDat1(diff*4)) allocate (topDat2(diff*4)) allocate (topDat3(diff*4)) call mkl_domatcopy('r', 't', diff, 4, 1.d0, opDat1(4*sliceStart:4*sliceEnd-1), 4, topDat1, diff) call mkl_domatcopy('r', 't', diff, 4, 1.d0, opDat2(4*sliceStart:4*sliceEnd-1), 4, topDat2, diff) call mkl_domatcopy('r', 't', diff, 4, 1.d0, opDat3(4*sliceStart:4*sliceEnd-1), 4, topDat3, diff) call system_clock(time2, count_rate, count_max) print *, "### not counted", time2 - time1 call update_kernel_caller(topDat1, topDat2, topDat3, opDat4(sliceStart:sliceStart+diff-1), opDat5, diff) call system_clock(time1, count_rate, count_max) call mkl_domatcopy('r', 't', 4, diff, 1.d0, topDat1, diff, opDat1(4*sliceStart:4*sliceEnd-1), 4) call mkl_domatcopy('r', 't', 4, diff, 1.d0, topDat2, diff, opDat2(4*sliceStart:4*sliceEnd-1), 4) call mkl_domatcopy('r', 't', 4, diff, 1.d0, topDat3, diff, opDat3(4*sliceStart:4*sliceEnd-1), 4) deallocate (topDat1) deallocate (topDat2) deallocate (topDat3) call system_clock(time2, count_rate, count_max) print *, "### not counted", time2 - time1 end subroutine [/fortran] I apologize for the horrible spacing... in the forum editor it appears nicely, but somehow it becomes double spaced when posting... I'll see if I can do something about it... Regards, Alex
0 Kudos
Reply