cannot find offload entry __offload_entry__XXX

[vincent_b_]

Hi,

I'm trying to offload a loop in a fortran program.

!dir$ offload begin target(mic) 
DO j = 1,10
!$omp parallel do
        DO i = 1,10
                c(i,j) = a(i,j) + b(i,j)
        END DO
END DO
!dir$ end offload

There is no function inside the offloaded loop.

It compiles but then when I run the executable I get this error :
offload error: cannot find offload entry __offload_entry_zdfgls_f90_212zdfgls_mp_zdf_gls_ifort9461363648KyrNb

When I look into my zdfgls.o file for symbols, I get :
__offload_entry_zdfgls_f90_212zdfgls_mp_zdf_gls_ifort9461363648KyrNb_$entry

The sole difference is the "_$entry" added at the end of the symbol.

Here is what I obtain in the whole zdfgls.o file :
3:0000000000000000 d .2.13_2__offload_var_desc1_p.117
4:0000000000000880 d .2.13_2__offload_var_desc2_p.122
215:0000000000000000 d __offload_entry_zdfgls_f90_212zdfgls_mp_zdf_gls_ifort9461363648KyrNb_$entry
216:                 U __offload_offload1
217:                 U __offload_target_acquire

Thanks for your help.

Vincent

[Kevin_D_Intel]

Interesting. Pretty simple offload and the directives are correct. The compiler creates an outlined function containing the offload code that is part of the offload image and that relates to the entry symbols you noted.

Which particular ifort do you have (ifort -V)?

Are you able to reproduce this with a small complete program that you could share with me?

[vincent_b_]

Hi Kevin,

Are you saying that the added suffix "_$entry" is normal for the symbol? 

This is the result of ifort -V:
Intel(R) Fortran Intel(R) 64 Compiler XE for applications running on Intel(R) 64, Version 14.0.3.174 Build 20140422
Copyright (C) 1985-2014 Intel Corporation.  All rights reserved.

The offload is pretty simple yes. But the code is much bigger.
In fact, I'm offloading a loop with some +, x and sqrt operations on matrix arrays but nothing really complicated.

However, what might bring some complexity is that the object file is then used with other non-offload object files to create a static library with xiar.

Thank you.

Vincent
 

[vincent_b_]

Note that I added "-qoffload-build" to the xiar command without success.

Thank you.

Vincent

[Kevin_D_Intel]

Yes the symbol is correct. Building/mixing objects with and without offload within the static library should not be an issue. Whatever your combination is, it looks like a defect to me.

Did you add any USE mic_lib for using the compiler's offload module?

[vincent_b_]

Hi Kevin,

I've tried with and without "USE mic_lib" without success.
By the way "USE mic_lib" in fortran and "#include <offload.h>" in C/C++ aren't used only if you use offload functions such as :
__Offload_number_of_devices() , etc. ?

If the symbol __offload_entry_zdfgls_f90_212zdfgls_mp_zdf_gls_ifort9461363648KyrNb_$entry is correct, then it shouldn't look for __offload_entry_zdfgls_f90_212zdfgls_mp_zdf_gls_ifort9461363648KyrNb when running the binary you mean? Or is it normal to look for that symbol minus "_$entry" and the error is that it should not be missing it?

Thanks,

Vincent

[Kevin_D_Intel]

Ok. I asked because I have come across a weird case that must be different that yours where including mic_lib leads to the error you see.

Yes, mic_lib and offload.h are included when using the offload APIs but may also be included w/o harm.

I cannot speak too much to the different symbols in the object versus the error since I don't have a lot of in-depth knowledge the internals of how the offload run-time. Based on what I see in working programs, I do not believe the symbols have any relation to the error you are experiencing. From working programs, I see the same symbol difference you note. The $entry symbol you noted is in the data section of the CPU (host) object and that is *not* the object that corresponds to the Phi (target) executable image. The <file>MIC.o is the Phi (target) object where you find the $entry form of the symbol (in the data section) and the form w/o the $entry tag in the text (code) section which I expect equates to the symbol reported in the error. Again, I don’t believe the symbols have any relation to the error you are experiencing but I do feel like there is a compiler defect related to your failure; however, w/o having a reproducer I am at a loss to explain it or offer any work around at the moment.

Is it possible you can share the code w/me that’s causing the error?  I believe our Forum Private message facility is still not working so you can share the code in private with us using our Intel Premier system and request the Premier issue be assigned to me if you wish.

[vincent_b_]

Ok. I can't share the code (licence issue). But that might help...

We had an error before this one :
undefined reference to `__offload_target_acquire'
undefined reference to `__offload_offload1'

We thought we had fixed it by giving include and lib paths for MIC at compilation.
And that gave the issue we discussed earlier.

However, if I look at the order of the symbols in the object file :
215:0000000000000000 d __offload_entry_zdfgls_f90_212zdfgls_mp_zdf_gls_ifort9461363648KyrNb_$entry
216:                 U __offload_offload1
217:                 U __offload_target_acquire

it seems that symbol _offload_entry_zdfgls_f90_212zdfgls_mp_zdf_gls_ifort9461363648KyrNb_$entry is "tested" before the "offload_offload1" symbol. So manually adding paths for mic include and lib maybe has not fixed our first error.

Hope you understood my point.

PS: I tested some examples from intel sample00C.c, etc and some test cases of mine and had no issue so I don't think that might come from the environment variables for MIC. (we're sourcing the compilervars.sh)

Thank you

[Kevin_D_Intel]

I haven't been able to re-create your error but my sense is that I don't believe the way to resolve the first error you mentioned with the unresolved offload symbols should be via path/include changes. I believe you should be trying to do that via the compiler driver and maybe forcing -offload option on the final executable link so the driver can include the proper libraries. That those offload routines were unresolved suggests that the main executable is not aware of the offloading in your libraries and it needs to be which is why you'd force the -offload option.

Are you linking your final executable using the ifort compiler driver or ld directly?

When linking to create the final executable, does that only link static libraries that contain the offloaded code you showed earlier or are there shared libraries involved here also?

[Kevin_D_Intel]

I just reproduced both of your errors. What I think is happening somehow is either you are still building your static library using only ar or there is a malfunction of some sort (most likely involving the <file>MIC.o files) associated with creating the static library containing offloaded code when using xiar.

If you build a static library using only ar and not xiar, where a routine in the library contains offload code, the link for the final executable fails with something like this:

$ ifort -o main_exe main.f90 -L. -lfoobar -openmp
./libfoobar.a(bar.o): In function `barm_mp_bar_':
bar.f90:(.text+0x2e): undefined reference to `__offload_target_acquire'
bar.f90:(.text+0x60): undefined reference to `__offload_offload1'

If one simply forces linking of -loffload (or uses the -offload option) the link succeeds; however, execution fails as you noted earlier where the entry associated with the subroutine containing the offload code is not found. In my case, bar() contains offload code and foo() calls bar(). So, when I run the executable (where my static lib is build using ar and my main is built with -offload) it fails as follows:

$ ifort -o main_exe main.f90 -L. -lfoobar -openmp -offload
$ ./main_exe
offload error: cannot find offload entry __offload_entry_bar_f90_4barm_mp_bar_ifort411230073Ao6hIL
offload error: process on the device 0 unexpectedly exited with code 1

In your case, the static library with the routine containing offloaded code was probably not built as needed.

From additional testing I also found that the same set of errors above will occur if somehow the <file>MIC.o is not included in the static library when it is built. That includes when building using xair -qoffload-build and you will not receive any sort of error at library creation time that a <file>MIC.o was not found.

Double check that you are using xiar with -qoffload-build and also make certain you are not manipulating any <file>MIC.o files in any way. Ensure all <file>MIC.o files are present alongside their host counterpart <file>.o files when xiar executes.

In our (15.0 compiler) release due out later this year, we a single create “fat” .o files containing both the CPU and MIC object files so users will no longer see the MIC object.

[vincent_b_]

Hi Kevin,

That was indeed the problem.
The build would compile F90 files and move only *.o files (not *MIC.o) to the directory were the static library was created.

Thanks again!
You did a great job!

[Kevin_D_Intel]

Excellent!  I'm happy to hear that. Thank you for the kind words too. Hopefully our exchange can prove useful to others who may face a similar error. As I noted the "fat" files coming in our next (15.0) release will help others avoid this in the future.

I wish you well moving forward.

[aketh_t_]

hi all,

is it possible to get build and run successfully using ar instead of xair.

[Kevin_D_Intel]

No. xiar is offload and IPO aware, ar is not.