undefined reference to `sgx_oc_cpuidex'

[Ko__Ronny]

Hi,

I am trying to compile sgx-nbench from my machine. In its Makefile, its Enclave_Link_Flags uses the deprecated -lsgx_tstdcxx library, so I modified this to -lsgx_tcxx and then tried "make". But I get the following error:

```
g++ Enclave/encl_emfloat.o Enclave/Enclave_t.o Enclave/Enclave.o -o enclave.so -m64 -O2 -Wl,--no-undefined -nostdlib -nostartfiles -L/opt/intel/sgxsdk/lib64 -Wl,--whole-archive -lsgx_trts -Wl,--no-whole-archive -Wl,--start-group -lsgx_tstdc -lsgx_tcxx -lsgx_tcrypto -lsgx_tservice -Wl,--end-group -Wl,-Bstatic -Wl,-Bsymbolic -Wl,--no-undefined -Wl,-pie,-eenclave_entry -Wl,--export-dynamic -Wl,--defsym,__ImageBase=0 -Wl,--version-script=Enclave/Enclave.lds
/opt/intel/sgxsdk/lib64/libsgx_tstdc.a(se_cpuid.o): In function `sgx_cpuidex':
se_cpuid.c:(.text.sgx_cpuidex+0x6): undefined reference to `sgx_oc_cpuidex'
/opt/intel/sgxsdk/lib64/libsgx_tstdc.a(se_cpuid.o): In function `sgx_cpuid':
se_cpuid.c:(.text.sgx_cpuid+0x8): undefined reference to `sgx_oc_cpuidex'
collect2: error: ld returned 1 exit status
Makefile:199: recipe for target 'enclave.so' failed
make: *** [enclave.so] Error 1
```
I wonder if anybody could give me a hint what went wrong in the Makefile, because I spent 2 hours but still couldn't figure out why...

[Rodolfo_S_]

Hi, Ronny.

I've just checked that Makefile and I noticed that it also includes "$(SGX_SDK)/include/stlport". You should change that to "$(SGX_SDK)/include/libcxx".

Did you also make that change? There might be differences in the signatures of the functions of those libs.

Regards,

Rodolfo

[Ko__Ronny]

Hi Rodolfo,

Thanks for the comment. I changed "$(SGX_SDK)/include/stlport" to "$(SGX_SDK)/include/libcxx", but it still generates the same error..

[Rodolfo_S_]

Hi Ronny.

I have just compiled that code inside a Docker container (rodolfoams/sgx-base:sgx_2.5) with just a few modifications on that code. That Docker image has SGX SDK version 2.5 installed on it. here is the patch file that I generated:

diff --git a/App/App.cpp b/App/App.cpp
index fd84726..0b0af3d 100644
--- a/App/App.cpp
+++ b/App/App.cpp
@@ -5,7 +5,7 @@
 #include <pwd.h>
 #define MAX_PATH FILENAME_MAX
 #include "sgx_urts.h"
-#include "sgx_status.h"
+//#include "sgx_status.h"
 #include "App.h"
 #include "Enclave_u.h"
 
diff --git a/Makefile b/Makefile
index 5abf919..5da2152 100644
--- a/Makefile
+++ b/Makefile
@@ -89,13 +89,13 @@ else
 endif
 
 Enclave_Cpp_Files := Enclave/Enclave.cpp $(wildcard Enclave/Edger8rSyntax/*.cpp) $(wildcard Enclave/TrustedLibrary/*.cpp)
-Enclave_Include_Paths := -IInclude -IEnclave -I$(SGX_SDK)/include -I$(SGX_SDK)/include/tlibc -I$(SGX_SDK)/include/stlport -I$(NBENCHDIR)
+Enclave_Include_Paths := -IInclude -IEnclave -I$(SGX_SDK)/include -I$(SGX_SDK)/include/tlibc -I$(SGX_SDK)/include/libcxx -I$(NBENCHDIR)
 
 Enclave_C_Flags := $(SGX_COMMON_CFLAGS) -nostdinc -fvisibility=hidden -fpie -fstack-protector $(Enclave_Include_Paths)
 Enclave_Cpp_Flags := $(Enclave_C_Flags) -std=c++03 -nostdinc++
 Enclave_Link_Flags := $(SGX_COMMON_CFLAGS) -Wl,--no-undefined -nostdlib -nodefaultlibs -nostartfiles -L$(SGX_LIBRARY_PATH) \
 	-Wl,--whole-archive -l$(Trts_Library_Name) -Wl,--no-whole-archive \
-	-Wl,--start-group -lsgx_tstdc -lsgx_tstdcxx -l$(Crypto_Library_Name) -l$(Service_Library_Name) -Wl,--end-group \
+	-Wl,--start-group -lsgx_tstdc -lsgx_tcxx -l$(Crypto_Library_Name) -l$(Service_Library_Name) -Wl,--end-group \
 	-Wl,-Bstatic -Wl,-Bsymbolic -Wl,--no-undefined \
 	-Wl,-pie,-eenclave_entry -Wl,--export-dynamic  \
 	-Wl,--defsym,__ImageBase=0 \

Regards,

Rodolfo

[Ko__Ronny]

Hi Rodolfo,

Thanks for the patch output.  I indeed applied all those changes, yet it still doesn't work... Could you check if my attached patched package compiles okay on your machine:

Also, I cannot find any Enclave's library that defines the problematic sgx_oc_cpuidex() function:

$ find /opt/intel/sgxsdk/lib64/ -exec sh -c 'nm {} | grep "cpuidex" ' \; -print 2> /dev/null
000000000000b630 T sgx_oc_cpuidex
/opt/intel/sgxsdk/lib64/libsgx_urts_sim.so
0000000000000000 T sgx_cpuidex
                 U sgx_oc_cpuidex
/opt/intel/sgxsdk/lib64/libsgx_tstdc.a
0000000000000a50 T sgx_oc_cpuidex
/opt/intel/sgxsdk/lib64/libsgx_urts.so

libsgx_urts.so is the App's library (not the Enclave's).
Does your machine output the same?

[Rodolfo_S_ · Accepted solution]

Hi, Ronny.

Yes, I do get exactly the same output as you do, but that is the expected behavior because the "sgx_oc_cpuidex" is an ocall, so it shouldn't be defined in the trusted libraries.

What sdk version are you using? Also, are there any other errors when compiling that code?

I'm afraid I can't reproduce your error here (compiling the code in a container with sdk 2.5 installed).

Regards,
Rodolfo

[Ko__Ronny]

Hi Rodolfo,

I retested after installing SDK 2.5, but still gets the same error:
 

/opt/intel/sgxsdk/lib64/libsgx_tstdc.a(se_cpuid.o): In function `sgx_cpuidex':
se_cpuid.c:(.text.sgx_cpuidex+0x6): undefined reference to `sgx_oc_cpuidex'
/opt/intel/sgxsdk/lib64/libsgx_tstdc.a(se_cpuid.o): In function `sgx_cpuid':
se_cpuid.c:(.text.sgx_cpuid+0x8): undefined reference to `sgx_oc_cpuidex'

I think this error occurs because libsgx_tstdc.a uses the untrusted sgx_oc_cpuidex function, which none of the below libraries define:

Enclave_Link_Flags := $(SGX_COMMON_CFLAGS) -Wl,--no-undefined -nostdlib -nodefaultlibs -nostartfiles -L$(SGX_LIBRARY_PATH)  -Wl,--whole-archive -l$(Trts_Library_Name) -Wl,--no-whole-archive -Wl,--start-group -lsgx_tstdc -lsgx_tstdcxx -l$(Crypto_Library_Name) -l$(Service_Library_Name) -Wl,--end-group  -Wl,--start-group -lsgx_tstdc -lsgx_tcxx -l$(Crypto_Library_Name) -l$(Service_Library_Name) -Wl,--end-group  -Wl,-Bstatic -Wl,-Bsymbolic -Wl,--no-undefined -Wl,-pie,-eenclave_entry -Wl,--export-dynamic   -Wl,--defsym,__ImageBase=0

And nbench's Makefile's $(Enclave_Name) target ties to compile it with the following command:

g++ Enclave/encl_emfloat.o Enclave/Enclave_t.o Enclave/Enclave.o -o enclave.so -m64 -O2 -Wl,--no-undefined -nostdlib -nodefaultlibs -nostartfiles -L/opt/intel/sgxsdk/lib64 -Wl,--whole-archive -lsgx_trts -Wl,--no-whole-archive -Wl,--start-group -lsgx_tstdc -lsgx_tcxx -lsgx_tcrypto -lsgx_tservice -Wl,--end-group -Wl,-Bstatic -Wl,-Bsymbolic -Wl,--no-undefined -Wl,-pie,-eenclave_entry -Wl,--export-dynamic -Wl,--defsym,__ImageBase=0 -Wl,--version-script=Enclave/Enclave.lds

and therefore we get the following error (because the above command's input files defines the sgx_oc_cpuidex() function):

/opt/intel/sgxsdk/lib64/libsgx_tstdc.a(se_cpuid.o): In function `sgx_cpuidex': se_cpuid.c:(.text.sgx_cpuidex+0x6): undefined reference to `sgx_oc_cpuidex' /opt/intel/sgxsdk/lib64/libsgx_tstdc.a(se_cpuid.o): In function `sgx_cpuid': se_cpuid.c:(.text.sgx_cpuid+0x8): undefined reference to `sgx_oc_cpuidex' collect2: error: ld returned 1 exit status

Is my reasoning correct? If so, I'm not sure how to solve this problem..

[Ko__Ronny]

I found a solution to fix this problem. In the file Enclave/Enclave.edl, I added the following in the enclave{} block:
 

from "sgx_tstdc.edl" import sgx_oc_cpuidex;

We need this because the trusted library (libsgx_tstdc.a) is calling this untrusted function. By adding the above, the Makefile generates a Ocall for this function with the enclave can call. But I'm still not sure why your compilation worked without adding the above in Enclave.edl.

[Jose__Kevin]

Thank You , I was looking for this fix.

You save my life. Thanks again.

Kevin79,
^{https://www.todotech20.com}