Illegal instruction on sgx_create_ex() for Switchless Library

[Jaemin_Park__Jaemin]

Dear,

I'm implementing a prototype to use Switchless Library.

A sample code of Switchless in SDK works fine in my machine (NUC8i7HVK) that runs Ubuntu Desktop 16.04.5 (64-bit).

However, my prototype receives SIGILL when the prototype invokes sgx_create_ex().

I followed the sample code of Switchless to implement my prototype.

Unfortunately, I cannot open all about my prototype, but I hope I can get any hint to fix this problem.

o Environment: Intel NUC8i7HVK(running Ubuntu Desktop 16.04.5 (64-bit)), newest linux SGX SDK and SGXSSL

o Makefile (parts of it)

 SGX_COMMON_FLAGS += -Wall -Wextra -Winit-self -Wpointer-arith -Wreturn-type \
                    -Waddress -Wsequence-point -Wformat-security \
                    -Wmissing-include-dirs -Wfloat-equal -Wundef -Wshadow \
                    -Wcast-align -Wcast-qual
SGX_COMMON_CFLAGS := $(SGX_COMMON_FLAGS) -Wjump-misses-init -Wstrict-prototypes -Wunsuffixed-float-constants

###### SGX SSL library #####
USGXSSL_LIBRARY := sgx_usgxssl
USGXSSL_CFLAGS := -fpic -fpie -fstack-protector -Wformat -Wformat-security -Wno-attribute
COMMON_SGXSSL_LINK_FLAGS := -Wl,-z,noexecstack -Wl,-z,relro -Wl,-z,now -pie

URTS_LIB_NAME := sgx_urts
SERVICE_LIB_NAME := sgx_uae_service

CFLAGS += $(SGX_COMMON_CFLAGS) $(USGXSSL_CFLAGS) 
CFLAGS += -I$(SGX_SDK)
CFLAGS += -I$(SGX_SDK)/include
LDLIBS += $(SGX_COMMON_CFLAGS) $(COMMON_SGXSSL_LINK_FLAGS)
LDLIBS += -L$(SGX_LIBRARY_PATH) -l$(URTS_LIB_NAME) -lpthread -l$(SERVICE_LIB_NAME)
LDLIBS += -L$(SGXSSL_LIBRARY_PATH) -l$(USGXSSL_LIBRARY) 

##### SGX Enclave settings #####
TRTS_LIB_NAME := sgx_trts
TSERVICE_LIB_NAME := sgx_tservice
CRYPTO_LIB_NAME := sgx_tcrypto
TSGXSSL_LIB_NAME := sgx_tsgxssl
TSGXSSL_CRYPTO_LIB_NAME := sgx_tsgxssl_crypto

ENCLAVE_C_SRC := $(wildcard $(ENCLAVE_PATH)/*.c) 
ENCLAVE_C_OBJ := $(ENCLAVE_C_SRC:.c=.o)

ENCLAVE_INC := -I. -I$(ENCLAVE_PATH) -I$(SGX_SDK) -I$(SGX_SDK)/include -I$(SGX_SDK)/include/tlibc -I$(SGX_SDK)/include/stlport -I$(SGXSSL_PACKAGE)/include

COMMON_C_CPP_FLAGS := -DOS_ID=1 $(SGX_COMMON_CFLAGS) -nostdinc -fvisibility=hidden -fpic -fpie -fstack-protector -fno-builtin-printf -Wformat -Wformat-security $(ENCLAVE_INC) -include "tsgxsslio.h" 

ENCLAVE_C_FLAGS := $(COMMON_C_CPP_FLAGS) -Wno-implicit-function-declaration -std=c11
ENCLAVE_CPP_FLAGS := $(COMMON_C_CPP_FLAGS) -std=c++11 -nostdinc++

TSGXSSL_LINK_FLAGS := -L$(SGXSSL_LIBRARY_PATH) -Wl,--whole-archive -l$(TSGXSSL_LIB_NAME) \
                      -Wl,--no-whole-archive -l$(TSGXSSL_CRYPTO_LIB_NAME)

ENCLAVE_LFLAGS := $(SGX_COMMON_CFLAGS) -Wl,--no-undefined -nostdlib -nodefaultlibs -nostartfiles \
          -Wl,--whole-archive -lsgx_tswitchless -l$(TRTS_LIB_NAME) -Wl,--no-whole-archive \
          $(COMMON_SGXSSL_LINK_FLAGS) \
          $(TSGXSSL_LINK_FLAGS) \
          $(SGXSSL_LIBRARY) -L$(SGX_LIBRARY_PATH) \
                  -Wl,--start-group -lsgx_tstdc -lsgx_tcxx -l$(CRYPTO_LIB_NAME) -l$(TSERVICE_LIB_NAME) -Wl,--end-group \
                  -Wl,-Bstatic -Wl,-Bsymbolic -Wl,--no-undefined \
                  -Wl,-pie,-eenclave_entry -Wl,--export-dynamic \
                  -Wl,--defsym,__ImageBase=0 \
          -Wl,--version-script=$(ENCLAVE_PATH)/enclave.lds

ENCLAVE_NAME := enclave.so
SIGNED_ENCLAVE_NAME := enclave.signed.so
ENCLAVE_CONFIG_FILE := enclave.config.xml

Build_Mode := HW_DEBUG

[Park__Seonghyun1]

Hi, I am encountering similar issue here.

Simply linking libtsgx_ssl libraries with switchless library causes an error (Illegal instruction)

Did you solve this issue?

[Jaemin_Park__Jaemin]

Today, I think I figure out this issue.

I referred to this post (https://github.com/intel/linux-sgx/issues/414).

Then, I rebuilt SGX SDK with "make USE_OPT_LIBS=0" and also PSE ("make").

I installed SDK and also PSE, and rebuilt linux-sgx-driver (and also installed it).

Finally, I could create my enclave with switchless.

I hope this can help others.

[Jaemin_Park__Jaemin]

Today I attempted to use `sgx-gdb` for some debugs.

And then I came back to the previous status, but unfortunately, I faced the same error, illegal instruction, in sgx_create_ex().

Now I'm finding a way to fix it.

[Jaemin_Park__Jaemin]

Finally, I could figure out my problem.

I re-built SGX-SDK and also PSW with "USE_OPT_LIBS=0" option as follows:

$ cd linux-sgx

$ make USE_OPT_LIBS=0

$ make sdk_install_pkg USE_OPT_LIBS=0

$ make deb_pkg USE_OPT_LIBS=0

Installing the built SDK and PSW and re-building my enclave that leverages Switchless could fix the problem.

I hope this can help others.

[Shao__Qifeng]

I am encountering similar issue too.

That may be a conflict between switchless libraries and libtsgx_ssl libraries