Linking against libsgx_tprotected_fs.a fails

[Marion_M_Intel]

Hello,

I've been trying to use the protected_fs API from SGX within a test enclave project. The enclave finds the header file just fine, but when instructing the linker to link against -lsgx_tprotected_fs I get a whole lot of undefined reference errors. My experience in Linux development is close to 0, I would suspect something went wrong in building sgx_tprotected_fs.a, but really no idea. 

How can I resolve this?

The enclave linker flags:

Enclave_Link_Flags := $(SGX_COMMON_CFLAGS) -Wl,--no-undefined -nostdlib -nodefaultlibs -nostartfiles -L$(SGX_LIBRARY_PATH) \

-Wl,--whole-archive -l$(Trts_Library_Name) -Wl,--no-whole-archive \

-Wl,--start-group -lsgx_tstdc -lsgx_tstdcxx -l$(Crypto_Library_Name) -l$(Service_Library_Name) -lsgx_tprotected_fs -Wl,--end-group \

-Wl,-Bstatic -Wl,-Bsymbolic -Wl,--no-undefined \

-Wl,-pie,-eenclave_entry -Wl,--export-dynamic  \

-Wl,--defsym,__ImageBase=0 

The gcc output:

GEN  =>  App/Enclave_u.c

CC   <=  App/Enclave_u.c

CXX  <=  App/App.cpp

CXX  <=  App/sgx_utils/sgx_utils.cpp

LINK =>  app

GEN  =>  Enclave/Enclave_t.c

CC   <=  Enclave/Enclave_t.c

CXX  <=  Enclave/Enclave.cpp

CXX  <=  Enclave/Sealing/Sealing.cpp

[...]sdk/lib64/libsgx_tprotected_fs.a(file_flush.o): In function `protected_fs_file::write_recovery_file()':

file_flush.cpp:(.text._ZN17protected_fs_file19write_recovery_fileEv+0x45): undefined reference to `u_sgxprotectedfs_recovery_file_open'

file_flush.cpp:(.text._ZN17protected_fs_file19write_recovery_fileEv+0x9e): undefined reference to `u_sgxprotectedfs_fwrite_recovery_node'

file_flush.cpp:(.text._ZN17protected_fs_file19write_recovery_fileEv+0xee): undefined reference to `u_sgxprotectedfs_fwrite_recovery_node'

file_flush.cpp:(.text._ZN17protected_fs_file19write_recovery_fileEv+0x111): undefined reference to `u_sgxprotectedfs_fclose'

file_flush.cpp:(.text._ZN17protected_fs_file19write_recovery_fileEv+0x121): undefined reference to `u_sgxprotectedfs_remove'

file_flush.cpp:(.text._ZN17protected_fs_file19write_recovery_fileEv+0x16e): undefined reference to `u_sgxprotectedfs_fclose'

file_flush.cpp:(.text._ZN17protected_fs_file19write_recovery_fileEv+0x179): undefined reference to `u_sgxprotectedfs_remove'

file_flush.cpp:(.text._ZN17protected_fs_file19write_recovery_fileEv+0x18d): undefined reference to `u_sgxprotectedfs_fclose'

file_flush.cpp:(.text._ZN17protected_fs_file19write_recovery_fileEv+0x198): undefined reference to `u_sgxprotectedfs_remove'

file_flush.cpp:(.text._ZN17protected_fs_file19write_recovery_fileEv+0x1bb): undefined reference to `u_sgxprotectedfs_fclose'

file_flush.cpp:(.text._ZN17protected_fs_file19write_recovery_fileEv+0x1c6): undefined reference to `u_sgxprotectedfs_remove'

file_flush.cpp:(.text._ZN17protected_fs_file19write_recovery_fileEv+0x1e1): undefined reference to `u_sgxprotectedfs_fwrite_recovery_node'

file_flush.cpp:(.text._ZN17protected_fs_file19write_recovery_fileEv+0x20b): undefined reference to `u_sgxprotectedfs_fclose'

file_flush.cpp:(.text._ZN17protected_fs_file19write_recovery_fileEv+0x216): undefined reference to `u_sgxprotectedfs_remove'

file_flush.cpp:(.text._ZN17protected_fs_file19write_recovery_fileEv+0x236): undefined reference to `u_sgxprotectedfs_fclose'

file_flush.cpp:(.text._ZN17protected_fs_file19write_recovery_fileEv+0x24d): undefined reference to `u_sgxprotectedfs_fclose'

file_flush.cpp:(.text._ZN17protected_fs_file19write_recovery_fileEv+0x258): undefined reference to `u_sgxprotectedfs_remove'

/opt/intel/sgxsdk/lib64/libsgx_tprotected_fs.a(file_flush.o): In function `protected_fs_file::set_update_flag(bool)':

file_flush.cpp:(.text._ZN17protected_fs_file15set_update_flagEb+0x43): undefined reference to `u_sgxprotectedfs_fwrite_node'

file_flush.cpp:(.text._ZN17protected_fs_file15set_update_flagEb+0xa5): undefined reference to `u_sgxprotectedfs_fflush'

file_flush.cpp:(.text._ZN17protected_fs_file15set_update_flagEb+0xd5): undefined reference to `u_sgxprotectedfs_fwrite_node'

/opt/intel/sgxsdk/lib64/libsgx_tprotected_fs.a(file_flush.o): In function `protected_fs_file::clear_update_flag()':

file_flush.cpp:(.text._ZN17protected_fs_file17clear_update_flagEv+0x31): undefined reference to `u_sgxprotectedfs_fwrite_node'

file_flush.cpp:(.text._ZN17protected_fs_file17clear_update_flagEv+0x42): undefined reference to `u_sgxprotectedfs_fflush'

/opt/intel/sgxsdk/lib64/libsgx_tprotected_fs.a(file_flush.o): In function `protected_fs_file::write_all_changes_to_disk(bool)':

file_flush.cpp:(.text._ZN17protected_fs_file25write_all_changes_to_diskEb+0x5a): undefined reference to `u_sgxprotectedfs_fwrite_node'

file_flush.cpp:(.text._ZN17protected_fs_file25write_all_changes_to_diskEb+0xcd): undefined reference to `u_sgxprotectedfs_fwrite_node'

file_flush.cpp:(.text._ZN17protected_fs_file25write_all_changes_to_diskEb+0x145): undefined reference to `u_sgxprotectedfs_fwrite_node'

file_flush.cpp:(.text._ZN17protected_fs_file25write_all_changes_to_diskEb+0x1b5): undefined reference to `u_sgxprotectedfs_fflush'

/opt/intel/sgxsdk/lib64/libsgx_tprotected_fs.a(file_flush.o): In function `protected_fs_file::erase_recovery_file()':

file_flush.cpp:(.text._ZN17protected_fs_file19erase_recovery_fileEv+0x45): undefined reference to `u_sgxprotectedfs_remove'

/opt/intel/sgxsdk/lib64/libsgx_tprotected_fs.a(file_init.o): In function `protected_fs_file::file_recovery(char const*)':

file_init.cpp:(.text._ZN17protected_fs_file13file_recoveryEPKc+0x3f): undefined reference to `u_sgxprotectedfs_fclose'

file_init.cpp:(.text._ZN17protected_fs_file13file_recoveryEPKc+0x9e): undefined reference to `u_sgxprotectedfs_do_file_recovery'

file_init.cpp:(.text._ZN17protected_fs_file13file_recoveryEPKc+0xc8): undefined reference to `u_sgxprotectedfs_exclusive_file_open'

file_init.cpp:(.text._ZN17protected_fs_file13file_recoveryEPKc+0x11e): undefined reference to `u_sgxprotectedfs_fread_node'

/opt/intel/sgxsdk/lib64/libsgx_tprotected_fs.a(file_init.o): In function `protected_fs_file::init_existing_file(char const*, char const*, unsigned char const (*) [16])':

file_init.cpp:(.text._ZN17protected_fs_file18init_existing_fileEPKcS1_PA16_Kh+0x46): undefined reference to `u_sgxprotectedfs_fread_node'

file_init.cpp:(.text._ZN17protected_fs_file18init_existing_fileEPKcS1_PA16_Kh+0x1a1): undefined reference to `u_sgxprotectedfs_fread_node'

/opt/intel/sgxsdk/lib64/libsgx_tprotected_fs.a(file_init.o): In function `protected_fs_file::protected_fs_file(char const*, char const*, unsigned char const (*) [16], unsigned char const (*) [16])':

file_init.cpp:(.text._ZN17protected_fs_fileC2EPKcS1_PA16_KhS4_+0x1c7): undefined reference to `u_sgxprotectedfs_check_if_file_exists'

[..... output omitted .....]

[you_w_]

Hi Marion M:

As I can see, all undefined reference starts with "u_", so did you link the untrusted part with-lsgx_uprotected_fs? Check it !

I have write a very simple demo about how to use protected_fs API : https://github.com/light1021/SGX-protected-fs-demo . You can make a reference to it. 

Regards

you

[Marion_M_Intel]

Hello!

Many thanks for the quick reply! I indeed had not linked the lsgx_uprotected_fs lib, awesome. I found some other things I was missing in your sample code too, again aweseome. I still cant compile now though, for another set of undefined references that seem.. C++ related. Do you maybe have a clue what is wrong here too?

Cheers, Marion

GEN  =>  App/Enclave_u.c

CC   <=  App/Enclave_u.c

CXX  <=  App/App.cpp

CXX  <=  App/sgx_utils/sgx_utils.cpp

LINK =>  app

GEN  =>  Enclave/Enclave_t.c

CC   <=  Enclave/Enclave_t.c

CXX  <=  Enclave/Enclave.cpp

CXX  <=  Enclave/Sealing/Sealing.cpp

/opt/intel/sgxsdk/lib64/libsgx_tprotected_fs.a(lru_cache.o): In function `std::__1::__hash_table<std::__1::__hash_value_type<unsigned long, _map_node*>, std::__1::__unordered_map_hasher<unsigned long, std::__1::__hash_value_type<unsigned long, _map_node*>, std::__1::hash<unsigned long>, true>, std::__1::__unordered_map_equal<unsigned long, std::__1::__hash_value_type<unsigned long, _map_node*>, std::__1::equal_to<unsigned long>, true>, std::__1::allocator<std::__1::__hash_value_type<unsigned long, _map_node*> > >::rehash(unsigned long)':

lru_cache.cpp:(.text._ZNSt3__112__hash_tableINS_17__hash_value_typeImP9_map_nodeEENS_22__unordered_map_hasherImS4_NS_4hashImEELb1EEENS_21__unordered_map_equalImS4_NS_8equal_toImEELb1EEENS_9allocatorIS4_EEE6rehashEm[_ZNSt3__112__hash_tableINS_17__hash_value_typeImP9_map_nodeEENS_22__unordered_map_hasherImS4_NS_4hashImEELb1EEENS_21__unordered_map_equalImS4_NS_8equal_toImEELb1EEENS_9allocatorIS4_EEE6rehashEm]+0x77): undefined reference to `std::__1::__next_prime(unsigned long)'

lru_cache.cpp:(.text._ZNSt3__112__hash_tableINS_17__hash_value_typeImP9_map_nodeEENS_22__unordered_map_hasherImS4_NS_4hashImEELb1EEENS_21__unordered_map_equalImS4_NS_8equal_toImEELb1EEENS_9allocatorIS4_EEE6rehashEm[_ZNSt3__112__hash_tableINS_17__hash_value_typeImP9_map_nodeEENS_22__unordered_map_hasherImS4_NS_4hashImEELb1EEENS_21__unordered_map_equalImS4_NS_8equal_toImEELb1EEENS_9allocatorIS4_EEE6rehashEm]+0x104): undefined reference to `std::__1::__next_prime(unsigned long)'

collect2: error: ld returned 1 exit status

Makefile:206: recipe for target 'enclave.so' failed

make: *** [enclave.so] Error 1

[you_w_]

Hi Marion：

I have never see this error before. Can your share the content of “lru_cache.cpp" ? It seems like you enclave didn't link some library that lru_cache need.

Thanks

You. 

[you_w_]

Hello:

I'm sorry. I find that Lru_cache is a part of protected_fs. It seems that "std::__1::__* " is not a part of stdc++.  Lru uses libc++ you  can install it by run "apt install libc++-dev", Then you need to add -lc++ in your enclave link flag.Try it！

$(Enclave_Name): $(Enclave_C_Objects) enclave_t.o
	@echo $^
	@$(CC) $^ -o $@ $(Enclave_Link_Flags) -lc++
	@echo "LINK =>  $@"

Regards 

you

[Marion_M_Intel]

Hey,

I found this other thread https://software.intel.com/en-us/forums/intel-software-guard-extensions-intel-sgx/topic/711404https://software.intel.com/en-us/forums/intel-software-guard-extensions-intel-sgx/topic/711404

which seems to talk about the same issue. I did change my stdlib to sgx_tcxx.lib and now it compiles fine. I'll switch to your proposed solution though and see how this works out, many thanks!