SIGBUS when running multiple threads in enclave

[xunf]

Hi,

I wonder if anyone have ever encountered bus errors when running multiple threads inside enclave.

My application launches multiple threads to enter enclave, each handling a request. Inside enclave they decrypt buffers of AES GCM encrypted contents in parallel and access shared data coordinated by mutexes. When I increase concurrency to 4 or 8 threads, some time the apps is terminated by SIGBUS signal. And the sgx-gdb points to locations with memory operations like memcpy/memset. I tried to implement similar logics outside SGX and no memory leak is detected with repeated testing using valgrind/asan.

I am using SGX SDK 2.14 on ubuntu 20.04 with in-kernel driver. Any suggestions is greatly appreciated.

Best regards,

xunf

[Sahira_Intel]

Hi Xunf,

Can you share the exact error messages you are getting?

Sincerely,

Sahira

[xunf]

Hi Sahira,

The message is like  "Bad memory access (SIGBUS) bus error".

I have managed to get rid of the problem by rolling back to an older kernel version that worked for me: linux 5.13.0-41-generic. The problematic combination was linux 5.15.0-46 with SGX SDK 2.14. I am still not sure what caused the problem, any thoughts?

Btw, is there any table that describes the compatibility between SDK versions and kernel versions?

Best regards,

xunf

[Sahira_Intel · Accepted solution]

Hi Xunf,

There's not any table listing compatibility specifically for the SGX SDK, but it is listed the mainline Linux kernel has built-in SGX support since release 5.11, and the in-kernel driver requires the platform to support and be configured for flexible launch control. If you are able to, I would suggest upgrading your SDK version to the latest 2.17 and trying again. 

If you are still getting the error, would you be able to share a sample of your code so I can run it on my end?

Sincerely,

Sahira