Intel® Software Guard Extensions (Intel® SGX)
Use hardware-based isolation and memory encryption to provide more code protection in your solutions.

Slow initialization of SGX Enclave

shmoo
Novice
619 Views

Hello!

I am using the newest SGX with the newest SDK version on a pretty fast machine that supports SGX 2.0. I wrote a test that creates and closes 100 enclaves and measures the initialization time. 

Creating an enclave with 

<HeapInitSize>0x1000000</HeapInitSize> // 16 MB

and its first initialization e_call (does nothing special, but is necessary because otherwise internally some sgx calls are not called and the enclave performance test is useless)

takes between 59 and 65 milliseconds. Ok. Its not super fast but it is acceptable. 

Creating an enclave with 

<HeapInitSize>0x40000000</HeapInitSize> // 1 GB

takes between 2000 and 6000 milliseconds! 

Why does it take that long? Can I improve the initialization time somehow?

Thank you!

0 Kudos
1 Solution
JesusG_Intel
Moderator
576 Views

Hello shmoo,


It is expected that initializing large amounts of memory for an enclave heap will take a long time. Initializing and adding memory to the Enclave Page Cache (EPC) for an enclave is much more complex than a simple memory allocation in a regular application. Not getting into too much detail here, before a chunk of memory is added to the EPC it must go through several security checks, including being measured using a 256-bit SHA-2 secure hash function.

 

A good place to start for more details is the SGX Explained paper (sections 5.1, 5.3, 5.6). Also look at the EADD and EEXTEND instructions in Intel SGX Enclave Lifecycle.

 

Following the SGX code:

 

  1. First the signtool determines enclave layout. It places the layout information in the metadata. Here is where the heap is added and set to EADD only: manage_metadata#L775
  2. During enclave load, the untrusted Run-time System (uRTS) loader loops through the layout entries and adds appropriately: loader.cpp#L382
  3. Each enclave page is added by calling the driver, which calls EADD: loader.cpp#L311


Sincerely,

Jesus G.

Intel Customer Support


View solution in original post

6 Replies
JesusG_Intel
Moderator
601 Views

Hello shmoo,


Since you have an SGX2 CPU (has EDMM), you should probably set HeapMaxSize to 1GB, and set HeapInitSize to something minimal. HeapInitSize is the amount to actually fully initialize (ie EADD the page then EEXTEND it) at enclave launch, which takes time. With a minimal Init size, it would be quicker, and the rest of the memory would only be added utilizing the SGX2 instructions when malloc'd/needed by the enclave during runtime.


Sincerely,

Jesus G.

Intel Customer Support


shmoo
Novice
592 Views

Hello!

Yeah I know. We will do this once the servers where we deploy our solution also support SGX2
But now we have to initialize it that big. And I think a few seconds for allocating some memory is quite long? Or do you experience the same issues?

Sincerely,

David

JesusG_Intel
Moderator
577 Views

Hello shmoo,


It is expected that initializing large amounts of memory for an enclave heap will take a long time. Initializing and adding memory to the Enclave Page Cache (EPC) for an enclave is much more complex than a simple memory allocation in a regular application. Not getting into too much detail here, before a chunk of memory is added to the EPC it must go through several security checks, including being measured using a 256-bit SHA-2 secure hash function.

 

A good place to start for more details is the SGX Explained paper (sections 5.1, 5.3, 5.6). Also look at the EADD and EEXTEND instructions in Intel SGX Enclave Lifecycle.

 

Following the SGX code:

 

  1. First the signtool determines enclave layout. It places the layout information in the metadata. Here is where the heap is added and set to EADD only: manage_metadata#L775
  2. During enclave load, the untrusted Run-time System (uRTS) loader loops through the layout entries and adds appropriately: loader.cpp#L382
  3. Each enclave page is added by calling the driver, which calls EADD: loader.cpp#L311


Sincerely,

Jesus G.

Intel Customer Support


Bronze_me
Novice
313 Views

HiJesus G,

 

The EDMM feature of SGX 2.0 actually can be a good solution to reduce the overhead of initializing an enclave which contains  large amounts of EPC page.

 

However, it seems the upstream linux kernel  had not support the SGX 2.0 (EDMM) yet.

 

The Intel-sgx-driver  was deprecated.

 

Is there any ways to enable the EDMM feature on a SGX 2.0 CPU? and how do we use it?

 

Thanks.

 

shmoo
Novice
556 Views

Hi!

Thanks for the detailed answer. 

Long story short -> it is just slow. Either we reduce enclave size or we switch to SGX 2.0.

Thanks!
David

JesusG_Intel
Moderator
527 Views

This thread has been marked as answered and Intel will no longer monitor this thread. If you want a response from Intel in a follow-up question, please open a new thread.


Reply