Intel® Software Guard Extensions (Intel® SGX)
Discussion board focused on hardware-based isolation and memory encryption to provide extended code protection in solutions.
1448 Discussions

Why do pointers passed into ECALL do not cause EPC paging?

kai__chi
Novice
1,204 Views

Hi,

I performed a simple ECALL:

public void test_count([in, count=cnt] int* ptr, unsigned cnt);

The ptr pointed to a large data structure (let's say 1GB - way more than EPC memory). Inside of the ecall I sorted the data without allocating any additional data. I noticed that the enclave did not perform any EPC paging during the sort. If the EPC is only 128MB and I access 1GB of data, how is that possible that it doesn't trigger EPC paging?

In the Developer Reference Section "Pointer Handling in ECALLS" it states:

 When a pointer to the untrusted memory with the in attribute is passed to the
 enclave, the trusted bridge allocates memory inside the enclave and copies
 the memory pointed to by the pointer from outside to the enclave memory.

 Where does the trusted bridge allocate the memory for the data? If it's in EPC than why doesn't it trigger EPC paging? Is there any resource that explains this mechanism?

I did a second version of the ECALL, where I allocate memory inside of the enclave, copy the ptr data to the new structure, and sort it. I see enormous amounts of EPC paging in this scenario. Is EPC paging related ONLY to the memory allocated (i.e., malloc'ed) inside the enclave?

0 Kudos
1 Solution
JesusG_Intel
Moderator
1,159 Views

Hello chi__kai,


Section Structures, Enums, and Unions in the Intel SGX Developer Reference Guide for Linux explains how to achieve a deep copy of the structure elements into the trusted domain. The way you are declaring your structures is causing a shallow copy, i.e. "s SGX not copy the data into the EPC but copies only the pointer which points to the memory outside of EPC."


If you want the structure data to be copied into the EPC, declare your structures in the EDL file as below using count and size. I show here your structure definitions and the declarations that achieve a deep copy (you determine the values of count and size


   struct pair_t {

       uint32_t key;

       uint32_t value;

   };


   struct table_t {

       struct pair_t* pairs;

       uint32_t num_pairs;

   };


   struct deep_table_t{

       [count = 1, size = 12] struct pair_t* pairs;

       uint32_t num_pairs;

   };


   trusted {

       public sgx_status_t ecall_sort_table([in] struct table_t * rel);

       public sgx_status_t ecall_deep_sort_table([in, count = 1] struct deep_table_t * rel);

   };


Check the file enclave_t.c to see the generated proxy functions. The function sgx_ecall_deep_sort_table shows the iterative deep copy of the structure from untrusted memory to trusted memory.


The proxy functions are automatically generated by the edger8 tool before the code is compiled. You must tell the edger8 tool how much memory to copy in the proxy functions via the count and size parameters in the EDL file.


Sincerely,

Jesus G.

Intel Customer Support



View solution in original post

0 Kudos
5 Replies
JesusG_Intel
Moderator
1,192 Views

Hello Chi__Kai,


How did you measure paging within the EPC?


If you check the trusted proxy code in the generated proxy file enclave_t.c (or similar), you will find code like this:

if (_tmp_ptr != NULL && _len_ptr != 0) {

      _in_ptr = (void*)malloc(_len_ptr);

      if (_in_ptr == NULL) {

         status = SGX_ERROR_OUT_OF_MEMORY;

         goto err;

      }


      if (memcpy_s(_in_ptr, _len_ptr, _tmp_ptr, _len_ptr)) {

         status = SGX_ERROR_UNEXPECTED;

         goto err;

      }


 }


First, memory is allocated via malloc with size _len_ptr. _len_ptr is calculated from your ecall argument cnt. Data is then copied from _tmp_ptr to _in_ptr.


You can check how much memory your enclave uses with the Enclave Memory Measurement Tool described in the Developer Reference Guide.


Sincerely,

Jesus G.

Intel Customer Support





0 Kudos
kai__chi
Novice
1,167 Views

Hi Jesus,

Thanks for a quick response.

I measure EPC paging on the level of the SGX driver. I monitor invocation of the EWB instruction, which triggers paging out. There is an open question of mine about it [1].

I measured the memory with EMMT and for approx 1.1 GB of data passed into the enclave I got these results:

 

 

  [Peak stack used]: 13 KB
  [Peak heap used]:  302192 KB
  [Peak reserved memory used]:  0 KB

 

 

I could also see some EPC paging. I still don't understand how 1.1GB of memory in the untrusted app became 302192 KB in the enclave but at least I saw some EPC paging, allocated memory, etc.

Now, in the original question I wasn't fully precise. I was not passing an array pointer but a structure with an array pointer inside. So my ECALL looks like this now:

 

 

public sgx_status_t ecall_sort_table([in] struct table_t * rel);

 

 

This is more or less how my structures look like:

 

 

struct pair_t {
    uint32_t key;
    uint32_t value;
};

struct table_t {
    struct pair_t* pairs;
    uint32_t num_pairs;
};

 

 

When I call ecall_sort_table with a structure that allocates 1.1 GB I get these results from EMMT:

 

 

  [Peak stack used]: 49 KB
  [Peak heap used]:  10240 KB
  [Peak reserved memory used]:  0 KB

 

 

What does that mean? Does SGX not copy the data into the EPC but copies only the pointer which points to the memory outside of EPC?

I have troubles understanding this. I would appreciate any help!

 

[1] https://community.intel.com/t5/Analyzers/How-to-count-EPC-paging-EPC-hits-and-EPC-misses/m-p/1255304#M19989

0 Kudos
JesusG_Intel
Moderator
1,160 Views

Hello chi__kai,


Section Structures, Enums, and Unions in the Intel SGX Developer Reference Guide for Linux explains how to achieve a deep copy of the structure elements into the trusted domain. The way you are declaring your structures is causing a shallow copy, i.e. "s SGX not copy the data into the EPC but copies only the pointer which points to the memory outside of EPC."


If you want the structure data to be copied into the EPC, declare your structures in the EDL file as below using count and size. I show here your structure definitions and the declarations that achieve a deep copy (you determine the values of count and size


   struct pair_t {

       uint32_t key;

       uint32_t value;

   };


   struct table_t {

       struct pair_t* pairs;

       uint32_t num_pairs;

   };


   struct deep_table_t{

       [count = 1, size = 12] struct pair_t* pairs;

       uint32_t num_pairs;

   };


   trusted {

       public sgx_status_t ecall_sort_table([in] struct table_t * rel);

       public sgx_status_t ecall_deep_sort_table([in, count = 1] struct deep_table_t * rel);

   };


Check the file enclave_t.c to see the generated proxy functions. The function sgx_ecall_deep_sort_table shows the iterative deep copy of the structure from untrusted memory to trusted memory.


The proxy functions are automatically generated by the edger8 tool before the code is compiled. You must tell the edger8 tool how much memory to copy in the proxy functions via the count and size parameters in the EDL file.


Sincerely,

Jesus G.

Intel Customer Support



0 Kudos
kai__chi
Novice
1,156 Views

thanks for a fully comprehensive reply!

0 Kudos
JesusG_Intel
Moderator
1,145 Views

This thread has been marked as answered and Intel will no longer monitor this thread. If you want a response from Intel in a follow-up question, please open a new thread.


0 Kudos
Reply