- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
I have to implement an SGX software which has to dynamically change the memory location of the code section. If I do this in a regular C program, I need to allocate a new memory page, call mprotect() and set the page's PROT_EXEC bit to 1, copy the original code section to the newly allocated memory, and jump to here.
However, I do "#include <sys/mman.h>" which defines mprotect(), the SGX can't find this header. Does this mean SGX has no way to change the memory page permission bit of a dynamically allocated memory?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Here are some thoughts:
1. You can allocate a section (using pragmas) in your .so enclave file.
2. Then you can modify the permissions on that section to be executable.
3. After making those modifications, use the sgx signing tool to sign your enclave.
For 2, you web searches show some usages of objcopy, ld -N, or writing your own program that modifies them. I believe there's also a way to write it in assembly that ensures the permissions stay RWX.
SGX has to do this for its "Protected Code Loader" feature, but I am not familiar with its implementation, I think it is done by calling a program built with the first link below.
https://github.com/intel/linux-sgx/blob/master/sdk/encrypt_enclave/encryptip.cpp , function "update_flags"
The sample and the underlying library it uses are:
https://github.com/intel/linux-sgx/tree/master/SampleCode/SampleEnclavePCL
https://github.com/intel/linux-sgx/tree/master/sdk/protected_code_loader
Thanks,
Francisco
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Some processors support EMODPE and EMODPR instructions as described in the developer manual.
https://software.intel.com/sites/default/files/managed/48/88/329298-002.pdf
If you have such a processor, you can use sgx_virtual_protect in Windows, declared C:\Program Files (x86)\Intel\IntelSGXSDK\include\sgx_tedmm.h. There is an "SGX2Enclave" sample included in the Windows SDK.
In Linux, there is an SGX2 driver https://github.com/intel/linux-sgx-driver/tree/sgx2 and I believe the API is trts_mprotect - https://github.com/intel/linux-sgx/blob/d10cabebb5512878e84f5d21cdf27c39c428ffe2/sdk/trts/trts_ecall.cpp
Hope this helps.
Francisco
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks very much for your reply. I tried to use trts_mprotect() as you suggested, but I get an "Illegal instruction" error when calling the ENCLU instruction from file sdk/trts/linux/trts_pic.S:
DECLARE_GLOBAL_FUNC do_emodpe SE_PROLOG mov $SE_EMODPE, %eax ENCLU SE_EPILOG
Does this mean my computer's processor doesn't support ENCLU instruction? If so, is there absolutely no way for me to executable dynamic memory page from within the enclave? Would it be not possible even if I pass the enclave's dynamic memory page's address to the untrusted code (by using ocall) to ask for modifying the permission bit to PROT_EXEC?
Ronny
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
It appears EMOD instructions are not supported on your system. You can look at the CPUID check for this:
https://github.com/intel/linux-sgx/blob/master/psw/urts/linux/edmm_utility.cpp
is_cpu_support_edmm()
The NUCs below should have processors that support SGX2 (EDMM) instructions.
https://ark.intel.com/content/www/us/en/ark/products/126135/intel-nuc-kit-nuc7cjyh.html
https://ark.intel.com/content/www/us/en/ark/products/126137/intel-nuc-kit-nuc7pjyh.html
If you do not have hardware support, then no, you cannot modify the permissions of a EPC page.
Thanks,
Francisco
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks for further help. As you said, is_cpu_support_edmm() returns false on my computer.
However, I could manage to change the enclave's memory permission by implementing an ocall. I allocated a memory page in Enclave.cpp:
void * allocate_aligned(size_t size, size_t alignment) { const size_t mask = alignment - 1; const uintptr_t mem = (uintptr_t) malloc(size + alignment); return (void *) ((mem + mask) & ~mask); }
void ocall_create_executable_memory(char* s) { char* page = allocate_alligned(0x1000, 0x1000); ocall_pass_memory((size_t)page); }
and I passed this memory page's address to App.cpp and let it change this page's permission:
void ocall_create_executable_memory(size_t s) { if (mprotect((char*)s, 0x1000, PROT_NONE)) { perror("Couldn’t mprotect"); exit(errno); } }
I change the page permission to PROT_NONE from within App.cpp, after then Enclave.cpp cannot read this page (it exits with an error). So I think it works as expected. Do you see any logical flaws in my approach?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
mprotect will modify the OS permissions on that page , but not the Enclave Page Cache Map (EPCM) permissions. You need EMOD instructions to modify the EPCM permissions on that page.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I understand that the OS can't change the memory permission of the Enclave, but if I run the code, it somehow works. I attached my code. You can run it and see:
$ unzip sgx-mprotect.zip $ cd sgx-mprotect $ make $ ./App_exe is_cpu_support_edmm(): 0 pagesize: 4096 bytes, function1()'s size: 15 bytes, function2()'s size: 39 bytes ocall_mprotect(): received memory address 0x7fc02185d000 from SGX ocall_mprotect(): modified page permission of memory address 0x7fc02185d000 to 7 SGX's dynamic_execute(): copied function1() to memory address 0x7fc02185d000 SGX's dynamic_execute(): executed original function1() --> returned 1 SGX's dynamic_execute(): executed dynamically created function1()--> returned 1 SGX's dynamic_execute(): copied function1() to memory address 0x7fc02185d000 SGX's dynamic_execute(): executed original function2() --> returned 83 SGX's dynamic_execute(): executed dynamically created function2()--> returned 83 Done
However, if you comment out Enclave.cpp's line 30: "sgx_mprotect(mem_aligned, PAGE_SIZE, 7);" and run it again:
$ make $ ./App_exe is_cpu_support_edmm(): 0 pagesize: 4096 bytes, function1()'s size: 15 bytes, function2()'s size: 39 bytes SGX's dynamic_execute(): copied function1() to memory address 0x7f67e31d9000 SGX's dynamic_execute(): executed original function1() --> returned 1 Segmentation fault (core dumped)
Does this imply that the OS can change the page permission of the Enclave?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
I just realized that the memory address allocated by SGX's malloc() call can be also accessed by the OS. I thought malloc() within SGX is the Enclave's own memory, but is this actually the OS's memory space?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi Ronny.
It appears from your Makefile that you are running in SIM (simulation) mode (you've commented out "HW")... can you confirm? If so, then yes, the OS would be able to access this memory. You will need to build and run in HW mode for the SGX protections to be used.
Regards.
Scott
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Yes, you are correct, it was my flaw that I compiled in SIM Mode. As I compile in HW mode, the OS can't change the SGX's page permission successfully anymore and the SGX trying to execute the memory page ends with a segmentation fault. Thanks a lot for finding out this bug for me.
I really need an executable memory page from within the Enclave... Just before the Enclave program starts, could it be possible to configure the Enclave to make it statically pre-allocate huge memory pages whose executable bit is 1? If this is ever possible, which (either configuration or source code) file should I modify to enforce this?
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Here are some thoughts:
1. You can allocate a section (using pragmas) in your .so enclave file.
2. Then you can modify the permissions on that section to be executable.
3. After making those modifications, use the sgx signing tool to sign your enclave.
For 2, you web searches show some usages of objcopy, ld -N, or writing your own program that modifies them. I believe there's also a way to write it in assembly that ensures the permissions stay RWX.
SGX has to do this for its "Protected Code Loader" feature, but I am not familiar with its implementation, I think it is done by calling a program built with the first link below.
https://github.com/intel/linux-sgx/blob/master/sdk/encrypt_enclave/encryptip.cpp , function "update_flags"
The sample and the underlying library it uses are:
https://github.com/intel/linux-sgx/tree/master/SampleCode/SampleEnclavePCL
https://github.com/intel/linux-sgx/tree/master/sdk/protected_code_loader
Thanks,
Francisco
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
Thanks very much for your kind directions. As you hinted me, I allocated a huge char[] array as a global variable in my Enclave application C code and compiled it to create an enclave.so. Then I inserted the following code in the update_flag() function in the linux-sgx/sdk/encrypt_enclave/encryptip.cpp file:
if (dat->phdr[segidx].p_type == 1 && dat->phdr[segidx].p_flags == (PF_W | PF_R)) dat->phdr[segidx].p_flags = (Elf64_Word)(PF_W | PF_R | PF_X);
to change the page permission of enclave.so's ELF data section to be readable & writable & executable, and compiled this program by typing "make". Then I ran this compiled memory permission modifier program with the above enclave.so as an input to make its ELF data section executable, and then compiled it again (by typing "make") to re-sign the modified enclave.so. Now the finalized enclave can dynamically write code to its global char[] array and execute it properly. I attached my Enclave memory permission modifier program for anyone's reference, and the following is the output I get:
$ cd ~ $ unzip sgx-app.zip $ mv elf_data_segment_mark_x.zip ~/linux-sgx/sdk $ cd ~/linux-sgx/sdk $ unzip elf_data_segment_mark_x.zip $ cd elf_data_segment_mark_x $ make $ cd ~/sgx-app $ make clean $ make $ ~/linux-sgx/sdk/elf-data-segment-mark-x/sgx_data_segment_mark_x -i enclave.so -o enclave.so $ make # This re-creates enclave.signed.so based on the modified enclave.so $ ./App_exe is_cpu_support_edmm(): 0 pagesize: 4096 bytes, function1()'s size: 15 bytes, function2()'s size: 39 bytes ocall_mprotect(): received memory address 0x7ff379231000 from SGX ocall_mprotect(): modified page permission of memory address 0x7ff379231000 to 7 SGX's dynamic_execute(): copied function1() to memory address 0x7ff379224960 SGX's dynamic_execute(): executed original function1() --> returned 1 SGX's dynamic_execute(): executed dynamically created function1()--> returned 1 SGX's dynamic_execute(): copied function1() to memory address 0x7ff379224960 SGX's dynamic_execute(): executed original function2() --> returned 83 SGX's dynamic_execute(): executed dynamically created function2()--> returned 83 Done
Thanks very much for all of your great support :)
![](/skins/images/45C6C2D737ED71F4C51F0145C8CB1E9C/responsive_peak/images/icon_anonymous_message.png)
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page