Question about EPC page encryption in SGX

[Insu_J_]

Dear,

I am studying how EPC page is used in SGX, and I am qurious about the position in which encryption by hardware is done.

https://software.intel.com/sites/default/files/332680-002.pdf
Seeing the 33rd page of this ISCA '15 presentation, it says "Data and code outside CPU package is encrypted and/or integrity checked" and "External memory reads and bus snoops see only encrypted data". This means even EPC page in PRM should also be encrypted, because EPC is also a part of memory.

However, seeing 81st page of the same presentation, it says "When page is reloaded: The processor decrypts, and integrity checks the page, using crypto metadata". This means EPC page contains the plaintext of code or data.

I think these two statements don't make sense by each other. Assuming EPC pages contain decrypted data, attackers can steal the data by snooping memory buses.

Could you explain what is the answer? Where is encryption done?

[Insu_J_]

I also saw ELDB/ELDU instruction explanation in Intel SGX Programming Reference, uploaded in [here].

ELDB/ELDU: Load an EPC page and Marked its State
Description
This leaf function copies a page from regular main memory to the EPC. As part of the copying process, the page is cryptographically authenticated and decrypted.

Then it means every data in EPC, which is a part of DRAM, is decrypted. Then how SGX can protect the data from DRAM-cache bus snooping attack?

Thanks!

[Juan_d_Intel]

I think the first sentence is clear. Data in EPC is encrypted and integrity protected so even a cold boot attack won't be able to retrieve enclave data.

The CPU decrypts and integrity checks EPC memory when the enclave reads from memory before it's stored in the processor cache. Similarly, data is encrypted inside the CPU when the enclave writes to EPC memory. The metadata that is used to verify the integrity of the data is itself also integrity protected. The integrity information is stored in a tree, the root of which is stored in SRAM, inside the processor.

You may read the article "A Memory Encryption Engine Suitable for General Purpose Processors" that explains how to Integrity Tree is maintained.

Does this answer your question?

[Insu_J_]

Thank you for the information, Juan.
It is clear for me that EPC page is encrypted, however, I still don't understand explanation about encryption during page swapping.

https://software.intel.com/en-us/articles/innovative-instructions-and-software-model-for-isolated-execution

Section 7.2. (Evicting the enclave page) in this article explains that EWB instruction encrypts the EPC page using the paging encryption key.
Does it mean EWB encrypts an encrypted EPC page again?

If yes, why SGX should encrypt data twice?

If no, why page swapping is so burden? The paper [SCONE: Secure Linux Container] illustrates that when allocated EPC size is over than 92MB, so page swapping is necessary, memory access overhead is 1000x times higher than just accessing within L3 cache. It is even 100x times slower than memory access with L3 cache miss.
If encryption is only done between L3 cache and memory, This high overhead due to page swapping cannot be explained, I think. Then which job during page swapping makes it so slow?

Thank you!

[Juan_d_Intel]

From the CPU standpoint, data in EPC is unencrypted, because the MEE sits transparently between the CPU and the PRM. In other words, the data in EPC is encrypted because it's outside the CPU package. However, it doesn't need to be this way. For instance, a CPU with special on-chip memory wouldn't need the MEE and the EPC memory wouldn't have to be encrypted.

When a page is evicted it's a different story. The page is placed outside EPC, and therefore it must be encrypted, and integrity and replay protected. This is what the EWB does and what makes this operation expensive, compared to normal memory page swaps. Going back to the example of the EPC located in some special on-chip memory. In this case EWB would still have to encrypt & protect the page before it's evicted.

In summary, encrypting EPC memory is an implementation artifact, the result of having the EPC in DRAM that requires MEE. EWB encrypting the page before eviction is architectural.

I hope this answer your question.

[Insu_J_]

Juan D. (Intel) wrote:

From the CPU standpoint, data in EPC is unencrypted, because the MEE sits transparently between the CPU and the PRM. In other words, the data in EPC is encrypted because it's outside the CPU package. However, it doesn't need to be this way. For instance, a CPU with special on-chip memory wouldn't need the MEE and the EPC memory wouldn't have to be encrypted.

When a page is evicted it's a different story. The page is placed outside EPC, and therefore it must be encrypted, and integrity and replay protected. This is what the EWB does and what makes this operation expensive, compared to normal memory page swaps. Going back to the example of the EPC located in some special on-chip memory. In this case EWB would still have to encrypt & protect the page before it's evicted.

In summary, encrypting EPC memory is an implementation artifact, the result of having the EPC in DRAM that requires MEE. EWB encrypting the page before eviction is architectural.

I hope this answer your question.

Thank you for the explanation, but I'm afraid it is still not clear the difference between encryption by MEE and encryption due to EWB instruction.

The reason I think the encryption during eviction is "additional" is that, if EWB selects an EPC page to be evicted, my thought is that it must be the least recently accessed one (according to LRU policy), which means that it is highly possible that this EPC page is already outside the CPU package, as encrypted. Then encrypting an encrypted EPC is an additional operation. Am I wrong?

To make my question clear, let me define three categories that an EPC page can be in: inside the CPU package (i.e. on-chip memory or cache), in the EPC region (protected), outside the EPC region (unprotected).
1. MEE performs encryption of an EPC page when an EPC page goes from cache (inside the CPU package) to the EPC region (protected), because of small capacity of L3 cache.
2. EWB performs eviction of an EPC page from the EPG region (protected) to outside the EPC region (unprotected), because of small capacity of EPC.

I understand operation number 1. But documentations say that SGX ALSO encrypts the EPC page during the operation number 2.
Is there anything that I misknow?

Thank you again for the comment!

[Juan_d_Intel]

As a requirement, the CPU must encrypt and integrity and replay protect the contents of an EPC page before it is evicted (EWB).

However, the CPU doesn't see the EPC page as encrypted. From the CPU standpoint, EPC memory is not encrypted. The MEE (implementation detail) sits between the CPU and DRAM and encrypts/decrypts memory content.

EPC is part of the PRM  so there is no mechanism to simply transfer the EPC page out.

[Insu_J_]

Juan D. (Intel) wrote:

As a requirement, the CPU must encrypt and integrity and replay protect the contents of an EPC page before it is evicted (EWB).

However, the CPU doesn't see the EPC page as encrypted. From the CPU standpoint, EPC memory is not encrypted. The MEE (implementation detail) sits between the CPU and DRAM and encrypts/decrypts memory content.

EPC is part of the PRM  so there is no mechanism to simply transfer the EPC page out.

Thank you again for the explanation. So the key point was "the CPU standpoint". CPU sees that EPC memory is not encrypted because MEE transparently encrypts/decrypts the EPC memory. If EWB is executed, CPU explicitly orders encryption for the EPC page, while the EPC page is already encrypted and resides in EPC. (again, from the CPU standpoint, it is not encrypted, but actually encrypted from outside CPU package). Then when EPC page is swapped out to the unprotected memory region, it should mean the EPC page is encrypted twice (transparently by MEE, explicitly by EWB).

And if this evicted EPC page is swapped in the EPC, a CPU instruction ELDB or ELDU checks its integrity and decrypts it. After the instruction call, "from the CPU standpoint", it is unencrypted. But still from the view of the ouside CPU package, it is encrypted, as it is still outside the CPU package. And it will become plaintext through transparent decryption by MEE if it is accessed and copied into L3 cache.

Hence, from the view of the world outside, not the view of CPU, encryption and decryption are done in both borders: between cache and EPC, and between EPC and untrusted memory.

Am I correct?

Thanks!

[spring__z]

Insu J. wrote:

Quote:

Juan D. (Intel) wrote:

As a requirement, the CPU must encrypt and integrity and replay protect the contents of an EPC page before it is evicted (EWB).

However, the CPU doesn't see the EPC page as encrypted. From the CPU standpoint, EPC memory is not encrypted. The MEE (implementation detail) sits between the CPU and DRAM and encrypts/decrypts memory content.

EPC is part of the PRM  so there is no mechanism to simply transfer the EPC page out.

Thank you again for the explanation. So the key point was "the CPU standpoint". CPU sees that EPC memory is not encrypted because MEE transparently encrypts/decrypts the EPC memory. If EWB is executed, CPU explicitly orders encryption for the EPC page, while the EPC page is already encrypted and resides in EPC. (again, from the CPU standpoint, it is not encrypted, but actually encrypted from outside CPU package). Then when EPC page is swapped out to the unprotected memory region, it should mean the EPC page is encrypted twice (transparently by MEE, explicitly by EWB).

And if this evicted EPC page is swapped in the EPC, a CPU instruction ELDB or ELDU checks its integrity and decrypts it. After the instruction call, "from the CPU standpoint", it is unencrypted. But still from the view of the ouside CPU package, it is encrypted, as it is still outside the CPU package. And it will become plaintext through transparent decryption by MEE if it is accessed and copied into L3 cache.

Hence, from the view of the world outside, not the view of CPU, encryption and decryption are done in both borders: between cache and EPC, and between EPC and untrusted memory.

Am I correct?

Thanks!

Hi, Insu J.  

I also have this question. I have read all your comments and i think that is reasonable, but there are no replays about this question, so have you got the right answer?

[jamason]

Is the PRM memory encrypyed with the same key.

or is the PRM memory region for each enclave encrypted with a different key?

[Kumar__Dixit · Accepted solution]

Hi, Insu J.

Whatever understanding I have so far,

• Data belongs to enclaves, either in the EPC region or in the NON-EPC region is encrypted.
• When a page swap occurs from the EPC to the NON-EPC region, CPU first decrypts the EPC page, then again encrypts it with a new counter value. Once encryption is done, OS swap the newly encrypted page to the NON-EPC region.
• To provide a freshness guarantee and protection against replay attacks on enclave data, SGX might be doing multiple encryption/decryption operataion.

Although, I am also not sure about this, for more understanding you can refer:

"In case of Enclave page swap from EPC to NON-EPC region, Upon evicting page from the EPC, MEE decrypts the page, handing it to the CPU. The CPU then assigns a counter, encrypts the page again using combinations of counter and enclave key’s, and makes entry in the eviction tree."  {Reference: Vault}

Kind Regards,
Dixit