How to measure cache misses and paging in SGX?

[kai__chi]

Hi,

Is there a way to measure cache misses and paging in SGX? I'd like to measure both L3 misses and EPC misses (that lead to paging). In [1], the author talks about profiling the programs and measuring cache misses (Section 7) but does not explain how he does it.

I figured out I can measure the total number of allocated (and freed) pages in the linux-sgx-driver by counting invocations of sgx_alloc_page_fast in sgx_page_cache.c. That, unfortunately, is a sum for all the enclaves since the driver starts and includes initialization of enclaves. Is there a way to profile particular function invocations?

I also tried to use VTune with sgx-hotspots. That gives "Precise Clockticks" per function and also shows that sgx_encl_add_page and sgx_fault_page consume a lot of CPU but, again, it's for the whole program and the init phase is a great part of the results. Can I somehow filter the results for a specific time span?

[1] A Memory Encryption Engine Suitable for General Purpose Processors https://eprint.iacr.org/2016/204.pdf

[JesusG_Intel]

Hello Kai, visibility into SGX enclaves by external software is extremely limited by design to maintain security.

Regards,

[kai__chi]

Hi Jesus, Thanks for a quick response! So does that mean there is no way to measure paging and cache misses? I have found a similar tool [1] that measures the speed of paging. It's definitely a step towards my question. There's definitely possible something... Also, Shay Gueron somehow measured cache misses in his paper on MEE. Don't we have any info how he accomplished that? [1] https://www.fortanix.com/blog/2020/02/monitoring-intel-sgx-enclaves/

[JesusG_Intel]

Hello Kai,

Note that Fortanix has patched Intel's Linux SGX driver and written an app, sgxtop, that works with their version of the driver to report memory usage. The source code for the Fortanix tools is open source in Github, https://github.com/fortanix/linux-sgx-driver and https://github.com/fortanix/sgxtop. For more information on these tools, please refer to their respective gits or contact Fortanix.

It seems that these tools provide memory usage and paging statistics for enclaves as a whole but do not get inside the enclaves like what you are trying to do. As I wrote earlier, being able to get inside the enclave would violate their security properties. This is what it means for the EPC to be protected memory.

Regards,

[kai__chi]

Hey Jesus, Thanks for your response. Yes, I checked out the Fortranix solution and as you mentioned - it provides only overall statistics about the driver. I understand that measuring cache misses will violate enclave's security. However, the debugger is able to access debug enclaves so I thought there would also exists something that monitors cache misses. From your response I assume there's no publicly available tool to measure cache misses and/or paging. The tool used by Shy Gueron in [1] is either an Intel internal tool or his own. Please correct me if I'm wrong. [1] https://eprint.iacr.org/2016/204.pdf

[JesusG_Intel]

Hello Kai, I am checking internally for more info on that test.

Regards,

[kai__chi]

Hi Jesus, Thanks a lot. Looking forward to what comes out of it.

[kai__chi]

Hi Jesus, Any updates on this issue? Best, Kai

[JesusG_Intel]

Hello Kai,

I have not received a response yet. I'll ping them again today. Please stay tuned.

Regards,

[JesusG_Intel]

Hello Kai, the feedback that I received is that it is possible to measure enclave cache misses as long you run the enclave in DEBUG mode. Unfortunately, I do not have more specific VTune guidance. I recommend to post your question in the VTune forum, https://community.intel.com/t5/Analyzers-Intel-VTune-Profiler/bd-p/analyzers, where they should be able to help you further if needed.