Overview
To achieve full potential of computing solutions, several layers of software are integrated into a cohesive system. System BIOS or bootloaders, device firmware, virtualization management software, operating systems, orchestration software, applications, etc. The biggest the number of trusted layers, often referred to as Trusted Computing Base (TCB), more complex it is the job of protecting workloads.
Security is important in general, but even more critical for regulated segments and its workloads, such as financial information in retail transactions or patient records in healthcare deployments. Measures to help protect user data and privacy become even more important in the era of artificial intelligence (AI).
Intel® Software Guard Extensions (Intel® SGX) assists individuals and organizations to keep code integrity and data security at runtime by providing memory isolation mechanisms to create secure enclaves. Enclaves help to isolate workloads from each other and from privileged software such as operating system Kernel and hypervisors. It helps to assist in delivering solutions compatible with confidential computing.
In a deployment scenario without confidential computing, all the following components are in the trust boundary.
When Intel® SGX enabled workload is deployed, we only have the hardware (processor and memory) and the application within the trusted boundary.
A typical Intel® SGX deployment can use 2 different strategies:
- New application or modified application - Use of Intel® SGX SDK, by creating separate code portions to run as untrusted code in general memory space and trusted code to run inside the enclave.
- Wrap an application with abstraction layer, Library OS, and lift-and-shift the workload as-is into the enclave, with adjusted configurations set.
The first method is very efficient, since it is focused on adding into the enclave boundary only the code that handles sensitive data, such as personal and financial information, but less convenient since it requires application adaptation or rewrite.
Its advantage is to be able to run on both Windows* and Linux and provide fine grain control.
It uses the concept of:
- Untrusted Run-Time System (uRTS) – code that executes outside of the Intel(R) SGX enclave environment and performs functions such as:
- Loading and managing an enclave.
- Making calls to an enclave and receiving calls from within an enclave.
- Trusted Run-Time System (tRTS) – code that executes within an Intel(R) SGX enclave environment and performs functions such as:
- Receiving calls into the enclave and making calls outside of an enclave.
- Managing the enclave itself.
- Standard C/C++ libraries and run-time environment.
It leverages edge instructions enabled in the code to bridge the untrusted and trusted code with appropriate calls:
- ECall: “Enclave Call” a call made into an interface function within the enclave.
- OCall: “Out Call” a call made from within the enclave to the outside application.
The second method is more convenient as it leverages additional software layer, acting as a middleware, such as Gramine project, which is a library OS that maps operating system syscalls and transports the necessary execution flow into the enclave without the need to modify the application.
Gramine is the framework used in this article to demonstrate how to encapsulate container applications that run in Intel® SGX enclave without necessity of application code modifications.
Gramine is an Open-Source project, Linux-compatible Library OS for multi-process applications:
- Designed to run single application.
- Minimum host requirements.
- Enables to run unmodified applications in trusted environment.
This article will focus on Intel® SGX protected workload, but it is important to always strive to achieve security through defense in-depth where multiple layers are applied to the solution to make it more robust overall.
An example of typical multi-layer protection can be found in the following picture:
Now that introduction is complete, it is possible to shift to the activation of the example workload with Intel® SGX and Gramine.
Enabling Requirements
Base hardware and software requirements for Intel® SGX are highlighted at https://community.intel.com/t5/Blogs/Products-and-Solutions/Security/Enhancing-Security-at-the-Edge-Intel-Software-Guard-Extensions/post/1567552
Gramine Library OS
Gramine documentation page establishes the usage of Gramine very well, so it will be reproduced here to set the baseline: https://gramine.readthedocs.io/en/stable/
“Gramine is a lightweight guest OS that is designed to run a single Linux application with minimal host requirements. Gramine can run applications in an isolated environment with benefits comparable to running a complete OS in a virtual machine, including guest customization, ease of porting to different host OSes, and process migration.
Gramine supports running Linux applications using the Intel SGX (Software Guard Extensions) technology. Gramine is able to run unmodified applications inside SGX enclaves, without the toll of manually porting the application to the SGX environment. Applications running inside the Gramine SGX enclave become protected against a malicious host.”
Gramine does not come pre-installed with Linux distributions at the moment, so manual installation is required. For Ubuntu 22.04 the following steps are necessary:
sudo curl -fsSLo /usr/share/keyrings/gramine-keyring.gpg https://packages.gramineproject.io/gramine-keyring.gpg
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/gramine-keyring.gpg] https://packages.gramineproject.io/ $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/gramine.list
sudo curl -fsSLo /usr/share/keyrings/intel-sgx-deb.asc https://download.01.org/intel-sgx/sgx_repo/ubuntu/intel-sgx-deb.key
echo "deb [arch=amd64 signed-by=/usr/share/keyrings/intel-sgx-deb.asc] https://download.01.org/intel-sgx/sgx_repo/ubuntu $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/intel-sgx.list
sudo apt-get -y update
sudo apt-get -y install gramine git
For latest information on Gramine installation, see https://gramine.readthedocs.io/en/stable/installation.html
Gramine install a utility that can be used to check if Intel® SGX is properly enabled in the system and can be executed via the following command in the terminal:
is-sgx-available
Intel® SGX enclaves require signing and Gramine provides an utility tool to facilitate the private key used for the signing. It can be done via command:
gramine-sgx-gen-private-key
The private key is generated at $HOME/.config/gramine/enclave-key.pem
At this point, if all pre-requisites were met, it is possible to run an Intel® SGX enabled application, but since it requires access to /dev/sgx_enclave device loaded by Linux kernel, an unprivileged user cannot access the resource and an error would be thrown.
It is possible to configure the current authenticated user with access to the SGX device resource by performing the following steps in the terminal:
sudo groupadd -r sgx
sudo gpasswd -a $USER sgx
sudo groupadd -r sgx_prv
sudo gpasswd -a $USER sgx_prv
echo 'SUBSYSTEM=="misc",KERNEL=="sgx_enclave",MODE="0660",GROUP="sgx"' | sudo tee -a /etc/udev/rules.d/65-gramine-sgx.rules
echo 'SUBSYSTEM=="misc",KERNEL=="sgx_provision",MODE="0660",GROUP="sgx_prv"' | sudo tee -a /etc/udev/rules.d/65-gramine-sgx.rules
sudo udevadm trigger
After that, a system reboot is required to complete the loading of the new configuration.
The sample workload
For this article a simple Java application is used as sample workload. The application reads environment variable to acquire how many iterations should process a simple count with print to standard output.
After the max_count number is reached, application restarts the cycle.
For convenience the reference source code can be obtained at https://github.com/intel/esate_blog_sgx
Java sample is located under sample/java directory.
Workload Execution
It is possible to launch a native application encapsulated into enclave to achieve better protections while running from the host OS, but given container deployment have become so common, it is demonstrated the configuration enclaves within a container image context.
Instructions for Docker installation on Ubuntu can be found at https://docs.docker.com/engine/install/ubuntu/.
In order to run Docker with better settings such as not requiring privileged access (root), it is recommended the post-install steps to be performed as well, documented at https://docs.docker.com/engine/install/linux-postinstall/.
With Docker engine properly installed, gramine ready and access to SGX devices configured it is time to explore the workload deployment.
Native Java application running in Docker container
To run the native application, not in the enclave it is possible to execute the following command to build and run:
docker build -t java-native -f Docker.Native .
docker run -it --rm --name java-native java-native
In order to increase the max_count in the application, it is possible to use the command -e via Docker run to include the environment variable (increase from 10 to 60 iterations):
docker run -it --rm -e max_count=60 --name java-native java-native
Increasing the max_count helps to keep the application running for a bit longer between iterations in order to connect to bash in the container and be able to explore which command has been used to launch the workload. We can use the following command in a separate window:
docker exec -it java-native bash
To check on the running process:
ps aux | grep java
We can see that the java application was launched directly.
It is possible to terminate the instance by using interruption via Ctrl + C in the window executing the counting.
Java application running inside Intel(r) SGX enclave via Docker container
Given enclaves require signing there are a few changes required. First, it is needed to create a private key to sign the enclave. This can be done with OpenSSL in the local terminal, for development purposes:
openssl genrsa -out enclave_private_key.pem -3 3072Note the disclaimer about best practices when handling keys provided at https://github.com/intel/esate_blog_sgx/blob/main/README.md#enclave-signing-private-key
The sample provides an example Gramine manifest, which is needed in order to describe the setup for Gramine instance. It can be found at https://github.com/intel/esate_blog_sgx/blob/main/sample/java/java.manifest.template
For full details on how to create a manifest, with all options, consult Gramine's documentation at https://gramine.readthedocs.io/en/stable/manifest-syntax.html.
Gramine by default will not accept untrusted arguments via command line, so the max_count environment variable is moved to the manifest:
loader.env.max_count = “60”
Gramine executes the workload based on the definitions provided in the manifest, so, developers need to pay attention to the files and directories needed to be mapped and set as trusted to be executed.
Finally, the docker run command includes the passing of the SGX devices from the host into the container (Note that it is also needed to map the sgx_enclave device so the launch happens correctly. It is also possible to provide additional devices needed for provisioning and attestation, but those are out of scope for this article):
docker build -t java-gramine-sgx -f Dockerfile.Gramine.SGX .
docker run -it --rm --name java-gramine-sgx --device=/dev/sgx_enclave --device=/dev/sgx_provision -v /var/run/aesmd/aesm.socket:/var/run/aesmd/aesm.socket java-gramine-sgx
The count output starts to show in the terminal:
Same as in the native application, we can inspect the container by using:
docker exec -it java-gramine-sgx bashAnd again, to check the running process use:
ps aux | grep javaThe different is that now the application is no longer launched via Java call directly, but using Gramine SGX loader:
We have completed now the setup of an initially native Java launched application and moved same workload into Intel SGX enclave with a few modifications in the Dockerfile and creation of Gramine manifest, but without need to change application source code which is very convenient when trying to port existing applications to run in confidential computing enclaves.
Notes for developers
It is important to understand the current state of Gramine when developing and deploying applications. Gramine features are listed at https://gramine.readthedocs.io/en/stable/devel/features.html, including list of syscalls currently implemented.
While implementing solutions based on Gramine and Intel® SGX, it is important to verify if all the necessary syscalls to run the application have a secure implementation in Gramine.
Being Gramine an open-source project with permissive license, it allows for companies and individuals to implement customizations and implement additional layers not yet available in Gramine. It may be a great opportunity to get involved in the Gramine community and collaborate in the project.
Summary, remarks, and call to action
In this article it was explored the details on how to package a Java application in a Docker container, configure Gramine and enable Intel® SGX.
Finally, details were provided on how to look for supported syscalls and additional places to look for when porting additional software.
As a next step, if thinking on moving into cluster deployments with Kubernetes, one may find useful to investigate the Intel Device Plugins for Kubernetes that has a specific plugin to enable Intel® SGX and other technologies:
https://intel.github.io/intel-device-plugins-for-kubernetes/README.html
https://intel.github.io/intel-device-plugins-for-kubernetes/README.html#device-plugins-operator
https://intel.github.io/intel-device-plugins-for-kubernetes/README.html#sgx-device-plugin
https://intel.github.io/intel-device-plugins-for-kubernetes/README.html#demos
For Intel® SGX additional details, visit: https://www.intel.com/content/www/us/en/architecture-and-technology/software-guard-extensions.html.
For a list of commercial solutions and services around Intel® SGX visit :https://www.intel.com/content/www/us/en/architecture-and-technology/sgx-product-offerings.html.
References
Overview of an Intel Software Guard Extensions Enclave Life Cycle
Intel® Software Guard Extensions – Linux Overview
Gramine Documentation – main page
Gramine Documentation - Features
Gramine Documentation – installation options
Gramine Documentation – SGX setup
Gramine Documentation – Linux kernel drivers
Gramine Documentation – Docker image
Gramine Shielded Containers Documentation
Install Docker Engine on Ubuntu
Linux post-installation steps for Docker Engine
Seccomp security profiles for Docker
Notices and Disclaimers
Intel technologies may require enabled hardware, software or service activation.
No product or component can be absolutely secure.
© Intel Corporation. Intel, the Intel logo, and other Intel marks are trademarks of Intel Corporation or its subsidiaries. Other names and brands may be claimed as the property of others.
No license (express or implied, by estoppel or otherwise) to any intellectual property rights is granted by this document, with the sole exception that code included in this document (reference sample) is subject to the license under BSD-3-Clause with license file available at https://github.com/intel/esate_blog_sgx/blob/main/LICENSE