How Open Source Gramine Accelerates Expanding Confidential Computing Market

Published February 14th, 2022

Mona Vij is a Principal Engineer and Cloud and Data Center Security Research Manager at Intel Labs, where she focuses on Scalable Confidential Computing for end-to-end Cloud to Edge security.

The need for end-to-end data protection has never been greater. As computing moves from on-premises to the public cloud and to the edge, individuals, companies and organizations are becoming increasingly vulnerable to hacks, data breaches and malicious attacks. It is no wonder that the confidential computing market is projected to grow at a CAGR of 90%-95% to reach $54 billion in 2026. That is why Intel is so excited about the Gramine Project, which has been accepted into the Confidential Computing Consortium (CCC). We think that Gramine and the emergence of confidential computing with Intel® Software Guard Extensions (Intel® SGX) elevates computational security to a level befitting our world today.

The Gramine Project recently announced the latest production-ready version – Gramine 1.1. This open-source library OS (LibOS) enables the protection of sensitive workloads with SGX and it has a flexible architecture to enable other back-end systems like Intel® Trust Domain Extensions (Intel® TDX) in the future. Put simply, Gramine fast-tracks secure deployment of complex software stacks within SGX by eliminating any additional developer effort. It also provides tools for developing end-to-end secure solutions with SGX enclaves that shield proprietary code and sensitive data from hackers, whether the data is in a state of use, transit, or rest.

Gramine allows running unmodified Linux applications/binaries on different platforms, including Intel® SGX enclaves. It serves as a minimal bootloader and an emulation layer between the enclaved application and the underlying host OS. It can be thought of as a minimal re-implementation of the Linux kernel, striving to resolve most application requests within the LibOS and to carefully vet those application requests that must be processed by the host OS.

Gramine wasn't born yesterday. It began as a research prototype at Stony Brook University in 2011 and the SGX port was done in collaboration with Intel Labs in 2015. Since then, the Gramine maintainer community has grown to include several universities and companies, such as Intel, Invisible Things Lab, University of North Carolina, and Texas A&M University, as well as several individuals. It also boasts a growing list of well-tested applications, including Artificial Intelligence (AI)/Machine Learning (ML)  frameworks, databases, web servers, and several programming languages.

 Simplified Security

Applications can benefit from the confidentiality and integrity guarantees of Intel® SGX, but developers need to be very skilled for effective partitioning and code modification for SGX environments. Gramine makes life easier for the application developer and the end-user by enabling end-to-end enhanced security for unmodified applications without any additional code development. The user/developer only needs to specify the security policies that they want to enforce for their workload in the manifest file of Gramine and Gramine does the rest. That’s why we say Gramine allows you to “lift and shift” an application from one environment, like a native Linux, to another environment, like a protected SGX.

It is this “lift and shift” capability that fuels the broad range adoptability of SGX, providing much needed simplicity for those who are eager to move forward with their endeavors but want to ensure hacker-proof environments for data and compute.

Gramine provides several tools and infrastructure components for developing end-to-end protected solutions within SGX. With Gramine’s support for remote attestation, secret provisioning  and protected files features, we can enable a large class of workloads that operate on sensitive code and data. Gramine’s protected file system feature makes the encryption and decryption of input/output files transparent to the application. Using Gramine and its remote attestation with secret provisioning and protected file system techniques, owners or developers can build a generic framework to protect any application that operates on sensitive data.

Gramine supports dynamically loaded libraries and runtime linking and is one of the few frameworks that provides full support for fork/clone/execv system calls for multi-process abstraction, thus enabling a broad range of applications. It is highly optimized with exitless system calls and with other features that minimize performance overhead, while maintaining the confidentiality and integrity of all code and data within an SGX enclave.

Gramine supports Docker integration via a tool called Gramine Shielded Containers (GSC) that automatically converts Docker images to Gramine images. Containers built with GSC can be deployed via Kubernetes for confidential containers and microservices. Gramine also supports cloud deployment with Azure Confidential VMs and integrates with Azure Kubernetes Services in Azure cloud.

 AI/ML Relevant Use Cases

While Gramine can support large classes of workloads and language runtimes, we see the most traction with AI/ML applications used in healthcare, finance, and several other industries. Gramine makes development of secure and confidential AI/ML solutions simple with its growing features set.

Figure 1: End-to-End Enhanced Secure Machine Learning

Consider a ML application where the data owner may not trust the platform where the computations will take place and wants to protect the confidentiality and integrity of all sensitive data (see Figure 1).  First, as a provisioning step, the owner can use Gramine to encrypt all confidential inputs with a secret key in a trusted location. At the deployment time, Gramine automatically runs the unmodified application inside an SGX enclave, but before control is transferred to the ML application, Gramine’s SGX remote attestation feature generates an attestation report and authenticates to a verifier and key server that securely provisions a cryptographic key to the SGX enclave over a secure TLS channel. At this point control is transferred to the application and the inputs are automatically decrypted with the provisioned key. Any output is stored and transparently encrypted with the same key. Since the plain text inputs and outputs are only accessible inside the enclave, the untrusted platform owner or system software (e.g. operating system) cannot steal them.

There are also cases where a model owner wants to protect proprietary model intellectual property (IP). With Gramine we can enable a variety of secure and private use cases such as:

Training with private data. A streaming company provides a personalized contents recommendation system. The company wants to train its models without collecting user data directly because the customers do not want their data to be used for marketing. In this case, the training ML application may consume user data to train the model but should not allow the data to be used for other purposes.

Inference with proprietary model. A company developed a video editing application with a patented ML-based object detection feature. The application runs on the client side, thus the client’s data never leaves the local computer. However, the company is worried that the model could be stolen by competitors.

Inference with private data. A company trained a model that predicts road traffic based on the real-time GPS information of a few nearby users. The company wants to provide a map service using this model. However, collecting real-time location of individuals violates the privacy law. The company needs to make sure that the customers’ GPS data will never be disclosed, but still wants to use this data to predict traffic.

With Gramine’s rich feature set, tools and infrastructure components, we can support majority of above use cases. Gramine can be used not only for confidentiality of model IP and private user data but can also be used for protecting the integrity of AI/ML workloads, e.g. in a federated learning scenario Gramine can ensure that each of the nodes follow the rules of federation by guaranteeing the integrity of execution.

As one example of a production-ready application deployment framework with Gramine, Intel’s OpenVINO toolkit now supports OpenVINO security add-on (OVSA) that provides tools for model IP protection during model development as well as deployment, storage, and at runtime. There are several places in the model development and deployment pipeline where a model can be copied, cloned, or reverse engineered. OVSA integrates encryption and trusted execution protections with model licensing flows to mitigate model theft. OVSA enables model IP trusted execution protection by using Gramine with SGX.

Gramine is a growing project with several new features in the pipeline. The one to highlight is generic device communication support that can be used to extend the secure solutions from CPUs to hardware accelerators like GPUs.

There are several large and small companies developing their secure solutions with Gramine. We invite you to join the Gramine community; we welcome contributions of all types. Try experimenting with Gramine as you develop your own security solutions. Please share your experiences with the Gramine community and feel free to let us know of any additional requirements you may have.