Intel® Software Guard Extensions (Intel® SGX)
Discussion board focused on hardware-based isolation and memory encryption to provide extended code protection in solutions.

Can a Docker container or other non-C program run as a Trusted (Enclave) Component?

Harris__David
Beginner
1,251 Views

Hello,

I'm new to Intel SGX and Docker.  I'm trying to determine if I can run "sensitive" programs and data within a Docker container and run that container as a trusted (enclave) component in the SGX hardware. I plan to use the latest Linux version of the SGX platform.

I've been reading through the SGX developer's guide, reference, programming reference, etc. and am not sure how I'd use things like the Enclave Definition Language to invoke a container as a trusted component.The SCONE project has a technical approach, but it appears to not use the Enclave Definition Language, ECALLS, OCALLS, Intel's SDK and secure C libraries and other guidelines I'm reading in the Intel SGX documentation.  Also, previous questions I've found on using SGX with Docker containers have been mixed.

So, is the answer:

"No, you can't run a Docker container as a trusted enclave component, using Intel SGX development guidelines"

"Yes, you can run a Docker container as a trusted enclave component, using Intel SGX development guidelines and here's how to do it...."

Related question:  SGX appears to only run C/C++ programs.  So, languages like Python (not using Ctypes) or Java (not using JNI) won't run as a trusted component, right?   

Also, is there anything besides SCONE that is a viable example of running a Docker container as a trusted component in SGX?  Thanks for your help!

0 Kudos
1 Solution
Dr__Greg
Super User
1,251 Views

Good morning David, I hope this comment finds your week starting well.

To cut to the chase, "No, you can't run a Docker container as a trusted component using the Intel SGX development environment/guidelines.

Enclaves are statically linked self-contained execution domains.  This is the reason the Intel SDK provides its own version of the standard C library, one that has had system call dependencies removed.  In order to access those types of services the enclave has to do an OCALL in order to temporarily exit the enclave to access the untrusted resources and return back into the enclave with the results.

Currently the only way to create trusted enclave based applications is to write the application in C/C++ with a Rust based development environment evolving.  As a result it isn't possible to create trusted applications in Python or Java.

As you noted, the SCONE initiative is a platform designed to run containers inside of a secure enclave.  This initiative and quasi-related efforts such as PANOPOLY facilitate the execution of enclaves by providing an enhanced standard C library that essentially stubs out all of the operating system calls that can be possibly implemented into untrusted userspace. The end result is the ability to run 'containers' in an enclave as long as they respect fundamental limitations imposed by the lack of system calls such as fork().

In order to get acceptable performance, worker threads sit in untrusted userspace waiting to have service requests dispatched to them by the library OS inside of the enclave placing a request for service on a work queue.  The work threads process the request and place the result on the queue which is then interrogated by the enclave.  All of this is possible since the enclave can read/write untrusted memory.  Intel supports something equivalent in their SDK with their 'switchless' architecture which I previously wrote an extensive response on in this forum.

The upside is that you can use this to run some definition of a container inside of an enclave.  The downside is that all of this infrastructure increases the Trusted Computing Base (TCB) of the containerized application such that it is probably equivalent to that of a well secured container platform.  The functional security guarantees of this model also have to be considered in the face of how 'leaky' current platforms are in the face of micro-architectural vulnerabilities, ie. Spectre/Meltdown type vulnerabilities.

The other impact is on performance which is documented to be about 60-80% of native containers.  The other operational issue is the limited amount of Enclave Page Cache memory which is just under 128 megabytes on currently available platforms.  This places an effective total limit on the total container footprint that can be supported.

In order to address all of these issues we just spent two years of development on an alternative which we refer to as Autonomous Introspection Canisters, yes canisters...:-)  In our model we run standard containers, each of which is paired with an SGX enclave running a modeling engine which models the information exchange events (actor/subject) interactions that occur inside of the canister/container.  If the enclave detects an extra-dimensional behavior inside of the canister it sets a flag in the task control structure of the process which then applies very granular disciplining with a per process LSM, also controlled by the modeling enclave.

The behavior modeling is implemented as an independent namespace which allows each canister to have its own behavioral definition.  We run a very small security supervisor in the root behavior namespace which is also introspected by its own enclave.  All of this allows a guarantee that neither the platform or any of the canisters that it runs could have executed a behavior contrary to the intention of its designer/developers.  We believe this provides a superior security guarantee to the 'shielding' strategy implemented by SCONE.

The most significant impact is that there are no limitations on the size of canisters that can be run and the canisters run at close to native speed.  Applications running inside of a canister are free to implement enclaves in order to provide integrity and confidentiality guarantees for privileged data/code.  With the advantage of a documented guarantee that a side-channel attack could have not been conducted against the enclave or any other platform component.

I've mentioned in other posts that we wrote an independent implementation of the PSW to support this work.  We use the ability of that PSW to implement direct enclave<->enclave attestation and communications in order to provide remote launch and telemetry of canisters.

So I believe that pretty much covers the spectrum of how SGX is being used to address the execution of containers.  Hopefully the information is useful to your efforts.

Best wishes for a productive week.

Dr. Greg

View solution in original post

0 Kudos
3 Replies
Dr__Greg
Super User
1,252 Views

Good morning David, I hope this comment finds your week starting well.

To cut to the chase, "No, you can't run a Docker container as a trusted component using the Intel SGX development environment/guidelines.

Enclaves are statically linked self-contained execution domains.  This is the reason the Intel SDK provides its own version of the standard C library, one that has had system call dependencies removed.  In order to access those types of services the enclave has to do an OCALL in order to temporarily exit the enclave to access the untrusted resources and return back into the enclave with the results.

Currently the only way to create trusted enclave based applications is to write the application in C/C++ with a Rust based development environment evolving.  As a result it isn't possible to create trusted applications in Python or Java.

As you noted, the SCONE initiative is a platform designed to run containers inside of a secure enclave.  This initiative and quasi-related efforts such as PANOPOLY facilitate the execution of enclaves by providing an enhanced standard C library that essentially stubs out all of the operating system calls that can be possibly implemented into untrusted userspace. The end result is the ability to run 'containers' in an enclave as long as they respect fundamental limitations imposed by the lack of system calls such as fork().

In order to get acceptable performance, worker threads sit in untrusted userspace waiting to have service requests dispatched to them by the library OS inside of the enclave placing a request for service on a work queue.  The work threads process the request and place the result on the queue which is then interrogated by the enclave.  All of this is possible since the enclave can read/write untrusted memory.  Intel supports something equivalent in their SDK with their 'switchless' architecture which I previously wrote an extensive response on in this forum.

The upside is that you can use this to run some definition of a container inside of an enclave.  The downside is that all of this infrastructure increases the Trusted Computing Base (TCB) of the containerized application such that it is probably equivalent to that of a well secured container platform.  The functional security guarantees of this model also have to be considered in the face of how 'leaky' current platforms are in the face of micro-architectural vulnerabilities, ie. Spectre/Meltdown type vulnerabilities.

The other impact is on performance which is documented to be about 60-80% of native containers.  The other operational issue is the limited amount of Enclave Page Cache memory which is just under 128 megabytes on currently available platforms.  This places an effective total limit on the total container footprint that can be supported.

In order to address all of these issues we just spent two years of development on an alternative which we refer to as Autonomous Introspection Canisters, yes canisters...:-)  In our model we run standard containers, each of which is paired with an SGX enclave running a modeling engine which models the information exchange events (actor/subject) interactions that occur inside of the canister/container.  If the enclave detects an extra-dimensional behavior inside of the canister it sets a flag in the task control structure of the process which then applies very granular disciplining with a per process LSM, also controlled by the modeling enclave.

The behavior modeling is implemented as an independent namespace which allows each canister to have its own behavioral definition.  We run a very small security supervisor in the root behavior namespace which is also introspected by its own enclave.  All of this allows a guarantee that neither the platform or any of the canisters that it runs could have executed a behavior contrary to the intention of its designer/developers.  We believe this provides a superior security guarantee to the 'shielding' strategy implemented by SCONE.

The most significant impact is that there are no limitations on the size of canisters that can be run and the canisters run at close to native speed.  Applications running inside of a canister are free to implement enclaves in order to provide integrity and confidentiality guarantees for privileged data/code.  With the advantage of a documented guarantee that a side-channel attack could have not been conducted against the enclave or any other platform component.

I've mentioned in other posts that we wrote an independent implementation of the PSW to support this work.  We use the ability of that PSW to implement direct enclave<->enclave attestation and communications in order to provide remote launch and telemetry of canisters.

So I believe that pretty much covers the spectrum of how SGX is being used to address the execution of containers.  Hopefully the information is useful to your efforts.

Best wishes for a productive week.

Dr. Greg

0 Kudos
Harris__David
Beginner
1,251 Views

Thanks, Dr. Greg, your answer clarified a number of things and brought up a couple other questions on the landscape of projects that are working with SGX.   My intent is not to rate these projects in general, but understand their technical features in light of my project's specific objectives.  Those objectives are to maintain compatibility with the Intel SGX tools, platform and SDK as closely as possible, and related, to ensure the smallest possible attack surface of the untrusted, as well as trusted components of the production application. 

Some of the other projects I've come across appear to invert the (trusted-untrusted) application-within-SGX enclave model with a SGX within a Container model, but I'm not completely sure of the implications of their approaches.  I'm thinking of github.com/tozd/docker-sgx and github.com/sebva/docker-sgx, and github.com/sebva/sgx-orchestrator.  Do these sgx-within-docker or kubernetes approaches ultimately support or break the SGX security model? Also, how compatible is Asylo'S API at https://asylo.dev/docs with the underlying SGX platform and SDK?

Also, what is the availability of the Autonomous Introspection Canisters? 

 

 

0 Kudos
Dr__Greg
Super User
1,251 Views

Good afternoon David, I hope your day has gone well.

From a security perspective, the work by sebva (docker-sgx) and tozd (docker-sgx) are about creating Docker environments that support running SGX based applications inside of a container.  As such, they containerize the Intel PSW and the infrastructure necessary to load, initialize, run and support an enclave and do not have a significant impact on enclave security.  The ability of SGX to attain the required security guarantees is a function of how the enclave is architected.  The sgx-orchestrator work by sebva is about providing a framework to implement SGX resource management in the context of a Kubernetes framework, ie. allocation of EPC resources across a cluster of SGX nodes.  Once again, the actual security of the managed enclave application will be dependent on the size of the Trusted Computing Base (TCB) the application architect implements.

The Asylo project concerns itself with providing a standardized API to a Trusted Execution Environment (TEE).  It can be thought of as a wrapper framework that enables application developer to not have to concern themselves with whether the 'trusted' component of their application is rooted in an SGX enclave or an alternate TEE such as TrustZones.  The 'backend' uses an implementation of the Intel PSW to support enclave management.  As is the case with the previously discussed environments, the inherent security guarantee afforded to code and data will be dependent on how much trust the application developer is willing to convey on the operating system and untrusted userspace.

From a security perspective, one of the challenges with wrapping the PSW is the inherent size of the PSW.  This was an issue that was raised in the original BlackHat review of SGX:

https://www.blackhat.com/docs/us-16/materials/us-16-Aumasson-SGX-Secure-Enclaves-In-Practice-Security-And-Crypto-Review.pdf

That review notes, at the time, that the PSW+SDK clocked in at around 170 KLOC, it has grown a bit since then.  Those numbers don't address the fact that the AESM is not exactly petite with respect to what it depends on, at current count it clocks in at a dependency on 35 separate shared libraries.

These issues, and the need to address minimum footprint embedded applicationsm motivated us to build an alternate PSW implementation for our SGX/AI work.  We come in at around 25-30 KLOC for everything needed to provision EPIDs and to load, initialize and attest enclaves.  The only external dependencies we have are the MUSL C library, OpenSSL and libelf.  Using this infrastructure we have been able to implement 'fire and forget' architectures where everything needed to implement an enclave based application can be packaged into a singe 'fat' binary, which has implications for SGX enabled container applications.

Just to be clear, our primary focus isn't on supporting the use of SGX in an enclave.  Our primary focus is to use an SGX enclave to guarantee the behavioral integrity of a container, or in our parlance, a canister.  We also support SGX enabled applications inside of a canister in order to provide additional confidentiality and integrity (CI) guarantees for sensitive data and code.  As I have previously noted, the ability to demonstrate that a platform has not been able to execute unknown code means the final SGX CI guarantees are as absolute as they can be, particularly with respect to confidentiality.

With respect to general availability of our work, that is currently an open question.  All of this work was done in order to implement a functional prototype for an SGX mediated architecture for secure command and control of critical infrastructure based on Intelligent Network Endpoint Devices (INED's) which are capable of compromise self-detection and self-healing.  That work has now been presented and we are waiting to see if there is an appetite to tackle this problem on a national level  The outcome of those conversations will determine whether it makes sense to license the technology or if some type of open-source strategy makes sense.

I would conclude by applauding the concerns that you and others have demonstrated in this forum about making good security architecture decisions.  As the industry embraces TEE's it will be critical that architects make very good decisions with respect to TCB analysis, otherwise all of this very interesting technology will fail to produce the desired outcomes.

Have a good day.

Dr. Greg

0 Kudos
Reply