Intel® Software Guard Extensions (Intel® SGX)
Use hardware-based isolation and memory encryption to provide more code protection in your solutions.

TCB_OUT_OF_DATE despite TCB recovery

CLauverjat
Beginner
277 Views
Hello,

 

We are experiencing problems with the attestation failing because of a TCB_OUT_OF_DATE. We are aware that Intel has updated the TCB due to the discovery of vulnerabilities that require MCU patch (INTEL-SA-00615, INTEL-SA-00657). And we have applied the TCB Recovery procedure, but despite that the attestation still fails due to a TCB_OUT_OF_DATE

 

Machine description:
OS : Ubuntu 20.04 LTS
Model : DELL PowerEdge-R750xs
CPU : Intel(R) Xeon(R) Gold 6334 CPU @ 3.60GHz
We are using intel-microcode Ubuntu package to apply microcode updates.

 

Here is the procedure we followed:
Here are the SGX PSW + DCAP software versions after upgrade : (output from apt list --installed)
libsgx-ae-epid/unknown,now 2.18.101.1-focal1 amd64 [installed,automatic]
libsgx-ae-id-enclave/unknown,now 1.15.100.3-focal1 amd64 [installed]
libsgx-ae-le/unknown,now 2.18.101.1-focal1 amd64 [installed,automatic]
libsgx-ae-pce/unknown,now 2.18.101.1-focal1 amd64 [installed,automatic]
libsgx-ae-qe3/unknown,now 1.15.100.3-focal1 amd64 [installed]
libsgx-ae-qve/unknown,now 1.15.100.3-focal1 amd64 [installed]
libsgx-ae-tdqe/unknown,now 1.15.100.3-focal1 amd64 [installed]
libsgx-aesm-ecdsa-plugin/unknown,now 2.18.101.1-focal1 amd64 [installed,automatic]
libsgx-aesm-epid-plugin/unknown,now 2.18.101.1-focal1 amd64 [installed,automatic]
libsgx-aesm-launch-plugin/unknown,now 2.18.101.1-focal1 amd64 [installed,automatic]
libsgx-aesm-pce-plugin/unknown,now 2.18.101.1-focal1 amd64 [installed,automatic]
libsgx-aesm-quote-ex-plugin/unknown,now 2.18.101.1-focal1 amd64 [installed,automatic]
libsgx-dcap-default-qpl/unknown,now 1.15.100.3-focal1 amd64 [installed]
libsgx-dcap-ql/unknown,now 1.15.100.3-focal1 amd64 [installed]
libsgx-dcap-quote-verify/unknown,now 1.15.100.3-focal1 amd64 [installed]
libsgx-enclave-common-dev/unknown,now 2.18.101.1-focal1 amd64 [installed]
libsgx-enclave-common/unknown,now 2.18.101.1-focal1 amd64 [installed]
libsgx-epid/unknown,now 2.18.101.1-focal1 amd64 [installed]
libsgx-headers/unknown,now 2.18.101.1-focal1 amd64 [installed,automatic]
libsgx-launch/unknown,now 2.18.101.1-focal1 amd64 [installed]
libsgx-pce-logic/unknown,now 1.15.100.3-focal1 amd64 [installed]
libsgx-qe3-logic/unknown,now 1.15.100.3-focal1 amd64 [installed]
libsgx-quote-ex/unknown,now 2.18.101.1-focal1 amd64 [installed]
libsgx-ra-network/unknown,now 1.15.100.3-focal1 amd64 [installed]
libsgx-ra-uefi/unknown,now 1.15.100.3-focal1 amd64 [installed]
libsgx-tdx-logic/unknown,now 1.15.100.3-focal1 amd64 [installed]
libsgx-uae-service/unknown,now 2.18.101.1-focal1 amd64 [installed]
libsgx-urts/unknown,now 2.18.101.1-focal1 amd64 [installed]
sgx-aesm-service/unknown,now 2.18.101.1-focal1 amd64 [installed,automatic]
sgx-dcap-pccs/unknown,now 1.15.100.3-focal1 amd64 [installed]
sgx-pck-id-retrieval-tool/unknown,now 1.15.100.3-focal1 amd64 [installed]
Concerning the microcode here is the tool version :
intel-microcode/focal-updates,focal-security,now 3.20220809.0ubuntu0.20.04.1 amd64 [installed]
It looks like the right microcode update is applied see dmesg output below :
dmesg | grep "microcode"
[ 1.610253] microcode: sig=0x606a6, pf=0x1, revision=0xd00037b
[ 1.610401] microcode: Microcode Update Driver: v2.2.
The microcode version indicated is 0x37b
and according to your documentation https://www.intel.com/content/www/us/en/developer/topic-technology/software-security-guidance/proces... for our processor we need (at least) 2022.2: 0x375. So it should be ok, right ?

 

But despite the procedure we still encounter the TCB_OUT_OF_DATE error.

 

At this point we were a bit lost, so we modified the QVL to extract the TCB values from the PCK certificate to get a better picture of the TCB, and we got :
certPceSvn = 11
# Tcb[i] indicates the SVN for each level of the TCB
Tcb[0] = 4
Tcb[1] = 4
Tcb[2] = 3
Tcb[3] = 3
Tcb[4] = 255
Tcb[5] = 255
Tcb[6] = 0
Tcb[7] = 0
Tcb[8] = 0
Tcb[9] = 0
Tcb[10] = 0
Tcb[11] = 0
Tcb[12] = 0
Tcb[13] = 0
Tcb[14] = 0
Tcb[15] = 0
By comparing that to the TCBInfo.json we can see that up-to-date TCB should have a PCESVN of 13 but we only have a PCESVN of 11.
 
Is there something we overlooked ? How can we resolve this TCB issue ?

 

Thank you
Labels (4)
0 Kudos
1 Solution
CLauverjat
Beginner
158 Views

Hello,

 

After further investigation, we were able to solve the issue. I only had remote access to the server, so I was only able to access the BIOS via the racadm tool that allow remote control to the DELL server. Because of that I could not find the CPU revision, but I expect the revision to be the last one since we have done all relevant BIOS updates. 

By I'll cut to the chase, when I was in the (remote) BIOS settings I still found a SysSecurity.SgxFactoryReset setting that could be turned on. I did that (which resetted SGX keys), and after a restart it finally worked, the TCB_OUT_OF_DATE went away, and we know have a STATUS_TCB_SW_HARDENING_NEEDED which is to be expected ! 

 

I was somewhat surprised this worked, so I did some digging and I found this document https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/reso... which states :

If the 3rd Generation Intel® Xeon® Scalable Processors, code name Icelake-SP Post Launch Release 3 (PLR3) microcode update (MCU) is loaded at reset (otherwise known as FIT-loaded1and the platform then executes the TCB Recovery Boot Flow2, a microcode (MCheck) error occurs that causes Intel® Software Guard Extensions (Intel® SGX) to be disabled.

 

I don't now for sure if the problem I encountered was related to the issue described in the document because in our case the Intel SGX was never disabled... but I expect a SgxFactoryReset to inititate a Initial Platform Establishment (IPE) flow, which the document says work fine contrary to the TCB Recovery Boot Flow (which was what we were doing before).

Anyway with the caveat that we lost the sealing keys during the reset (fortunately we could do that because we did not have any important data sealed on the server), it solves the issue for us.

View solution in original post

4 Replies
KFPW_Intel
Moderator
200 Views

Hi,

 

Thank you for your interest in Intel® SGX.

 

Recently there is another posting asked a similar issue regards to TCB OutOfDate. Do you see any errors that can share to us?

 

Based on the last similar issue, we suggested to check the version of the uCode loaded by the BIOS, which can be found in the BIOS setup menu. The latest cpu_svn[0] is 0x7.

 

In the meantime, we are investigating this issue with the development teams regards to the TCB Recovery mentioned, especially for Intel® Xeon® Gold 6334 Processor.

 

Please allow some time for us to investigate, thank you for your patience.

 

Regards,

Ken


KFPW_Intel
Moderator
172 Views

Hi,

 

Is the suggested method to check the uCode loaded by the BIOS works for you? Hope that it helps.

 

Furthermore you can get the SGX TCB Info via curl -v -X GET "https://api.trustedservices.intel.com/sgx/certification/v4/tcb?fmspc={}" based on the documentation that may assist you for your use cases.

 

Please inform us if you have any questions regards to this issue. Thank you.

 

Regards,

Ken


CLauverjat
Beginner
159 Views

Hello,

 

After further investigation, we were able to solve the issue. I only had remote access to the server, so I was only able to access the BIOS via the racadm tool that allow remote control to the DELL server. Because of that I could not find the CPU revision, but I expect the revision to be the last one since we have done all relevant BIOS updates. 

By I'll cut to the chase, when I was in the (remote) BIOS settings I still found a SysSecurity.SgxFactoryReset setting that could be turned on. I did that (which resetted SGX keys), and after a restart it finally worked, the TCB_OUT_OF_DATE went away, and we know have a STATUS_TCB_SW_HARDENING_NEEDED which is to be expected ! 

 

I was somewhat surprised this worked, so I did some digging and I found this document https://www.intel.com/content/www/us/en/developer/articles/technical/software-security-guidance/reso... which states :

If the 3rd Generation Intel® Xeon® Scalable Processors, code name Icelake-SP Post Launch Release 3 (PLR3) microcode update (MCU) is loaded at reset (otherwise known as FIT-loaded1and the platform then executes the TCB Recovery Boot Flow2, a microcode (MCheck) error occurs that causes Intel® Software Guard Extensions (Intel® SGX) to be disabled.

 

I don't now for sure if the problem I encountered was related to the issue described in the document because in our case the Intel SGX was never disabled... but I expect a SgxFactoryReset to inititate a Initial Platform Establishment (IPE) flow, which the document says work fine contrary to the TCB Recovery Boot Flow (which was what we were doing before).

Anyway with the caveat that we lost the sealing keys during the reset (fortunately we could do that because we did not have any important data sealed on the server), it solves the issue for us.

KFPW_Intel
Moderator
145 Views

Hi,

 

This is amazing! It is good to know that issue can be solved.

Thank you for sharing the solutions to us.

This thread will be marked as answered and Intel will no longer monitor this thread.

Please start a new thread if you need further help.

 

Regards,

Ken


Reply