Getting Intel PCS server returns error(404)

[jgnoonan]

I am confused on how this is supposed to work with Azure Confidential Computing with Intel SGX virtual machines.  I am running this VM with Ubuntu 20.04.  I have installed the Intel SGX SDK, as well as the AESM and PCCS Services.  My sgx_default_qncl.conf file looks like this:

{
  "pccs_url": "https://localhost:8081/sgx/certification/v3/",

  "use_secure_cert": false,

  "collateral_service": "https://api.trustedservices.intel.com/sgx/certification/v3/",

  "pccs_api_version": "3.1",

  "retry_times": 6,

  "retry_delay": 5,

  "local_pck_url": "http://169.254.169.254/metadata/THIM/sgx/certification/v3/",

  "pck_cache_expire_hours": 48,

  "custom_request_options" : {
       "get_cert" : {
          "headers": {
              "metadata": "true"
          },
          "params": {
             "api-version": "2021-07-22-preview"
         }
      }
   }
}

When I run the PCKIDRetrievalTool, it generates the csv and repoorts that the file was successfully sent to the cache server.  It seems that a part of the process is missing, namely registering the server (VM).  In my application, I am getting a failed to renew attestation error, attestation data invalid: No evidence provided on handshake start.  Any guidance would be greatly appreciated.

[Scott_R_Intel]

Hello.

When running in Azure, you do not need to use the PCK Cert ID Retrieval Tool nor do you need to register the platform.  Microsoft has already registered the platform for you.  If you need to get the platform PCK Cert, you will need to get it from their THIM service directly, or use the "local_pck_url" to get it.  This is a local service running on the host of your VM that has the platform PCK Cert.

When using that QCNL config file, the QCNL will automatically first look to the "local_pck_url" to try to get the platform PCK Cert for quote generation and for quote verification collateral.

BTW, the endpoint versions you're using (v3) are old...  you should be using v4 now (this may be the issue?).  Our latest config file for running in Azure can be found here.

Regards.

[jgnoonan]

Scott:  Thanks.  I changed QCNL config file to what was at the link you sent.  I am not seeing the same error but am now seeing this in the log:

Dec  9 15:27:00 jgnoonan aesm_service[24739]: [QCNL] JSON config file /etc/sgx_default_qcnl.conf is loaded successfully.

Dec  9 15:27:01 jgnoonan aesm_service[24739]: [QCNL] JSON config file /etc/sgx_default_qcnl.conf is loaded successfully.

Dec  9 15:27:01 jgnoonan aesm_service[24739]: [QCNL] Getting pck certificate and chain.

Dec  9 15:27:01 jgnoonan aesm_service[24739]: [QCNL] JSON config file /etc/sgx_default_qcnl.conf is loaded successfully.

Dec  9 15:27:01 jgnoonan aesm_service[24739]: [QCNL] Request URL http://169.254.169.254/metadata/THIM/sgx/certification/v4/pckcert?qeid=3B23371D966CD8619A8B5413028B0396&encrypted_ppid=D308CDFECE281909FC5CDB66033751EBB9F487723B9D39CBEF8BED58D9E5261D2D64D924D87510F8A9DB7F9677CC5B00E2D72AD56EE65C601815E777ACD363697F8267A0602BD0BCF0DA4D7787EFFED1DB7FFD7D2F14FD1B1E04FC18137A3C2D22D00FB6A792B11A3CBE0F61A87DD9F99E169B9...

Dec  9 15:27:02 jgnoonan aesm_service[24739]: [QCNL] HTTP status code: 200

Dec  9 15:27:02 jgnoonan aesm_service[24739]: [QCNL] Successfully fetched certificate from primary URL: 'http://169.254.169.254/metadata/THIM/sgx/certification/v4/'.

Dec  9 15:27:02 jgnoonan aesm_service[24739]: [QCNL] Header 'sgx-tcbm' not found.

Dec  9 15:27:02 jgnoonan aesm_service[24739]: [QCNL] Header 'sgx-pck-certificate-issuer-chain' not found.

Dec  9 15:27:02 jgnoonan aesm_service[24739]: [QCNL] sgx-Tcbm: 1515020401800e0000000000000000000D00

Dec  9 15:27:02 jgnoonan aesm_service[24739]: [QCNL] pckCert: -----BEGIN CERTIFICATE-----#012MIIEjTCCBDOgAwIBAgIUPK/aHNlgp3hqf2Na6tdHIItyKBUwCgYIKoZIzj0EAwIw#012cTEjMCEGA1UEAwwaSW50ZWwgU0dYIFBDSyBQcm9jZXNzb3IgQ0ExGjAYBgNVBAoM#012EUludGVsIENvcnBvcmF0aW9uMRQwEgYDVQQHDAtTYW50YSBDbGFyYTELMAkGA1UE#012CAwCQ0ExCzAJBgNVBAYTAlVTMB4XDTI0MTAzMTA4MDYzMloXDTMxMTAzMTA4MDYz#012MlowcDEiMCAGA1UEAwwZSW50ZWwgU0dYIFBDSyBDZXJ0aWZpY2F0ZTEaMBgGA1UE#012CgwRSW50ZWwgQ29ycG9yYXRpb24xFDASBgNVBAcMC1NhbnRhIENsYXJhMQswCQYD#012VQQIDAJDQTELMAkGA1UEBhMCVVMwWTATBgcqhkjOPQIBBggqhkjOPQMBBwNCAASr#012o1ywHxWXP8BHGdTKg3TMewuDIRWH1gpx8l0kHxr2S+Rf3qjJ5plMPOnjA5pu10or#012K5I0AaLuhxO00GjBlyyKo4ICqDCCAqQwHwYDVR0jBBgwFoAU0Oiq2nXX+S5JF5g8#012exRl0NXyWU0wbAYDVR0fBGUwYzBhoF+gXYZbaHR0cHM6Ly9hcGkudHJ1c3RlZHNl#012cnZpY2VzLmludGVsLmNvbS9zZ3gvY2VydGlmaWNhdGlvbi92NC9wY2tjcmw/Y2E9#012cHJvY2Vzc29yJmVuY29kaW5nPWRlcjAdBgNVHQ4EFgQU1GQZ9zmm8OxCboGZwOJR#0126sIiyAEwDgYDVR0PAQH/BAQDAgbAMAwGA1UdEwEB/wQCMAAwggHUBgkqhkiG+E0B#012DQEEggHFMIIBwTAeBgoqhkiG+E0BDQEBBBCMTD8iJ02erEUq9RPwbnCmMIIBZAYK#012KoZIhvhNAQ0BAjCCAVQwEAYLKoZIhvhNAQ0BAgECARUwEAYLKoZIhvhNAQ0BAgIC#012ARUwEAYLKoZIhvhNAQ0BAgMCAQIwEAYLKoZIhvhNAQ0BAgQCAQQwEAYLKoZIhvhN#012AQ0BAgUCAQEwEQYLKoZIhvhNAQ0BAgYCAgCAMBAGCyqGSIb4TQENAQIHAgEOMBAG#012CyqGSIb4TQENAQIIAgEAMBAGCyqGSIb4TQENAQIJAgEAMBAGCyqGSIb4TQENAQIK#012AgEAMBAGCyqGSIb4TQENAQILAgEAMBAGCyqGSIb4TQENAQIMAgEAMBAGCyqGSIb4#012TQENAQINAgEAMBAGCyqGSIb4TQENAQIOAgEAMBAGCyqGSIb4TQENAQIPAgEAMBAG#012CyqGSIb4TQENAQIQAgEAMBAGCyqGSIb4TQENAQIRAgENMB8GCyqGSIb4TQENAQIS#012BBAVFQIEAYAOAAAAAAAAAAAAMBAGCiqGSIb4TQENAQMEAgAAMBQGCiqGSIb4TQEN#012AQQEBgCQbtUAADAPBgoqhkiG+E0BDQEFCgEAMAoGCCqGSM49BAMCA0gAMEUCIQDr#012qM5LWeyBPOkyNOlTKnECFSMwRc3tpLAXkcrc9ORM3QIgacl3JIPzTIf2YRY9K9ia#012szaRwIuj+NSvS1yhCQa7djA=#012-----END CERTIFICATE-----#012

Dec  9 15:27:02 jgnoonan aesm_service[24739]: [QCNL] sgx-Pck-Certificate-Issuer-Chain: -----BEGIN CERTIFICATE-----#012MIICmDCCAj6gAwIBAgIVANDoqtp11/kuSReYPHsUZdDV8llNMAoGCCqGSM49BAMC#012MGgxGjAYBgNVBAMMEUludGVsIFNHWCBSb290IENBMRowGAYDVQQKDBFJbnRlbCBD#012b3Jwb3JhdGlvbjEUMBIGA1UEBwwLU2FudGEgQ2xhcmExCzAJBgNVBAgMAkNBMQsw#012CQYDVQQGEwJVUzAeFw0xODA1MjExMDUwMTBaFw0zMzA1MjExMDUwMTBaMHExIzAh#012BgNVBAMMGkludGVsIFNHWCBQQ0sgUHJvY2Vzc29yIENBMRowGAYDVQQKDBFJbnRl#012bCBDb3Jwb3JhdGlvbjEUMBIGA1UEBwwLU2FudGEgQ2xhcmExCzAJBgNVBAgMAkNB#012MQswCQYDVQQGEwJVUzBZMBMGByqGSM49AgEGCCqGSM49AwEHA0IABL9q+NMp2IOg#012tdl1bk/uWZ5+TGQm8aCi8z78fs+fKCQ3d+uDzXnVTAT2ZhDCifyIuJwvN3wNBp9i#012HBSSMJMJrBOjgbswgbgwHwYDVR0jBBgwFoAUImUM1lqdNInzg7SVUr9QGzknBqww#012UgYDVR0fBEswSTBHoEWgQ4ZBaHR0cHM6Ly9jZXJ0aWZpY2F0ZXMudHJ1c3RlZHNl#012cnZpY2VzLmludGVsLmNvbS9JbnRlbFNHWFJvb3RDQS5kZXIwHQYDVR0OBBYEFNDo#012qtp11/kuSReYPHsUZdDV8llNMA4GA1UdDwEB/wQEAwIBBjASBgNVHRMBAf8ECDAG#012AQH/AgEAMAoGCCqGSM49BAMCA0gAMEUCIQCJgTbtVqOyZ1m3jqiAXM6QYa6r5sWS#0124y/G7y8uIJGxdwIgRqPvBSKzzQagBLQq5s5A70pdoiaRJ8z/0uDz4NgV91k=#012-----END CERTIFICATE-----#012-----BEGIN CERTIFICATE-----#012MIICjzCCAjSgAwIBAgIUImUM1lqdNInzg7SVUr9QGzknBqwwCgYIKoZIzj0EAwIw#012aDEaMBgGA1UEAwwRSW50ZWwgU0dYIFJvb3QgQ0ExGjAYBgNVBAoMEUludGVsIENv#012cnBvcmF0aW9uMRQwEgYDVQQHDAtTYW50YSBDbGFyYTELMAkGA1UECAwCQ0ExCzAJ#012BgNVBAYTAlVTMB4XDTE4MDUyMTEwNDUxMFoXDTQ5MTIzMTIzNTk1OVowaDEaMBgG#012A1UEAwwRSW50ZWwgU0dYIFJvb3QgQ0ExGjAYBgNVBAoMEUludGVsIENvcnBvcmF0#012aW9uMRQwEgYDVQQHDAtTYW50YSBDbGFyYTELMAkGA1UECAwCQ0ExCzAJBgNVBAYT#012AlVTMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAEC6nEwMDIYZOj/iPWsCzaEKi7#0121OiOSLRFhWGjbnBVJfVnkY4u3IjkDYYL0MxO4mqsyYjlBalTVYxFP2sJBK5zlKOB#012uzCBuDAfBgNVHSMEGDAWgBQiZQzWWp00ifODtJVSv1AbOScGrDBSBgNVHR8ESzBJ#012MEegRaBDhkFodHRwczovL2NlcnRpZmljYXRlcy50cnVzdGVkc2VydmljZXMuaW50#012ZWwuY29tL0ludGVsU0dYUm9vdENBLmRlcjAdBgNVHQ4EFgQUImUM1lqdNInzg7SV#012Ur9QGzknBqwwDgYDVR0PAQH/BAQDAgEGMBIGA1UdEwEB/wQIMAYBAf8CAQEwCgYI#012KoZIzj0EAwIDSQAwRgIhAOW/5QkR+S9CiSDcNoowLuPRLsWGf/Yi7GSX94BgwTwg#012AiEA4J0lrHoMs+Xo5o/sX6O9QWxHRAvZUGOdRQ7cvqRXaqI=#012-----END CERTIFICATE-----#012

Dec  9 15:27:02 jgnoonan aesm_service[24739]: [QCNL] JSON config file /etc/sgx_default_qcnl.conf is loaded successfully.

The application (Signal's Contact Discover Service) does start but displays the following to the screen:

2024-12-09 15:33:35,512 [main] INFO  org.signal.cdsi.enclave.Enclave.jni - Initialized enclave with 1 shards and 268435456 bytes of EPC memory

2024-12-09 15:33:35,559 [io-executor-thread-1] INFO  org.signal.cdsi.account.RandomAccountPopulator - Populated enclave with 0 random accounts in 0 milliseconds.

2024-12-09 15:33:35,568 [main] INFO  io.micronaut.logging.PropertiesLoggingLevelsConfigurer - Setting log level 'WARN' for logger: 'com.azure.cosmos'

2024-12-09 15:33:35,569 [main] INFO  io.micronaut.logging.PropertiesLoggingLevelsConfigurer - Setting log level 'TRACE' for logger: 'org.signal.cdsi.enclave'

2024-12-09 15:33:35,569 [main] INFO  io.micronaut.logging.PropertiesLoggingLevelsConfigurer - Setting log level 'TRACE' for logger: 'org.signal.cdsi'

2024-12-09 15:33:35,569 [main] INFO  io.micronaut.logging.PropertiesLoggingLevelsConfigurer - Setting log level 'TRACE' for logger: 'software.amazon.awssdk'

2024-12-09 15:33:35,615 [enclave-jni-executor-thread-2] DEBUG org.signal.cdsi.enclave.Enclave.jni - Renewing enclave attestation

2024-12-09 15:33:35,637 [enclave-jni-executor-thread-2] ERROR org.signal.cdsi.enclave.Enclave.jni - resource12717708207972734601::OE_UNSUPPORTED [/source/openenclave/enclave/core/sgx/report.c:sgx_create_report:131]

2024-12-09 15:33:35,637 [enclave-jni-executor-thread-2] ERROR org.signal.cdsi.enclave.Enclave.jni - resource12717708207972734601::OE_UNSUPPORTED [/source/openenclave/enclave/core/sgx/report.c:_get_local_report:189]

2024-12-09 15:33:35,637 [enclave-jni-executor-thread-2] ERROR org.signal.cdsi.enclave.Enclave.jni - resource12717708207972734601::OE_UNSUPPORTED [/source/openenclave/enclave/core/sgx/report.c:oe_get_remote_report:294]

2024-12-09 15:33:35,637 [enclave-jni-executor-thread-2] ERROR org.signal.cdsi.enclave.Enclave.jni - resource12717708207972734601::OE_UNSUPPORTED [/source/openenclave/enclave/core/sgx/report.c:_oe_get_report_internal:388]

2024-12-09 15:33:35,638 [enclave-jni-executor-thread-2] ERROR org.signal.cdsi.enclave.Enclave.jni - resource12717708207972734601::OE_UNSUPPORTED [/source/openenclave/enclave/core/sgx/report.c:oe_get_report_v2_internal:443]

2024-12-09 15:33:35,638 [enclave-jni-executor-thread-2] ERROR org.signal.cdsi.enclave.Enclave.jni - resource12717708207972734601:SGX Plugin: Failed to get OE report. OE_UNSUPPORTED (oe_result_t=OE_UNSUPPORTED) [/source/openenclave/enclave/sgx/attester.c:_get_evidence:165]

2024-12-09 15:33:35,638 [enclave-jni-executor-thread-2] ERROR org.signal.cdsi.enclave.Enclave.jni - resource12717708207972734601::OE_UNSUPPORTED [/source/openenclave/enclave/attest_plugin.c:oe_get_evidence:116]

2024-12-09 15:33:35,640 [scheduled-executor-thread-1] WARN  org.signal.cdsi.enclave.Enclave - Failed to renew attestation

java.util.concurrent.CompletionException: org.signal.cdsi.enclave.CdsiEnclaveException: CDSI enclave code 105

at org.signal.cdsi.enclave.Enclave.lambda$renewAttestation$3(Enclave.java:194)

at io.micrometer.core.instrument.composite.CompositeTimer.record(CompositeTimer.java:141)

at org.signal.cdsi.enclave.Enclave.lambda$runAsync$18(Enclave.java:440)

at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804)

at io.micrometer.core.instrument.composite.CompositeTimer.record(CompositeTimer.java:141)

at io.micrometer.core.instrument.Timer.lambda$wrap$0(Timer.java:196)

at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)

at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)

at java.base/java.lang.Thread.run(Thread.java:840)

Caused by: org.signal.cdsi.enclave.CdsiEnclaveException: CDSI enclave code 105

at org.signal.cdsi.enclave.Enclave.nativeEnclaveAttest(Native Method)

at org.signal.cdsi.enclave.Enclave.lambda$renewAttestation$3(Enclave.java:182)

... 8 common frames omitted

2024-12-09 15:33:35,873 [main] INFO  io.micronaut.runtime.Micronaut - Startup completed in 3503ms. Server Running: http://localhost:8080

It appears that the evidence report isn't being generated and there are some headers missing.  Any ideas?  Thanks!

[Scott_R_Intel]

The very first error there (OE_UNSUPPORTED [/source/openenclave/enclave/core/sgx/report.c:sgx_create_report:131) per the OE source, looks like you're running in SGX Simulation mode for some reason, hence the OE_UNSUPPORTED return:  https://github.com/openenclave/openenclave/blob/master/enclave/core/sgx/report.c#L131

[jgnoonan]

I fixed that, thanks! Server is running, but generating the following error.  Sorry for all the questions:

2024-12-09 16:40:05,314 [scheduled-executor-thread-1] WARN  org.signal.cdsi.enclave.Enclave - Failed to renew attestation
java.util.concurrent.CompletionException: org.signal.libsignal.cds2.DcapException: failure to attest remote SGX enclave code: AttestationError { message: "(evidence -> quote -> SgxPckExtension) could not parse required extension from PCK certificate: 1.2.840.113741.1.13.1.6" }
	at java.base/java.util.concurrent.CompletableFuture.encodeThrowable(CompletableFuture.java:315)
	at java.base/java.util.concurrent.CompletableFuture.completeThrowable(CompletableFuture.java:320)
	at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1807)
	at io.micrometer.core.instrument.composite.CompositeTimer.record(CompositeTimer.java:141)
	at io.micrometer.core.instrument.Timer.lambda$wrap$0(Timer.java:196)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: org.signal.libsignal.cds2.DcapException: failure to attest remote SGX enclave code: AttestationError { message: "(evidence -> quote -> SgxPckExtension) could not parse required extension from PCK certificate: 1.2.840.113741.1.13.1.6" }
	at org.signal.libsignal.internal.Native.Cds2Metrics_extract(Native Method)
	at org.signal.libsignal.cds2.Cds2Metrics.extract(Cds2Metrics.java:31)
	at org.signal.cdsi.enclave.Enclave.publishAttestationMetrics(Enclave.java:216)
	at org.signal.cdsi.enclave.Enclave.lambda$renewAttestation$3(Enclave.java:192)
	at io.micrometer.core.instrument.composite.CompositeTimer.record(CompositeTimer.java:141)
	at org.signal.cdsi.enclave.Enclave.lambda$runAsync$18(Enclave.java:440)
	at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1804)
	... 5 common frames omitted

[Scott_R_Intel]

It seems you're running on a DCSv2 series instance which are based on Xeon E CPUs..  These instances will not have that OID it's complaining about.  You can see all the OIDs in our PCK Cert Spec. As far as I know, OE should support those CPUs, but you'd need to debug OE or ask the OE maintainers about this.

Regards.

[jgnoonan]

Perfect.  BTW, what series do you recommend so I don't run into this issue?  I need to request a quota change from Microsoft.  Thanks!!

[Scott_R_Intel]

The DCSv3 series is based on our 3rd Gen Xeon Scalable (aka Ice Lake Server) CPUs.  It will have this OID, as all Xeon Scalables do, so it should get passed this particular issue.

[jgnoonan]

OK, so I provisioned a Standard DC2sv3 (2 vcpus, 16 GiB memory) but PCCS is showing the following errors in the log at PCCS Startup:

Intel PCS server returns error(404).

Dec 09 17:47:27 Signal-SGX-SRVR node[28819]: 2024-12-09 17:47:27.794 [error]: Error: No cache data for this platform.

Dec 09 17:47:27 Signal-SGX-SRVR node[28819]:     at Module.getPckCertFromPCS (file:///opt/intel/sgx-dcap-pccs/services/logic/commonCac>

Dec 09 17:47:27 Signal-SGX-SRVR node[28819]:     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Dec 09 17:47:27 Signal-SGX-SRVR node[28819]:     at async ReqCachingMode.registerPlatforms (file:///opt/intel/sgx-dcap-pccs/services/c>

Dec 09 17:47:27 Signal-SGX-SRVR node[28819]:     at async Module.registerPlatforms (file:///opt/intel/sgx-dcap-pccs/services/platforms>

Dec 09 17:47:27 Signal-SGX-SRVR node[28819]:     at async postPlatforms (file:///opt/intel/sgx-dcap-pccs/controllers/platformsControll>

Dec 09 17:47:27 Signal-SGX-SRVR node[28819]: 2024-12-09 17:47:27.796 [info]: 127.0.0.1 - - [09/Dec/2024:17:47:27 +0000] "POST /sgx/cer>

~

 
Intel PCS server returns error(404).

Dec 09 17:47:27 Signal-SGX-SRVR node[28819]: 2024-12-09 17:47:27.794 [error]: Error: No cache data for this platform.

Dec 09 17:47:27 Signal-SGX-SRVR node[28819]:     at Module.getPckCertFromPCS (file:///opt/intel/sgx-dcap-pccs/services/logic/commonCac>

Dec 09 17:47:27 Signal-SGX-SRVR node[28819]:     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

Dec 09 17:47:27 Signal-SGX-SRVR node[28819]:     at async ReqCachingMode.registerPlatforms (file:///opt/intel/sgx-dcap-pccs/services/c>

Dec 09 17:47:27 Signal-SGX-SRVR node[28819]:     at async Module.registerPlatforms (file:///opt/intel/sgx-dcap-pccs/services/platforms>

Dec 09 17:47:27 Signal-SGX-SRVR node[28819]:     at async postPlatforms (file:///opt/intel/sgx-dcap-pccs/controllers/platformsControll>

Dec 09 17:47:27 Signal-SGX-SRVR node[28819]: 2024-12-09 17:47:27.796 [info]: 127.0.0.1 - - [09/Dec/2024:17:47:27 +0000] "POST /sgx/cer>

~

 

So here is my question:  Based on the sqx_default_qcnl.conf, we're not even using a local PCCS service.  Do I need to worry about this?

[jgnoonan]

Scott:  Never mind about the last post.  I have a V3 and it's working perfectly now.  Thanks so much for your help!!

[jgnoonan]

Scott:  Is there a way to pull the certificate into a pk12 file?  I am getting a certificate error on the client side and I think I need to load the certificate into the client application's jks file.  The error is:

failure to attest remote enclave: AttestationError { message: “TCB contains unmitigated unaccepted advisory ids: ["INTEL-SA-00615"]” }

Thanks!

[Scott_R_Intel]

This means that there is an actual vulnerability for your platform/enclave that you need to understand.  If you do an internet search for that Security Advisory (SA), you will find information about it:

INTEL-SA-00615

That will provide all the relevant CVEs about this issue, as well as links to Intel SGX Attestation Technical Details and TCB Recovery Guidance that will also help you understand what you need to do for the given SA.

Th output you gave doesn't show it, but you would have gotten one of two possible verification results from that particular SA:  "SWHardeningNeeded" or "ConfigurationAndSWHardeningNeeded"

SWHardeningNeeded means you need to do something when building your enclave (ie. special toolchain, build options, etc.) to fully mitigate the vulnerability.

ConfigurationAndSWHardeningNeeded means you need to do something when building your enclave as above but also need to disable Hyperthreading.

Generally, once a vulnerability is found with a particular CPU that results in a SWHardeningNeeded verification result, you will forever and always get that result when verifying.  It is not an actual "failure" if you have done whatever work is required when building your enclave to mitigate the vulnerability.  Your quote verification policy needs to understand this.  Meaning, once you've mitigated the issue in your enclave and bumped the SVN (Security Version Number) of your enclave, your verification policy will need to understand that this result is ok because you've done what you need to do.

Finally, it is up to the platform owner and the enclave builder to read about and understand anything you may need to do for any given SA.

[jgnoonan]

We used the Azure ACC VM type DC2s_v3. Do you know which type does not have this issue, or do I need to contact Microsoft? Thanks!

[jgnoonan]

Scott:  Any advice on my last question?  Thanks and I really appreciate all of your help!

Warmest regards,

Joe Noonan

[Scott_R_Intel]

Hi again Joe.

If your ultimate goal is to get a fully "UP_TO_DATE" verification result with no SAs, that is going to be a tough task.  This becomes easily understandable if you go look at our Software Security Guidance page.  I am not aware of any clouds/instances that can do this currently, off the top of my head.

My advice, as I mentioned before, is to make sure you build your enclave(s) with all mitigations, and your verification code (looks to be Signal's) needs to be able to set and handle policies around this.  My educated guess is that their code probably has this ability, but I've never personally looked into it.

Regards.

[jgnoonan · Accepted solution]

Scott:  Thanks.  So it turns out my client did not install the Intel Microcode updates to their VM.  They claimed they had, but in fact, hadn't.  Once that was done, and the system rebooted, we rebuilt the enclaves including the flags recommended by the CVE docs, the issue was resolved.  Everything is up and running and I will accept the solution.  Thanks again so much for your help.

Problem resolution:

1.  Ran 

sudo apt update && sudo apt install intel-microcode -y

2.  Rebooted Virtual Machine

3.  Rebuilt enclaves.

[Roman888]

@jgnoonan Hi, I'm also trying setup this CDS service, but facing with little bit another error which comes from mobile app:

invalidAttestationData("SGX operation failed: attestation data invalid: Evidence does not fit expected format")

After debugging it looks like mobile app after initial connect to CDS service trying make enclave attestation, but service (or enclave) sends back some zero "ereport" data, and mobile app abort websocket connection with that error above.

And that's it, no errors from CDS backend, just this one from mobile client.

So, if everything works for you, could you please help me and describe your final setup by answering my questions:

1. What is your VM size? Is it still Standard_DC2s_v3? And what is OS? is it Ubuntu 20.04?

2. What is your final sgx_default_qcnl.conf looks like?

3. Where I can find same aesm_service logs as from this post ? https://community.intel.com/t5/Intel-Software-Guard-Extensions/Getting-Intel-PCS-server-returns-error-404/m-p/1648249/highlight/true#M6237

4. What exact libs you installed on VM? Could you show the result of the command: apt list --installed | grep 'sgx-.*'

5. Did you install Intel SGX SDK and/or open-enclave SDK?

6. Did you install az-dcap-client lib?

7. You said you installed local PCCS service, but it's not worked... Have you fixed it or you find out that it's not needed and removed it?

8. Did you setup at Azure control panel "attestation provider" service with some policies..?

9. You said "we rebuilt the enclaves including the flags recommended by the CVE docs" -- could you please provide more details how you did it? I mean what is the flags and enclave build command looks like?

[jgnoonan]

@Roman888 the answers to your questions are quite lengthy. I did get it to work and it requires both the Intel SGX SDK and Openenclave. I'll respond with a more detailed post shortly.

[Roman888]

thanks, will waiting for your answer..

[jgnoonan]

1. Yes, used DC2s_v3 (2 CPU 8GB RAM) with Ubuntu 20.04
2. See below:

{
  // *** ATTENTION : This file is in JSON format so the keys are case sensitive. Don't change them.
  
  // This is a typical config file when used in Microsoft Azure environment

  "pccs_url": "https://global.acccache.azure.net/sgx/certification/v4/",
  //"pccs_url": "https://localhost:8099/sgx/certification/v4/",

  "use_secure_cert": false,

  "collateral_service": "https://api.trustedservices.intel.com/sgx/certification/v4/",

  "pccs_api_version": "3.1",

  "retry_times": 6,

  "retry_delay": 5,

  "local_pck_url": "http://169.254.169.254/metadata/THIM/sgx/certification/v4/",

  "pck_cache_expire_hours": 48,

  "verify_collateral_cache_expire_hours": 48,

  "custom_request_options" : {
       "get_cert" : {
          "headers": {
              "metadata": "true"
          },
          "params": {
             "api-version": "2021-07-22-preview"
         }
      }
   }
}

•  The aesmd service writes logs to /var/log/syslog with the identifier aesmd
•   See below:

libsgx-ae-epid/unknown,now 2.25.100.3-focal1 amd64 [installed]
libsgx-ae-id-enclave/unknown,now 1.22.100.3-focal1 amd64 [installed,automatic]
libsgx-ae-le/unknown,now 2.25.100.3-focal1 amd64 [installed]
libsgx-ae-pce/unknown,now 2.25.100.3-focal1 amd64 [installed]
libsgx-ae-qe3/unknown,now 1.22.100.3-focal1 amd64 [installed]
libsgx-ae-qve/unknown,now 1.22.100.3-focal1 amd64 [installed]
libsgx-ae-tdqe/unknown,now 1.22.100.3-focal1 amd64 [installed]
libsgx-aesm-ecdsa-plugin/unknown,now 2.25.100.3-focal1 amd64 [installed]
libsgx-aesm-epid-plugin/unknown,now 2.25.100.3-focal1 amd64 [installed]
libsgx-aesm-launch-plugin/unknown,now 2.25.100.3-focal1 amd64 [installed]
libsgx-aesm-pce-plugin/unknown,now 2.25.100.3-focal1 amd64 [installed]
libsgx-aesm-quote-ex-plugin/unknown,now 2.25.100.3-focal1 amd64 [installed]
libsgx-dcap-default-qpl-dev/unknown,now 1.22.100.3-focal1 amd64 [installed]
libsgx-dcap-default-qpl/unknown,now 1.22.100.3-focal1 amd64 [installed]
libsgx-dcap-ql-dev/unknown,now 1.22.100.3-focal1 amd64 [installed]
libsgx-dcap-ql/unknown,now 1.22.100.3-focal1 amd64 [installed]
libsgx-dcap-quote-verify-dev/unknown,now 1.22.100.3-focal1 amd64 [installed]
libsgx-dcap-quote-verify/unknown,now 1.22.100.3-focal1 amd64 [installed]
libsgx-enclave-common-dev/unknown,now 2.25.100.3-focal1 amd64 [installed]
libsgx-enclave-common/unknown,now 2.25.100.3-focal1 amd64 [installed]
libsgx-epid-dev/unknown,now 2.25.100.3-focal1 amd64 [installed]
libsgx-epid/unknown,now 2.25.100.3-focal1 amd64 [installed]
libsgx-headers/unknown,now 2.25.100.3-focal1 amd64 [installed,automatic]
libsgx-launch-dev/unknown,now 2.25.100.3-focal1 amd64 [installed]
libsgx-launch/unknown,now 2.25.100.3-focal1 amd64 [installed]
libsgx-pce-logic/unknown,now 1.22.100.3-focal1 amd64 [installed]
libsgx-qe3-logic/unknown,now 1.22.100.3-focal1 amd64 [installed]
libsgx-quote-ex-dev/unknown,now 2.25.100.3-focal1 amd64 [installed]
libsgx-quote-ex/unknown,now 2.25.100.3-focal1 amd64 [installed]
libsgx-ra-network-dev/unknown,now 1.22.100.3-focal1 amd64 [installed]
libsgx-ra-network/unknown,now 1.22.100.3-focal1 amd64 [installed]
libsgx-ra-uefi-dev/unknown,now 1.22.100.3-focal1 amd64 [installed]
libsgx-ra-uefi/unknown,now 1.22.100.3-focal1 amd64 [installed]
libsgx-tdx-logic-dev/unknown,now 1.22.100.3-focal1 amd64 [installed]
libsgx-tdx-logic/unknown,now 1.22.100.3-focal1 amd64 [installed]
libsgx-uae-service/unknown,now 2.25.100.3-focal1 amd64 [installed]
libsgx-urts/unknown,now 2.25.100.3-focal1 amd64 [installed]
sgx-aesm-service/unknown,now 2.25.100.3-focal1 amd64 [installed]
sgx-dcap-pccs/unknown,now 1.21.100.3-focal1 amd64 [installed]
sgx-pck-id-retrieval-tool/unknown,now 1.22.100.3-focal1 amd64 [installed]
sgx-ra-service/unknown,now 1.22.100.3-focal1 amd64 [installed]

• Yes.  Installed both though I ran the ansible scripts located on this page.  It installs SGX and then you build and install OpenEnclave.
  
  1. https://github.com/openenclave/openenclave/blob/master/docs/GettingStartedDocs/Contributors/SGX1FLCGettingStarted.md
• The ansible scripts from step 6 installs the az-dcap-client, but we're using the Intel Quote Provider Library
• You do not need the local PCCS server since the pccs_url is using the Azure caching service.
• No.  You don't need that.
• You don't need to worry about that.  The enclaves are built already mitigated.  That said, the enclave ID is hard-coded into libsignal (yes, you read that correctly).  The hard-coded value is 

  0f6fd79cdfdaa5b2e6337f534d3baf999318b0c462a7ac1f41297a3e4b424a57.  You need to put this in pom.xml when you run the command to build "enclave-release". Just add ENCLAVE_ID=

  0f6fd79cdfdaa5b2e6337f534d3baf999318b0c462a7ac1f41297a3e4b424a57 to the command under that section of pom.xml.
  
  

I hope this helps.

[Roman888]

Thank you so much!

I have few more questions:

1. Libsignal looks like use hardcoded signals domains, because IOS app ignores changed domains in it settings and keep sending request to signals cdsi domain.. so have you also forked libsignal and change domains in it to yours and rebuild it or how you deal with it?

2. What about dockerfiles of cds service? Do you use them without any extra added libs to install or added something? I mean besides sgx libs something like aesm service, az-dcap-client, etc.. ?