Relation between TCSNUM and Switchless num_tworkers

[Anandakumar]

Hi Team,

I'm asking this for clarification because I couldn't find any explicit mentions of the relations between the num_tworkers (Trusted worker threads) and the TCSNUM configurations.

TCSNUM is configured during Enclave compilation time, which is the number of threads created during the enclave initialization. Exceeding the parallel request to the Enclave than the TCSNUM will result in Error code: 0x1003 (The enclave is out of TCS). 

num_tworkers is passed to sgx_create_enclave_ex to create the enclave to support the switchless calls. num_tworkers is the number of threads in the trusted worker pool. The number of parallel switchless calls to the Enclave exceeding the num_tworkers  will result in switchless ECALL fallback to regular ECall with the Enclave switch.

So from my understanding, the Trusted Pool contains a portion of Trusted threads that are created/initialized during the Enclave init by using the TCSNUM from the `enclave.config` and the TCSNUM should be the max limit for the num_tworkers.

Case 1: The num_tworkers amount of Trusted threads will be assigned to the Trusted Pool to handle the switchless calls. The remaining Trusted threads will be available for the normal ECall and the fallback Ecall from the failed to serve switchless Ecall request.

Case 2: If the num_tworkers is equal to TCSNUM then there won't be any available Trusted threads to handle normal ECalls and fallback ECalls.

Case 3: Also, if the num_tworkers is greater than the TCSNUM, then enclave initialization should fail.

But when I try case 3, enclave initialization is successful with num_tworkers > TCSNUM and no issues in the runtime.

Somewhere I'm missing the context. So, please explain the relation between TCSNUM and num_tworkers.

Thanks,

Anand