sgx-gdb not pausing on exceptions

[kai__chi]

Hi,

I am trying to debug an SGX program. The debugger (sgx-gdb) generally works - I am able to set breakpoints and see the variables. However, the debugger does not break on exceptions. For example, I set a dummy assertion in my code

assert(1 == 0)

and the debugger does not stop on it. It simply quits. I will add that I am running the same program on an old SGXv1 machine and the debugger works like a charm.

specs of the non-working machine (SGXv2):
Intel(R) Xeon(R) Silver 4416+
Ubuntu 22.04.3 LTS 5.15.0-87-generic
SGX SDK 2.24
sgx-gdb Ubuntu 12.1-0ubuntu1~22.04.2

specs of the working machine (SGXv1):
Intel (R) Core(TM) i7-8565U
Ubuntu 20.04.6 LTS 5.4.0-190-generic
SGX-SDK 2.24
sgx-gdb Ubuntu 10.2-0ubuntu1~18.04~2

This is the log from the gdb log from the working machine:

Thread 1 "app" received signal SIGILL, Illegal instruction.
0x00007fff803ab948 in abort ()
(gdb) bt
#0  0x00007fff803ab948 in abort ()
#1  0x00007fff8039231f in __assert ()
#2  0x00007fff8006a965 in ecall_join (res=0x7fff80ea0000, relR=0x7fff80e9c020, 
    relS=0x7fff80e9c030, algorithm_name=0x7fff80ea2000 "J2r", config=0x7fff80ea4000)
    at Enclave/Enclave.cpp:247
#3  0x00007fff80061e0d in sgx_ecall_join (pms=0x7fffffffd060) at Enclave/Enclave_t.c:402
#4  0x00007fff8015107f in do_ecall ()
#5  0x00007fff803ab015 in enter_enclave ()
#6  0x00007fff803ab607 in enclave_entry ()
#7  0x00007ffff7cf8ccc in __morestack () from /usr/lib/x86_64-linux-gnu/libsgx_urts.so.2
#8  0x00007ffff7d01e29 in do_ecall(int, void const*, void const*, CTrustThread*) ()
   from /usr/lib/x86_64-linux-gnu/libsgx_urts.so.2
#9  0x00007ffff7cfc10a in CEnclave::ecall(int, void const*, void*, bool) ()
   from /usr/lib/x86_64-linux-gnu/libsgx_urts.so.2
#10 0x00007ffff7cfe2a2 in _sgx_ecall(unsigned long, int, void const*, void*, bool) ()
   from /usr/lib/x86_64-linux-gnu/libsgx_urts.so.2
#11 0x00005555555596af in ecall_join (eid=2, result=0x7fffffffd140, relR=0x7fffffffd0d0, 
    relS=0x7fffffffd0e0, algorithm=0x7fffffffd1a8 "J2r", config=0x7fffffffd110)
    at App/Enclave_u.c:603
#12 0x000055555555a4fd in main (argc=11, argv=0x7fffffffd898) at App/App.cpp:241
(gdb) list
160	
161		cmpl	$7,%r11d
162		jb	.Lno_extended_info
163		movl	$7,%eax
164		xorl	%ecx,%ecx
165		cpuid
166		btl	$26,%r9d
167		jc	.Lnotknights
168		andl	$0xfff7ffff,%ebx
169	.Lnotknights:

This is the gdb log running the same code on the non-working machine. As you can see gdb does not stop on the failing assert and terminates the program:

(gdb) run
Starting program: ...
detect urts is loaded, initializing
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
add-symbol-file '/enclave.signed.so' 0x7fff80061000 -s .interp 0x7fff800002e0  -s .note.gnu.build-id 0x7fff800002fc  -s .gnu.hash 0x7fff80000320  -s .dynsym 0x7fff80000358  -s .dynstr 0x7fff80000418  -s .gnu.version 0x7fff80000490  -s .gnu.version_d 0x7fff800004a0  -s .rela.dyn 0x7fff800004d8  -s .plt 0x7fff80060000  -s .plt.got 0x7fff80060010  -s .nipx 0x7fff803a4800  -s .rodata 0x7fff803a9000  -s .niprod 0x7fff803f5c80  -s .eh_frame_hdr 0x7fff803f65c0  -s .eh_frame 0x7fff8040a410  -s .gcc_except_table 0x7fff804736c0  -s .tbss 0x7fff80477480  -s .init_array 0x7fff80477480  -s .fini_array 0x7fff804774c0  -s .data.rel.ro 0x7fff80477500  -s .dynamic 0x7fff804c6cd0  -s .got 0x7fff804c6e60  -s .data 0x7fff804c7000  -s .nipd 0x7fff804cb860  -s .bss 0x7fff804cb880 
[  2.6854][ INFO] Enclave id = 2
[New Thread 0x7ffff774a640 (LWP 3288886)]
Error: The enclave is crashed.
Enclave: "./enclave.signed.so"
  [Peak stack used]: 24 KB
  [Peak heap used]:  9216 KB
  [Peak reserved memory used]:  0 KB
remove-symbol-file -a 140735341268992
[Thread 0x7ffff774a640 (LWP 3288886) exited]
[Inferior 1 (process 3288881) exited normally]

 I tried to enforce breakpoints via gdb, use different gdb versons to compile but so far nothing has worked. Any idea how to solve it?

Thanks!

[kai__chi]

Additional check.

I just added an assert in the main app and the debugger correctly stops on the failed assert. This means that the breaks are not made only inside the enclaves.

[kai__chi · Accepted solution]

I received help in a github issue [1]. The problem was that vdso was enabled, which in turn, hindered throwing exceptions. The solution was to disable vdso.

[1] https://github.com/intel/linux-sgx/issues/1042