__security_check_cookie called inside exp call

TD8 · ‎09-22-2023

Hi,

I noticed in the disassembly of __libm_exp (from intel numerical routines) that there are a few calls to __security_check_cookie, which seems to mean that it was compiled with security checks on.

Shouldn't such analytical functions be highly optimized and free from unnecessary security features? I mean how could __libm_exp possibly end up corrupting the stack? It's taking one floating point argument.

here's an extrait:

00007FF6FB255FD6 vldmxcsr dword ptr [rsp+28h]
00007FF6FB255FDC mov rcx,qword ptr [rsp+38h]
00007FF6FB255FE1 xor rcx,rsp
00007FF6FB255FE4 call __security_check_cookie (07FF6FB7B1C20h)
00007FF6FB255FE9 vmovapd xmm0,xmm6
00007FF6FB255FED vmovups xmm6,xmmword ptr [rsp+40h]
00007FF6FB255FF3 add rsp,58h
00007FF6FB255FF7 ret

This is a from a release build using Intel Compiler 2023.2.0 on Windows.

VaishnaviV_Intel · ‎09-25-2023

Hi,

Thanks for posting on Intel communities.

Could you please share us the following details so that we can investigate more on your issue,

1. >>I noticed in the disassembly of __libm_exp (from intel numerical routines)

What are the Intel numerical routines you are using?

2. And also, how are you able to check the disassembly of __libm_exp?

3.Could you please share us the sample reproducer and steps to reproduce your issue?

4.Are you using Visual Studio? If yes, Could you please let us know the VS version and its subversion.

Thanks & Regards,

Vankudothu Vaishnavi.

TD8 · ‎09-25-2023

1. I'm using the numerical routines from Intel Compiler 2023.2.0 on Windows.

2. I'm stepping through the "Disassembly" window from Visual Studio's debugger.

3. To "reproduce" the issue, you need to look inside the relevant dll. Install Intel Compiler 2023.2.0, and disassemble the function "__libm_exp", found in "...\Intel\oneAPI\compiler\2023.2.0\windows\redist\intel64_win\compiler\libmmd.dll". You can use any disassembler / debugger. The function __libm_exp begins at offset 0x53C0, but it then dispatches to other functions depending on the FPU capabilities, I guess. In my case it dispatches to "__libm_exp_l9", at offset 0xcbca0.

here's a part of the function (I'm just copy-pasting from Visual Studio disassembly window):
00007FF9D0E8BCA0 endbr64
00007FF9D0E8BCA4 sub rsp,58h
00007FF9D0E8BCA8 vmovups xmmword ptr [rsp+40h],xmm6
00007FF9D0E8BCAE mov rax,qword ptr [__security_cookie (07FF9D112BD58h)]
00007FF9D0E8BCB5 xor rax,rsp
00007FF9D0E8BCB8 mov qword ptr [rsp+38h],rax
00007FF9D0E8BCBD vmovq rax,xmm0
00007FF9D0E8BCC2 shr rax,30h
00007FF9D0E8BCC6 and eax,7FFFh
00007FF9D0E8BCCB lea ecx,[rax-4084h]
00007FF9D0E8BCD1 cmp ecx,0FFFFFBFCh
00007FF9D0E8BCD7 jb __libm_exp_l9+128h (07FF9D0E8BDC8h)
00007FF9D0E8BCDD vstmxcsr dword ptr [rsp+34h]
00007FF9D0E8BCE3 mov ecx,dword ptr [rsp+34h]
00007FF9D0E8BCE7 mov eax,ecx
00007FF9D0E8BCE9 and eax,6000h
00007FF9D0E8BCEE je __libm_exp_l9+60h (07FF9D0E8BD00h)
00007FF9D0E8BCF0 and ecx,0FFFF9FFFh
00007FF9D0E8BCF6 mov dword ptr [rsp+30h],ecx
00007FF9D0E8BCFA vldmxcsr dword ptr [rsp+30h]
00007FF9D0E8BD00 vmovsd xmm1,qword ptr [__real@40571547652b82fe (07FF9D0F4DC70h)]
00007FF9D0E8BD08 vfmadd213sd xmm1,xmm0,mmword ptr [__real@433800000000ffc0 (07FF9D108CA60h)]
00007FF9D0E8BD11 vmovq rcx,xmm1
00007FF9D0E8BD16 lea edx,[rcx+rcx]
00007FF9D0E8BD19 and edx,7Eh
00007FF9D0E8BD1C and ecx,3FFC0h
00007FF9D0E8BD22 lea r8d,[rdx+1]
00007FF9D0E8BD26 vaddsd xmm1,xmm1,mmword ptr [__real@c33800000000ffc0 (07FF9D108CA68h)]
00007FF9D0E8BD2E vfmadd231sd xmm0,xmm1,mmword ptr [__real@bf862e42fefa0000 (07FF9D0F7A9E8h)]
00007FF9D0E8BD37 vfmadd231sd xmm0,xmm1,mmword ptr [__real@bd1cf79abc9e3b3a (07FF9D0F7A9F0h)]
00007FF9D0E8BD40 vmovd xmm1,ecx
00007FF9D0E8BD44 vpsllq xmm1,xmm1,2Eh
00007FF9D0E8BD49 vmovsd xmm2,qword ptr [__real@3f8111126eeb729a (07FF9D108CA70h)]
00007FF9D0E8BD51 vfmadd213sd xmm2,xmm0,mmword ptr [__real@3fa555570aa70f87 (07FF9D108CA78h)]
00007FF9D0E8BD5A vmovsd xmm6,qword ptr [__real@3fc55555555541d3 (07FF9D108CA80h)]
00007FF9D0E8BD62 vfmadd213sd xmm6,xmm0,mmword ptr [__real@3fdfffffffffe5bc (07FF9D108CA88h)]
00007FF9D0E8BD6B vmulsd xmm3,xmm0,xmm0
00007FF9D0E8BD6F lea rcx,[__real@3fdfffffffffe5bc+8h (07FF9D108CA90h)]
00007FF9D0E8BD76 vmovq xmm4,mmword ptr [rcx+rdx*8]
00007FF9D0E8BD7B vpor xmm1,xmm1,xmm4
00007FF9D0E8BD7F vaddsd xmm0,xmm0,mmword ptr [rcx+r8*8]
00007FF9D0E8BD85 vfmadd231sd xmm6,xmm3,xmm2
00007FF9D0E8BD8A vfmadd213sd xmm6,xmm3,xmm0
00007FF9D0E8BD8F vfmadd132sd xmm6,xmm1,xmm1
00007FF9D0E8BD94 test eax,eax
00007FF9D0E8BD96 je __libm_exp_l9+10Ch (07FF9D0E8BDACh)
00007FF9D0E8BD98 vstmxcsr dword ptr [rsp+2Ch]
00007FF9D0E8BD9E or eax,dword ptr [rsp+2Ch]
00007FF9D0E8BDA2 mov dword ptr [rsp+28h],eax
00007FF9D0E8BDA6 vldmxcsr dword ptr [rsp+28h]
00007FF9D0E8BDAC mov rcx,qword ptr [rsp+38h]
00007FF9D0E8BDB1 xor rcx,rsp
00007FF9D0E8BDB4 call __security_check_cookie (07FF9D0F2E370h)
00007FF9D0E8BDB9 vmovapd xmm0,xmm6
00007FF9D0E8BDBD vmovups xmm6,xmmword ptr [rsp+40h]
00007FF9D0E8BDC3 add rsp,58h
00007FF9D0E8BDC7 ret

Note that the addresses here are in-memory addresses, not offsets in the dll. Visual studio says the dll was loaded to the base address 0x00007FF9D0E8BCA0, so you can subtract 0x00007FF9D0E8BCA0 to get the offset in the dll.

Anyway, you can see the setup of the security cookie, and the call to __security_check_cookie().

4. Yes I'm using Microsoft Visual Studio Community 2022 (64-bit) - Version 17.6.2

ShanmukhS_Intel · ‎10-03-2023

Hi,

Thanks for sharing the details.

We are able to reproduce the issue at our end. We will get back to you soon with an update.

Best Regards,

Shanmukh.SS

ShanmukhS_Intel · ‎10-09-2023

Hi,

We have tried a sample code and could see the __security_check_cookie__. However, could you please let us know the numerical routines being used by you? You could share with us a sample reproducer with the numerical routines being used by you, as it helps us triage the issue better.

Best regards,

Shanmukh.SS

TD8 · ‎10-10-2023

As I said, it is just calling into `exp`.

So, that should do:

#include <iostream>

#include <cmath>

int main(int argc, char** argv) {

std::cout << std::exp(1.23456789 * argc);

}

Compile with Visual Studio Community 2022 (64-bit) - Version 17.6.2, (but VS version is somewhat irrelevant), selecting 'Intel Compiler 2023.2.0' as the compiler.

VaishnaviV_Intel · ‎10-12-2023

Hi,

Thanks for providing the sample code.

You can disable a security check with the following steps:

1. Open the property pages

2. Go to Configuration Properties -> C/C++ -> Code Generation

3. Set "Security Check" to "Disable Security Check (/GS-)"

If you still have any issues, please let us know.

Thanks & Regards,

Vankudothu Vaishnavi.

TD8 · ‎10-13-2023

I think there is a misunderstanding.

Changing the security options on a project that links to the intel libraries won't change anything, because the security checks I am mentioning are inside Intel's distributed DLL code, namely libmmd.dll. Changing the options of my VS project and rebuilding it won't touch libmmd.dll. The security checks will still be done by the exp function inside libmmd.dll.

The VS project I'm using already has security features disabled, by the way.

The issue has to come from the way the dlls were built by Intel. It's probably due to security features being enabled when libmmd.dll was compiled by Intel.

VaishnaviV_Intel · ‎10-18-2023

Hi,

We have tried executing your code by disabling the security_check cookie in VS properties. But, we couldn't find the security_check call in the assembly code.

So, could you please share us your VS project by disabling the security_check cookie so that we can investigate more on your issue?

>>Changing the options of my VS project and rebuilding it won't touch libmmd.dll. The security checks will still be done by the exp function inside libmmd.dll.

As you are observing security_checks from libmmd.dll. So, Do you want to disable _security_check_cookie in libmmd.dll?

Thanks & Regards,

Vankudothu Vaishnavi.

TD8 · ‎10-18-2023

You couldn't find the security_call in the main function? That's expected. As I said, the _security_check_cookie call is in libmmd.dll, not in the code which calls into exp, so it won't appear in the code of the main function.

But you're almost there, you need to step in the 'exp' call. Just hit F11 until you are inside a function named "__libm_exp_l9" (assuming the machine has a cpu with the right SIMD features, otherwise "exp" might end up calling into another specialized exp function). The _security_check_cookie call I came across happens in "__libm_exp_l9".

What I want, or rather what any person would expect from performance-critical math routines, is that an "exp" function is free from irrelevant work, such as checking that the stack is not corrupted. So yes, I'd like to disable _security_check_cookie in libmmd.dll. But that's not something I can do, regardless of my compiler flags, because libmmd.dll is already a compiled binary, it already has gone through compilation, Intel is distributing these routines as compiled binaries, no compiler flags on my end can change the already compiled code inside libmmd.dll. I would expect Intel engineers to know that.

VaishnaviV_Intel · ‎10-24-2023

Hi,

It was the 2014 decision to add security features to Compiler libraries.

The reason why the Compiler generates those calls to checks is that __libm_exp_l9 is not an isolated function.

It calls another kernel __libm_exp_e7 from itself while this one calls __libm_error_support with callback 'matherr' function overloadable by users which is definitely a potential vulnerability (https://man7.org/linux/man-pages/man3/matherr.3.html).

Could you please let us know why you want to disable the security feature? Does this severely impact the performance of your workloads, can you provide us with a replicator, and how much performance will drop?

Thanks & Regards,

Vankudothu Vaishnavi.

TD8 · ‎10-25-2023

As I said, I would expect highly optimized math routines to be free from unnecessary security features. There is no valid reason an exp function must perform stack-corruption checks.

I see there are some "features" that "justify" the security checks. But here it seems *all* users pay for a very niche feature (the matherr callback function).

It probably is a rather small overhead, but still, again, highly optimized math routines should have ZERO overhead, period. It's disappointing to see Intel get it wrong.

VaishnaviV_Intel · ‎10-30-2023

Hi,

Thank you for your feedback. We have provided your feedback to the relevant team. At this moment there is no visibility when it will be implemented and available for use. Please let me know if we can go ahead and close this thread?

Thanks & Regards,

Vankudothu Vaishnavi.

TD8 · ‎03-27-2024

Hi,

It's been a little while, but today I was profiling some HPC code and I stumbled upon significant work done in __security_check_cookie...

Guess where this came from?

Yep, from our beloved and highly optimized __libm_pow_l9 from intel math routines.

Anyway, it would be great if intel developers could do something about it.

Having pow() spend 4% of its time doing irrelevant work is a bit disappointing as I imagine even a 1% optimization of pow() is considered significant...