LEA and MOV behaving differently in Windows and Linux

[yax99]

Hi,

I am converting a big Windows dll to work on both Windows and Linux. The dll has a lot of assembly (and SS2 instructions) for video manipulation.

The code now compiles fine on both Windows and Linux using Intel compiler included in Intel ComposerXE-2011 on Windows and Intel ComposerXE-2013 SP1 on Linux.

The execution, however, crashes in Linux when trying to call a function pointer. I traced the code in gdb and indeed the function pointer doesn't point to the required function (whereas in Windows in does). Almost everything else works fine.

This is the sequence of code:

...
mov    rdi, this
lea    rdx, [rdi].m_sSomeStruct
...
lea    rax, FUNCTION_NAME                # if replaced by 'mov', works in Linux but crashes in Windows
mov    [rdx].m_pfnFunction, rax
...
call   [rdx].m_pfnFunction               # crash in Linux

where:

1) 'this' has a struct member m_sSomeStruct.

2) m_sSomeStruct has a member m_pfnFunction, which is a pointer to a function.

3) FUNCTION_NAME is a free function in the same compilation unit.

4) All those pure assembly functions are declared as naked.

5) 64-bit environment.

What is confusing me the most is that if I replace the 'lea' instruction that is supposed to load the function's address into rax with a 'mov' instruction, it works fine on Linux but crashes on Windows. I traced the code in both Visual Studio and gdb and apparently in Windows 'lea' gives the correct function address, whereas in Linux 'mov' does.

I tried looking into the Intel assembly reference but didn't find much to help me there (unless I wasn't looking in the right place).

___________________________________________________________________________

Few remarks:

1) I tried using square brackets

lea    rax, [FUNCTION_NAME]

but that didn't change the behaviour in Windows nor in Linux.

2) I looked at the disassembler in gdb and Windows, seem to both give the same instructions that I actually wrote. What's even worse is that I tried putting both lea/mov one after the other, and when I look at them in disassembly in gdb, the address printed after the instruction after a # sign (which I'm assuming is the address that's going to be stored in the register) is actually the same, and is NOT the correct address of the function.

It looked like this in gdb disassembler

lea  0xOffset1(%rip), %rax   # 0xSomeAddress
mov  0xOffset2(%rip), %rax   # 0xSomeAddress

where both (SomeAddress) were identical (and incorrect) and both offsets were off by the same amount of difference between lea and mov instructions, But somehow, the when I check the contents of the registers after each execution, mov seem to put in the correct value!!

3) The member variable m_pfnFunction is of type LOAD_FUNCTION which is defined as

typedef void (*LOAD_FUNCTION)(const void*, void*);

4) The function FUNCTION_NAME is declared in the .h (within a namespace) as

void FUNCTION_NAME(const void* , void*);

and implemented in .cpp as

__declspec(naked) void namespace_name::FUNCTION_NAME(const void* , void*)
{
...
}

5) I tried turning off optimizations by adding

#pragma optimize("", off)

but I still have the same issue.

Thanks in advance.

[Bernard]

Can you try to add offset directive to the Windows code? I am not sure if this will help to fix the crash.

lea    rax, offset FUNCTION_NAME 

[jimdempseyatthecove]

On Windows, you typically do not generate PIC, rather you let the linker or DLL loader fix-up the offsets. On Linux you can either do the same, or use PIC. I am not experienced at this but, if you are using PIC I think you will be required to use a more convoluted addressing expression. Something like (but not necessarily what is listed below)

lea (_FUNCTION_NAME - $)(%rip), %rax

The above should be equivalent to OFFSET, which would be preferred.

Jim Dempsey

[yax99]

iliyapolak wrote:

Can you try to add offset directive to the Windows code? I am not sure if this will help to fix the crash.

lea    rax, offset FUNCTION_NAME 

Thanks. I tried that but it doesn't even compile. I receive an error: "Unsupported instruction form in asm instruction lea.".

[yax99]

Thanks Jim for your answer. We do in fact use PIC in Linux.

Unfortunately, I tried the following

mov   rax, offset FUNCTION_NAME

but that doesn't compile either: "internal error: 04010003_1159". I tried looking for that error but didn't find anything.

[Bernard]

yax99 wrote:

Quote:

iliyapolak wrote:

Can you try to add offset directive to the Windows code? I am not sure if this will help to fix the crash.

lea    rax, offset FUNCTION_NAME 

Thanks. I tried that but it doesn't even compile. I receive an error: "Unsupported instruction form in asm instruction lea.".

I probably made a mistake.I think that offset directive should be used with mov instruction.I suppose that ml64.exe can compile this code.

mov rax ,offset FUNCTION_NAME

eventually you can try this:

lea rax , [FUNCTION_NAME]

[Bernard]

yax99 wrote:

Thanks Jim for your answer. We do in fact use PIC in Linux.

Unfortunately, I tried the following

mov   rax, offset FUNCTION_NAME

but that doesn't compile either: "internal error: 04010003_1159". I tried looking for that error but didn't find anything.

I do not understand why this code is causing internal compiler error.I suppose that this could be some kind of compiler bug.

[yax99]

iliyapolak wrote:

Quote:

yax99 wrote:

Quote:

iliyapolak wrote:

Can you try to add offset directive to the Windows code? I am not sure if this will help to fix the crash.

lea    rax, offset FUNCTION_NAME 

Thanks. I tried that but it doesn't even compile. I receive an error: "Unsupported instruction form in asm instruction lea.".

I probably made a mistake.I think that offset directive should be used with mov instruction.I suppose that ml64.exe can compile this code.

mov rax ,offset FUNCTION_NAME

eventually you can try this:

lea rax , [FUNCTION_NAME]

I've already tried both actually. With mov and offset I get a compilation error "internal error: 04010003_1159", and with lea and square brackets it compiles but still crashes.

[yax99]

iliyapolak wrote:

Quote:

yax99 wrote:

Thanks Jim for your answer. We do in fact use PIC in Linux.

Unfortunately, I tried the following

mov   rax, offset FUNCTION_NAME

but that doesn't compile either: "internal error: 04010003_1159". I tried looking for that error but didn't find anything.

I do not understand why this code is causing internal compiler error.I suppose that this could be some kind of compiler bug.

In fact, this doesn't work in Windows either. It compiles, but fails to link with error:
error LNK2017: 'ADDR32' relocation to '?FUNCTION_NAME@namespace@@YAXPEBXPEAX@Z' invalid without /LARGEADDRESSAWARE:NO.

I also tried mov   rax, qword ptr offset FUNCTION_NAME, same issue.
 

[Melanie_B_Intel]

Try writing a little function in C++ that does what you want your assembly code to do, then compile it with the Linux compiler, using the -S switch to generate assembly, and check the instruction sequence.

[jimdempseyatthecove]

Follow Melanie's suggestion with C++ code that gets the address of FUNCTION_NAME. This will give you the decorated name (assuming you are not using extern "C" ...).

Jim Dempsey

[Bernard]

Another suggestion could writing simple code in 64-bit assembly which calls indirectly function and compile it with ml64.exe  assembler. This can be done in order to verify that specific instructions like mov rax , lea rax are compiled.I would also advise to check with VS debugger the address which is being hold inside RAX when LEA RAX,[FUNCTION_NAME] executes. It seems that you only investigated the issue with gdb.

[Iacovache__Andrei]

For anyone else experiencing this problem: it has to do with large address aware. The way I fixed it was simply disabling Large Adress aware from properties > linker > system. This feature makes sure that the program can handle adresses bigger than 4GB. Since a normal 64 bit register can not hold a value large enough to point to an adress outside of 4GB, it is impossible to use lea and offsets normally while this feature is enabled. The compiler gives an error for a good reason: if your program does end up using more than 4GB of ram, it will not correctly use it since the pointers you use can't actually work with so much memory, and it will crash, unexpectedly. So if you want to have more than 4GB ram you have to use something else, if not just disable large address aware. I know I'm late, but I'm leaving this here for anyone who might have the same problem again.

[Iacovache__Andrei]

I don't know if my first comment posted so: basically LEA and offsets only work within 64bit values. 64 bit values can't be pointers for memories greater than 4GB. So when LARGEADDRESSAWARE is enabled, it prevents you from using them, because they are simply meant to crash as soon as you make use of more memory than 4GB. Of course you can disable LARGEADDRESSAWARE from Properties>Linker>System. Doing this you admit that your program is not designed to handle more than 4GB of RAM, therefore LEA and offset work correctly, and can be used. A workaround is possible but also likely to crash the program later on. I know the problem is old and probably solced already, I just wanted to make this clear for people looking for this, since I found it hard to solve this.