Intel® ISA Extensions
Use hardware-based isolation and memory encryption to provide more code protection in your solutions.
1093 Discussions

Invalid address/instruction in mapped DLL

ketanm
Beginner
414 Views
Hi,

I am seeing some strange problem in mapped function ptrs. What I would like to know is the difference in following two instruction sets.

Working:
00000000`69555260 4883ec28 sub rsp,28h
00000000`69555264 65488b042548120000 mov rax,qword ptr gs:[1248h]
00000000`6955526d ff9070680000 call qword ptr [rax+6870h]
00000000`69555273 4883c428 add rsp,28h
00000000`69555277 c3 ret
00000000`69555278 cc int 3
00000000`69555279 cc int 3
00000000`6955527a cc int 3

Invalid:
00000000`69555264 65488b042548120000 mov rax,qword ptr gs:[1248h]
00000000`6955526d ff9070680000 call qword ptr [rax+6870h] ds:00000000`00006870=????????????????
00000000`69555273 4883c428 add rsp,28h
00000000`69555277 c3 ret
00000000`69555278 cc int 3
00000000`69555279 cc int 3

What does bold line mean?


Background:

What I am doing is as follows:
1. process starts..
2. Makes call to my.dll
3. My.dll is initialized and created (singleton)
4. MyClass maps OpenGL extensions functions (e.g. glDeletePrograms) by calling wglGetProcAddress, hence has valid function ptrs..
4. Uses OpenGL..and mapped functions
5. Process exists out of main
6. Runs destructor for MyClass
7. Try to call function initialized before , e.g glDeletePrograms and It crashes as has invalid information (shown in above block)

Couple of points:
when function is called, OpenGL and nvoglnt.dll are still loaded.
Address space shown above is where nvoglnt.dll is mapped.

Thanks
Ketan

0 Kudos
3 Replies
jimdempseyatthecove
Honored Contributor III
414 Views

What is the value of rax at each call?

My guess at what is happening is rax (loaded from gs:[1248h]) is supposed to hold the address of the DLL dispatch table. In the working case I would venture to guess that rax was not 0, and in the failing case rax is 0. i.e. the DLL was not loaded at the time of the call.

Jim Dempsey
0 Kudos
ketanm
Beginner
414 Views

What is the value of rax at each call?

My guess at what is happening is rax (loaded from gs:[1248h]) is supposed to hold the address of the DLL dispatch table. In the working case I would venture to guess that rax was not 0, and in the failing case rax is 0. i.e. the DLL was not loaded at the time of the call.

Jim Dempsey

Yes, you are right. rax was indeed zero at that time. This is also what I have concluded. So I need to figure out why dll is unloaded at that time.

thanks
Ketan
0 Kudos
levicki
Valued Contributor I
414 Views
Quoting - ketanm

Yes, you are right. rax was indeed zero at that time. This is also what I have concluded. So I need to figure out why dll is unloaded at that time.

thanks
Ketan

Out of curiosity, do you have DisableThreadLibraryCalls() in your DllMain()?
If you don't perhaps MyClass gets destroyed earlier on DLL_THREAD_DETACH message.
Another more likely possibility is that you have destroyed the OpenGL context. In that case function pointers aren't valid anymore.

0 Kudos
Reply