Shared memory in a DLL in IVF

[schapman]

Hello,
We are in the process of converting a very lage CVF application to IVF. The CVF application uses a shared memory DLL containing module data, with the main program using the memory allocated in the module.

In CVF, we could directly access the memory in the DLL in which it was declared, and also in the main, but not in other DLLs (they allocated new memory spaces of their own when the modules were included in them, so data was not shared). Has the syntax in IVF changed so that module data from one DLL can be directly accessed in both the main and in other DLLs? If so, what is the new syntax?

Thanks
Steve

[Steven_L_Intel1]

There's been no change in this. The way to handle this is to have the other DLLs that reference the share data to link to the one that defines the shared data.

[schapman]

Steve,
How do you make that link happen? At the current time, our shared memory DLL and our main program both allcoate the memory spaces, but only the one in the DLL gets used. This makes the total program data 200 MB, but onlyhalf of that gets used. Each time we add the structures to another DLL, the total size goes up by another 100 MB, but this time the memory is not in common. We must be doing something wrong in the project settings.

[Steven_L_Intel1]

There's no project settings I know of. You need to specify DLLIMPORT directives for the variables or COMMON and then link against the export library of the DLL that exports the variable. If the variable is defined in a module with a DLLEXPORT, that turns into a DLLIMPORT when the module is USEd.

[schapman]

"If the variable is defined in a module with a DLLEXPORT, that turns into a DLLIMPORT when the module is USEd."

This is the situation that we are in. It works as advertised when we use a data module with DLLEXPORT in ONE DLL and in the main.

However, if we use that same module in multiple DLLs, each DLL produces its OWN data module, and the main links to whichever one is first in the load order. For example, we have created a data module with DLLFXPORT and included it in

main.for
DLL1.for
DDL2.for
DLL3.for

If you examine the size of the executable and DLL files resulting, each of them has the data allocated in its image. This is easy to see if you make the size of the data be 100 MB, where the bloat shows up in the file sizes.

If you then execute the program, the main will share the data space with the data from whichever DLL is loaded first (we can change the order by changing project names, and we see the shared memory move around with the changing load order). The other DLLs wind up with their own copies of the memory which are NOT shared. It appears that the main program's copy is allocated but never used, while the ones in the extra DLLs are used but are not shared. We have not figured out any way to get around this problem.

Steve

[Steven_L_Intel1]

I'll have to try some examples and see what I find. Do note that the main and all the DLLs need to link against the DLL that exports the variable.

[schapman]

Hello Steve,



I have attached a CVF project that illustrates the problem. This project contains a main and 3 DLLs, each one of which includes a data module declared using DLL export. As you can see from file sizes, the data was allocated in EACH image:



D: est_shared_modulesdebug>dir


Volume in drive D is DATA


Volume Serial Number is 6462-A133


Directory of D: est_shared_modulesdebug


21/12/2005 09:21 PM.


21/12/2005 09:21 PM..


21/12/2005 09:21 PM 40,022,075 dll1.dll


21/12/2005 09:21 PM 40,022,075 dll2.dll


21/12/2005 09:21 PM 40,022,075 dll3.dll


21/12/2005 09:21 PM 40,353,851 main.exe


4 File(s) 160,420,076 bytes


2 Dir(s) 5,588,439,040 bytes free




The main program looks as follows:


PROGRAM main


USE data_module


USE interfaces


IMPLICIT NONE


! Print the value of x


WRITE(*,*) 'At start in main, x = ', test_data%x


! Change x


test_data%x = 2.


! Print the value of x


WRITE(*,*) 'In main after changing value, x = ', test_data%x


! Call DLL 1


CALL dll1_sub()


! Call DLL 2


CALL dll2_sub()


! Call DLL 3


CALL dll3_sub()


! Print the value of x


WRITE(*,*) 'In main, x = ', test_data%x


END PROGRAM main

<


When this program is executed, the results are:




D: est_shared_modulesdebug>main


At start in main, x = 1.000000


In main after changing value, x = 2.000000


In DLL1, x = 2.000000


In DLL1, x = 1.000000


In DLL1, x = 1.000000


In main, x = 2.000000




When the value of x was changed in the main program, it also changed in DLL1, since they are sharing a common memory space. However, it did NOT change in DLL2 or DLL3--those two are using their own copies of memory. The behaviour that we get for module data is a follows:




1. All main programs and DLLs allocate the data structure, even if (in the case of the main program) it is not used.

2. The main and the FIRST loaded DLL share a memory space; all other DLLs have their own independent copies. Experiments have proved that each DLL's copy is independent of all other DLLs that allocate the memory. Also, if we change the order of loading, the DLL whose memory is shared will change.


This is the behaviour that has been giving us fits, and that we hope IVF will fix. We need all main and DLLs to share a COMMON memory, not just some of them. It would also be good if the extra unused data structures were not allocated. We can live with it if they are, but the very large size of our structures makes it preferable if they weren't.




Thanks

Steve

Message Edited by schapman@ieee.org on 12-21-2005 02:43 AM

Message Edited by schapman@ieee.org on 12-21-2005 02:47 AM

[Steven_L_Intel1]

Steve,

Thanks for attaching the projects. What you're actually doing doesn't match your description. Rather than reference the module from one of the DLLs, you are repeating the module declaration in each of the projects. That will cause the memory to be duplicated.

I modified things as follows:

1. Removed data_module.f90 from each of the projects except dll1.
2. Added ..dll1debug to the "Additional include directories" property of each of the projects except dll1. This allows the USE DATA_MODULE to be fulfilled.
3. Used the project dependencies menu to make dll1 a dependent of all of the other projects. This causes them to link against the DLL that actually defines the data.

When I do that, only dll1 has the large data allocated and all projects see the same copy of the variable.

[schapman]

Thanks, Steve. We'll try that in our large project today.

[schapman]

Steve,

We tried the changes that you specified in our big application, and they worked. The duplicate memory allocation went way down. However, the program slowed down by 60% when it executed!

We found that if we leave the duplicate memory allocation in the main program and one DLL, then the program sped back up again. It looks as though the compiler is generating a faster type of access thinking that the memory is local, and then gets linked to the DLL at runtime. When the memory is not compiled into the main program, the compiler seems to be generating extra instructions with every access, adn slowing everything down. Does this make sense to you?

Steve

Message Edited by schapman@ieee.org on 12-21-2005 04:27 PM

[Steven_L_Intel1]

Yes. Access to DLLIMPORTed variables requires an indirect memory access, and this can slow down code. I would not have expected 60%, but I can imagine how it might happen since this indirection probably disables other optimizations.

You might consider setting up a pointer in the local code, assigning the pointer to the data once, and then referencing through the pointer.