DLL Search Algorithm Flawed

[Michael_D_]

We have upgraded from IPP 5.1 to 6.02. We also want to be compatible with Windows Vista or Windows 7. This means system32 is "off limits" as an install location for the IPP DLLs. We also don't like the PATH environment solution, since anyone can change the PATH and break the DLL search algorithm. Why doesn't the "waterfall" procedure look in the current directory? This would solve our run-time problem of missing DLLs. Please advise.

[elhefe38]

Quoting - mdl61

We have upgraded from IPP 5.1 to 6.02. We also want to be compatible with Windows Vista or Windows 7. This means system32 is "off limits" as an install location for the IPP DLLs. We also don't like the PATH environment solution, since anyone can change the PATH and break the DLL search algorithm. Why doesn't the "waterfall" procedure look in the current directory? This would solve our run-time problem of missing DLLs. Please advise.

Well I guess you just have to resort to static linking then...

Regards

[Vladimir_Dudnik]

Hello,

according our expert:

"Waterfall" looks to the current directory, if no success it try to find according to Windows* dll search rules.

Regards,
Vladimir

[Michael_D_]

Thank you Vladimir. I need to clarify our requirements since my original post was slightly incorrect. We can not depend on the "current directory" since our end-product DLLs need to be registered without the benefit of the current directory being set.

You had mentioned that the IPP waterfall procedure follows the "Windows DLL search rules," but we believe this is not the case. For example, if I use regsvr32.exe or LoadLibrary() to register a DLL given a full path, all dependent DLLs sharing that same path (i.e. co-located) will be found even though the current directory or PATH environment is not set:

If dllA requires to load dllB and they both reside in c:mypath, then:

regsvr32 c:mypathdllA.dll
LoadLibrary(c:mypathdllA.dll);

will work properly given the "Windows DLL search rules" even if PATH is not set and the current directory of the calling process is not c:mypath.

We have found that LoadLibrary() on the first IPP DLL works because the full path is specified; but when the waterfall procedure tries to locate the other dependent IPP DLLs, the original full path specified is not used and DLL location fails.

We believe that the IPP waterfall search needs to emulate the Windows OS behavior exactly in order to resolve this issue.

Please let me know if I have misunderstood or if there is any more information that you need.

Regards,
mdl61

[pvonkaenel]

Quoting - mdl61

Thank you Vladimir. I need to clarify our requirements since my original post was slightly incorrect. We can not depend on the "current directory" since our end-product DLLs need to be registered without the benefit of the current directory being set.

You had mentioned that the IPP waterfall procedure follows the "Windows DLL search rules," but we believe this is not the case. For example, if I use regsvr32.exe or LoadLibrary() to register a DLL given a full path, all dependent DLLs sharing that same path (i.e. co-located) will be found even though the current directory or PATH environment is not set:

If dllA requires to load dllB and they both reside in c:mypath, then:

regsvr32 c:mypathdllA.dll
LoadLibrary(c:mypathdllA.dll);

will work properly given the "Windows DLL search rules" even if PATH is not set and the current directory of the calling process is not c:mypath.

We have found that LoadLibrary() on the first IPP DLL works because the full path is specified; but when the waterfall procedure tries to locate the other dependent IPP DLLs, the original full path specified is not used and DLL location fails.

We believe that the IPP waterfall search needs to emulate the Windows OS behavior exactly in order to resolve this issue.

Please let me know if I have misunderstood or if there is any more information that you need.

Regards,
mdl61

I do not think the search order you are listing is correct: while DLLs will be loaded from the directed the application was started in, it will not necessarily search in the directory that DLLs were loaded from. When you list the regsrv32 example, I'm not sure if that is considered the application directory or not, but the basic search order can be found in the msdn at

http://msdn.microsoft.com/en-us/library/ms682586.aspx

One thing I have done in the path is store the startup directory when the application starts using GetCurrentDirectory(). Then change directory using SetCurrentDirectory() before calls to LoadLibrary() so that dependent DLLs in the same directory will load. The only problem I can see with this is if your startup sequence is threaded and each thread is trying load its own set of DLLs. If you can keep the startup single threaded, you should be able to avoid this problem.

Hope this helps,

Peter

[matthieu_darbois]

Hi, we're having the same problem with activeX dlls linked against ipp dlls (5.x, 6).
As the executable that hosts the activeX is not in the ActiveX directory and, the ActiveX directory is not the default working directory, ipp fails to load optimized modules and shows the "Waterfall..." error message.
IPP should only try to load optimized module side-by-side with the dispatch dll (which is loaded correctly)
Modify the path is not an option for us (as well as putting ipp dll in System32).
We'd like not to link statically because :
- there's not just one module using Ipp (that would be n times ipp size instead of only 1)
- this will seriously increase the size for updates

Has this issue been taken into account or do I need to make a request using Premier services ?

Thanks,

Matthieu

[noemata]

Quoting - matthieu.darbois

Hi, we're having the same problem with activeX dlls linked against ipp dlls (5.x, 6).
As the executable that hosts the activeX is not in the ActiveX directory and, the ActiveX directory is not the default working directory, ipp fails to load optimized modules and shows the "Waterfall..." error message.
IPP should only try to load optimized module side-by-side with the dispatch dll (which is loaded correctly)
Modify the path is not an option for us (as well as putting ipp dll in System32).
We'd like not to link statically because :
- there's not just one module using Ipp (that would be n times ipp size instead of only 1)
- this will seriously increase the size for updates

Has this issue been taken into account or do I need to make a request using Premier services ?

Thanks,

Matthieu

The only way to be absolutely certain that you are getting DLLs loaded in the manner needed by your application is to have a private copy of those DLLs alongside your application or in a common folder in the case where a third party vendor does not provide a merge module ora universalinstaller.

Even Microsoft has had problems addressing this issue, a good example being the MFC DLLs. Adobe has done a good job with Flash, but perhaps has taken things too far by requiring that only their installers be used todeliver the needed components onto a users system.

We have implemented our own installer for IPP. With a private copy of needed DLLs the OS is smart enough to load only a single instance of DLLs of the same version even if several copies of those DLLs exist in different directories. The key advantage of using the private copy approach is that it will allow your application to run correctly after a user installs updated DLLs that might otherwise break compatibility provided they install the update somehwere other than where your private copy is located.

The bottom line ... the safest way to install 3rd party DLLs is in a common folder within your application's "Program Files" install directory. Any other approach requires coordination with 3rd parties and may eventually break your app.

Hard disk space is cheap, it's safe to abuse it.

[Vladimir_Dudnik]

Please note that you can build IPP custom DLL which may only contain functions used in your application. As a result it will have much less size and will not be updated with newer versions of IPP you or your customer may have installed in the system.

Regards,
Vladimir

[matthieu_darbois]

Quoting - Vladimir Dudnik (Intel)

Please note that you can build IPP custom DLL which may only contain functions used in your application. As a result it will have much less size and will not be updated with newer versions of IPP you or your customer may have installed in the system.

Regards,
Vladimir

This is not a project I usually work on. I told the project manager the different options (static, custom dll) offered to us but they don't want to change anything.
I'm sorry to say this but I won't work around flaws in IPP.
The dispatch DLL should at least (if not only) search for optimized modules in the directory it resides.

Regards,
Matthieu

[Vladimir_Dudnik]

Yes, it should and it does (at least have to) when IPP DLL is loaded from application. there is a difference when IPP DLL is loaded from other DLL because application 'current folder' is unknown.

Note, youmay fill your issue report (or feature request) through Intel Premier Support

Regards,
Vladimir

[matthieu_darbois]

Quoting - Vladimir Dudnik (Intel)

Yes, it should and it does (at least have to) when IPP DLL is loaded from application. there is a difference when IPP DLL is loaded from other DLL because application 'current folder' is unknown.

Note, youmay fill your issue report (or feature request) through Intel Premier Support

Regards,
Vladimir

Thanks for your reply,
I submitted an issue (number 545821) through Intel Premier Support

Regards,
Matthieu

[elhefe38]

Quoting - matthieu.darbois

Thanks for your reply,
I submitted an issue (number 545821) through Intel Premier Support

Regards,
Matthieu

IMHO, you are facing a Windows behavior more likely than an IPP one. You seem to be facing this dll search path method, which does look inside the current directory of the process trying to load the dll.
Which may roughly translates to "you're screwed" (at least from what I understand about your situation)...

Anyway, let us know if the support request gives any workaround for this, I might be interested :)

regards

[matthieu_darbois]

Quoting - elhefe38

Quoting - matthieu.darbois

Thanks for your reply,
I submitted an issue (number 545821) through Intel Premier Support

Regards,
Matthieu

IMHO, you are facing a Windows behavior more likely than an IPP one. You seem to be facing this dll search path method, which does look inside the current directory of the process trying to load the dll.
Which may roughly translates to "you're screwed" (at least from what I understand about your situation)...

Anyway, let us know if the support request gives any workaround for this, I might be interested :)

regards

Hi,
I know that we're facing a Windows behaviour. What I'm saying is that IPP, which developers know this behaviour (or so I hope), should adapt to this behaviour : Retrieve the full path to the DLL being loaded, strip out the dll name, and use this path to try to load optimized modules. It is possible to do so, thus, it should do it.

Regards,
Matthieu

ps : not so sure how to retrieve path for dll being loaded, but as ippcore-x.x is loaded before that, it's sure possible to retrieve the right path (check GetModuleFileName in MSDN)

[elhefe38]

Quoting - matthieu.darbois

I know that we're facing a Windows behaviour. What I'm saying is that IPP, which developers know this behaviour (or so I hope), should adapt to this behaviour : Retrieve the full path to the DLL being loaded, strip out the dll name, and use this path to try to load optimized modules. It is possible to do so, thus, it should do it.

Regards,
Matthieu

ps : not so sure how to retrieve path for dll being loaded, but as ippcore-x.x is loaded before that, it's sure possible to retrieve the right path (check GetModuleFileName in MSDN)

Sounds like a possible workaround yes (provided you manage to actually load the first ipp dll from the correct location, as GetModuleFileName is only for "modules loaded by the current process"). I'd hate to have to rely on this though :)

a+

[pvonkaenel]

Quoting - elhefe38

IMHO, you are facing a Windows behavior more likely than an IPP one. You seem to be facing this dll search path method, which does look inside the current directory of the process trying to load the dll.
Which may roughly translates to "you're screwed" (at least from what I understand about your situation)...

Anyway, let us know if the support request gives any workaround for this, I might be interested :)

regards

I agree that this is a Windows issue, and additionally an application design issue - you might have had additional 3rd party DLLs that depend on yet more DLLs. I think it is a bit unfair to identify it as an "IPP flaw".

Anyhow, if you are able to identify your DLL directory, and cannot add it to the path or change to it using SetCurrentDirectory(), then you might also want to consider using SetDllDirectory(). This will add directories to the DLL search path.

Peter

[Michael_D_]

Quoting - matthieu.darbois

Thanks for your reply,
I submitted an issue (number 545821) through Intel Premier Support

Regards,
Matthieu

Matthieu,
I was about to log the same through the Intel Premier Support site. I'm having trouble logging in right now, but I will check for the status on 545821 to see if any resolution is forthcoming. I appreciate your sticking with this; it is an important consideration which doesn't seem to be well understood or appreciated by most.
Regards,
Mike

[elhefe38]

Quoting - mdl61

Matthieu,
I was about to log the same through the Intel Premier Support site. I'm having trouble logging in right now, but I will check for the status on 545821 to see if any resolution is forthcoming. I appreciate your sticking with this; it is an important consideration which doesn't seem to be well understood or appreciated by most.
Regards,
Mike

Reading again your previous email, the behavior you describe is perfectly consistent with Windows' way of loading dlls... What exactly have we not understood ? I tend to agree with Peter that blaming this on IPP is a bit unfair as it looks more like a design issue, I however understand that you may need a workaround in IPP to solve this mess.

regards

[pvonkaenel]

Quoting - mdl61

Matthieu,
I was about to log the same through the Intel Premier Support site. I'm having trouble logging in right now, but I will check for the status on 545821 to see if any resolution is forthcoming. I appreciate your sticking with this; it is an important consideration which doesn't seem to be well understood or appreciated by most.
Regards,
Mike

One developers fix is another developers bug. I for one would rather Intel not change the standard Windows behavior (yes I do understand the issue and there are ways around it - several mentioned in this thread).The proposed IPP changemay help some people work around this issue, but it may also cause other developers new problems which are even more difficult to understand because the behavior is non-standard. Instead let's continue discussing additional possible work-arounds in this thread. Does anyone else have ideas?

Peter

[matthieu_darbois]

Quoting - pvonkaenel

Quoting - mdl61

Matthieu,
I was about to log the same through the Intel Premier Support site. I'm having trouble logging in right now, but I will check for the status on 545821 to see if any resolution is forthcoming. I appreciate your sticking with this; it is an important consideration which doesn't seem to be well understood or appreciated by most.
Regards,
Mike

One developers fix is another developers bug. I for one would rather Intel not change the standard Windows behavior (yes I do understand the issue and there are ways around it - several mentioned in this thread).The proposed IPP changemay help some people work around this issue, but it may also cause other developers new problems which are even more difficult to understand because the behavior is non-standard. Instead let's continue discussing additional possible work-arounds in this thread. Does anyone else have ideas?

Peter

I really don't think that, when loading "dummyPathippi-x.x.dll", trying to load "dummyPathippioptimizedversion-x.x.dll" should be considered a non-standard behaviour. It may be non-standard behaviour for WINDOWS but should be standard for IPP. We have several ActiveX which use IPP and don't work with the dynamic link model. It's simply not acceptable.
You state ahead that : "The proposed IPP changemay help some people work around this issue, but it may also cause other developers new problems which are even more difficult to understand because the behavior is non-standard.". I do not agree with you, it will surely help developers to know that the optimized module loaded is the one which resides in the same path as the dispatch DLL. If you have for example IPP 6.0 in your path and, along with your activeX, IPP 6.0 Update 2, you might wonder why ippiTrueDistanceTransform fails with large images...

Regards,
Matthieu

[elhefe38]

Quoting - pvonkaenel

One developers fix is another developers bug. I for one would rather Intel not change the standard Windows behavior (yes I do understand the issue and there are ways around it - several mentioned in this thread).The proposed IPP changemay help some people work around this issue, but it may also cause other developers new problems which are even more difficult to understand because the behavior is non-standard. Instead let's continue discussing additional possible work-arounds in this thread. Does anyone else have ideas?

Maybe Intel could release the code that performs this "waterfall" s.t. people could adapt it to their needs, without breaking IPP ? Or maybe adding a function in the core library to perform a waterfall explicitely with a given path (if this function is not called, the waterfall procedure is done as usual) ?

Regards

[pvonkaenel]

Quoting - matthieu.darbois

I really don't think that, when loading "dummyPathippi-x.x.dll", trying to load "dummyPathippioptimizedversion-x.x.dll" should be considered a non-standard behaviour. It may be non-standard behaviour for WINDOWS but should be standard for IPP. We have several ActiveX which use IPP and don't work with the dynamic link model. It's simply not acceptable.
You state ahead that : "The proposed IPP changemay help some people work around this issue, but it may also cause other developers new problems which are even more difficult to understand because the behavior is non-standard.". I do not agree with you, it will surely help developers to know that the optimized module loaded is the one which resides in the same path as the dispatch DLL. If you have for example IPP 6.0 in your path and, along with your activeX, IPP 6.0 Update 2, you might wonder why ippiTrueDistanceTransform fails with large images...

Regards,
Matthieu

Fair enough. I cannot come up with an example of when this "fix" might interfere with another linking model. I just get nervous when a low-level change is introduced that has the potential to cause unexpected side effects - I've introduced very elegant "fixes" to standard behavior in the past which ended up biting me later on and at the expense of very difficult debugging (and failing to remember I'd implemented the elegant "fix").

Out of curiosity, what did your management have against the static linking solution?

Peter