Remove dependency from MS C++ runtime lib MSVCR80.dll

[Blindleistung]

Hi all.

I am rather new to this but have successfully built a little app using TBB on Windows x64. It works fine on the dev machine.

Now the tbb.dll uses the C++ runtime dlls of microsoft which are called msvc*80.dll and reside in C:\WINDOWS\WinSxS\... (called "Side-by-Side Assemblies")

The problem I have with that is that these DLLs refuse to load if I copy them into my app-dir. They need to be installed. And they seem to make problems for many people (lots of hits on google :-).


I would like to keep this program in a way that it runs on any PC without installation. Just unpack and run. I have found that there are already topics in the forum discussing this.

Alexey's hint to copy the 'Microsoft.VC80.CRT' folder from the redist folder to my exe's target folder also did not help. The libs won't load and windows does not tell me why.

But I don't want to know how to get the libs loaded or installed, but how can I get rid of this dependency alltogether? My attempts to build tbb into a static library failed (It compiled, but crashed when in use.)

Is there a pre-compiled static lib? Is there another way to avoid these M$-dlls?


Thanks for your help, guys!


Achim.

[pvonkaenel]

Quoting - Blindleistung

Hi all.

I am rather new to this but have successfully built a little app using TBB on Windows x64. It works fine on the dev machine.

Now the tbb.dll uses the C++ runtime dlls of microsoft which are called msvc*80.dll and reside in C:WINDOWSWinSxS... (called "Side-by-Side Assemblies")

The problem I have with that is that these DLLs refuse to load if I copy them into my app-dir. They need to be installed. And they seem to make problems for many people (lots of hits on google :-).


I would like to keep this program in a way that it runs on any PC without installation. Just unpack and run. I have found that there are already topics in the forum discussing this.

Alexey's hint to copy the 'Microsoft.VC80.CRT' folder from the redist folder to my exe's target folder also did not help. The libs won't load and windows does not tell me why.

But I don't want to know how to get the libs loaded or installed, but how can I get rid of this dependency alltogether? My attempts to build tbb into a static library failed (It compiled, but crashed when in use.)

Is there a pre-compiled static lib? Is there another way to avoid these M$-dlls?


Thanks for your help, guys!


Achim.

You don't want to create a static library version of TBB - very dangerous to have multiple versions linked into a single application. Instead you want to statically like with the MS run-time. You can do that by changing the /MD in the makefile to /MT (check the MS_CRT_KEY definition in either windows_cl.inc or windows_icl.inc). I have been using this with TBB 2.1 for quite a while without problem.

[RafSchietekat]

"Instead you want to statically like with the MS run-time."
From years ago I remember that this can cause major pain, if some memory is allocated in one runtime and deallocated in another, resulting in the requirement to link with the same DLL version throughout the application, much like the whole application must dynamically link with TBB. I'm not saying this is incorrect, but I am surprised.

[pvonkaenel]

Quoting - Raf Schietekat

"Instead you want to statically like with the MS run-time."
From years ago I remember that this can cause major pain, if some memory is allocated in one runtime and deallocated in another, resulting in the requirement to link with the same DLL version throughout the application, much like the whole application must dynamically link with TBB. I'm not saying this is incorrect, but I am surprised.

This is correct. One of the possible negative side-effects of statically linking with the MS run-time is that you must deallocate memory in the same module you allocated it in - note that there are some other potential pitfalls with statically linking with the run-time which are available from http://msdn.microsoft.com/en-us/library/abx4dbyh(VS.80).aspx. In my opinion, you should always deallocate in the same module you allocate in - in-fact I would recommend carefully controling all resource allocation/deallocations, but that's a different discussion.

[RafSchietekat]

"you should always deallocate in the same module you allocate in"
That seems like a big hassle... how do you even do that reliably (semi-rhetorical): always override new and delete, non-inline of course? As if we don't have enough yet to worry about! Isn't it infinitely easier to consistently use the same runtime (semi-rhetorical again)?

[pvonkaenel]

Quoting - Raf Schietekat

"you should always deallocate in the same module you allocate in"
That seems like a big hassle... how do you even do that reliably (semi-rhetorical): always override new and delete, non-inline of course? As if we don't have enough yet to worry about! Isn't it infinitely easier to consistently use the same runtime (semi-rhetorical again)?

Hi Raf,

Well, I tend to wrap resources (including memory allocations) inside of classes and then expose allocate() and deallocate() methods (non-inline) to protect them from cross-module contamination. It can, however, become difficult to control if you start passing growable STL containers (such as strings) around as parameters from module to module. Other than that I've had a lot of luck keeping resource allocation/deallocation within a module.

As for using the same run-time across all modules, my opinion has bounced between /MT and /MD for several years now. Most people use /MD (it is the default after all) and several issues do go away. However, I've also run into cases where a customer has one version of the run-time DLLs but not the correct version or even a corrupt version. At this point the whole SxS installation may take care of things, but what happens when you run into a 3rd party library that was compiled with VS2005 or VS2003 (with old run-time DLL dependencies) and you're using VS2008? In this case I'd rather know that my module does not violate any of the multi-run-time rules and have it statically linked using /MT. I'm still on the fence on this one (even though I'm currently using /MT), but I could be convinced back to /MD.

Peter

[Blindleistung]

Quoting - pvonkaenel

You don't want to create a static library version of TBB - very dangerous to have multiple versions linked into a single application. Instead you want to statically like with the MS run-time. You can do that by changing the /MD in the makefile to /MT (check the MS_CRT_KEY definition in either windows_cl.inc or windows_icl.inc). I have been using this with TBB 2.1 for quite a while without problem.

Hi.

I tried selecting the static link in the Visual studio project settings. But that not only caused the TBB lib to output a warning, but that only linked my code to the static lib. The pre-compiled TBB still wanted the dll. So I did not drop the dependency.

Is there something I have missed? Can I statically link the runtime into the tbb.dll lib?

Of course the use of two runtimes is dangerous concerning allocation. But I think I can keep this clean as I always try to put allocate/deallocate pairs into one file. As long as the TBB lib does not allocate stuff for me and expects me to free it like some Windows Driver APIs do...

I think I'll try to dump all the TBB sources into my project next... Maybe that works.

[pvonkaenel]

Quoting - Blindleistung

Hi.

I tried selecting the static link in the Visual studio project settings. But that not only caused the TBB lib to output a warning, but that only linked my code to the static lib. The pre-compiled TBB still wanted the dll. So I did not drop the dependency.

Is there something I have missed? Can I statically link the runtime into the tbb.dll lib?

Of course the use of two runtimes is dangerous concerning allocation. But I think I can keep this clean as I always try to put allocate/deallocate pairs into one file. As long as the TBB lib does not allocate stuff for me and expects me to free it like some Windows Driver APIs do...

I think I'll try to dump all the TBB sources into my project next... Maybe that works.

If you're building from the command line using make then you need to change the /MD to /MT in the windows.cl.inc or windows.icl.inc file. If you're building using a Visual Studio Solution/Project then you need to go to project properties, C/C++, code generation, and change the run-time library for /MTd for debug and /MT for release. As far as I know, this is all you need to do to remove dependency on the MS run-time DLLs.

[Alexey-Kukanov]

Quoting - Blindleistung

Hi.

I tried selecting the static link in the Visual studio project settings. But that not only caused the TBB lib to output a warning, but that only linked my code to the static lib. The pre-compiled TBB still wanted the dll. So I did not drop the dependency.

Is there something I have missed? Can I statically link the runtime into the tbb.dll lib?

Of course the use of two runtimes is dangerous concerning allocation. But I think I can keep this clean as I always try to put allocate/deallocate pairs into one file. As long as the TBB lib does not allocate stuff for me and expects me to free it like some Windows Driver APIs do...

I think I'll try to dump all the TBB sources into my project next... Maybe that works.

The precompiled TBB DLLs will always have the dependency on the MSVC DLLs. You can remove the manifest embedded into TBB binaries (it is just a resource, and also there is a special tool from MS that deals with manifests), but the dependency will still remain, as TBB needs some functions from the C++ runtime (such as __beginthreadex, for example).

Dumping all the TBB sources into your project essentially means statically linking with, which is generally not recommended, not supported, and not guaranteed to work. If you want to have the dependency resolved at link time rather than at runtime, you need to rebuild TBB (and not only your app) with /MT instead of /MD, as it was suggested already. Well, TBB built with /MT is not officially supported either, but I would expect less problems there, and also you might run the TBB test suite for sanity check.

[thevinn]

Seriously what is all this jibber-jabber about static linking being in some way, shape, or form BAD?

User experiences with dynamic linking is NEVER better, only worse than the statically linked equivalent. As a developer of user-oriented desktop applications for over a decade, I can say without reservation that it is always the goal to produce an application that has no external library dependencies whatsoever. Especially in Windows, the introduction '.DLL' files inevitably precludes some portion of users from being able to run. Whether its a version conflict, a botched install, corrupted file, or any one of a number of gremlins that seem to pop up at the most inopportune time, .DLLs are a headache, period.

Now, someone said that linking statically causes multiple versions of the TBB library to get linked in. How is that possible if an executable links with no other shared library? Someone told me the same thing about linking statically with OpenMP. I would like a decent explanation of this situation if possible.

Intel: Lets make statically linked libraries a priority. Resolve the duplicate library issue by whatever means necessary, or document the problem more fully instead of the one line explanation in the help file.

Dynamic linking: A headache for users

Static linking: A smooth, pleasurable experience for all involved.

[thevinn]

Quoting - Alexey Kukanov (Intel)

TBB built with /MT is not officially supported either, but I would expect less problems there, and also you might run the TBB test suite for sanity check.

Lets add some official support then!

[robert-reed]

Quoting - thevinn

Seriously what is all this jibber-jabber about static linking being in some way, shape, or form BAD?

I don't think that anyone suggested that static linking is outright bad! It's just that it can lead to some unfortunate circumstances, that can get libraries like TBB confused. TBB needs a singleton object in order to build one copy of a data structure used for holding a thread pool and managing the tasks it executes. You might be able to do singletons in static libraries but only with a lot of hair an marginal programming to make it bulletproof. It's a lot easier to guarantee the uniqueness of the data structure by putting its manager in a dynamic library that only gets loaded once, rather than if static and loaded by every library whose developers like your library.

Quoting - thevinn

User experiences with dynamic linking is NEVER better, only worse than the statically linked equivalent.

Oh, sounds like a survivor of the notorious Microsoft DLL-hell wars. Not all shared objects are bad, just some implementations. :-(

Quoting - thevinn

Now, someone said that linking statically causes multiple versions of the TBB library to get linked in. How is that possible if an executable links with no other shared library? Someone told me the same thing about linking statically with OpenMP. I would like a decent explanation of this situation if possible.

I admit that if you can somehow guarantee that only one copy of TBB is going to be linked in to the entire application, there are some benefits to be had from static linking, and I think it is with this mindset that Alexey even suggested using TBB statically. All too often though, we live ina world of multiple libraries, each used in myriad combinations with various applications. To keep life sane, TBB so far has held to the convention of only linking dynamically. Source code is available for anyone who wants to blunder, uh,... venture down that path.

[thevinn]

Static linking worked great!

But are you saying that if there are two completely different applications that statically link in TBB the performance will be worse than if they both used a common scheduler?

[robert-reed]

Quoting - thevinn

But are you saying that if there are two completely different applications that statically link in TBB the performance will be worse than if they both used a common scheduler?

No, not different applications but applications that use multiple libraries. We have a variety of libraries at Intel that are starting to use or already using TBB as a basic threading resource or a basic memory allocation resource, or both. Applications may want to use multiple of these libraries in order to make use of their functions, and therein lies the rub. If each of those libraries has a static copy of TBB, the singleton requirement will not be met and within the application process may be divisions in memory pools as well as multiple thread pools.

[pvonkaenel]

Quoting - thevinn

Static linking worked great!

But are you saying that if there are two completely different applications that statically link in TBB the performance will be worse than if they both used a common scheduler?

If you are able to guarentee that your application does not link with any DLLs, then you are probably fine. The problem comes when you have an application module that statically links with TBB, and that application links with a DLL that statically links with TBB. In this case you end up with two copies of TBB in your application each with it's own schedule and set of worker threads.

[thevinn]

Quoting - pvonkaenel

If you are able to guarentee that your application does not link with any DLLs, then you are probably fine. The problem comes when you have an application module that statically links with TBB, and that application links with a DLL that statically links with TBB. In this case you end up with two copies of TBB in your application each with it's own schedule and set of worker threads.

I can guarantee that for my app - I am in control of all the libraries save two, and those are open source and don't use threads.

However, how is the case that you described (two client libraries in the same app each with their own instance of the TBB task manager singleton) different from the case of two separate processes, each with their own instance of the TBB task manager singleton?

[RafSchietekat]

Quoting - thevinn

However, how is the case that you described (two client libraries in the same app each with their own instance of the TBB task manager singleton) different from the case of two separate processes, each with their own instance of the TBB task manager singleton?

One of them has an easy fix.

[pvonkaenel]

Quoting - thevinn

I can guarantee that for my app - I am in control of all the libraries save two, and those are open source and don't use threads.

However, how is the case that you described (two client libraries in the same app each with their own instance of the TBB task manager singleton) different from the case of two separate processes, each with their own instance of the TBB task manager singleton?

You have control over when to start processes, but no control over the modules within a single process. In both cases you will oversubscribe your cores, but at least within a single application you can prevent this by making sure you only have a single TBB scheduler running.

[seaephpea]

Quoting - thevinn

Static linking worked great!

But are you saying that if there are two completely different applications that statically link in TBB the performance will be worse than if they both used a common scheduler?

Did you do do anything non-obvious to get static linking to work (i.e. beyond just changing the configuration type)?

Thanks,

Tom

[pvonkaenel]

Quoting - seaephpea

Did you do do anything non-obvious to get static linking to work (i.e. beyond just changing the configuration type)?

Thanks,

Tom

In terms of TBB there are a couple possible meaning of static linking: rebuilding the TBB library to statically link with the MS run-time, and rebuilding TBB to be a static library which you link with your application. Which are you trying to do? To get TBB to statically link with the MS run-time, change the /MD to /MT in the makefile and rebuild TBB.

[seaephpea]

Quoting - pvonkaenel

In terms of TBB there are a couple possible meaning of static linking: rebuilding the TBB library to statically link with the MS run-time, and rebuilding TBB to be a static library which you link with your application. Which are you trying to do? To get TBB to statically link with the MS run-time, change the /MD to /MT in the makefile and rebuild TBB.

Yeah I'd got static linking to the MS run-time to work. I was asking about static linking to TBB, or code dumping, which is more likely to have a measurable impact on performance. The post from "thevinn" above made it sound like he'd got static linking TBB to work, which seemed like it ought to require quite a few changes (not least doing something with DllMain).