Intel® oneAPI Threading Building Blocks
Ask questions and share information about adding parallelism to your applications when using this threading library.

TBB For MinGW

dharaj
Beginner
1,545 Views
This problem seems to have been asked before without a satisfactory solution. There is no way of building for windows and mingw currently distributed with TBB.

I've tried to make my own inc files to get it working but I am, how you say, bad at it.

I have an angle on how it could be done, but I'd appreciate it if a more skilled person either helped or did it himself. If I knew what TBB required of the compiler and how to satisfy those needs via GCC, this would be trvial. Alas, the inc files are a little opaque to my amateur eyes.

I know of one issue, that is that the windows inc file called detect.js. The thing is, this script gets information like the runtime version by calling the VC++ compiler. Not good. I'm not sure how to get such info for MinGW and the linux gcc inc isn't very helpful.

Basically the inc files have to be spliced together; part linux gcc, part windows cl. I don't know how to do it.

I do know that mingw uses the same runtime as VC++, so it should be possible with a little work.

"Halp."
0 Kudos
30 Replies
RafSchietekat
Valued Contributor III
1,106 Views
Please consider starting with the "Additions to atomic" patch, which is for a large part concerned with porting.

0 Kudos
robert-reed
Valued Contributor II
1,106 Views
This is not the first discussion we've had here regarding minGW. Did you know about this apparently successful attempt? It seems that what they did was pretty rudimentary and certainly less than a port, but maybe it would work for you?
0 Kudos
balage
Beginner
1,106 Views
In it's current form, it's impossible to build TBB natively with anything else than MSVC or Intel compiler on windows. And it's FAR from a build system issue. (Cygwin is another story, it's not native).

Even if you could write the necessary detection stuff (which you can actually skip by explicitly setting the necessary arch and compiler env vars on the command line), and the relevant .inc files, the build is still going to fail when compiling the sources. If you just take a quick look at the sources, there are #ifdefs and the like all over the place, some of which test for the platform (WIN32, __GNUC__, MSC_VER, etc.), and currently, all these tests assume, that "windows = msvc or intel". The compiler (intel, gcc, msvc, digital mars, watcom etc.) and platform (windows, linux, mac, etc.) specific settings should have been decoupled from the start. But for windows, it currently implicitly assumes msvc or intel.

And even if the build succeeds with mingw (that's a whole lot of work), there is still the question whether it actually works...

To be able to fully support MINGW, someone who is already familiar with the internal needs and workings of the library should help. Otherwise, we won't see any mingw support for at least a few years.

--
Greets,
B.

0 Kudos
balage
Beginner
1,106 Views
I'm trying stuff, and report if I succeed.

--
Greets,
B.
0 Kudos
RafSchietekat
Valued Contributor III
1,106 Views
You can lead a horse to water, but you can't make it drink... smiley [:-)]
0 Kudos
balage
Beginner
1,106 Views
Well, after modifying some of the sources and build files, I was able to build TBB with MinGW (gcc 4.2.1-sjlj precisely), and the unit tests as well. Some of them work, some seem to livelock, some crash.

For now, I won't get to look what might be the problem, perhaps sometime next week.

Here's a test run log (though I suspect it would be different on a second run):
http://digitus.itk.ppke.hu/~oroba/tbb/tests.log

--
Greets,
B.
0 Kudos
balage
Beginner
1,106 Views
After a bit more testing, it seems that everything works, except exception handling.
I don't know if it has anything to do with SEH, because MinGW doesn't support it (the relevant keywords __try, etc. are a noop -- #defined to be empty).

I don't yet know how exceptions are dealt with in a MT environment.

For a start, can someone explain what exactly is the purpose of the __try and __finally block in task.cpp around line 2673? Does it have anything to do with C++ exceptions or is it only for SEH (or both)?

Thanks in advance.

--
Greets,
B.

0 Kudos
Alexey-Kukanov
Employee
1,106 Views

Dear B., thanks for exploring the possibility of MinGW port. I am glad you have made a good progress there. We have identified the problem with MinGW being use of _WIN32 to process compiler-specific portions of code, but have not yet put resources to improve it. If you contribute the patch, we will be glad to merge it back to TBB.

The problem, however, is how to ensure that any subsequent changes do not break it again, having inmind that we do not officially support MinGW and do not test with it. So it would be good if youor someone else provides some advice to follow in order to keep MinGW support healthy. Even better if someone could keep an eye on TBB updates and make sure MinGW support is not broken, once implemented.

As for the question: the try-finally block is there to catch any exception (as C++ exceptions are based on SEH in MS implementation). It is necessary to perform proper cleanup in case an exception is thrown in a worker thread and not intercepted.

0 Kudos
lazalong
Beginner
1,106 Views
Hello,

I would be very interested if you could tell or send me how you managed to compile with mingw32-gcc.exe.
You can contact me at spatlabor at gmail.com

If you prefer sending me the .a and .dll would be fine also.

Thanks
0 Kudos
balage
Beginner
1,106 Views
What I have actually done, is that I took the low-level assembly implementations from linux_ia32.h, and put it inside windows_ia32.h with a #ifdef __MINGW32__ guard (well, more-or-less). Other modifications were necessary in other files, but basically, that's the core of it.
This is of course not the best solution, as it is redundant. Whatever modification is made on the linux low-lever routines, also have to be made on the windows MINGW routines, as they are the same. Perhaps some common file, something like "gcc_ia32.h" would be better, which contains common GCC stuff, which is included by both windows_ia32.h and linux_ia32.h. The same goes for other architectures.

I'll pack up a zip with the modified sources and description soon, so others can also try it, and someone with better TBB understanding can begin to ponder on how to integrate it into official TBB (currently my solution feels a bit hackish).

Stay tuned :)

EDIT: Almost forgot: How could that __try/__finally block be fixed for MinGW in task.cpp? I'm sure that's the source of some of the crashes. How is it done on linux? What happens exactly if one of the threads throw an exception (linux/windows)? Does it propagate "out" from the thread?

--
Greetings,
Balazs
0 Kudos
tricaric
Beginner
1,106 Views
MADakukanov:

Dear B., thanks for exploring the possibility of MinGW port. I am glad you have made a good progress there. We have identified the problem with MinGW being use of _WIN32 to process compiler-specific portions of code, but have not yet put resources to improve it. If you contribute the patch, we will be glad to merge it back to TBB.



This is something I honestly don't understand. MinGW is GCC for Windows, that is, the most popular open source compiler for the most popular platform. And Intel treats it just as some kind of toy compiler, not worth the time or the work to support it. Intel, the biggest computing company, worth I don't know how many billion dollars, fails to recognize priorities and allocate resources to support what users keep asking. How long is it going to take before you finally decide that maybe MinGW is something Intel needs to support, instead of asking users with no clue of the TBB internals to port it and keep supporting it for you? There are things that users/volunteers can contribute, but major support should come from the main developers.

Look at Trolltech/Qt and their support for compilers and platforms. Smaller company than Intel for sure, with maybe more developers than TBB? OK, but Qt is a much much (much) larger project than TBB, with all the due respect, so things should even out on that, leaving the MinGW support problem on Intel/TBB's shoulders.

Pasquale

0 Kudos
RafSchietekat
Valued Contributor III
1,106 Views
If so many people use MinGW, maybe one of them can base a port on the latest "Additions to atomic" patch (which is largely concerned with porting, not just with tbb::atomic), currently version 2008-09-15? Last intervention here, I promise, but the earlier suggestion of gcc_ia32.h looks eerily like gcc_x86.h.

0 Kudos
Alexey-Kukanov
Employee
1,106 Views

balage:
How could that __try/__finally block be fixed for MinGW in task.cpp? I'm sure that's the source of some of the crashes.

I discussed it with Arch Robison and we agreed the best thing is to change it over to RAII instead of __try/__finally. I.e. create a local object which destructor does the necessary cleanup.

How is it done on linux? What happens exactly if one of the threads throw an exception (linux/windows)? Does it propagate "out" from the thread?

On Linux (or more correctly, with POSIX threads) you could specify a cleanup handler with pthread_cleanup_{push,pop}.

Exception handling changed a lot in TBB 2.1, while the code in question dates probably back to 1.0. To keep it short, now we try to intercept exceptions in tasks executed by a worker thread, andpropagate those to the master thread that called a TBB algorithm. For safety reasons, however, I would stillkeep that old cleanup code.

0 Kudos
balage
Beginner
1,106 Views
First of all, sorry for the long delay, this week has been exceedingly heavy.

To tricaric:
Don't be too surprised. Most bigger projects received MinGW support only later, I'm talking about wxWidgets, the OGRE 3D engine and lots of others, but eventually, all of them got pretty good MinGW support, so I'm confident TBB will have excellent MinGW support sooner or later, just stay tuned :)

As to the progress:
At latest tomorrow I'll pack the things together and upload somewhere with instructions.

As for the exception handling:
The problem is, MinGW doesn't have SEH at all (__try/__finally is noop), and because it's not posix, there are no cleanup handlers either. What could be the solution? What's exactly the RAII solution? Does that mean, that the exception is simply not catched at all?

--
Greetings,
Balzs

0 Kudos
dharaj
Beginner
1,106 Views
I'm glad somebody has decided to look into this seriously. Much appreciation for you!

Now, MinGW doesn't support SEH, but couldn't you use it via inlined ASM? It's just a thought. I'm going to do some research and see if such a solution has been successfully implemented in other projects.
0 Kudos
Alexey-Kukanov
Employee
1,106 Views

balage:
As for the exception handling:
The problem is, MinGW doesn't have SEH at all (__try/__finally is noop), and because it's not posix, there are no cleanup handlers either. What could be the solution? What's exactly the RAII solution? Does that mean, that the exception is simply not catched at all?

In simple, RAII is the C++ technique to deal with the kind of problems that __finally is intended to help, i.e. to execute some code independently on whether an exception was caught or not. Read the Wikipedia article for RAIIto get acquainted with it. The idea is simple: put cleanup code into a destructor of an object in the scope of interest, and this destructor will be called. In the particular case, __try/__finally should be removed, a special class (probably local tothe worker_routine function) should be written in a way that its destructors does all required cleanup, and an object of this class should be created in the scope where __try/__finally had resided.

You might as well leave this exercise to us in the TBB team if you do not feel confident about what should be done. If I have little more time to implement it later I can post it here.

0 Kudos
balage
Beginner
1,106 Views
I'm familiar with RAII, it's just that I forgot, that there is no catch now either, so the exception is not catched now either :)
I'll try this then, and see if it works...

--
Greets,
Balzs
0 Kudos
balage
Beginner
1,106 Views
Okay, sorry for the delay again.
As promised, here's the patch, attached to this post.
To build using MinGW:
  • First rename (or copy) mingw32-g++.exe and mingw32-gcc.exe in the in directory to g++.exe and gcc.exe respectively.
  • Copy over the attached archive's contents to the TBB source tree (tbb21_20080605oss) overwriting files.
  • Launch a cmd prompt in the TBB root, then type:
set tbb_os=windows
set arch=ia32
set compiler=gcc
set runtime=unknown
mingw32-make

That will build the libs, to build and run the tests, type:
mingw32-make test
Of course, some tests will live-lock, others simply crash. The source of problems is centered around exception handling issues I believe.

Other issues I encountered so far:
  • Some tests use __declspec(thread) in WIN mode for TLS. MinGW doesn't support this, yet only issues a warning, hence the resulting program will be incorrect. I modified one of these (namely test_task_scheduler_observer.cpp) to use the WinAPI Tls*() functions.
  • _M_IX86 (and others similar) are defined by the compiler under windows by MSVC and Intel, but NOT by MinGW. However, the MinGW winapi headers DO define these. The problems occur, when sometimes the windows.h header gets included before _M_IX86 is used, so sometimes it works, but when windows.h is not included, compilation fails. I added __i386__ next to where _M_IX86 is used to correct this.
  • __m128 needs some special alignment, and GCC (or just MinGW, I don't know) does alignment differently. I didn't fully research the subject, but if you do a google on "__m128 mingw align" you'll see what I mean. Also, here's a short discussion: http://readlist.com/lists/lists.sourceforge.net/mingw-users/0/2675.html
Well, that's it so far. Whoever can, experiment with this stuff.

I used gcc 4.2.1-sjlj for compilation.

--
Greets,
Balzs

0 Kudos
Alexey-Kukanov
Employee
1,106 Views

Thanks Balzs for the great job done. Could you please also officially contribute your patch to the project? For that, open http://www.threadingbuildingblocks.org/contribution_first.phpand follow the instructions. If the official contribution is not made, we can not incorporate the patch to the main TBBtrunk.

0 Kudos
balage
Beginner
1,000 Views

What I plan to do, is to get the latest development version, and see if I can do the necessary modifications.

Anyway, the "patch" can be downloaded from here (I cannot see my attachment to my prev post - do they work?):

http://digitus.itk.ppke.hu/~oroba/tbb/tbb21_20080605oss_mingw_patch_1.zip

Again, I have to stress, that my solution is a bit hackish, and also requires a lot of testing. So please, whoever has MinGW and some time, experiment with this stuff. Or just report, I'm curious if anyone found this thing useful.

--

Greets,

Balzs

0 Kudos
Reply