- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
What would I have to know to begin wrapping my head around the idea of Parallel Programing? Lets see what shakes loose.
Cheers,
Jerry
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
OpenMP has regained momentum as a basis for shared memory parallelism; it's the basis for the Intel C, C++, and Fortran compiler auto-parallel options, Parallel Studio, and MKL threading. MPI also gained momentum; it's been running well ahead of general server market growth. For those who stay within C++, TBB has proven to be useful. You'll find a lot of both promotional and solid reference material on all of those in these forum areas.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
What would I have to know to begin wrapping my head around the idea of Parallel Programing? Lets see what shakes loose.
Cheers,
Jerry
To pin down - How to learn Parallel Programming, few things would be needed -
(a) Learn first to choice the architecture on which you wish to execute the Parallel Program - SMP, Distributed MP, Cluster, NUMA & Distributed SMP. Identifying the architecture 'll make you to execute & write Parallel Program.
(b) If you are with SMP, try learning the behaviour of Implicit & Explicit Vectorizations. Also, try learning Auto-Parallelization andknowing the threads (POSIX, OpenMP, etc.) behaviour.
(c) If the need is Distributed MP, try learning MPI programming with Send & Recive API to start with.
(d) If the need is Cluster computing, try learning both OpenMP & MPI.
Think ofachieving maximum scalability for your Parallel Programming by using better scientific libraries (MKL, ACML, Atlas, GoToBlas, etc.) and reliable network stack (IB, etc.).
Off-course tools for Parallel Programming is also needed, like Intel VTune, Intel Thread Checker, Intel Trace Analyzer & Checker, SPEC-MPI2007, IMB-v3.1, Total View Debugger, Marmot, etc.
Never align with people's solution of HPC Parallel Languages like - Fortress, Cilk, Manticore, Ct, etc...all tried to address the HPC needs for Parallel programming but they missed to address the issues of performance for HPC. These languages are good only in Research Labs but not as commercial.
~BR
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
What would I have to know to begin wrapping my head around the idea of Parallel Programing? Lets see what shakes loose.
Cheers,
Jerry
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Parallelism is nothing more than the management of a collection of serial tasks
where
management refers to the policies by which (a) tasks are scheduled, (b) premature
terminations are handled, (c) preemptive support is provided, (d) communication primitives
are enabled/disabled, and (e) the manner in which resources are obtained and released,
and
serial tasks are classified in terms of either (a) Coarse grain -- where tasks may not
communicate prior to conclusion, or (b) Fine grain -- where tasks may communicate
prior to conclusion
That's it! Of course, the devil is in the details (both hardware and software).
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
where
management refers to the policies by which (a) tasks are scheduled, (b) premature
terminations are handled, (c) preemptive support is provided, (d) communication primitives
are enabled/disabled, and (e) the manner in which resources are obtained and released,
and
serial tasks are classified in terms of either (a) Coarse grain -- where tasks may not
communicate prior to conclusion, or (b) Fine grain -- where tasks may communicate
prior to conclusion
That's it! Of course, the devil is in the details (both hardware and software).
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Well, that's really all any parallel program is, a collection of piecewise serial programs that do their work and the more of them that can do their work without interfering with their peers, the higher the concurrency level (scaling) the application can achieve. Think of a parallel for running across an array of values: each thread takes a section and serially runs a regular for loop over a portion of the array--it's a serial program for all intents and purposes.
The problem of course is that it's only piecewise serial: when each of the serial sections in our parallel-for finish, they need to rendezvous (join) with the other workers finishing other sections of the array so that code following the parallel for can run with the assumption that all the array elements have been processed. And the serial sections can be even shorter: an algorithm that needs to update some shared common state each time through the loop can only count as serial the portion from one state update to the next. And the more frequent the interruptions, the more time is spent in a truly serial state, where only one thread can progress through some critical section, the greater the limitation on the scalability of that algorithm.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
There is a tight coupling between
+) Scheduling of tasks (across resources -- threads/processes) on one hand, and
+) Communication primitives any of the said tasks may utilize on the other-hand
I.e. they are two sides of the same coin.
So, for example, if I want to schedule a task in isolation of any other task, I need to
ensure that the said task will not be able to communicate with any other task.
Conversely, for example, if I want N tasks to communicate among themselves, I have
no choice but to manage the N resources as a single unit; i.e. if one resource goes
down, the rest of them are forcibly shutdown.
Take this approach to its logical conclusion, and you have a wide and rich variety of
(scheduler + communication primitives) tuples.
Again, note that everything stated transcends the moment.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
>>So, for example, if I want to schedule a task in isolation of any other task, I need to
ensure that the said task will not be able to communicate with any other task.
Additionally you want to inhibit non-communication side effects such as multiple threads inadvertantly using the same temporary variables, inadvertantly working on (modifying) objects that they may share, inapropriate file writes, etc...
>>Conversely, for example, if I want N tasks to communicate among themselves, I have
no choice but to manage the N resources as a single unit; i.e. if one resource goes
down, the rest of them are forcibly shutdown.
Not necessarily. It is better if the N resources are independently managed i.e you have N mutex or N+1 mutex (one for each, + one for all)
Jim Dempsey
![](/skins/images/D2683F18326913BBA0436CB7114DD569/responsive_peak/images/icon_anonymous_message.png)
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page