OpenCL for IVF?

[cfrieler]

Steve,

Some years ago, I asked about support for both OpenMP and MPI within the Intel Fortran compilers and you said I "might as well ask for a pony, too."

Well, Intel's Cluster OpenMP project is very close to the pony I was hoping for. I have a large cluster of quad CPU boxes that I have been doing hybrid MPI/OpenMP coding for. I use the Intel Linux compilers and am going to give Cluster OpenMP a try on a small test code and see if it simplifies my life. Unfortunately, it requires EM64 compatible hardware and I will only be able to run it on a couple of newer nodes, as the majority of my cluster is older Xeon hardware.

But as always, when a child gets one shiny new toy, there is always another on the horizon. If Intel is going to support OpenCL in any of it's compiler products in the short term, it may stop me from purchasing the PGI Workstation product which was recently announced with this capability. Any hints you can provide on when/if?

If the Cluster OpenMP technology (DSM) could be evolved to use the OpenCL APIs, that would be a powerful extension of the concept. It no-doubt changes the programming model somewhat, but being able to use the same code for either a cluster or a GPGPU accelerated workstation is very attractive. Any show-stoppers I can't see?

Regards, Cliff

[Steven_L_Intel1]

Did I really say that? Doesn't sound like something I would say.

Sorry, I don't have anything I can tell you regarding OpenCL.

[TimP]

Successful hybrid MPI/OpenMP products using Intel Fortran with Windows MPI have been released since Cluster OpenMP support faded.
OpenCL and alternatives such as Ct are likely to emphasize extensions of C++ rather than Fortran, and Intel support in that area is likely to center on Intel hardware products rather than incompatible competing products.
You seem to be asking for something which covers future 64-bit and past 32-bit products under Windows, rather than current ones. If PGI did that to your satisfaction, I wouldn't bet on anyone competing with it. I don't read the announcement the way you do. For one thing, it's clear about supporting linux, not Windows. For another, it involves writing in explicit accelerator coding calls, and doesn't handle treating accelerators as cluster nodes.

[Les_Neilson]

Steve Nuchia asked for a pony back in January 2008 (see How to change multiple project settings at once ). Christmas has come and gone I wonder if he got his wish.

Les

[cfrieler]

Steve,

Yes, you did say that. But it was many years ago, say '02 or thereabouts, and I didn't let it put me off. I'd been working on massively parallel architectures, and even then I knew that parallelism was going to become more important in the future. Not that the insight has made me rich or anything ;)

Tim,

"Successful hybrid MPI/OpenMP products using Intel Fortran with Windows MPI have been released since Cluster OpenMP support faded."

Brings up two questions -- Why has "support faded"?

and -- Do you have any specific "Successful" products/technologies I could look into?

Regarding OpenCL - "likely to emphasize extensions of C++ rather than Fortran"

C is much more likely, witness CUDA and AMD's Stream (Brook?) GPGPU endeavors. But the motivation for my question is that PGI just released a Fortran interface for GPGPU acceleration;

http://www.pgroup.com/lit/articles/insider/v1n1a4.htm

The offering is CUDA specific right now, but PGI promises to move to OpenCL as soon as possible. And don't count Intel's interest out, as PGI specifically mentions Larrabee as a future target.

"it involves writing in explicit accelerator coding calls, and doesn't handle treating accelerators as cluster nodes." True, but I think you have the problem backwards, and my comment about Cluster OpenMP "evolving" to use the OpenCL API was backwards as well. I would like a product that treated a cluster node as an accelerator, and ClusterMP is clearly at higher level in the coding heirarchy than OpenCL is right now.

OpenCL doesn't include an accelerator model, but rather a data localization and task model that maps to a broad range of hardware. The key element that both it and ClusterMP bring to the table IMO is Distributed Shared Memory. Understanding that the compiler capability to map task/data structures to optimal execution capabilities in a heterogeneous system transparently to the user is still a long way off, I accept that I will have to give substantial hints in the form of compiler directives. The hope is that once I have dissected my problem, the device drivers which expose OpenCL interfaces will compare notes and decide where to most efficiently execute each part of the program. That is how the nVidia and AMD GPGPU drivers support OpenCL currently, I think. If the ClusterMP technology was in the form of a driver (kernel module) and an OpenCL interface were developed, seems like a winner to me.

In any case, I would strongly encourage Intel to support OpenCL while they develop tools to support Larrabee. If OpenCL could be worked into IVF, I think it would increase the product's value.

Cheers, Cliff

[Steven_L_Intel1]

Yours is the first request I have seen for OpenCL support in Fortran, but as I think it is mainly an API, I don't see why you couldn't use it with a suitable library.

From our perspective, based on input from numerous customers, we are focusing on the Fortran 2008 Co-Array feature to provide parallelism in a "Fortran" way.

[cfrieler]

Steve,

"Did I really say that?"

Your reply about OpenCL being "just an API" sounded uncannily familiar, so I did some searching;

http://software.intel.com/en-us/forums/showthread.php?t=45495

I was wrong, it was '01. Let's see, I asked in 2001 and Intel has a research project in 2008.

Although I don't have enough direct experience to recognize it's limitations, Co-Arrays seem like a nice high-level syntax for handling DSM. If the back-end implementation was based on calls to OpenCL, operations could be passed to GPGPUs or other nodes on a cluster, depending on suitability. That would be my pony and an elephant, too!

But if I have to wait till 2016 for it, I may retire first. (I hope)

Tim,

I'd still be interested in your views on (experience with?) ClusterMP and any references to similar successful efforts.

Thanks All, Cliff

[Steven_L_Intel1]

Cluster OpenMP (CLOMP) is useful for a certain set of applications - those coded to the OpenMP API, scale well on a SMP system and where the user wants to expand execution across a cluster. CLOMP works best when there is relatively little data exchange across the tasks and each task does a lot of work. It ended up not being as popular as we hoped, so it was removed from the compiler product and made available through the whatif.intel.com site (you still need a compiler license). If I recall correctly, it supports only Intel 64 or IA-64 and Linux.

I don't know why you are focused on OpenCL specifically.

Wow - that post you found was back in the Compaq days, eight years ago. But what I said about MPI is true - and people are using it with Intel Fortran today.

[cfrieler]

Steve,

"I don't know why you are focused on OpenCL specifically."

If I were writing exclusively graphics, I would be asking about OpenGL, or perhaps f90GL. If I were looking for a convenient method of multi-threading and getting the most out of my SMP box, I would be asking about OpenMP. If I were running on a cluster and Argonne National Labs hadn't been out front (in the day) with MPICH, I would be asking about OpenMPI.

Eight years ago, I wanted to know how to conveniently write Distributed Shared Memory (DSM) programs that would distribute across a cluster of SMP boxes, yet get top performance out of each box. That was despite not knowing to call it DSM. I've spent eight years writing hybrid OpenMP/MPI programs to accomplish this. It hasn't been convenient, and I had given up looking for a commercial tool to make it so. It looks like CLOMP might be that tool, so I'm sorry I missed it's introduction a year ago and will take a look at it.

Now however, my cluster is getting a bit creaky and I find that the most cost effectively solution for my problems is to use my graphics card. You just can't compete with 2Tflops peak for less than $500!! I can eliminate an entire room of racks and run a handful of dual Nehalem workstations with SLI X2 nVidia GTX295 cards. Total acquisition cost ~$25k and power savings of $30k/yr.

So now I'm looking for a convenient method of programming for that capability and I'm asking about OpenCL. I know CUDA and Brook already exist, but I want to code in Fortran and have as much of the memory handling done by the compiler/drivers as possible. I'm hoping that Fortran compiler support for OpenCL and a few years of evolution will get me there. I'm also hoping that I don't need to switch to PGI compilers to get going.

This seems very compelling to me, but if I've missed something or there is a better alternative, I'm eager to learn!

Regards, Cliff

[Steven_L_Intel1]

Cliff,

Thanks for your comments. Intel is headed that way, perhaps not quite in the manner you envision, but the goal is the same. I can't talk about details, but you should follow what we are saying about Larrabee. I'll be sure to pass along your comments to our team, but I don't have anything to offer you in the near term.

[j0e]

Well, I'm certainly not much in the way of a parellel programer (a little openMP and MPI), but all the talk about GPU accelaration got my interest several months ago, especially the low hardware costs. I also saw the PGI announcement of native GPU fortran support, which is how I came across this thread, as I too would rather use the Intel compiler.

It does seem for at least some computational problems, the GPU route seems important (IBM's Roadrunner is currently the top supercomputer because of it). Anyway, I'd also support the idea of openCL support from Intel.

cheers,
-joe

[cfrieler]

"Intel is headed that way, perhaps not quite in the manner you envision, but the goal is the same.

I can't talk about details, but you should follow what we are saying about Larrabee."

Steve,

It's been almost a year, and what I've heard about Larrabee is not very encouraging.

Anything positive to report about Intel compilers and OpenCL?

Regards, Cliff

BTW; In the past year I have added a single workstation with dual nVidia cards, and for the right problem (and painstakingly managed memory access) the performance is stunning. I still get the bulk of my work done with hybrid OpenMP/MPICH on the larger cluster, but hardware investments for that style of computing is over for me!!

[Steven_L_Intel1]

Cliff,

I don't have anything new to say here. Sorry.