mpiifort / mpiicc / mpiicpc script error

mrentropy · ‎12-16-2009

I get the following strange error with mpiifort:

[shell]$ mpiifort foo.F90
[: 478: no: unexpected operator
$[/shell]

and likewise with mpiicc:

[shell]$ mpiicc -c foo.c 
[: 382: no: unexpected operator
$[/shell]

and with mpiicpc:

[shell]$ mpiicpc -c foo.c 
[: 389: no: unexpected operator
$[/shell]

This is on Intel64 (i.e. x86_64) Ubuntu 9.10, which has this annoying problem that libstdc++5 is missing, but I fixed that.

In all scripts (mpiifort / mpiicc / mpiicpc) the offending bit of code seems to be this:

[shell]if [ $_force_rel_version == "yes" ]; then
    if [ $MPILIBNAME != "mpi_log" ]; then
        # Forse linking with release version of the MPI library
        MPILIBNAME="mpi"
    fi
fi
[/shell]

For reference,

[shell]$ uname -a
Linux monostatos 2.6.31-16-generic #53-Ubuntu SMP Tue Dec 8 04:02:15 UTC 2009 x86_64 GNU/Linux
[/shell]

And:

[shell]$ mpiifort --version
[: 478: no: unexpected operator
ifort (IFORT) 11.1 20091012
Copyright (C) 1985-2009 Intel Corporation.  All rights reserved.

$ mpiicc --version
[: 382: no: unexpected operator
icc (ICC) 11.1 20091012
Copyright (C) 1985-2009 Intel Corporation.  All rights reserved.

$ mpiicpc --version
[: 389: no: unexpected operator
icpc (ICC) 11.1 20091012
Copyright (C) 1985-2009 Intel Corporation.  All rights reserved.

$ 
[/shell]

Hmm. Any ideas?

Thanks,
Peter

mrentropy · ‎12-16-2009

P.S. I should note that on an identical system, also running Ubuntu 9.10, I do NOT have this problem. On that system, I installed Intel MPI when it was 9.04, then upgraded to 9.10 and fixed the libstdc++5 problem. On the system on which I have problems, I installed Intel MPI after upgrading to 9.10 and doing the libstdc++5 fix.

I must have messed up something on the install.

I did the install with the -no-rpm flag, since otherwise, rpm complained with this error:

[shell]rpm: please use alien to install rpm packages on Debian, if you are really sure use --force-debian switch. See README.Debian for more details.[/shell]

Gergana_S_Intel · ‎12-16-2009

Hi Peter,

I would like to preface this by saying that the Intel MPI Library does not currently support Ubuntu or other Debian-based systems.

Can you tell me which version of the Intel MPI Library you're using? I suspect the latest one (3.2 Update 2) but you can make sure by viewing the contents of the mpisupport.txt file located in the install directory.

Also, does this happen if you just run the standalone compiler (e.g. icc, ifort) outside the mpiicc and mpiifort scripts? Just trying to isolate the problem - it's either the Intel MPI Library scripts, or the underlying compiler.

If you suspect corruption during installation, then uninstalling and running the install scripts again is pretty easy. There should be a uninstall.sh file located in the Intel MPI Library install directory. Speaking of, did you simply run the install script on your problemless system? We posted a short HowTo on installing the Cluster Tools on Debian, but we advise using alien. It'll be interesting to know if you've succeeded in a different way.

Let me know how it goes.

Regards,
~Gergana

mrentropy · ‎12-16-2009

Quoting - Gergana Slavova (Intel)

I would like to preface this by saying that the Intel MPI Library does not currently support Ubuntu or other Debian-based systems.

Understood. My real compute cluster uses CentOS 4, but I find it a pain to administer so I use Ubuntu for everything else since it's usually brain-dead easy for people who aren't linux experts like myself :) Thanks so much for offering to help.

Quoting - Gergana Slavova (Intel)

Can you tell me which version of the Intel MPI Library you're using? I suspect the latest one (3.2 Update 2) but you can make sure by viewing the contents of the mpisupport.txt file located in the install directory. (etc etc)

Package ID: l_mpi_p_3.2.2.006

Compilers (icc/icpc/ifort) work just fine. Problem seems isolated to scripts. Here's another one, this time with mpirun:

[plain]$ mpirun -np 4 ./gypc
/opt/intel/impi/3.2.2.006/bin64/mpirun: 63: Syntax error: "(" unexpected (expecting "fi")
$ [/plain]

uninstall.sh does not work. Out of the box, it's not executable, but even doing chmod still have problems:

[plain]$ chmod +x uninstall.sh
$ sudo ./uninstall.sh 
[sudo] password for pwilliam: 
ERROR: File "./impi.uninstall.config" does not exist.
$ [/plain]

I don't remember how I did the install on the Ubuntu 9.04 -> 9.10 system where IntelMPI/mpirun/mpiifort etc work. It was four months ago or so. Sorry. Good question, since I am almost certain that it worked right out of the box, without any tweaking.

I already found your notes at: http://software.intel.com/en-us/articles/installing-cluster-tools-on-debian/
Thanks very much.

Note however that the --extract option does not seem to work:

[plain]$ mkdir myrpms
$ sudo ./install.sh --extract=./myrpms/
Extracting components...

Which one of the following would you like?

    1.  Install cluster node software on an automounted shared file system
    2.  Install cluster node software on every node of your cluster
    x.  Exit

Please type a selection (1/2/x) [1]: 1

Illegal option -- extract
$ 
[/plain]

However, it's probably a moot point, since I have the rpm's already (I'm guessing distro changed to include rpms in tar file):

[plain]$ ls rpm
intel-mpi-em64t-3.2.2p-006.x86_64.rpm  intel-mpi-ipf-3.2.2p-006.ia64.rpm         intel-mpi-rt-ia32-3.2.2p-006.i486.rpm
intel-mpi-ia32-3.2.2p-006.i486.rpm     intel-mpi-rt-em64t-3.2.2p-006.x86_64.rpm  intel-mpi-rt-ipf-3.2.2p-006.ia64.rpm
$ 
[/plain]

I'll follow the remainder of your notes on that page. My only question is whether I need to do an install (with dpkg, e.g.) of the runtime component as well. I'm a bit confused that there's an -rt- version and a non-rt version of each rpm. I don't know what that means.

Thanks,
Peter

P.S. note this install's not actually on a cluster. It's on a single SMP machine that I use for code development, before I try build/run on the big machine. Not that that makes any difference.....

mrentropy · ‎12-16-2009

OK I did an install using dpkg of my alien-created intel-mpi-em64t_3.2.2p-7_amd64.deb.

alien generated an error BTW but then seemed to chug along fine and created a .deb nonetheless:

[plain]$ sudo alien --scripts intel-mpi-em64t-3.2.2p-006.x86_64.rpm 
error: incorrect format: unknown tag
intel-mpi-em64t_3.2.2p-7_amd64.deb generated
$[/plain]

I did not try an uninstall, but doing the dpkg should have written over the old install. In fact can verify that this is a new install just doing ls -l to get the modification date:

[plain]$ ls -l `which mpiifort`
-rwxr-xr-x 1 root root 20345 2009-12-16 13:21 /opt/intel/impi/3.2.2.006/bin64/mpiifort
$[/plain]

i.e. the mod date is six minutes ago, when I did the install using dpkg (my clock now says 13:27).

However, problem remains:

[plain]$ mpiifort -c painout.F90
[: 478: no: unexpected operator
$ [/plain]

That might not be so bad, but this is:

[plain]$ mpirun ./gypc
/opt/intel/impi/3.2.2.006/bin64/mpirun: 63: Syntax error: "(" unexpected (expecting "fi")
$ [/plain]

It's possible the gzipped tar was corrupted, although it seems unlikely that corruption would have led to such a subtle error. So I think I'll try installing an earlier release and let you know how that works.

If that fails, my plan is to go home (the functional computer is my home computer), copy everthing in the mpi directory onto a USB drive, and then copy it over to the non-functional system, since they have identical OS's and hardware. (should work, right?)

Thanks,
Peter

Gergana_S_Intel · ‎12-16-2009

Hi Peter,

To reply to your first post:

Quoting - mrentropy

My only question is whether I need to do an install (with dpkg, e.g.) of the runtime component as well. I'm a bit confused that there's an -rt- version and a non-rt version of each rpm. I don't know what that means.

I guess you figured it out but the -rt- version contains the runtimes only. The other RPM should extract to the full Intel MPI Library kit, so you can just use that one. We provide the runtime RPM so those can be redistributed by vendors.

As far as your second install, I'm starting to lean towards some sort of corruption of the scripts during install. It seems like it's unable to parse the code itself:

[plain]$ mpirun ./gypc
/opt/intel/impi/3.2.2.006/bin64/mpirun: 63: Syntax error: "(" unexpected (expecting "fi")
$ [/plain]

You have Python, a Python XML module, and an XML parser installed on this machine, correct?

I don't know if you want to keep playing with it, but I would recommend tossing everything and starting from scratch:

Remove the existing Intel MPI directory by doing "$ rm -rf /opt/intel/impi/3.2.2.006"
Re-download the tar package (it doesn't hurt to try anew) by logging into the Intel Registration Center
Untar, then go into the rpm/ dir, create the .deb file, and try the install again

If it compains at some point about the Intel MPI Library prefix being taken, go ahead and remove a file called intel_sdp_products.db. It should be located in the root home directory, under intel/.

On my side, I'll see if I can get access to an Ubuntu machine here, and try the install as well.

Feel free to call me, if you can't access the package from the Intel Registration Center, or have questions.

Regards,
~Gergana

mrentropy · ‎12-16-2009

Gergana,

Thanks so much for your help!!!!!

Problem solved.

I tried installing old versions, tinkering with this and that.... I learned the hard way that you can't undo packages by just doing an "rm -rf" on the whole directory tree (how do packages work? oh well). Ok several hours blown. But....

What finally worked - going from the hint that the scripts weren't being parsed right - is that I just hand-edited the scripts (mpirun, mpiifort, mpicc, etc etc) to use bash instead of sh:

Change:
#!/bin/sh

to:
#!/bin/bash

...on first line of each script.

This worked and seems to have fixed the problem entirely. I am now able to compile and run with mpirun -n 8 (say) my code, just fine!

Thanks again,
Peter

Gergana_S_Intel · ‎12-16-2009

Quoting - mrentropy

What finally worked - going from the hint that the scripts weren't being parsed right - is that I just hand-edited the scripts (mpirun, mpiifort, mpicc, etc etc) to use bash instead of sh

Ah, the joys of computers :)

I'm glad you have it working, Peter. I wonder if doing a symlink from /bin/sh to /bin/bash would have worked as well? But then you'll be overwriting sh on your system so maybe not ...

Have fun and let us know if you hit other problems.

Regards,
~Gergana

mrentropy · ‎12-16-2009

By the way, although Debian (eg Ubuntu) derived linux isn't supported, I really would like to thank you so much for all of your help.

Hope somebody else finds the info useful. Not sure why sh failed to parse where bash succeeded. Perhaps the fault lies on my end. I'd be curious if any other Ubuntu 9.10 / Intel MPI users find the same problem, or if mine was an isolated incident.

Gergana_S_Intel · ‎12-22-2009

Hi Peter,

I was recently able to get access to a local Ubuntu 9.10 system and ran the installer for the Intel MPI Library. Unsurprisingly, I ran into the same issue as you have:

gsslavov@backuppc:~$ mpirun -np 4 hostname
/home/gsslavov/intel/impi/3.2.2.006/bin/mpirun: 63: Syntax error: "(" unexpected (expecting "fi")

So I did some more digging and this is what I found out:

gsslavov@backuppc:~/intel/impi/3.2.2.006$ ls -l /bin/sh
lrwxrwxrwx 1 root root 4 2009-11-22 03:55 /bin/sh -> dash

It seems that with version 9.10, Ubuntu has decided to change the default shell from bash to dash. More info is available from the link in the excerpt below:

In Ubuntu 6.10, the default system shell, /bin/sh, was changed to dash (the Debian Almquist Shell); previously it had been bash (the GNU Bourne-Again Shell). This document explains this change and what you should do if you encounter problems.

The default login shell remains bash.

As it so happens, the two shells are not fully compatible (since bash is quite more extensive than dash), thus leading to the errors you and I are seeing.

This is actually a fairly easy fix on your side. Simply, you need to change the /bin/sh symlink to point from dash to bash:

$ rm /bin/sh
$ ln -s /bin/bash /bin/sh

This seems to be an acceptable solution based on the Ubuntu article.

I hope this helps. I'll also update the knowledge base article I wrote to include this information.

Happy Holidays,
~Gergana

shawn85 · ‎11-11-2011

Hi,

I am trying to use the Intel MPI for linux on a debian system Ubuntu 10.04, my cpu is running on 32-bit.
ifort, icc etc all work fine. I am using the non-commercial version for the intel compilers.

I downloaded the latest version (4.0.3.008) of the Intel MPI library and untarred it...
In the rpm folder i used 'alien --scripts' to create this file 'intel-mpi-ia32_4.0.3p-9_i386.deb'
An error was shown first: error: incorrect format: unknown tag

Then i used 'dpkg --intsall intel-mpi-ia32_4.0.3p-9_i386.deb'

Selecting previously deselected package intel-mpi-ia32.
(Reading database ... 161071 files and directories currently installed.)
Unpacking intel-mpi-ia32 (from intel-mpi-ia32_4.0.3p-9_i386.deb) ...
Setting up intel-mpi-ia32 (4.0.3p-9) ...

I tried to use mpiifort

mpiifort: command not found

I have no idea what the problem is, hope you can provide some advice

Shawn

TimP · ‎11-11-2011

Assuming ifort and Intel MPI are both correctly installed, you must still set up the environment of each by source-ing both environment scripts, including e.g.
source /opt/intel/impi/4.0.3/bin/mpivars.sh
Intel software install policies require that no MPI path is set up by default, so that it will be possible to select the MPI of your choice by adding it to your paths. You could do that e.g. in .bashrc if that fits your needs.