Intel® MPI Library
Get help with building, analyzing, optimizing, and scaling high-performance computing (HPC) applications.
2154 Discussions

Bug with setvars.sh in latest version (or a subscript)

L__D__Marks
New Contributor II
4,752 Views

I downloaded and installed yesterday the latest base and HPC toolkits. I noticed today a bug in one or more of the scripts. After "source /opt/intel/oneapi/setvars.sh" (using --force does not matter), I am getting for "echo $1 ; echo $2 ; echo $9" (or similar)

advisor=latest
ccl=latest
dpl=latest

 

In other words the values that would be printed out are being exported into my environment, which is obviously not what I want!

0 Kudos
1 Solution
L__D__Marks
New Contributor II
3,990 Views

I did not respond since he clearly did not believe anything that I was reporting.

 

I looked more carefully at the setvars.sh script, and it is a classic example of a "Greedy Algorithm", Quoting from https://en.wikipedia.org/wiki/Greedy_algorithm (similar can be found elsewhere):

 

"A greedy algorithm is any algorithm that follows the problem-solving heuristic of making the locally optimal choice at each stage"

 

Greed can be good, but often also bad. As an illustration, suppose you want a total of 41 cents from a combination of 25, 10 and 4 cent coins. The Greedy approach is to reduce this as much as possible with 25 cents, so we take one leaving 16 cents more needed. Next, we reduce this as much as possible with a 10 cent coin – leaving 6. Then we can only use a single 4 cent; oops, we can only make 39 cents!

 

The different versions of the setvars.sh do not have date or release number information (bad practice!), but quoting from one:

 

# Convert a list of '\n' terminated strings into a format that can be moved
# into the positional arguments array ($@) using the eval "set -- $array_var"

 

This is classic greedy coding, converting arguments into the general $1 $2 etc variables for local convenience, ignoring the possible wider implications of this. Rather than using global variables that might be used for something else, good coding is to use local variables which are not conflicting with anything else.

 

My patch for what I consider to be poor coding is to myself save then restore the variables that the Intel code should not be using, i.e.

 

var="$@"
source /opt/intel/oneapi/setvars.sh ...
eval set -- $var

View solution in original post

0 Kudos
33 Replies
ShivaniK_Intel
Moderator
3,325 Views

Hi,


Thanks for posting in the Intel forums.


Could you please provide us with the details of the OS you have been using?


Could you also please provide the steps you have followed to reproduce the error ?


Thanks & Regards

Shivani



0 Kudos
L__D__Marks
New Contributor II
3,320 Views

Standard bash/linux

[ldm@head PtF]$ bash --version
GNU bash, version 4.2.46(2)-release (x86_64-redhat-linux-gnu)

The steps are in the original email -- conventional source of the file, nothing Nivel.

0 Kudos
L__D__Marks
New Contributor II
3,317 Views

Nothing novel ...

0 Kudos
L__D__Marks
New Contributor II
3,281 Views

For reference, the "cure" (which I don't like) is to use in .bashrc

 

var="$@"
source /opt/intel/oneapi/setvars.sh --force > /dev/null
eval set -- $var

 

This then over-rides the incorrect setting of $1 $2 etc by the setvars.sh.

 

N.B. I tried to find where these where being set, but failed; my bash is not good enough.

N.N.B., Before someone asks, the "--force" is needed as some commands do not propogate environments and .bashrc has to be sourced a second time for safety. So it goes.

0 Kudos
ShivaniK_Intel
Moderator
3,269 Views

Hi,


Could you please let us know whether you want to use any specific component of Intel oneAPI or else all the components of Intel oneAPI ?



>>> " I noticed today a bug in one or more of the scripts"


Could you please let us know the bug you are referring to?



>>> "In other words, the values that would be printed out are being exported into my environment, which is obviously not what I want!"


When we execute setvars script, it would configure the local environment variables to reflect the needs of the installed Intel oneAPI 

development tools.


Thanks & Regards

Shivani


0 Kudos
L__D__Marks
New Contributor II
3,260 Views

Please read my previous email, where your question is already answered!

 

As stated previously, the setvars.sh script sets $1, $2...$9. It should not do this. My patch resets these but should not be needed. There is an error in the script.

 

For reference, I already found one page on the Internet which reported the same bug, there are probably others.

 

0 Kudos
PaulF_IntelCorp
Employee
3,205 Views

I recommend that you not source setvars.sh from within your .bashrc file. The `setvars.sh` and `env/varsh.sh` scripts are not designed for use in your shell initialization and login scripts. It should be run only after your main environment has been setup. If you want to run `setvars.sh` easily, you could define a short alias that sources `setvars.sh`, so you don't have to type the full path to `setvars.sh` each time you want to use the oneAPI tools.

The issue you are referring to was fixed ~2 years ago. I suspect the problem you are now seeing is due to some error messages that are being emitted by the pytorch and tensorflow components (assuming you have installed those components) which may be terminating the `setvars.sh` script prematurely within your `.bashrc` script. The `setvars.sh` script is designed to restore the original argument array, but that only works if it is able to exit normally. Since you are redirecting the output of `setvars.sh` to `/dev/null` you may not be seeing those error messages. If `setvars.sh` does terminate properly, it will restore the arguments array, but in your `.bashrc` context that may not be happening.

0 Kudos
L__D__Marks
New Contributor II
3,197 Views

No

To repeat, No.

 

1) This has nothing to do with the redirection, that is just for a cleaner output during login. It occurs without the redirection.

2) The script should be sourced in .bashrc to appropriately setup LD_LIBARARY_PATH and PATH. This, or some "module load ..." is needed.

3) There are no reported errors, including without the redirect.

4) Using the full path or not is user preference, that is not relevant to anything.

 

You may have cured this bug 2 years ago. However, as all coders know, bugs commonly reoccur due to operator error.

 

The bug is real. Sometimes the unwashed user is right.

0 Kudos
PaulF_IntelCorp
Employee
3,190 Views

Can you provide a copy of the output of setvars.sh and post it here? That is, remove the /dev/null redirection and copy the complete output text of setvars.sh. It should end with the phrase ":: oneAPI environment initialized ::" which should result in executing the restoration of the argument array. This is what I'm referring to at the end of the setvars.sh script:

echo ":: oneAPI environment initialized ::"
echo " "

if [ "$warning_tally" -ne 0 ] ; then
  echo ":: $warning_tally warnings issued: review warning messages."
  echo "   Possible incomplete environment initialization."
  # Restore original $@ array before return.
  eval set -- "$script_args" || true
  prep_for_exit 250 ; return
else
  # Restore original $@ array before return.
  eval set -- "$script_args" || true
  prep_for_exit 0 ; return
fi

And if you could perform the following immediately before the call to setvars.sh and immediately after it, in your .bashrc and include that output:

echo "\$# = " "$#"
echo "\$@ = " "$@"
0 Kudos
L__D__Marks
New Contributor II
3,182 Views

[ldm@head ~]$ echo "\$# = " "$#" echo "\$@ = " "$@"
$# = 0 echo $@ =
[ldm@head ~]$ source /opt/intel/oneapi/setvars.sh --force

:: initializing oneAPI environment ...
-bash: BASH_VERSION = 4.2.46(2)-release
args: Using "$@" for setvars.sh arguments: --force
:: advisor -- latest
:: ccl -- latest
:: clck -- latest
:: compiler -- latest
:: dal -- latest
:: debugger -- latest
:: dev-utilities -- latest
:: dpcpp-ct -- latest
:: dpl -- latest
:: inspector -- latest
:: intelpython -- latest
:: ipp -- latest
:: ippcp -- latest
:: itac -- latest
:: mkl -- latest
:: mpi -- latest
:: tbb -- latest
:: vtune -- latest
:: oneAPI environment initialized ::

[ldm@head ~]$ echo "\$# = " "$#" echo "\$@ = " "$@"
$# = 18 echo $@ = advisor=latest ccl=latest clck=latest compiler=latest dal=latest debugger=latest dev-utilities=latest dpcpp-ct=latest dpl=latest inspector=latest intelpython=latest ipp=latest ippcp=latest itac=latest mkl=latest mpi=latest tbb=latest vtune=latest

 

This is what I said before.

0 Kudos
L__D__Marks
New Contributor II
3,181 Views

Addendum.

 

In I use a configuration file with intelphython=exclude, i.e. 

 

source /opt/intel/oneapi/setvars.sh --force --config="$HOME/bin/oapi.txt"

 

The problem goes away. This localizes the issue

0 Kudos
PaulF_IntelCorp
Employee
3,173 Views

Excluding intelpython is interesting. It has a very complex initialization env/vars.sh script. However, I'm still not able to reproduce what you are seeing even though in my case I'm including intelpython.

My instructions were unclear. I meant to have you perform the test from within your .bashrc file, not at the Bash prompt. In other words, something like this inside your .bashrc file:

 

<other .bashrc statements>

echo "\$# = " "$#"
echo "\$@ = " "$@"

. /opt/intel/oneapi/setvars.sh

echo "\$# = " "$#"
echo "\$@ = " "$@"

<other .bashrc statements>

 

where the <other .bashrc statements> represent everything else that is already inside your .bashrc file.

And then paste the results here.

0 Kudos
PaulF_IntelCorp
Employee
3,173 Views

p.s. Don't copy and paste the suggested text from the email you get from this forum. It is converting multiple lines into a single line. Go to the forum page to see the properly formatted text.

0 Kudos
L__D__Marks
New Contributor II
3,166 Views

It does not matter if I do it in .bashrc or from the shell -- which it should not.

Also, of course I did not do a silly paste, I have some experience with coding (20+ years).

[ldm@head ~]$ bash --version
GNU bash, version 4.2.46(2)-release (x86_64-redhat-linux-gnu)

[ldm@head ~]$ uname -a
Linux head.cluster 3.10.0-1160.11.1.el7.x86_64 #1 SMP Fri Dec 18 16:34:56 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

 

[ldm@head ~]$ head -5 /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 85
model name : Intel(R) Xeon(R) Silver 4114 CPU @ 2.20GHz

0 Kudos
PaulF_IntelCorp
Employee
3,128 Views

I am still unable to reproduce this issue.

Is `setvars.sh` exiting with a non-zero exit code? If so, what is the exit code number?

Have you redefined the `cd` command with an alias?

0 Kudos
L__D__Marks
New Contributor II
3,120 Views
To repeat, there is no exit code.

No, I have not aliased cd. I suspect you have a non-standard bash or similar.
0 Kudos
L__D__Marks
New Contributor II
3,120 Views
I suggest that you supply a version of the intelpython setvars (or whatever) with debugging or a stack of embedded debug writes. (Standard when one knows nothing more than that a code is broken.)
0 Kudos
PaulF_IntelCorp
Employee
2,826 Views

You indicated that you have installed the latest version of the Base Kit and HPC Kit. Since you isolated the issue to the "intelpython" component, could you replace the following "vars.sh" script in your installation:

/opt/intel/oneapi/intelpython/python3.9/env/vars.sh

with the attached "vars.sh" script (inside the "vars.tar" file -- forum does not allow me to attach an sh file directly). This replacement includes a few simple additional lines, in a few key locations within the intelpython vars.sh script that might provide a sense as to what is going on. Obviously, you'll have to run without the setvars config file so the intelpython vars.sh script is called by setvars.sh and remove the output redirection to /dev/null.

The output should look something like the following:

$ . /opt/intel/oneapi/setvars.sh

:: initializing oneAPI environment ...
   bash: BASH_VERSION = 4.4.20(1)-release
   args: Using "$@" for setvars.sh arguments:
:: advisor -- latest
:: ccl -- latest
:: compiler -- latest
:: dal -- latest
:: debugger -- latest
:: dev-utilities -- latest
:: dnnl -- latest
:: dpcpp-ct -- latest
:: dpl -- latest
:: intelpython -- latest
LINENO: 16
SETVARS_CALL=1
LINENO: 201
SETVARS_CALL=1
LINENO: 228
SETVARS_CALL=1
LINENO: 312
SETVARS_CALL=1
:: ipp -- latest
:: ippcp -- latest
:: ipp -- latest
:: mkl -- latest
:: mpi -- latest
:: tbb -- latest
:: vpl -- latest
:: vtune -- latest
:: oneAPI environment initialized ::

The test above was run on an Ubuntu 18.04 Desktop system with just the Base Kit installed. As you can see from the script output, the shell is Bash version 4.4. No arguments were provided to the setvars.sh script.

0 Kudos
ShivaniK_Intel
Moderator
2,702 Views


Hi,


As we didn't hear back from you, Could you please let us know if your issue is resolved?


Thanks & Regards

Shivani


0 Kudos
L__D__Marks
New Contributor II
2,695 Views

The issue is not resolved.

 

The file that it was suggested I try was filtered out by the web based system, so the suggestion to use it was not useful.

0 Kudos
Reply