Bug with setvars.sh in latest version (or a subscript)

[L__D__Marks]

I downloaded and installed yesterday the latest base and HPC toolkits. I noticed today a bug in one or more of the scripts. After "source /opt/intel/oneapi/setvars.sh" (using --force does not matter), I am getting for "echo $1 ; echo $2 ; echo $9" (or similar)

advisor=latest
ccl=latest
dpl=latest

In other words the values that would be printed out are being exported into my environment, which is obviously not what I want!

[ShivaniK_Intel]

Hi,

Thanks for posting in the Intel forums.

Could you please provide us with the details of the OS you have been using?

Could you also please provide the steps you have followed to reproduce the error ?

Thanks & Regards

Shivani

[L__D__Marks]

Standard bash/linux

[ldm@head PtF]$ bash --version
GNU bash, version 4.2.46(2)-release (x86_64-redhat-linux-gnu)

The steps are in the original email -- conventional source of the file, nothing Nivel.

[L__D__Marks]

Nothing novel ...

[L__D__Marks]

For reference, the "cure" (which I don't like) is to use in .bashrc

var="$@"
source /opt/intel/oneapi/setvars.sh --force > /dev/null
eval set -- $var

This then over-rides the incorrect setting of $1 $2 etc by the setvars.sh.

N.B. I tried to find where these where being set, but failed; my bash is not good enough.

N.N.B., Before someone asks, the "--force" is needed as some commands do not propogate environments and .bashrc has to be sourced a second time for safety. So it goes.

[ShivaniK_Intel]

Hi,

Could you please let us know whether you want to use any specific component of Intel oneAPI or else all the components of Intel oneAPI ?

>>> " I noticed today a bug in one or more of the scripts"

Could you please let us know the bug you are referring to?

>>> "In other words, the values that would be printed out are being exported into my environment, which is obviously not what I want!"

When we execute setvars script, it would configure the local environment variables to reflect the needs of the installed Intel oneAPI 

development tools.

Thanks & Regards

Shivani

[L__D__Marks]

Please read my previous email, where your question is already answered!

As stated previously, the setvars.sh script sets $1, $2...$9. It should not do this. My patch resets these but should not be needed. There is an error in the script.

For reference, I already found one page on the Internet which reported the same bug, there are probably others.

[PaulF_IntelCorp]

I recommend that you not source setvars.sh from within your .bashrc file. The `setvars.sh` and `env/varsh.sh` scripts are not designed for use in your shell initialization and login scripts. It should be run only after your main environment has been setup. If you want to run `setvars.sh` easily, you could define a short alias that sources `setvars.sh`, so you don't have to type the full path to `setvars.sh` each time you want to use the oneAPI tools.

The issue you are referring to was fixed ~2 years ago. I suspect the problem you are now seeing is due to some error messages that are being emitted by the pytorch and tensorflow components (assuming you have installed those components) which may be terminating the `setvars.sh` script prematurely within your `.bashrc` script. The `setvars.sh` script is designed to restore the original argument array, but that only works if it is able to exit normally. Since you are redirecting the output of `setvars.sh` to `/dev/null` you may not be seeing those error messages. If `setvars.sh` does terminate properly, it will restore the arguments array, but in your `.bashrc` context that may not be happening.

[L__D__Marks]

No

To repeat, No.

1) This has nothing to do with the redirection, that is just for a cleaner output during login. It occurs without the redirection.

2) The script should be sourced in .bashrc to appropriately setup LD_LIBARARY_PATH and PATH. This, or some "module load ..." is needed.

3) There are no reported errors, including without the redirect.

4) Using the full path or not is user preference, that is not relevant to anything.

You may have cured this bug 2 years ago. However, as all coders know, bugs commonly reoccur due to operator error.

The bug is real. Sometimes the unwashed user is right.

[PaulF_IntelCorp]

Can you provide a copy of the output of setvars.sh and post it here? That is, remove the /dev/null redirection and copy the complete output text of setvars.sh. It should end with the phrase ":: oneAPI environment initialized ::" which should result in executing the restoration of the argument array. This is what I'm referring to at the end of the setvars.sh script:

echo ":: oneAPI environment initialized ::"
echo " "

if [ "$warning_tally" -ne 0 ] ; then
  echo ":: $warning_tally warnings issued: review warning messages."
  echo "   Possible incomplete environment initialization."
  # Restore original $@ array before return.
  eval set -- "$script_args" || true
  prep_for_exit 250 ; return
else
  # Restore original $@ array before return.
  eval set -- "$script_args" || true
  prep_for_exit 0 ; return
fi

And if you could perform the following immediately before the call to setvars.sh and immediately after it, in your .bashrc and include that output:

echo "\$# = " "$#"
echo "\$@ = " "$@"

[L__D__Marks]

[ldm@head ~]$ echo "\$# = " "$#" echo "\$@ = " "$@"
$# = 0 echo $@ =
[ldm@head ~]$ source /opt/intel/oneapi/setvars.sh --force

:: initializing oneAPI environment ...
-bash: BASH_VERSION = 4.2.46(2)-release
args: Using "$@" for setvars.sh arguments: --force
:: advisor -- latest
:: ccl -- latest
:: clck -- latest
:: compiler -- latest
:: dal -- latest
:: debugger -- latest
:: dev-utilities -- latest
:: dpcpp-ct -- latest
:: dpl -- latest
:: inspector -- latest
:: intelpython -- latest
:: ipp -- latest
:: ippcp -- latest
:: itac -- latest
:: mkl -- latest
:: mpi -- latest
:: tbb -- latest
:: vtune -- latest
:: oneAPI environment initialized ::

[ldm@head ~]$ echo "\$# = " "$#" echo "\$@ = " "$@"
$# = 18 echo $@ = advisor=latest ccl=latest clck=latest compiler=latest dal=latest debugger=latest dev-utilities=latest dpcpp-ct=latest dpl=latest inspector=latest intelpython=latest ipp=latest ippcp=latest itac=latest mkl=latest mpi=latest tbb=latest vtune=latest

This is what I said before.

[L__D__Marks]

Addendum.

In I use a configuration file with intelphython=exclude, i.e. 

source /opt/intel/oneapi/setvars.sh --force --config="$HOME/bin/oapi.txt"

The problem goes away. This localizes the issue

[PaulF_IntelCorp]

Excluding intelpython is interesting. It has a very complex initialization env/vars.sh script. However, I'm still not able to reproduce what you are seeing even though in my case I'm including intelpython.

My instructions were unclear. I meant to have you perform the test from within your .bashrc file, not at the Bash prompt. In other words, something like this inside your .bashrc file:

<other .bashrc statements>

echo "\$# = " "$#"
echo "\$@ = " "$@"

. /opt/intel/oneapi/setvars.sh

echo "\$# = " "$#"
echo "\$@ = " "$@"

<other .bashrc statements>

where the <other .bashrc statements> represent everything else that is already inside your .bashrc file.

And then paste the results here.

[PaulF_IntelCorp]

p.s. Don't copy and paste the suggested text from the email you get from this forum. It is converting multiple lines into a single line. Go to the forum page to see the properly formatted text.

[L__D__Marks]

It does not matter if I do it in .bashrc or from the shell -- which it should not.

Also, of course I did not do a silly paste, I have some experience with coding (20+ years).

[ldm@head ~]$ bash --version
GNU bash, version 4.2.46(2)-release (x86_64-redhat-linux-gnu)

[ldm@head ~]$ uname -a
Linux head.cluster 3.10.0-1160.11.1.el7.x86_64 #1 SMP Fri Dec 18 16:34:56 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

[ldm@head ~]$ head -5 /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 85
model name : Intel(R) Xeon(R) Silver 4114 CPU @ 2.20GHz

[PaulF_IntelCorp]

I am still unable to reproduce this issue.

Is `setvars.sh` exiting with a non-zero exit code? If so, what is the exit code number?

Have you redefined the `cd` command with an alias?

[L__D__Marks]

To repeat, there is no exit code.

No, I have not aliased cd. I suspect you have a non-standard bash or similar.

[L__D__Marks]

I suggest that you supply a version of the intelpython setvars (or whatever) with debugging or a stack of embedded debug writes. (Standard when one knows nothing more than that a code is broken.)

[PaulF_IntelCorp]

You indicated that you have installed the latest version of the Base Kit and HPC Kit. Since you isolated the issue to the "intelpython" component, could you replace the following "vars.sh" script in your installation:

/opt/intel/oneapi/intelpython/python3.9/env/vars.sh

with the attached "vars.sh" script (inside the "vars.tar" file -- forum does not allow me to attach an sh file directly). This replacement includes a few simple additional lines, in a few key locations within the intelpython vars.sh script that might provide a sense as to what is going on. Obviously, you'll have to run without the setvars config file so the intelpython vars.sh script is called by setvars.sh and remove the output redirection to /dev/null.

The output should look something like the following:

$ . /opt/intel/oneapi/setvars.sh

:: initializing oneAPI environment ...
   bash: BASH_VERSION = 4.4.20(1)-release
   args: Using "$@" for setvars.sh arguments:
:: advisor -- latest
:: ccl -- latest
:: compiler -- latest
:: dal -- latest
:: debugger -- latest
:: dev-utilities -- latest
:: dnnl -- latest
:: dpcpp-ct -- latest
:: dpl -- latest
:: intelpython -- latest
LINENO: 16
SETVARS_CALL=1
LINENO: 201
SETVARS_CALL=1
LINENO: 228
SETVARS_CALL=1
LINENO: 312
SETVARS_CALL=1
:: ipp -- latest
:: ippcp -- latest
:: ipp -- latest
:: mkl -- latest
:: mpi -- latest
:: tbb -- latest
:: vpl -- latest
:: vtune -- latest
:: oneAPI environment initialized ::

The test above was run on an Ubuntu 18.04 Desktop system with just the Base Kit installed. As you can see from the script output, the shell is Bash version 4.4. No arguments were provided to the setvars.sh script.

[ShivaniK_Intel]

Hi,

As we didn't hear back from you, Could you please let us know if your issue is resolved?

Thanks & Regards

Shivani

[L__D__Marks]

The issue is not resolved.

The file that it was suggested I try was filtered out by the web based system, so the suggestion to use it was not useful.