I downloaded and installed yesterday the latest base and HPC toolkits. I noticed today a bug in one or more of the scripts. After "source /opt/intel/oneapi/setvars.sh" (using --force does not matter), I am getting for "echo $1 ; echo $2 ; echo $9" (or similar)
In other words the values that would be printed out are being exported into my environment, which is obviously not what I want!
I did not respond since he clearly did not believe anything that I was reporting.
I looked more carefully at the setvars.sh script, and it is a classic example of a "Greedy Algorithm", Quoting from https://en.wikipedia.org/wiki/Greedy_algorithm (similar can be found elsewhere):
"A greedy algorithm is any algorithm that follows the problem-solving heuristic of making the locally optimal choice at each stage"
Greed can be good, but often also bad. As an illustration, suppose you want a total of 41 cents from a combination of 25, 10 and 4 cent coins. The Greedy approach is to reduce this as much as possible with 25 cents, so we take one leaving 16 cents more needed. Next, we reduce this as much as possible with a 10 cent coin – leaving 6. Then we can only use a single 4 cent; oops, we can only make 39 cents!
The different versions of the setvars.sh do not have date or release number information (bad practice!), but quoting from one:
# Convert a list of '\n' terminated strings into a format that can be moved
# into the positional arguments array ($@) using the eval "set -- $array_var"
This is classic greedy coding, converting arguments into the general $1 $2 etc variables for local convenience, ignoring the possible wider implications of this. Rather than using global variables that might be used for something else, good coding is to use local variables which are not conflicting with anything else.
My patch for what I consider to be poor coding is to myself save then restore the variables that the Intel code should not be using, i.e.
source /opt/intel/oneapi/setvars.sh ...
eval set -- $var
Thanks for posting in the Intel forums.
Could you please provide us with the details of the OS you have been using?
Could you also please provide the steps you have followed to reproduce the error ?
Thanks & Regards
For reference, the "cure" (which I don't like) is to use in .bashrc
source /opt/intel/oneapi/setvars.sh --force > /dev/null
eval set -- $var
This then over-rides the incorrect setting of $1 $2 etc by the setvars.sh.
N.B. I tried to find where these where being set, but failed; my bash is not good enough.
N.N.B., Before someone asks, the "--force" is needed as some commands do not propogate environments and .bashrc has to be sourced a second time for safety. So it goes.
Could you please let us know whether you want to use any specific component of Intel oneAPI or else all the components of Intel oneAPI ?
>>> " I noticed today a bug in one or more of the scripts"
Could you please let us know the bug you are referring to?
>>> "In other words, the values that would be printed out are being exported into my environment, which is obviously not what I want!"
When we execute setvars script, it would configure the local environment variables to reflect the needs of the installed Intel oneAPI
Thanks & Regards
Please read my previous email, where your question is already answered!
As stated previously, the setvars.sh script sets $1, $2...$9. It should not do this. My patch resets these but should not be needed. There is an error in the script.
For reference, I already found one page on the Internet which reported the same bug, there are probably others.
I recommend that you not source setvars.sh from within your .bashrc file. The `setvars.sh` and `env/varsh.sh` scripts are not designed for use in your shell initialization and login scripts. It should be run only after your main environment has been setup. If you want to run `setvars.sh` easily, you could define a short alias that sources `setvars.sh`, so you don't have to type the full path to `setvars.sh` each time you want to use the oneAPI tools.
The issue you are referring to was fixed ~2 years ago. I suspect the problem you are now seeing is due to some error messages that are being emitted by the pytorch and tensorflow components (assuming you have installed those components) which may be terminating the `setvars.sh` script prematurely within your `.bashrc` script. The `setvars.sh` script is designed to restore the original argument array, but that only works if it is able to exit normally. Since you are redirecting the output of `setvars.sh` to `/dev/null` you may not be seeing those error messages. If `setvars.sh` does terminate properly, it will restore the arguments array, but in your `.bashrc` context that may not be happening.
To repeat, No.
1) This has nothing to do with the redirection, that is just for a cleaner output during login. It occurs without the redirection.
2) The script should be sourced in .bashrc to appropriately setup LD_LIBARARY_PATH and PATH. This, or some "module load ..." is needed.
3) There are no reported errors, including without the redirect.
4) Using the full path or not is user preference, that is not relevant to anything.
You may have cured this bug 2 years ago. However, as all coders know, bugs commonly reoccur due to operator error.
The bug is real. Sometimes the unwashed user is right.
Can you provide a copy of the output of setvars.sh and post it here? That is, remove the /dev/null redirection and copy the complete output text of setvars.sh. It should end with the phrase ":: oneAPI environment initialized ::" which should result in executing the restoration of the argument array. This is what I'm referring to at the end of the setvars.sh script:
echo ":: oneAPI environment initialized ::" echo " " if [ "$warning_tally" -ne 0 ] ; then echo ":: $warning_tally warnings issued: review warning messages." echo " Possible incomplete environment initialization." # Restore original $@ array before return. eval set -- "$script_args" || true prep_for_exit 250 ; return else # Restore original $@ array before return. eval set -- "$script_args" || true prep_for_exit 0 ; return fi
And if you could perform the following immediately before the call to setvars.sh and immediately after it, in your .bashrc and include that output:
echo "\$# = " "$#" echo "\$@ = " "$@"
[ldm@head ~]$ echo "\$# = " "$#" echo "\$@ = " "$@"
$# = 0 echo $@ =
[ldm@head ~]$ source /opt/intel/oneapi/setvars.sh --force
:: initializing oneAPI environment ...
-bash: BASH_VERSION = 4.2.46(2)-release
args: Using "$@" for setvars.sh arguments: --force
:: advisor -- latest
:: ccl -- latest
:: clck -- latest
:: compiler -- latest
:: dal -- latest
:: debugger -- latest
:: dev-utilities -- latest
:: dpcpp-ct -- latest
:: dpl -- latest
:: inspector -- latest
:: intelpython -- latest
:: ipp -- latest
:: ippcp -- latest
:: itac -- latest
:: mkl -- latest
:: mpi -- latest
:: tbb -- latest
:: vtune -- latest
:: oneAPI environment initialized ::
[ldm@head ~]$ echo "\$# = " "$#" echo "\$@ = " "$@"
$# = 18 echo $@ = advisor=latest ccl=latest clck=latest compiler=latest dal=latest debugger=latest dev-utilities=latest dpcpp-ct=latest dpl=latest inspector=latest intelpython=latest ipp=latest ippcp=latest itac=latest mkl=latest mpi=latest tbb=latest vtune=latest
This is what I said before.
Excluding intelpython is interesting. It has a very complex initialization env/vars.sh script. However, I'm still not able to reproduce what you are seeing even though in my case I'm including intelpython.
My instructions were unclear. I meant to have you perform the test from within your .bashrc file, not at the Bash prompt. In other words, something like this inside your .bashrc file:
<other .bashrc statements> echo "\$# = " "$#" echo "\$@ = " "$@" . /opt/intel/oneapi/setvars.sh echo "\$# = " "$#" echo "\$@ = " "$@" <other .bashrc statements>
where the <other .bashrc statements> represent everything else that is already inside your .bashrc file.
And then paste the results here.
It does not matter if I do it in .bashrc or from the shell -- which it should not.
Also, of course I did not do a silly paste, I have some experience with coding (20+ years).
[ldm@head ~]$ bash --version
GNU bash, version 4.2.46(2)-release (x86_64-redhat-linux-gnu)
[ldm@head ~]$ uname -a
Linux head.cluster 3.10.0-1160.11.1.el7.x86_64 #1 SMP Fri Dec 18 16:34:56 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
[ldm@head ~]$ head -5 /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 85
model name : Intel(R) Xeon(R) Silver 4114 CPU @ 2.20GHz
You indicated that you have installed the latest version of the Base Kit and HPC Kit. Since you isolated the issue to the "intelpython" component, could you replace the following "vars.sh" script in your installation:
with the attached "vars.sh" script (inside the "vars.tar" file -- forum does not allow me to attach an sh file directly). This replacement includes a few simple additional lines, in a few key locations within the intelpython vars.sh script that might provide a sense as to what is going on. Obviously, you'll have to run without the setvars config file so the intelpython vars.sh script is called by setvars.sh and remove the output redirection to /dev/null.
The output should look something like the following:
$ . /opt/intel/oneapi/setvars.sh :: initializing oneAPI environment ... bash: BASH_VERSION = 4.4.20(1)-release args: Using "$@" for setvars.sh arguments: :: advisor -- latest :: ccl -- latest :: compiler -- latest :: dal -- latest :: debugger -- latest :: dev-utilities -- latest :: dnnl -- latest :: dpcpp-ct -- latest :: dpl -- latest :: intelpython -- latest LINENO: 16 SETVARS_CALL=1 LINENO: 201 SETVARS_CALL=1 LINENO: 228 SETVARS_CALL=1 LINENO: 312 SETVARS_CALL=1 :: ipp -- latest :: ippcp -- latest :: ipp -- latest :: mkl -- latest :: mpi -- latest :: tbb -- latest :: vpl -- latest :: vtune -- latest :: oneAPI environment initialized ::
The test above was run on an Ubuntu 18.04 Desktop system with just the Base Kit installed. As you can see from the script output, the shell is Bash version 4.4. No arguments were provided to the setvars.sh script.