Sourcing setvars.sh in singularity container

[yihu3230]

Hi all,

I would like to configure the singularity container such that the singularity image file will automatically run intel one api.

I successfully installed intel oneAPI in the singularity sandbox container. Within the container, I can use . /opt/intel/oneapi/setvars.sh to set the variables correctly and compile my program.

However, I want to configure the singularity sandbox container such that the singularity image file will automatically run the setvars.sh so I can use the image file and run my program directly from my host system.

According to the singularity documentation

https://sylabs.io/guides/3.6/user-guide/environment_and_metadata.html

I think what I need to do is to put the setvars.sh file in the /.singularity.d/env/ directory.

After I did that, when I start the container, the shell returns the message

"source: /.singularity.d/env/99-vars_mkl.sh:137:59: parameter expansion requires a literal"

Does anybody know how to resolve the issue? How to let singularity image run the setvars.sh directly?

Thanks!

[ShivaniK_Intel]

Hi,

Thanks for reaching out to us.

Could you please let us know from where you have taken the singularity image? If you are referring to the below GitHub link you can follow the below steps.

https://github.com/intel/oneapi-containers/tree/master/images/singularity

1.git clone https://github.com/intel/oneapi-containers/tree/master/images/singularity

2.cd basekit-devel-ubuntu18.04

3.simg="intel-oneapi-basekit-devel-ubuntu18.04.simg"

4.sudo singularity build "$simg" Singularity

5.singularity shell "$simg"

To check whether setvars.sh ran automatically or not you can check any compiler version.

For example:

dpcpp --version

If you still face any issues please let us know and provide steps to reproduce them.

Thanks & Regards

Shivani

[yihu3230]

Hi Shivani,

Thanks for the suggestion.

I am trying to build ubuntu20.04 by myself instead of pulling the image from the above Github. In principle, I think there should be a solution if I try to build different versions of ubuntu images and try to install intel one API on it. Are there solutions like that? I need to use other packages from 20.04. It will be ideal if I know how to configure intel one API properly in the singularity image.

Thanks.

[ShivaniK_Intel]

Hi,

We are working on it and will get back to you soon.

Thanks & Regards

Shivani

[cw_intel]

Thanks for contacting Intel® Support. We are working on it, and will let you know when we get the solutions.

[mija]

Hi,

OneAPI is not usable with Singularity when using package manager installation. At some point you will need to source the variables:

Example:

...

%post

yum install -y intel-hpckit-runtime-2021.3.0.x86_64

%test

source /opt/intel/oneapi/mpi/latest/env/vars.sh

%environment
source /opt/intel/oneapi/mpi/latest/env/vars.sh

%runscript
source /opt/intel/oneapi/mpi/latest/env/vars.sh

%startscript
source /opt/intel/oneapi/mpi/latest/env/vars.sh

...

But this leads to

source: /opt/intel/oneapi/setvars.sh:297:53: parameter expansion requires a literal

I could go through all envs and add the envvars manually, but this is annoying, would blow up the def file and could change in the feature, so that I would need to do that again.

Best,

Michael

[mija]

Sorry wrong def file:

%test

source /opt/intel/oneapi/setvars.sh

%environment

source /opt/intel/oneapi/setvars.sh

[mija]

Hi,

I could generate a working Singularity def file. Sourcing in %env is not possible (not sure if this came with Singularity v3.6). But in %run it works. I'm not sure if the problems was gone after I installed some extra packages or if it was a different problem I had before.

This works for Centos8:

Bootstrap: library
From: centos:8
OSVersion: 8

%post
    # some defaults for development
    yum install -y epel-release
    yum -y update
    yum -y install dnf-plugins-core
    yum config-manager --set-enabled powertools
    yum repolist
    yum group install -y --with-optional standard
    yum group install -y --with-optional development
    yum install -y redhat-lsb
    yum install -y yum-utils Lmod qperf
    # Infiniband, RDMA, etc.
    yum install -y numactl hwloc hwloc-devel rdma-core-devel infiniband-diags
    # optional: Omni-Path
    yum install -y opa-fastfabric opa-basic-tools
    # optional: UCX
    yum install -y ucx ucx-ib
    # Intel OneAPI repo
        tee > /tmp/oneAPI.repo << EOF
[oneAPI]
name=Intel(R) oneAPI repository
baseurl=https://yum.repos.intel.com/oneapi
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://yum.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB
EOF
    mv /tmp/oneAPI.repo /etc/yum.repos.d/
    rpm --import https://yum.repos.intel.com/intel-gpg-keys/GPG-PUB-KEY-INTEL-SW-PRODUCTS.PUB
    # Intel OneAPI Runtime 2021.3.0
    yum install -y intel-hpckit-runtime-2021.3.0.x86_64
    # optional: set modulefiles
    /opt/intel/oneapi/modulefiles-setup.sh --ignore-latest --force --output-dir=/etc/modulefiles

%test
    source /opt/intel/oneapi/setvars.sh
    mpirun hostname

%runscript
    source /opt/intel/oneapi/setvars.sh >/dev/null
    exec "$@"

%startscript
    source /opt/intel/oneapi/setvars.sh
    # add start script here

%labels
    Author mija
    Software Intel OneAPI Runtime
    Version 2021.3.0

%files

%environment
# sourcing in env not possible

%help
    Intel OneAPI Runtime 2021.3.0

[ShivaniK_Intel]

 Hi,

As we didn't hear back from you, could you please let us know whether your issue is resolved?

Thanks & Regards

Shivani

[ShivaniK_Intel]

Hi,

We have observed that the issue is resolved in CentOS 8. As we didn't hear back any update from you regarding Ubuntu 20.04, Intel will no longer monitor this thread. If you need further assistance, please post a new question. 

Thanks & Regards

Shivani

[PatL]

Probably related issue: Singularity %appenv "source: setvars.sh:230:17: parameter expansion requires a literal" - Intel Communities

In our case, when we put ". /opt/intel/oneapi/setvars.sh" into the "%environment" section of our Singularity definition file, the resulting container gives this error on startup:

source: /opt/intel/oneapi/setvars.sh:323:53: parameter expansion requires a literal

The problem is that bash is evaluating some zsh-specific code even though it never executes. I am not sure why; nested "source" calls maybe?

In any event, here is our work-around:

%environment

/bin/sh -c '. /opt/intel/oneapi/setvars.sh --ccl-configuration=cpu >/dev/null ; printenv' \
    | while read line ; do
    [ "${line#SINGULARITY_}" = "$line" ] || continue
    eval "export $line"
done

unset line

This will break if any environment variable has a weird character (like space or dollar sign) in its name or value. I am too lazy to fix this at the moment.