How to submit job script with different parameters, from another bash script on Intel devCloud

[Saharsa]

Hello,

I tried to read my file, the device (GPU or CPU), and the size of my vector from command line and then with respect to the chosen device, submit my job script with that parameters in to the queue of specific GPU or CPU. But after submitting my job to the queue, when I validate my script like:

bash job.sh

I saw that my parameters file, device, and size from submitjob.sh script did not pass to the job.sh script.

Could you please help me what is my mistake?!

My submitjob.sh:

#!/bin/sh -f
echo "Input The File!";
read file
echo "Choose Gpu Or Cpu!";
read device
echo "Input Vector Size!";
read size
echo
if [ "$device" = "Gpu" ]; then
qsub -I -l nodes=1:gen9:ppn=2 -d . ./job.sh file device size
else
qsub -I -l nodes=1:skl:ppn=2 -d . ./job.sh file device size
fi

-------

my job.sh:

#!/bin/bash

source /opt/intel/inteloneapi/setvars.sh

echo
echo start: $(date "+%y%m%d.%H%M%S.%3N")
echo
$file $device $size
echo
echo stop: $(date "+%y%m%d.%H%M%S.%3N")
echo

[RashmiP_Intel]

Hi Saharsa,

Thanks for reaching to Devcloud Support forum.

 I am mentioning here 2 options that could be useful for you:-

-v Paramater: Expands the list of environment variables that are exported to the job.

qsub -v file=$file,device=$device,size=$size -I -l nodes=1:gen9:ppn=2 -d . ./job.sh

-V Paramater: Declares that all environment variables in the qsub commands environment are to be exported to the batch job.

export file device size

qsub -V -I -l nodes=1:gen9:ppn=2 -d . ./job.sh

I am attaching the sample scripts for above 2 options here in this reply. Please try it from your end and tell us if it works for you.

Regards,

Rashmi

[Saharsa]

Thank you for your reply, but it does not work, and troughs an error:

su_job.sh: line 10: [cpu: command not found
qsub: script file 'device=cpu,' cannot be loaded - No such file or directory

and my script:

#!/bin/sh -f
echo "Input The File!";
read file
echo "Choose gpu Or cpu!";
read device
echo "Input Vector Size!";
read size
export file device size
echo
if ["$device" = "gpu" ]; then
qsub -v file=$file, device=$device, size=$size -I -l nodes=1:gen9:ppn=2 -d . ./job.sh
else
qsub -v file=$file, device=$device, size=$size -I -l nodes=1:skl:ppn=2 -d . ./job.sh
fi

[RashmiP_Intel]

Hi,

I was able to reproduce issue with the code you shared in last reply.
There is issue with spacing.
- Ensure that there is space between [ and "$device"
 if [ "$device" = "gpu" ]

- Please don't add spaces after parameters

file=$file,device=$device,size=$size

The script I mentioned in previous reply contains all this changes. Please try it and let me know.

Regards,
Rashmi

[Saharsa]

Thank you!

You are right, it was space issue.

but my main problem still remain. because when I submit my job on queue, after that when I do 

bash job.sh 

to validate my job, I can not see any thing.

I mean:

bash job.sh
:: WARNING: setvars.sh has already been run. Skipping re-execution.
To force a re-execution of setvars.sh, use the '--force' option.
Using '--force' can result in excessive use of your environment variables.

start: 210622.005619.973

stop: 210622.005619.984

but it should shows me the result of executing my code on CPU or GPU between start and stop time, but instead it is empty as you see.

my job.sh:

#!/bin/bash

source /opt/intel/inteloneapi/setvars.sh
file=$1
device=$2
size=$3
echo
echo start: $(date "+%y%m%d.%H%M%S.%3N")
echo
$file$device$size
echo
echo stop: $(date "+%y%m%d.%H%M%S.%3N")
echo

-----

by the way, for solving this problem I used other script which produce this script and now I have result. But I am interested in to resolve that which I pointed out above.

[RashmiP_Intel]

Hi,

The issue is with the job.sh script you are using in your last reply. The parameters I provided in previous reply(-v and -V) are for setting environment variable.
Environment variables can be accessed with just variable name(like $file). So there is no need of below lines:
file=$1
device=$2
size=$3

The script you provided in first reply should run. I am attaching the same here.

#!/bin/bash

source /opt/intel/inteloneapi/setvars.sh

echo
echo start: $(date "+%y%m%d.%H%M%S.%3N")
echo $file $device $size
echo
echo stop: $(date "+%y%m%d.%H%M%S.%3N")
echo

Please use above job.sh file to validate the result and let me know if it works for you.

Regards,
Rashmi

[RashmiP_Intel]

Hi,

We haven't heard back from you. Could you please confirm whether your issue is resolved?

Apart from the solution we provided above, I am attaching here an alternate solution(using command line arguments):
submitjob.sh:

#!/bin/sh -f
echo "Input The File!";
read file
echo "Choose gpu Or Cpu!";
read device
echo "Input Vector Size!";
read size
export file device size
echo
if [ "$device" = "gpu" ]; then
qsub -l nodes=1:gen9:ppn=2 -d . ./job.sh -F "$file $device $size"
else
qsub -l nodes=1:skl:ppn=2 -d ./job.sh -F "$file $device $size"
fi

job.sh:

#!/bin/bash

source /opt/intel/inteloneapi/setvars.sh
file=$1
device=$2
size=$3
echo
echo start: $(date "+%y%m%d.%H%M%S.%3N")
echo $file $device $size
echo
echo stop: $(date "+%y%m%d.%H%M%S.%3N")
echo

After submitting the job you can verify output in job.sh.o file generated in the the working directory.

Note: This command line argument in an alterative solution and solution I provided in previous replies(using environment variable) will also work.

However as we are passing job.sh script and parameters while submitting the job, you don't have to execute the job.sh script manually to validate results. As I mentioned earlier result can be verified in .o file generated in pwd.
Please try out these options and let us know whether it helped.

Regards,
Rashmi

[RashmiP_Intel]

Hi,

We haven't heard back from you.

Has the solution provided helped? Is your issue resolved? Can we discontinue monitoring this thread? Please let us know if the issue still persists"

Regards,

Rashmi

[RashmiP_Intel]

Hi,

We have not heard back from you, so we will close this inquiry now. If you need further assistance, please post a new question.

Thanks,

Rashmi