Intel® DevCloud
Help for those needing help starting or connecting to the Intel® DevCloud
Announcements
This community is designed for sharing of public information. Please do not share Intel or third-party confidential information here.
914 Discussions

How to submit job script with different parameters, from another bash script on Intel devCloud

Saharsa
New Contributor I
738 Views

Hello,

I tried to read my file, the device (GPU or CPU), and the size of my vector from command line and then with respect to the chosen device, submit my job script with that parameters in to the queue of specific GPU or CPU. But after submitting my job to the queue, when I validate my script like:

bash job.sh

I saw that my parameters file, device, and size from submitjob.sh script did not pass to the job.sh script.

Could you please help me what is my mistake?!

My submitjob.sh:

#!/bin/sh -f
echo "Input The File!";
read file
echo "Choose Gpu Or Cpu!";
read device
echo "Input Vector Size!";
read size
echo
if [ "$device" = "Gpu" ]; then
qsub -I -l nodes=1:gen9:ppn=2 -d . ./job.sh file device size
else
qsub -I -l nodes=1:skl:ppn=2 -d . ./job.sh file device size
fi

-------

my job.sh:


#!/bin/bash

source /opt/intel/inteloneapi/setvars.sh

echo
echo start: $(date "+%y%m%d.%H%M%S.%3N")
echo
$file $device $size
echo
echo stop: $(date "+%y%m%d.%H%M%S.%3N")
echo

 

 

 

0 Kudos
8 Replies
RashmiP_Intel
Moderator
697 Views

Hi Saharsa,

 

Thanks for reaching to Devcloud Support forum.

 

 I am mentioning here 2 options that could be useful for you:-

-v Paramater: Expands the list of environment variables that are exported to the job.

qsub -v file=$file,device=$device,size=$size -I -l nodes=1:gen9:ppn=2 -d . ./job.sh

 

-V Paramater: Declares that all environment variables in the qsub commands environment are to be exported to the batch job.

export file device size

qsub -V -I -l nodes=1:gen9:ppn=2 -d . ./job.sh

 

I am attaching the sample scripts for above 2 options here in this reply. Please try it from your end and tell us if it works for you.

 

Regards,

Rashmi

 

Saharsa
New Contributor I
688 Views

Thank you for your reply, but it does not work, and troughs an error:

su_job.sh: line 10: [cpu: command not found
qsub: script file 'device=cpu,' cannot be loaded - No such file or directory

 

and my script:

#!/bin/sh -f
echo "Input The File!";
read file
echo "Choose gpu Or cpu!";
read device
echo "Input Vector Size!";
read size
export file device size
echo
if ["$device" = "gpu" ]; then
qsub -v file=$file, device=$device, size=$size -I -l nodes=1:gen9:ppn=2 -d . ./job.sh
else
qsub -v file=$file, device=$device, size=$size -I -l nodes=1:skl:ppn=2 -d . ./job.sh
fi

 

 

RashmiP_Intel
Moderator
680 Views

Hi,

 

I was able to reproduce issue with the code you shared in last reply.
There is issue with spacing.
- Ensure that there is space between [ and "$device"
 if [ "$device" = "gpu" ]

- Please don't add spaces after parameters

file=$file,device=$device,size=$size

 

The script I mentioned in previous reply contains all this changes. Please try it and let me know.

 

Regards,
Rashmi

Saharsa
New Contributor I
675 Views

Thank you!

You are right, it was space issue.

but my main problem still remain. because when I submit my job on queue, after that when I do 

bash job.sh 

to validate my job, I can not see any thing.

I mean:

bash job.sh
:: WARNING: setvars.sh has already been run. Skipping re-execution.
To force a re-execution of setvars.sh, use the '--force' option.
Using '--force' can result in excessive use of your environment variables.

start: 210622.005619.973


stop: 210622.005619.984

 

but it should shows me the result of executing my code on CPU or GPU between start and stop time, but instead it is empty as you see.

my job.sh:

#!/bin/bash

source /opt/intel/inteloneapi/setvars.sh
file=$1
device=$2
size=$3
echo
echo start: $(date "+%y%m%d.%H%M%S.%3N")
echo
$file$device$size
echo
echo stop: $(date "+%y%m%d.%H%M%S.%3N")
echo

-----

by the way, for solving this problem I used other script which produce this script and now I have result. But I am interested in to resolve that which I pointed out above.

RashmiP_Intel
Moderator
667 Views

Hi,

The issue is with the job.sh script you are using in your last reply. The parameters I provided in previous reply(-v and -V) are for setting environment variable.
Environment variables can be accessed with just variable name(like $file). So there is no need of below lines:
file=$1
device=$2
size=$3

The script you provided in first reply should run. I am attaching the same here.

#!/bin/bash

source /opt/intel/inteloneapi/setvars.sh

echo
echo start: $(date "+%y%m%d.%H%M%S.%3N")
echo $file $device $size
echo
echo stop: $(date "+%y%m%d.%H%M%S.%3N")
echo

Please use above job.sh file to validate the result and let me know if it works for you.

Regards,
Rashmi

RashmiP_Intel
Moderator
597 Views

Hi,

We haven't heard back from you. Could you please confirm whether your issue is resolved?


Apart from the solution we provided above, I am attaching here an alternate solution(using command line arguments):
submitjob.sh:

 

#!/bin/sh -f
echo "Input The File!";
read file
echo "Choose gpu Or Cpu!";
read device
echo "Input Vector Size!";
read size
export file device size
echo
if [ "$device" = "gpu" ]; then
qsub -l nodes=1:gen9:ppn=2 -d . ./job.sh -F "$file $device $size"
else
qsub -l nodes=1:skl:ppn=2 -d ./job.sh -F "$file $device $size"
fi

 

 

job.sh:

 

#!/bin/bash

source /opt/intel/inteloneapi/setvars.sh
file=$1
device=$2
size=$3
echo
echo start: $(date "+%y%m%d.%H%M%S.%3N")
echo $file $device $size
echo
echo stop: $(date "+%y%m%d.%H%M%S.%3N")
echo

 

 

After submitting the job you can verify output in job.sh.o file generated in the the working directory.


Note: This command line argument in an alterative solution and solution I provided in previous replies(using environment variable) will also work.

 

However as we are passing job.sh script and parameters while submitting the job, you don't have to execute the job.sh script manually to validate results. As I mentioned earlier result can be verified in .o file generated in pwd.
Please try out these options and let us know whether it helped.


Regards,
Rashmi

RashmiP_Intel
Moderator
572 Views

Hi,


We haven't heard back from you.

Has the solution provided helped? Is your issue resolved? Can we discontinue monitoring this thread? Please let us know if the issue still persists"


Regards,

Rashmi


RashmiP_Intel
Moderator
530 Views

Hi,


We have not heard back from you, so we will close this inquiry now. If you need further assistance, please post a new question.


Thanks,

Rashmi


Reply