- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hey
I am submitting a job on different nodes and some of them do not have the latest version of CMake. My jobscript requires a minimum of CMake 3.12 and the nodes are running 3.10.2.
Can you update the version on the nodes?
I have encountered the issue on the following nodes:
s001-n142
s001-n143
s001-n144
s001-n090
s001-n088
s001-n087
I haven't had the chance to test all the nodes, but it seems like it is not limited to the ones above.
My jobscript is working on some of the other nodes with different properties than those of the ones above. So I don't think there is a problem with my script.
The reason why I am running my script on different nodes is because some of the nodes have trouble with c++ std::chrono and I am curious to know if it is a GPU related problem.
Thanks in advance
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
Thanks for reaching out to us.
We understood your concern. We have informed the DevCloud admin team about the cmake version (3.10.2) in cpu nodes.
You could use gpu nodes, which have cmake version 3.16.3
You can login to gpu node by using the below command:
qsub -I -l nodes=1:ppn=2:gpu -d .
Also as a workaround, you could install cmake in a conda environment and execute your workload. Please follow the below steps:
Step 1 : Create conda environment
conda create -n <env_name> python=<version>
Step 2 : Activate conda environment
conda activate <env_name>
Step3 : Install cmake
conda install -c anaconda cmake
Regarding the second query, we tried one c++ sample code having std::chrono and able to run without any error in both cpu and gpu nodes.
Could you please share the reproducer code so that we could try the same from our end.
Thanks.
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
Thanks for reaching out to us.
We understood your concern. We have informed the DevCloud admin team about the cmake version (3.10.2) in cpu nodes.
You could use gpu nodes, which have cmake version 3.16.3
You can login to gpu node by using the below command:
qsub -I -l nodes=1:ppn=2:gpu -d .
Also as a workaround, you could install cmake in a conda environment and execute your workload. Please follow the below steps:
Step 1 : Create conda environment
conda create -n <env_name> python=<version>
Step 2 : Activate conda environment
conda activate <env_name>
Step3 : Install cmake
conda install -c anaconda cmake
Regarding the second query, we tried one c++ sample code having std::chrono and able to run without any error in both cpu and gpu nodes.
Could you please share the reproducer code so that we could try the same from our end.
Thanks.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Thanks for the quick response. I will use the conda environment in the meantime.
I created a simple cpp called test to reproduce the issue with std:chrono. (Found in the end of this reply)
I ran test.cpp with the following command:
qsub -l nodes=[properties]:ppn=2 -d . test.sh
Where test.sh looks like this:
#!/bin/bash
g++ -o testfile ./test.cpp
vtune -collect hotspots ./testfile
When submitting the job to a node with the following properties:
xeon:cfl:e-2176g:ram64gb:net1gbe:gpu:gen9
The hotspot analysis will show:
Elapsed Time: 237.043s
std::chrono::_V2::system_clock::now 235.702s
On the other hand. If you submit the job to a node with following properties.
xeon:skl:ram384gb:net1gbe:renderkit
The hotspot analysis will show:
Elapsed Time: 1.135s
std::chrono::_V2::system_clock::now 0.344s
I have tried with interactive mode on the nodes as well. It gave the same result.
I have also tried with both std::chrono::steady_clock and std::chrono::system_clock and it also gave the same result. That is that std::chrono is slow on nodes with these properties: xeon:cfl:e-2176g:ram64gb:net1gbe:gpu:gen9.
I am curious to know why this happens. So please let me know if you find out why.
Best regards.
I had trouble uploading. It would not regonize the file as a cpp file, so here goes:
#include <stdio.h>
#include <stdlib.h>
#include <chrono>
int main (int argc, char* argv[]){
int n = 10000000;
auto arr = new int[n];
long long int total_time = 0;
//Perform simple loop
for (int i = 0 ; i < n ; i++){
//Stopwatch
auto start = std::chrono::high_resolution_clock::now();
arr[i] = i;
//Stopwatch
auto stop = std::chrono::high_resolution_clock::now();
total_time += std::chrono::duration_cast<std::chrono::nanoseconds>(stop - start).count();
}
printf("\nTotal time in nanoseconds: %lld\n\n", total_time);
return 0;
}
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
Thanks for sharing the details.
You are using two nodes with different properties, one is skylake(skl) and another one is coffee lake (cfl). Both have different CPUs and different architecture. This may be one of the reasons for the time difference.
CPU details of skylake node:
CPU details of coffee lake node:
If you have any further issue, please let us know.
Thanks.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
Could you please give us an update? Has the solution provided helped?
Thanks.
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
Hi,
Thank you for accepting it as solution. We would discontinue monitoring this thread, please raise a new thread if you have further issues.
Thanks.
![](/skins/images/7FC17B7B85029576C25F1E43CE255B51/responsive_peak/images/icon_anonymous_message.png)
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page