Intel® DevCloud
Help for those needing help starting or connecting to the Intel® DevCloud
1518 Discussions

$HOME storage is too slow.

User01
New Contributor II
612 Views

The Intel DevCloud compute and memory resources are very valuable. However, the I/O from $HOME is painfully slow. Copying to $PBS_SCRATCHDIR for jobs makes perfect sense, and the performance of $PBS_SCRATCHDIR is adequate. However, $HOME is so slow that staging data in $PBS_SCRATCHDIR is tedious. For comparison, a USB v3 5 Gbps external drive would give 625 MB/s. A 10 Gbps Ethernet connection to NFS would give 1,250 MBs. A fast NVMe PCIe drive would give 7,000 MB/s.

u123456@s001-n023:~$ dd if=/dev/zero of=~/tmp/output bs=8k count=100k; rm -f ~/tmp/output
102400+0 records in
102400+0 records out
838860800 bytes (839 MB, 800 MiB) copied, 8.19505 s, 102 MB/s
u123456@s001-n023:~$ dd if=/dev/zero of=${PBS_SCRATCHDIR}/output bs=8k count=100k; rm -f ${PBS_SCRATCHDIR}/output
102400+0 records in
102400+0 records out
838860800 bytes (839 MB, 800 MiB) copied, 0.662698 s, 1.3 GB/s

It would be very enabling if the compute nodes could read from $HOME at closer to at least 10 Gbps as is typical for slow NFS mounts.

0 Kudos
1 Solution
AthiraM_Intel
Moderator
544 Views

Hi,

 

Thank you for your feedback. We have informed the admin team regarding this. Current $HOME is on a 1Gbe connection for DevCloud. In future, Intel DevCloud  will be be on a faster 10Gbe connection.

 

Thank you for your feedback. Can we close this case?

 

Thanks.

 

 

 

 

 

View solution in original post

0 Kudos
4 Replies
AthiraM_Intel
Moderator
580 Views

Hi,


Thanks for posting in Intel communities.


We will inform the concerned team regarding this and get back to you soon.



Thanks


AthiraM_Intel
Moderator
545 Views

Hi,

 

Thank you for your feedback. We have informed the admin team regarding this. Current $HOME is on a 1Gbe connection for DevCloud. In future, Intel DevCloud  will be be on a faster 10Gbe connection.

 

Thank you for your feedback. Can we close this case?

 

Thanks.

 

 

 

 

 

0 Kudos
User01
New Contributor II
538 Views

That is great news! I am really looking forward to the 10 Gb transfer speed. Please post when the upgrade is complete. When will the upgrade occur?

0 Kudos
AthiraM_Intel
Moderator
497 Views

Hi,


The proposed Intel DevCloud upgrade is planned to be completed by the end of 2022. If you didn't see any update regarding the upgrade by this time, please let us know.

As of now your query is clarified, this thread will no longer be monitored by Intel. If you need further assistance, please post a new question.



Thanks


0 Kudos
Reply