Intel® C++ Compiler
Community support and assistance for creating C++ code that runs on platforms based on Intel® processors.

Using Asynchronous I/O in C

youn__kihang
Novice
982 Views

 

Hello, All

 

I have a code that creates a lot of small files with fopen in C. 

Because a large number of MPI ranks are written to the same folder, performance degradation occurs in shared storage.

In fortran I have found to use the asynchronous option on open. How do I modify the code to use asynchronous I/O?

Please let me know if there are any options to use with fopen.

 

Thanks

0 Kudos
4 Replies
jimdempseyatthecove
Honored Contributor III
967 Views

fopen (and it's accompanying writes) are not enhanced for asynchronous I/O.

This said, you are free to use a separate thread that uses fopen (and it's accompanying writes) to perform the I/O. Actually the fopen can be performed by the main thread of the rank and writes by an instantiated thread. Note, it will be your responsibility to coordinate the I/O such that the sequence is correct and that you do not destroy a buffer until after it has been written.

Hint, use only 1 thread per file and a FIFO queue from your Rank thread to your additional I/O thread.

Jim Dempsey

0 Kudos
SantoshY_Intel
Moderator
905 Views

Hi,

 

We need some more information so that we can investigate well. could you provide the below details?

  1. Are you trying to "write" a single file using multiple ranks? (or) multiple files using multiple ranks?
  2. Can you provide your system environmental details like OS & filesystem?
  3. Can you provide the size of your files approximately?

 

Awaiting your reply.

 

Thanks & Regards,

Santosh

 

0 Kudos
SantoshY_Intel
Moderator
860 Views

Hi,


As we didn't hear back from you, Is your issue resolved? If not, please provide the details that have been asked in my previous post.


Thanks & Regards,

Santosh


0 Kudos
SantoshY_Intel
Moderator
814 Views

Hi,

 

As we have not heard back from you, we are considering that your issue has been resolved. So we will no longer respond to this thread. If you require additional assistance from Intel, please start a new thread. Any further interaction in this thread will be considered community only.

 

Have a Good day!

 

Thanks & Regards,

Santosh

 

0 Kudos
Reply