Community
cancel
Showing results for 
Search instead for 
Did you mean: 
youn__kihang
Novice
196 Views

Using Asynchronous I/O in C

 

Hello, All

 

I have a code that creates a lot of small files with fopen in C. 

Because a large number of MPI ranks are written to the same folder, performance degradation occurs in shared storage.

In fortran I have found to use the asynchronous option on open. How do I modify the code to use asynchronous I/O?

Please let me know if there are any options to use with fopen.

 

Thanks

0 Kudos
4 Replies
jimdempseyatthecove
Black Belt
181 Views

fopen (and it's accompanying writes) are not enhanced for asynchronous I/O.

This said, you are free to use a separate thread that uses fopen (and it's accompanying writes) to perform the I/O. Actually the fopen can be performed by the main thread of the rank and writes by an instantiated thread. Note, it will be your responsibility to coordinate the I/O such that the sequence is correct and that you do not destroy a buffer until after it has been written.

Hint, use only 1 thread per file and a FIFO queue from your Rank thread to your additional I/O thread.

Jim Dempsey

SantoshY_Intel
Moderator
119 Views

Hi,

 

We need some more information so that we can investigate well. could you provide the below details?

  1. Are you trying to "write" a single file using multiple ranks? (or) multiple files using multiple ranks?
  2. Can you provide your system environmental details like OS & filesystem?
  3. Can you provide the size of your files approximately?

 

Awaiting your reply.

 

Thanks & Regards,

Santosh

 

SantoshY_Intel
Moderator
74 Views

Hi,


As we didn't hear back from you, Is your issue resolved? If not, please provide the details that have been asked in my previous post.


Thanks & Regards,

Santosh


SantoshY_Intel
Moderator
28 Views

Hi,

 

As we have not heard back from you, we are considering that your issue has been resolved. So we will no longer respond to this thread. If you require additional assistance from Intel, please start a new thread. Any further interaction in this thread will be considered community only.

 

Have a Good day!

 

Thanks & Regards,

Santosh

 

Reply