2

Need an async I/O Processing

Plan to use async I/O through aio* calls on Linux

The situation:

I have opened socket with AF_INET and SOCK_STREAM flags (TCP) Have limit high watermark for send buffers Want to write to that socket asynchronously, and when send buffer overflows, want to disconnect an socket

So, I have questions:

  1. When I made async call to aio_write on TCP socket, when I/O completion will arrives - when buffer written out into socket buffer or delivery is confirmed? How I can manage this behavior?

  2. How's best to handle this with lio_listio techniques

Regards, Andrew

westtrd
  • 23
  • 1
  • 5
  • out of interest have you looked at boost asio? – 111111 Mar 17 '12 at 15:49
  • Not sure about Linux. On Windows, MSDN: 'The successful completion of a WSASend does not indicate that the data was successfully delivered'. TBH, I was surprised by that - if the data has not been successfully delivered, it may need to be resent. If the kernel, driver etc. is returning buffers that have not been acked, then they must be keeping copies of the the data, so losing the 'no copying' advantage of queueing up user buffers :(( – Martin James Mar 17 '12 at 16:11
  • @MartinJames: It doesn't guarantee successful completion *using the return value of the `WSASend()` function*. Wait for the appropriate notification (callback, I/O completion notification, etc.) to get the transfer's actual result. This doesn't mean you can't use no-copying, you just need to wait for the transfer completes before re-using that specific buffer. – André Caron Mar 17 '12 at 17:27
  • @AndréCaron - OK, so a completion notification for a WSASend, (at least with TCP), indicates successful delivery? – Martin James Mar 18 '12 at 21:17
  • @MartinJames: it indicates that, to the best knowledge of the operating system, the contents were sent as requested. The completion notification basically tells you the same thing as the results of a call to the synchronous `send()` function. I'm not very knowledgeable about low-level TCP acknowledgement and when exactly the OS tells you it sent the data, so I'm not sure if this result means the data was succesfully *delivered* (as in, confirmed reception from the peer). – André Caron Mar 18 '12 at 21:28
  • @AndréCaron - OK, thanks anyway. That's two of us who don't know :( – Martin James Mar 18 '12 at 23:34

1 Answers1

4

You want to avoid AIO on Linux for anything real, at least for now, From aio(7):

The current Linux POSIX AIO implementation is provided in userspace by glibc. This has a number of limitations, most notably that maintaining multiple threads to perform I/O operations is expensive and scales poorly. Work has been in progress for some time on a kernel state-machine-based implementation of asynchronous I/O (see io_submit(2), io_setup(2), io_cancel(2), io_destroy(2), io_getevents(2)), but this implementation hasn't yet matured to the point where the POSIX AIO implementation can be completely reimplemented using the kernel system calls.

Instead, look into non-blocking IO with select(2)/poll(2)/epoll(7).

Nikolai Fetissov
  • 82,306
  • 11
  • 110
  • 171
  • Or, you can use a higher-level framework such as `boost::asio` which will use whatever the best implementation is for your system. – André Caron Mar 17 '12 at 17:25
  • What's about lio_listio? Thanks in advance – westtrd Mar 17 '12 at 18:46
  • Andre, it's assumed, but this time I need and want to isolate work with very many simultaneous network sockets – westtrd Mar 17 '12 at 18:49
  • This needs an update since for unbuffered files the `linux-aio` is being used with success. Also check the very recent `io_uring`, which seems to solve the linux dilema with asynchronous programming and can be used with any type of file (sockets, buffered, etc...) – MrIo Nov 17 '20 at 08:45