143
  1. Can we call send from one thread and recv from another on the same socket?
  2. Can we call multiple sends parallely from different threads on the same socket?

I know that a good design should avoid this, but I am not clear how these system APIs will behave. I am unable to find a good documentation also for the same.

Any pointers in the direction will be helpful.

Jay
  • 24,173
  • 25
  • 93
  • 141
  • 4
    why do you claim that doing so is a bad practice?. It looks fine to me because you listen to and receive from in different threads. – TheMathNoob Sep 02 '17 at 02:22

3 Answers3

106

POSIX defines send/recv as atomic operations, so assuming you're talking about POSIX send/recv then yes, you can call them simultaneously from multiple threads and things will work.

This doesn't necessarily mean that they'll be executed in parallel -- in the case of multiple sends, the second will likely block until the first completes. You probably won't notice this much, as a send completes once its put its data into the socket buffer.

If you're using SOCK_STREAM sockets, trying to do things a parallel is less likely to be useful as send/recv might send or receive only part of a message, which means things could get split up.

Blocking send/recv on SOCK_STREAM sockets only block until they send or recv at least 1 byte, so the difference between blocking and non-blocking is not useful.

Robert Harvey
  • 178,213
  • 47
  • 333
  • 501
Chris Dodd
  • 119,907
  • 13
  • 134
  • 226
  • What about blocking send / recv? Are they atomic? – Jay Dec 30 '09 at 18:16
  • this article (http://www.almaden.ibm.com/cs/people/marksmith/sendmsg.html) seems to confirm what you say about SOCK_STREAM but is not clear about SOCK_DGRAM, where exactly did you get your information from? – João Portela May 15 '10 at 18:22
  • 4
    @Joao: SOCK_DGRAM socket are documented as "preserving message boundaries", which isn't very clear. From looking at the linux kernel sources you can at least see that each send and recv deals with a single packet atomically (at least for udp). – Chris Dodd May 16 '10 at 02:33
  • @Chris, I am trying to do something similar using a single thread for reception of packets) and it's parent to send out the packets. Unfortunately, I see from the print log that reception is started only after all the packets are sent. It's not happening simultaneously or asynchronously. – KedarX May 23 '12 at 12:38
  • 3
    @Kedar: not sure what you mean. A `send` returns as soon as the data is placed into the send buffer, and the data is sent on through the netowrk stack and out onto the network asynchronously. So if you have one thread sending and one thread receiving, its entirely possible (even likely) for the sending thread to send many packets before the receiving thread receives the first packet. Its entirely asynchronous and not simultaneous. – Chris Dodd Jan 23 '13 at 17:30
  • 7
    @ChrisDodd, can you give a link for "POSIX defines send/recv as atomic operations"? – suitianshi Jul 19 '16 at 11:02
  • 4
    @suitianshi: The POSIX 1003.1c standard document lists all the functions in 1003.1 that are reentrant (safe to call from threads) and which are not. I'm not aware of a free online copy available anywhere, unfortunately. – Chris Dodd Jul 19 '16 at 15:16
  • 2
    @ChrisDodd I have found the copy on http://www.unix-systems.org/version4/ and I can see the list of System Interface Table at chapter 7.1 but do not see where it list the functions as being atomic operations. Not to doubt you, but can you please share/edit your answer to justify your point in the document? – user153882 Jul 05 '17 at 14:40
  • I am wondering how the described scenario when SOCK_STREAM is used and messages could get split up, is processed on a receiving side, in the case the receiver uses `MSG_WAITALL` flag? – RedSoft Aug 20 '18 at 18:10
  • 1
    @user153882: 2.9.1 under 'threads' list all the functions that may not be thread safe -- all others not listed there are thus thread-safe. – Chris Dodd Dec 06 '18 at 02:07
24

The socket descriptor belongs to the process, not to a particular thread. Hence, it is possible to send/receive to/from the same socket in different threads, the OS will handle the synchronization.

However, if the order of sending/receiving is semantically significant, you yourself (respectively your code) have to ensure proper sequencing between the operations in the different threads - as is always the case with threads.

Adrian Willenbücher
  • 923
  • 5
  • 14
3

I don't see how receiving in parallel could possibly accomplish anything. If you have a 3 bytes message, 1 thread could get the 1st 2 bytes and another the last byte, but you'd have no way of telling which was which. Unless your messages are only a byte long, there is no way you could reliably make anything work with multiple threads receiving.

Multiple sends might work, if you sent the entire message in a single call, but I'm not sure. It's possible that one could overwrite another. There certainly wouldn't be any performance benefit to doing so.

If multiple threads need to send, you should implement a synchronized message queue. Have one thread that does the actual sending that reads messages from the queue and have the other threads enqueue whole messages. The same thing would work for receiving, but the receive thread would have to know the format of the messages so it could deserialize them properly.

noah
  • 21,289
  • 17
  • 64
  • 88
  • 10
    If you're using SOCK_DGRAM sockets, each recv will get a single datagram; it will never be split between recvs – Chris Dodd Dec 30 '09 at 17:47
  • 2
    @noah, I agree parallel recvs cannot accomplish anything. That's why I haven't asked it. My question is send / recv parallely and then multiple sends parallely. Your answer does give an insight into parallel sends. Thanks for the same. – Jay Dec 30 '09 at 17:49
  • 2
    @Chris good point. I was assuming TCP. @Jay You might clarify the question "Can we call send / recv parallely" sounds like you want to receive in parallel. – noah Dec 30 '09 at 17:56