How to use single port for multiple logical data streams (Winsock)?

Question

I'm developing the client-server Winsock app (Visual C++) that should transmit the various kind of data (video stream, audio stream, service notifications, etc.) over the network. I know that the cleaner approach would be to use separate ports on separate threads for each individual data type (I call it "stream" here). But that would require occupying at least 5 different ports that would get problematic for some network infrastructures (firewall port forwarding, etc.).

So I'm trying to implement one-port connection (TCP), only one socket will be used to transmit different streams. The individual packets will have the info in header that will denote which stream it belongs to, total message size to be expected, etc. Say I have 5 different streams. I'm planning to use 5 threads to call send() of the same socket. Is this safe? I know the packets from different streams will arrive mixed but if I include necessary meta information in each sent packet, it could be reconstructed properly on the other side, right?

But the real problem is the receiving end. While it's probably OK to call send() from multiple threads on the same socket (though I'm not sure and need your confirmation!), calling recv() from multiple threads would not make much sense. So I should use one blocking recv() from one thread. But then, based on the packet header (identifying which stream the particular packet belongs to) I should fork the processing in different threads. Video stream should be processed by one thread, sound stream - by another, etc. But I don't quite know how to do this - how to fork the data processing from the receiving thread. Performance is a top priority so processing all streams by one thread can't be considered.

So to sum it up, I have three questions:

• Is it OK to call send() for same socket from multiple threads? (assuming there will be info in packet header about which sender thread (i.e. subsystem) it belongs).
• Having one blocking socket on the receiving end, calling recv() in loop from a single thread, how to "fork" the received packets of different logical streams to different worker threads?
• What would be your additional recommendations for implementing multiple-stream transfer through one port?

P.S.: Afaik, there's no way to use one port by multiple sockets, correct?

It's Windows platform, Winsock2, Visual C++ (if you could provide platform-specific hints that would be awesome).

= UPDATE =

• When you say to "lock socket", you mean serialize the access to send() function? e.g. with Critical Section?

• As for receiving end... I think I'll assemble messages (I call "message" the logical complete data structure, e.g. video frame, or sound sample, etc.) when recv()-ing them in loop (buffering messages from individual streams into individual buffers), and then I just pass the assembled messages (when one will be fully received), to forking threads. Now that is the question - how to pass them. The one way I've thought of is Event Objects: SetEvent() from receiver thread to trigger WaitForSingleObject() (which is in some loop) in different threads. Can you advice if this is the acceptable solution? Can you suggest anything better? Aren't there any faster solutions to do this ("trigger" another thread of the same application) than Event Objects? And how to pass the data?

Answer 1

• Is it OK to call send() for same socket from multiple threads? (assuming there will be info in packet header about which sender thread (i.e. subsystem) it belongs).

Of course you can call it from multiple threads. But remember that you have to put a lock on the socket to synchronize it. Otherwise you might end up getting information written inconsistent.

• Having one blocking socket on the receiving end, calling recv() in loop from a single thread, how to "fork" the received packets of different logical streams to different worker threads?

How about writing a BeginRecv and EndRecv as similar to boost asios library (or switching to Boost Asio. It is plattform independent!). When you have recieved a complete message you can send it away. I guess you would like to put that somewhere else as to continue recieving from the network? You could just use CreateThread and send your data via the lpParameter parameter and it would be someone elses problem than recv. There wouldn't be much sence in forking it prior to your data being downloaded because you still need to download the complete package before the next one is able to be.

• What would be your additional recommendations for implementing multiple-stream transfer through one port?

Wouldn't know much about it, but I have written an answer earlier regarding Winsock where I describe how uncertain recv() is. Don't forget to synchronize as to make sure that one package is written before the next one is.

When you say to "lock socket", you mean serialize the access to send() function? e.g. with Critical Section?

Yes indeed. You can do this pretty by using for instance RAII for the lock (then it would be automagically deleted/unlocked on function return). But yes, use a sendData function where you first lock an object and then send your data.

Now that is the question - how to pass them.

Well , You can just pass it as a new thread.

DWORD WINAPI myVideoProcessor(LPVOID theData){
    StructForKeepingVideoData* data = dynamic_cast<StructForKeepingVideoData*>(theData);
    // process the data that is passed in theData
    ...
}
...
void ReceiveData(){
    while (true){
        ...
        char buffer[SIZE];
        recv(mySocket, buffer, SIZE, 0);
        StructForKeepingVideoData* data = new StructForKeepingVideoData(buffer);

        HANDLE mId = CreateThread(NULL, 0, myVideoProcessor, data, 0, NULL);
        // now, the video processing will be done somewhere else. let's continue receiving data!
    }
}

I am kind of hazy here.. But I think you should check for constructions such as auto_ptr or shared_ptr when transferring the data - this will help you with the destruction of the data.

The one way I've thought of is Event Objects: SetEvent() from receiver thread to trigger WaitForSingleObject() (which is in some loop) in different threads. Can you advice if this is the acceptable solution?

Events would work as well, but it is not something i have ventured in to. Then you would just keep your thread going all the time instead of starting when you have data to process. Remember that you have to keep the data transfer synchronized as well. If you have some sort if queue you have to make sure that writes and reads are done after one another.

Can you suggest anything better? Aren't there any faster solutions to do this ("trigger" another thread of the same application) than Event Objects? And how to pass the data?

Wouldn't know of anything faster. But to pass the data, either do it when you create the thread or use some sort of queue that you keep synchronized.

Hope this helps.

Answer 2

Is it OK to call send() for same socket from multiple threads?

Sure, providing you lock the socket up. If I was doing this, I would queue up all the outgoing protocol units to one thread and send with that, possibly using a priority queue to enable some guarantee of prompt delivery to those streams that need it.

how to "fork" the received packets of different logical streams to different worker threads?

Your protocol is going to have to do that. Assemble a video/audio/whatever protocol-unit from the incoming byte-stream and queue the PDU to an appropriate handler thread.

Windows - IO completion ports at the server if you need high performance which, if you are attempting to stream video, you will.

Rgds, Martin