1

I'm trying to build the receiving end of my system, which sends UDP segments over a Gigabit Ethernet link. I have tried so far both Matlab and Python, but neither solution seems to be efficient. I always encounter huge drop rates, which cannot be explained by the fact that UDP is unreliable. I suppose that I don't read the incoming buffers fast enough and so new frames get discarded.

My question is, would a C/C++ implementation be faster that Matlab and Python?

And what about threads? Would that improve the situation? I'm thinking of a producer/consumer scheme. One thread for handling the UDP socket, another for processing the incoming segments. However I still have some doubts about this. For instance, if I use mutexes this means that the producer cannot write to the queue while the consumer reads from there. And that means that still the producer will be stalled by the operation of the consumer. Am I wrong in this?

nickagian
  • 673
  • 1
  • 5
  • 9

3 Answers3

1

If you use threads, you need a queue to put the messages in. This should definitely be protected by a mutex. But the consumer thread don't need to lock the queue for the whole time it is processing a message, just when removing a message from the queue. You should however be careful to check if the queue gets to big, you might need more than one consumer thread.

Some programmer dude
  • 400,186
  • 35
  • 402
  • 621
  • 1
    I mentioned you in my answer and elaborated a bit. My main point is that he should figure out why its slow, not to copy your part :) – Brady Jun 01 '12 at 10:47
1

As mentioned by @JoachimPileborg, a queue is the way to go. You could have several worker threads pulling off the queue. There are several options available for this queue:

  • A simple queue with a simple mutex protection for read/write operations
  • A lockless queue, which usually uses atomic operations for consistancy
  • A 2 queue system where one queue is used for writing and the other is used for reading. When the read queue is empty, switch the queues.

These are standard producer/consumer solutions.

More important than that is to determine why the solution is slow. Python should be fast enough, c++ too (personally I think it would probably be faster, but that could be argued), I dont know anything about matlab though.

What you should do is use a good profiler (both python and c++ have many options). Here is a related answer. To repeat from that answer: find the bottlenecks and focus on those.

Additionally, what are the requirements? Remember: "As fast as possible" is not a valid requirement. A few packet drops may be acceptable if you are conformant with the throughput requirements. I work on a telecom system for which we test the throughput requirements by raising the throughput until it starts dropping packets, and measure where we are. Point is, its ok to drop if you're within the limits :)

Community
  • 1
  • 1
Brady
  • 10,207
  • 2
  • 20
  • 59
0

If you are simply dumping UDP packets onto the network, then the vast majority are going to be dropped because you are pushing them into the socket faster than the network can send them.

Remember that UDP does not have any flow-control or error recovery. Your application must determine a suitable transmit rate, adjust that rate as needed, detect missing datagrams, arrange for missing datagrams to be resent, and assemble all the received datagrams into a coherent file.

That is not easy: See this answer to a similar question.

Community
  • 1
  • 1
Seth Noble
  • 3,233
  • 19
  • 31