0

I am developing a multi threaded application in C. In my application there are 40 threads which receives the UDP packets from 40 different ports and place the data into the queues (40 queues for 40 ports). Now there are another 40 data processing threads which takes the data from the respective queues and process them. The current scheduling of policy of my threads are SCHED_OTHER.

The problem is that some port listening threads are missing the packets. What can I do to solve this problem. I have some solutions in my mind:

a) Making the data reading threads real-time. How can I make it? (Will it starve the data processing threads from CPU?).

b) Increasing the nice level of my whole application (it is not solving the problem).

c) ?

I am using Pthreads.

halfer
  • 19,824
  • 17
  • 99
  • 186
Muhammad Awais Khalid
  • 59
  • 7
  • "Missing packets". Even with the buffering in kernel? Have you determined how long your packet reading is being starved of execution? – Joe Oct 30 '17 at 10:15
  • I don't know how to find the "starvation of execution". How I can buffer in kernel? – Muhammad Awais Khalid Oct 30 '17 at 10:17
  • try increasing the socket buffer size. See this question and answers: https://stackoverflow.com/questions/7968566/what-would-cause-udp-packets-to-be-dropped-when-being-sent-to-localhost – Sigi Oct 30 '17 at 10:26
  • I guess what I'm asking is, how have you determined your problem is down to your threads not executing to read a packet. 'Cos my gut feel is that that is not your issue... – Joe Oct 30 '17 at 10:26
  • Dear Joe, When I decrease the number of threads the system is running fine. So My feeling is that increasing the number of threads to 40 is causing the problem. How can I find that whether the thread is not executing the read a packet or is there any thing else (socket buffer size) as sigismondo suggested. – Muhammad Awais Khalid Oct 30 '17 at 10:32
  • Dear Sigismondo, Thanks for your answer, I am gonna check it. – Muhammad Awais Khalid Oct 30 '17 at 10:33
  • Please read [Under what circumstances may I add “urgent” or other similar phrases to my question, in order to obtain faster answers?](//meta.stackoverflow.com/q/326569) - the summary is that this is not an ideal way to address volunteers, and is probably counterproductive to obtaining answers. Please refrain from adding this to your questions. – halfer Oct 30 '17 at 10:34
  • Seems a bit of a strange design. 40 threads to read 40 ports, OK, but 40 queues with 40 processing threads? Probably not a big issue, assuming that the queues are 'real' producer-consumer queues, ie. block on empty, but still sounds a bit 'off' to me.. Please say that you are not using some nasty 'non-blocking' queue design:) – Martin James Oct 30 '17 at 10:54
  • How much time does the processing threads consume to extract the packets? How many packets arrive at every port? Maybe you should consider having 1 thread listening all ports at every time – k_kaz Oct 30 '17 at 11:03
  • Oh - another thing - your queues - they are queueing buffer pointers, yes? Not copying in/out all the data inside a lock? – Martin James Oct 30 '17 at 11:10

0 Answers0