4

In one of my parsing program i have to do the following(in C,C++,VC++):-

I have to connect 100 machines(dialysis machines) through socket programming. Send request to each machine and receive the response from the machine and the data that i will get from the machine i will have to parse that data and write the values in the a file.I have to send the request to each machine after every 5seconds.

So to accomplish the above task i am thinking to do it as:-

I will read the ip and port of each machine from the data base,make a thread to connect to each machine and in each thread a sub thread(child thread) will be made which will send and receive and parse the data from the machine after every 5 second(and write the values in a txt file). My parsing function will be common.

Is it the feasible solution. Please help me.Thnaks in advance.

Dany
  • 2,034
  • 8
  • 34
  • 54
  • For the writing of the data I suggest you send all the information back to 1 thread and write from there. – Minion91 Sep 18 '12 at 07:25
  • 2
    Threads is not the correct solution (especially with 100 threads). Use one thread to make the requests. When you get responses pass this to a work group for handling (the work group should have approx ` * 1.5` threads (of course measure to find the correct ration)) – Martin York Sep 18 '12 at 07:42
  • @LokiAstari - 100 threads is nothing. They are blocked for most of the 5 seconds, so it's a non-issue. – Martin James Sep 18 '12 at 09:01
  • 1
    @Minion91 - I would not do that, not for 100 independent connections. One machine that became unreachable would inflict a network timeout on all the remaining 99 machines. – Martin James Sep 18 '12 at 09:25
  • @MartinJames: Its not just the threads using CPU but the other resources that go with threads (like memory for the stack). Especially when it can all that can be done in a single thread (which also makes the code much simpler to write and read). We use three threads to maintain 1500 simultaneous connections. One threads makes requests and waits for responses. One thread downloads the data and one thread maintains the schedule and balances requests to sites to make sure we are [**always** polite](http://stackoverflow.com/q/8236046/14065) – Martin York Sep 18 '12 at 17:18

3 Answers3

4

Your solution sounds reasonable except for one point. You mention that you will create a thread to connect to each machine, then a sub-thread to manage sending, receiving, and parsing. I dont understand why you need to create a sub-thread. You should be able to handle everything in the connection thread. Also consider that 1 thread per connection may not scale well, if this application has to deal with a larger number of machines, a thread per machine should be avoided.

It may even be possible to achieve this with a simple thread pool instead of 1 thread per connection, which wont scale well. You could consider creating tasks that get put on a worker queue every 5 seconds, and the pool of threads would connect, read, disconnect, parse, and process. Assuming this is TCP/IP you probably shouldnt keep the connection open, but rather connect/disconnect for each read, similar to HTTP.

Here is a vc++ thread pool related question. And here is some more related info.

Another alternative could be to use libevent for the socket communication. As for the parsing, there are also other libraries that can be used like Apache Thrift or JSon, all of which are open source. The down side to these parsing libraries is that you may have to also modify the dialysis machines which may not be an option. If you can use something like Thrift, you can get everything from one library: socket comm and parsing.

Here is some code for the simple case of 1 thread per connection:

class ThreadInfo
{
public:
  ThreadInfo(const string &ipAddress, uint16_t port) : ipAddress_(ipAddress), port_(port) {}
  string getIpAddress() {return ipAddress_;}
  uint16_t getPort() {return port_;}
  string getRecvBuffer() {return recvBuffer_;}

private:
  string ipAddress_;
  uint16_t port_;
  string recvBuffer_;
};

void *threadEntryPoint(void *userData)
{
  ThreadInfo *threadInfo = (ThreadInfo*) userData;

  // You need to decide if you want to keep the connection open while sleeping
  // or open and close it for each transaction. Change code here accordingly.
  // Create socket with threadInfo->getIpAddress() and threadInfo->getPort()

  // while(1)
  //   Send request to each machine
  //   Get response from each machine and store in threadInfo->getRecvBuffer()
  //       The buffer could also be a local var in this function, decide accordingly
  //   parse data accordingly
  //   sleep 5 seconds
}

uint16_t getPort(int machineNum) { return 3456; }
string getIpAddress(int machineNum) { return string("192.168.1.2"); }
int main(int argc, char **argv)
{
   // 3 items that we need, and that you will have to plugin accordingly:
   //   1) Num threads, assuming 100 for now
   //   2) IP address of each external machine, implement getIpAddress() accordingly
   //   3) port of each machine, implement getPort() accordingly

   int numThreads(100);
   list<pthread_t> threadIdList;

   for(int i = 0; i < numThreads; ++i)
   {
      pthread_t threadId;
      ThreadInfo *threadInfo = new ThreadInfo(getIpAddress(i), getPort(i));
      pthread_create(&threadId, NULL, threadEntryPoint, threadInfo);
      threadIdList.push_back(threadId);
   }

   // Wait for the threads to finish
   std::list<pthread_t>::iterator iter = threadIdList.begin();
   while(iter != threadIdList.end())
   {
     pthread_t threadId = *iter++;
     pthread_join(threadId, NULL);
   }
}
Community
  • 1
  • 1
Brady
  • 10,207
  • 2
  • 20
  • 59
  • @Dany, Sorry I started but got interrupted. I'll update the answer in 5 mins. – Brady Sep 21 '12 at 11:35
  • @Dany, I added some code. I havent compiled it, so you'll have to test it. – Brady Sep 21 '12 at 11:59
  • thanks for the code i will run it and can i ask further queries when required related to code? – Dany Sep 21 '12 at 12:11
  • Siu have not pass the buffer to the thread,The biggest problem to pass is buffer,how to make 100 buffer and pass to each thread – Dany Sep 21 '12 at 12:21
  • @Dany, Im not sure which buffer you're referring to: the read buffer, or the command to send to the machine. Either way, these buffers can/should be in the ThreadInfo object. If its the command buffer to send to the machine, instantiate the ThreadInfo object with the command. – Brady Sep 21 '12 at 14:05
  • sir i made some code can u check that if it appropriate as per our requirement...should i paste it here? – Dany Sep 25 '12 at 06:58
  • @Dany, I would suggest one of 2 options for a code review: http://ideone.com/ for simple code examples, or even better use http://codereview.stackexchange.com. Try the codereview stackexchange option (with a link to this question) and post a link to the question in this comment. This way myself and others can help. – Brady Sep 25 '12 at 07:06
  • please check the following link http://codereview.stackexchange.com/questions/15901/is-this-appropriate-to-requirement – Dany Sep 25 '12 at 07:49
  • sir mine design somewhat changes now i do as this,i make a thread to each machine but before that i get an xml file from the server save that xml file that file has the following parameters BSTR AccountId;,BSTR MachineId;,char* Make1;,BSTR Model;,char* Model1;,BSTR SerialNumber;,char* IpAddress1;,int Port1;,BSTR LocationCode; using the make and model given in this xml i decide which type of machine is it and connect to it,but after parsing i have to send my parsed data to the database and update to databasefunction takes the following parameters that are machine id and status of machin – Dany Oct 18 '12 at 11:30
  • Can you please how can we check whether the connection is still there or not with the machine or not or please reply http://stackoverflow.com/questions/13090924/how-to-check-socket-connection-after-every-minute-in-c#comment17788910_13090924 – Dany Oct 26 '12 at 17:50
  • @Dany, when a TCP connection is closed, you'll receive a read of size 0 – Brady Oct 26 '12 at 18:31
  • sir did u check my design on http://stackoverflow.com/questions/13090924/how-to-check-socket-connection-after-every-minute-in-c#comment17788910_13090924,when connection is closed my send api return -1 – Dany Oct 26 '12 at 18:39
  • sir when i get a read zero and after few seconds if the connection became active can i send it data with the previous socket connection that i made or should i have to make new socket connection? – Dany Oct 26 '12 at 19:28
1

For 100 machines, polling every 5 seconds, one thread per machine is reasonable - the threads are going to be blocked on the Sleep(5000) for most of the time and blocked on socket I/O or disk I/O for most of the rest of the time. For this sort of loading, (or even five times the loading), I don't see any need to resort to async I/O or thread pools - why complicate matters unnecessarily?

As pointed out by @Brady, I don't see why you would need more than one thread per connection, assuming that the requirement is as you post - just polling every ~5 seconds and writing the replies to a text file.

I'm guessing, (hoping:), that the 5 second interval is not a safety-critical real-time requirement and that the dialysis machines will continue to operate normally if the period is occasionally 6 seconds due to some temporary software or network delay. I'm not a haematologist/nephrologist, but I would be surprised if any dialysis machine could make any significant change to overall treatment, (which takes hours), if a poll/instruction was occasionaly delayed for an extra second.

Edit - re. 'parsing function and writing data to a file function is common for all threads' - should be fine, assuming a different text file for each machine. If the logs are all written one log file, that's more of a problem - each log entry should really be queued off to one logger thread that, alone, writes to the log file. Using an existing, proven logger framework that already supports this kind of functionality would be the easiest solution.

Martin James
  • 24,453
  • 3
  • 36
  • 60
  • sir i made some code can u check that if it appropriate as per our requirement...should i paste it here? – Dany Sep 25 '12 at 06:57
  • please check the following link http://codereview.stackexchange.com/questions/15901/is-this-appropriate-to-requirement – Dany Sep 25 '12 at 08:09
0

If you'd use the boost::asio framework for both network and disk IO you would likely get away with far less threads than one per machine to handle.

Plus, it has a nice high-level interface for socket programming.

moooeeeep
  • 31,622
  • 22
  • 98
  • 187