0

I need to write a socket server that will be handling at least something about 1000 (much more in future) low-traffic permanent connections. I have made a draft version on PHP for testing purposes (we are developing a monitoring hardware, so we needed to develop and test a conversation protocol and hardware capabilities), which suited me very well when i had just a couple of clients connected. But when the amount of connections had grown to ten, some critical issues appeared. Here some info about server architecture:

I have a master process, which waits for socket connections and on connecting creates a child process (that serves this connection from now on) using pcntl_fork(). Also i am setting up a PDO connection to MySQL in master process. All the child processes are sharing the same single PDO object. At first i was afraid of getting some collisions during simultaneous queries, but i haven't encountered them, even through stress-test (10 children were making queries in the loop without stopping). But there is usleep(500000) in each child, so it could be luck, though i had this testing running for a couple of hours. But such load should not be present even at 1k clients connected, due to rare conversations between them and server.

So here is my first question: is it safe to use single PDO object for a big amount of child processes (ideally there would be around 1000)? I can use single connection for each child, but MySQL doesn't support nearly as much connections.

The second issue is in getting parasite MySQL connections. As i mentioned before, i have only one PDO object. But when i have more than one clients connected, and after they had run some queries, i see in mytop that there is more than one DB connection, and i could not find any correlation between the amount of connections and amount of child processes i have. For example i have 3 childs, and 5 DB connections. I tried to establish persistent connections, and it didn't changed anything.

Second question: Is it PDO who makes those additional connecitons to MySQL, or it is the MySQL driver? And is there a way to force them to use one connection? I don't think it could be my fault, my code prints an alert to console every time i call method which creates PDO object, and that happens only once, at the script start, before forking. After that i only run querys out of children, using parent's PDO object. Once again i can not afford to have so many connections due to MySQL limitations.

Third question: Will be one thousand of socket connections a problem by itself? Aside of the CPU and database load, i mean. Or i should do some amount of lesser servers (128 connections for example), that will tell the clients to connect to other one if max number of connects is exceeded?

Thanks in advance for your time and possible answers.

navij
  • 19
  • 1
  • 5
  • Why would you want to build a layer between the application and the database? Have you ever heard of connection pooling and connection reusing? I don't want to sound rude but it seems your experience is not enough to build services for a project as big as you describe. – Daniel W. Oct 25 '16 at 09:07
  • There is nothing rude about your comment, i really don't have much experience in that field, that's basically why i asked the question. But we all gotta start somwhere, you know. Also i am not that green to not know what a connection pooling is, but according to our calculations one DB connection will be just enough to fit our needs (there should'nt be more than 2 simple queries per second in worst cases). My problem was, as i see now, in a lack of understanding how forks work. – navij Oct 25 '16 at 16:53

1 Answers1

0

Currently your primary concern should be your socket server architecture. Forking a process for each client is super heavy. AFAIK an average PC can tolerate around 2000 threads and it's not going to work fast. Switching between processes means that CPU should save its state in memory, and if you have enormous amount of processes, CPU will be busy with memory IO and will have little time for actually doing stuff.

You may want to look at Apache for inspiration. In Apache they use a fixed amount of worker processes/threads, each process/thread working with multiple clients via select function and sockets in a non-blocking mode. This is a far more robust approach.

Regarding database IO, I would spawn a process/thread that would be the sole owner of database connections. Worker processes would communicate with the DB IO process using IPC (in case of processes) or lock-free queues (in case of threads). This approach makes you independent of PDO implementation details (if it is thread safe or does it spawn connections etc).

P. S. I suspect that you actually spawn new PDO objects with forking (forking merely means making a copy of a process with its memory and everything inside it) and PDO objects create and shut down connections on demand. It may explain why you're not seeing correlation between low traffic clients and DB connections.

u354356007
  • 3,205
  • 15
  • 25
  • You have some wordings wrong. Process forking spawns a new (child)process. A thread is not a process/fork. When you have 16 CPU cores, you shouldnt start more than 15 threads of the same programm concurrently. – Daniel W. Oct 25 '16 at 09:03
  • @DanFromGermany You're obviously correct with forking, but I don't see a contradiction between your comment and the answer. Also, I use terms _thread_ and _process_ interchangeably here because most of the post is true for both + I'm not convinced that "use processes for everything" is a correct approach to this task. – u354356007 Oct 25 '16 at 09:49
  • @DanFromGermany or maybe `fork` in PHP is different from `fork` in C? In C it creates a copy of a parent process, even with the instruction pointer pointing to the same instruction. The only different is `fork` return value - it's 0 for parent or smth like this. – u354356007 Oct 25 '16 at 09:52
  • PHP uses C fork internally ;-) It is indeed a copy of the process but it's called a child, because it inherits some stuff from the parent. I'm not sure now what was wrong lol nvm! You are right, fork vs thread have memory advantage/disadvantages. Threads should be used for smaller tasks that share some stuff in memory, fork makes a copy and does not share afaik. I found PHP threads better maintainable and faster than forks but it surely depends. See http://stackoverflow.com/questions/16354460/forking-vs-threading – Daniel W. Oct 25 '16 at 11:58
  • Yes, you were absolutely right - when you fork a process, it copies the PDO object, and it miraculously just works after that, of course with re-opening connection. Silly me. BTW, my server uses some kind of non-blocking socket reading (on each tick i use socket_recv() with MSG_PEEK & MSG_DONTWAIT and if socket is empty i run some other logic. I should use socket_set_nonblock() for this probably, but this scheme is working too), so i can rewrite it to serve a number of sockets if one worker, like nginx does. – navij Oct 25 '16 at 17:12
  • Also i have thought about single worker handling the query queue, and about using nginx-like architecture with a fixed number of workers , serving a bunch of clients each, from the very beginning of the project. But i had to use a simplier architecure for our proof of concept, and to develop a conversation protocol between server and client, so the people who develop hardware and it's firmware could start working. And now it seems that time had come to rewrite it in a proper way. – navij Oct 25 '16 at 17:22