long poll hanging all other server requests

Question

I have a simple long poll request on my website to check if there are any new notifications available for a user. As far as the request goes, everything seems to work flawlessly; the request is only fulfilled once the database is updated (and a notification has been created for that specific user), and a new request is sent out straight after.

The Problem

What I have noticed is that when the request is waiting for a response from the database (as long polls should), all other requests to the server will also hang with it - whether it be media files, AJAX requests or even new pages loading. This means that all requests to the server will hang until I close my browser and reopen it.

What is even stranger is that if I visit another one of my localhost sites (my long poll is on a MAMP virtualhost site, www.example.com), their is no problem and I can still use them as if nothing has happened - despite the fact they're technically on the same server.

My Code

This is what I have on my client side (longpoll.js):

window._Notification = {
    listen: function(){
        /* this is not jQuery's Ajax, if you're going to comment any suggestions, 
        *  please ensure you comment based on regular XMLHttpRequest's and avoid
        *  using any suggestions that use jQuery */
        xhr({
            url: "check_notifs.php",
            dataType: "json",
            success: function(res){
                /* this will log the correct response as soon as the server is 
                *  updated */
                console.log(res);
                _Notification.listen();
            }
        });
    }, init: function(){
        this.listen();
    }
}

/* after page load */
_Notification.init();

And this is what I have on my server side (check_notifs.php):

header("Content-type: application/json;charset=utf-8", false);

if(/* user is logged in */){
    $_CHECKED = $user->get("last_checked");

    /* update the last time they checked their notifications */
    $_TIMESTAMP = time();
    $user->update("last_checked", $_TIMESTAMP);

    /* give the server a temporary break */
    sleep(1);

    /* here I am endlessly looping until the conditions are met, sleeping every
    *  iteration to reduce server stress */
    $_PDO = new PDO('...', '...', '...');
    while(true){
        $query = $_PDO->prepare("SELECT COUNT(*) as total FROM table WHERE timestamp > :unix");
        if($query->execute([":unix" => $_CHECKED])){
            if($query->rowCount()){
                /* check if the database has updated and if it has, break out of
                *  the while loop */
                $total = $query->fetchAll(PDO::FETCH_OBJ)[0]->total;
                if($total > 0){
                    echo json_encode(["total" => $total]);
                    break;
                }
                /* if the database hasn't updated, sleep the script for one second,
                *  then check if it has updated again */
                sleep(1);
                continue;
            }
        }
    }
}   

/* for good measure */
exit;

I have read about NodeJS and various other frameworks that are suggested for long-polling, but unfortunately they're currently out of reach for me and I'm forced to use PHP. I have also had a look around to see if anything in the Apache configuration could solve my problem, but I only came across How do you increase the max number of concurrent connections in Apache?, and what's mentioned doesn't seem like it would be the problem considering I can still use my other localhost website on the same server.

Really confused as to how I can solve this issue, so all help is appreciated,
Cheers.

Your issue is most likely `if($total > 0){`, nothing is probably returning a value greater than zero so the loop never ends. — Get Off My Lawn, May 01 '19 at 15:36
Also, you query is always the same on every loop, so a loop seems useless here because you will always return the same data. To change that move the `$_CHECKED`, `$_TIMESTAMP` and `$user->...` stuff into the while loop. — Get Off My Lawn, May 01 '19 at 15:39
@GetOffMyLawn the point of the loop is to consistently check if the database is being updated :) But that's not the problem (the long poll itself works correctly), the problem is that it hangs every other request to that [virtualhost?] server — GROVER., May 01 '19 at 15:46
Please move `$_PDO = new PDO('...', '...', '...');` outside the `while`-loop and see if the problem persists. — huysentruitw, May 05 '19 at 12:52
@huysentruitw okay! the only reason I'm doing that in the first place is so I can close the connection - but not permanently :) see `$_PDO = null;` — GROVER., May 05 '19 at 12:53
Why do you remove the PDO connection and create a new one every iteration? Also I am inclined to think it is more a webserver configuration issue. Do you have specific Apache config? Do you use mod-php or php-fpm? If FPM please post that config too. — ferdynator, May 05 '19 at 12:53
@ferdynator am using the default config of MAMP currently. So I'm assuming that would be mod_php :) — GROVER., May 05 '19 at 12:55
IMO PHP scripts should not be hanging like this... The leaking of memory will probably be huge... Why don't you just query PHP every 10 seconds on your client side to check if new notifications exist? — Carlos Alves Jorge, May 06 '19 at 09:48
@CarlosAlvesJorge because I’m trying to avoid pointless HTTP requests. According to a large portion of the programming community (well at least from what I’ve seen), long polling is much more efficient than short polling. — GROVER., May 06 '19 at 09:52
@GROVER. It will never be more efficient if you have an endless loop querying your database... Basically you will be DDNS yourself with just a couple of users... — Carlos Alves Jorge, May 06 '19 at 09:57
@CarlosAlvesJorge but that’s the main idea of long polling... consistently checking if the database has been updated on the server side — GROVER., May 06 '19 at 09:58
@GROVER. Long pooling is used to consistently check event changes (that ideally are not resource intensive) to use it to perform endlessly "SELECT COUNT(*)" on a database is a call for disaster. If you cannot use websockets for real time notifications just do short pooling client side. If you want to keep your approach at least give a much bigger timeout than one second... — Carlos Alves Jorge, May 06 '19 at 10:08
@CarlosAlvesJorge how did Facebook, Twitter and Instagram do it back in the day when WebSockets weren’t available then? — GROVER., May 06 '19 at 10:09
Additionally having it controlled client side you can stop making requests when the window is not focused or when there is no mouse movement for longer than a period. I would want to have people it my database indefinitely when they forget a session open... — Carlos Alves Jorge, May 06 '19 at 10:10

Jorge Fuentes González · Accepted Answer · 2019-05-08T07:34:08.103

1

What is actually happening is that php is waiting for this script to end (locked) to serve the next requests to the same file.

As you can read here:

there is some lock somewhere -- which can happen, for instance, if the two requests come from the same client, and you are using file-based sessions in PHP : while a script is being executed, the session is "locked", which means the server/client will have to wait until the first request is finished (and the file unlocked) to be able to use the file to open the session for the second user.

the requests come from the same client AND the same browser; most browsers will queue the requests in this case, even when there is nothing server-side producing this behaviour.

there are more than MaxClients currently active processes -- see the quote from Apache's manual just before.

There's actually some kind of lock somewhere. You need to check what lock is happening. Maybe $_PDO is having the lock and you must close it before the sleep(1) to keep it unlocked until you make the next request.

You can try to raise your MaxClients and/or apply this answer

Perform session_write_close() (or corresponding function in cakephp) to close the session in the begin of the ajax endpoint.

edited May 08 '19 at 07:34

answered May 06 '19 at 09:18

Jorge Fuentes González

11,568
4
44
64

Whereabouts should I place the `session_write_close` function? – GROVER. May 06 '19 at 09:54
@GROVER. At the start of the file. But keep in mind that it was because it was the lock in that case. In you case you should look for your lock. Are you using sessions? – Jorge Fuentes González May 06 '19 at 10:11
I am using sessions :) but only for certain user data really – GROVER. May 06 '19 at 10:11
@GROVER. your `$user` maybe has the session lock. Try to unlock after the last `$user->update` or whatever thing that needs an open session. – Jorge Fuentes González May 06 '19 at 10:19
Congratulations on 8K reputation. ;) – GROVER. May 08 '19 at 07:22

long poll hanging all other server requests

The Problem

My Code

1 Answers1

Linked