0

i am running a webgame based on php . each player will create up to 100 attacks tasks that will be stored in db with details of it and end_time , every page load of any player on the website will call a function to check everything in db that supposed to end_time to be now or less . if found any , then it will start processing them ( make the actual attack and get its results ).

it is like travian if you know it . my issue happens when there are 20 task or more at the same moment then it will stuck and only process one task per page load ( which kills the game. )

i use foreach loop in order to process the array of queues that gotten by querying the db .

code is :

every page load will call :

 public function processQueue($type = 3, $playerId = 0)
    {
        global $gameConfig;
        $this->load_model('Mutex', 'mutex');
        $this->mutex->releaseOnTimeout();
        if ($this->mutex->lock()) { // locker to forbid 2 players from working on same rows and cause double values. 
            $this->processTaskQueue($type, $playerId);
            }
            $this->mutex->release();
        }
    }

Processing function :

public function processTaskQueue($type, $playerId)
    {
        $p_type = QS_ACCOUNT_DELETE . ',' . QS_MERCHANT_GO . ',' . QS_MERCHANT_BACK . ',' . QS_WAR_REINFORCE . ',' . QS_WAR_ATTACK . ',' . QS_WAR_ATTACK_PLUNDER . ',' . QS_WAR_ATTACK_SPY . ',' . QS_CREATEVILLAGE . ',' . QS_TATAR_RAISE . ',' . QS_SITE_RESET . ',' . QS_CROP_DELETE . ',' . QS_ARTEFACTS_RAISE;
        if ($type == 1) {
            $expr = "((q.player_id=" . $playerId . " AND q.proc_type!=" . QS_TROOP_TRAINING . ") OR q.proc_type IN (" . $p_type . "))";
        } elseif ($type == 2) {
            $expr = "(q.player_id=" . $playerId . " OR q.proc_type IN (" . $p_type . "))";
        } else {
            $expr = "q.proc_type!=" . QS_TROOP_TRAINING;
        }
        $result = db::get_all("SELECT  q.id, q.player_id, q.village_id, q.to_player_id, q.to_village_id, q.proc_type, q.building_id, q.proc_params, q.threads, q.execution_time, TIMESTAMPDIFF(SECOND, NOW(),q.end_date) remainingTimeInSeconds FROM p_queue q WHERE TIMESTAMPDIFF(SECOND, NOW(),(q.end_date - INTERVAL (q.execution_time*(q.threads-1)) SECOND)) <= 0 AND $expr ORDER BY TIMESTAMPDIFF(SECOND, NOW(),(q.end_date - INTERVAL (q.execution_time*(q.threads-1)) SECOND)) ASC");
        foreach ($result as $resultRow) {
            $remain = $resultRow['remainingTimeInSeconds'];
            if ($remain < 0) {
                $remain = 0;
            }
            $resultRow['threads_completed_num'] = $resultRow['execution_time'] <= 0 ? $resultRow['threads'] : floor(($resultRow['threads'] * $resultRow['execution_time'] - $remain) / $resultRow['execution_time']);
            if ($this->processTask($resultRow)) {
                unset($result);
                $this->processQueue($type, $playerId);
                break;
            }
        }
        unset($result);

    }

locker function :

define("__QS_LOCK_FS_", MODELS_DIR . "lock");

class Mutex_Model extends Model
{

    public function lock()
    {
        if (0 < db::count("UPDATE g_settings gs SET gs.qlocked=1, qlocked_date=NOW() WHERE gs.qlocked=0") && ($fp = fopen(__QS_LOCK_FS_, "r")) != FALSE) {
            if (flock($fp, LOCK_EX)) {
                fclose($fp);
                return TRUE;
            }
            fclose($fp);
        }
        return FALSE;
    }

    public function release()
    {
        $this->_releaseInternal();
        db::query("UPDATE g_settings gs SET gs.qlocked=0");
    }

    public function releaseOnTimeout()
    {
        if (0 < db::count("UPDATE g_settings gs SET gs.qlocked=0 WHERE gs.qlocked=1 AND TIME_TO_SEC(TIMEDIFF(NOW(), gs.qlocked_date)) > 120")) {
            $this->_releaseInternal();
        }
    }

    public function _releaseInternal()
    {
        if (($fp = fopen(__QS_LOCK_FS_, "r")) != FALSE) {
            flock($fp, LOCK_UN);
            fclose($fp);
        }
    }

}

what i need is to process all of the results in the array quickly ( could be 200 arrays in it or more ) and make it work all of the time without causing an issue ( tried ajax code to run every second but that cause missing stuff when a player logs out while it is running -- the row the script was handling while he is leaving will be miss ) and i need to process everything at once in it instead of the foreach loop ( it is very slow and will be stuck at every refresh )

i tried the following :

1- delete the break and make it continue running till it finished , but that gives players 500 http msg and make loading very very very slow for the one who got the locker opens ( mutex )

2- make cron job with php file that loops for 59s on processQueue function , but still it will only process like 10 - 20 tasks at one second maximum which is very slow

3- remove the locker which cause double values (meaning a. player send 50 solider they will return 100 )

my server infos are : 12 CPUs ( only 2 of them consumed ) 131,744,852 memory Configurations :

Start Servers [?] 5 default

10 Minimum Spare Servers [?] 5 default

10 Maximum Spare Servers [?] 10 default

20 Server Limit (Maximum: 20,000) [?] 256 default

8042 Max Request Workers [?] 150 default

8042 Max Connections Per Child [?] 10000 default

10000 Keep-Alive [?] On

PHP-FPM Pool options Max Requests
95000 cPanel Default: 20 Max Children
10000 cPanel Default: 5 Process Idle Timeout :30 i am really confused and hoping you can help me out here ..

Your Common Sense
  • 156,878
  • 40
  • 214
  • 345
Zeyd Harb
  • 11
  • 1
  • Question is not answerable... `what i need is to process all of the results in the array quickly` and `and make it work all of the time without causing an issue` is basically the wish of _every_ programmer out there. Making this actually happen is the Art in programming... writing good, fast and resilient code requires alot of experience, same applies for application or database design. – Honk der Hase Jul 16 '22 at 10:14
  • thanks for kind answer , i really tried everything i could thing of .. is there anyway to make at better at least ? – Zeyd Harb Jul 16 '22 at 11:35
  • First, definitely process this with a [service](https://stackoverflow.com/a/16577806/231316) or long-running process that doesn’t need to be invoked by a visitor (that method is often referred to as “poor man’s cron”). Second, consider switching to a true message queue system. Symfony’s [messenger component](https://symfony.com/doc/current/messenger.html) can be run without the full framework and supports a bunch of queues such as Redis, RabbitMQ or Beanstalkd. – Chris Haas Jul 16 '22 at 14:05
  • Another option is to never update or delete, and just insert only. Yes, this will store more data, but you should be able to calculate how much and your server sounds pretty beefy. This completely removes the need for those locks, although depending on your application it is technically possible for two select statements to get different results if an insert happens in between. For querying, having good indexes should remove any performance problems for selects. – Chris Haas Jul 16 '22 at 14:11
  • Are you using PHP's multi-threading? Is one database connection shared between threads (won't work!). Or are you launching new PHP requests for each 'attack'? It's unclear how much RAM you have. – Rick James Jul 16 '22 at 14:26
  • hello Chris Haas, thanks for replying . the issue is that it must delete / update on each task in the thread ,everything is well indexed and slow query log is empty it is all in the loop itself .. , i am going to read about the messenger component hoping it would be the answer , thanks a lot – Zeyd Harb Jul 16 '22 at 17:35
  • hello Rick James i am lunching new request for each task , my ram is 131 g – Zeyd Harb Jul 16 '22 at 17:38

0 Answers0