11

Currently, I tried to prevent an onlytask.php script from running more than once:

$fp = fopen("/tmp/"."onlyme.lock", "a+");
if (flock($fp, LOCK_EX | LOCK_NB)) {
  echo "task started\n";
  //
    while (true) {
      // do something lengthy
      sleep(10);
    }
  //
  flock($fp, LOCK_UN);
} else {
  echo "task already running\n";
}
fclose($fp);

and there is a cron job to execute the above script every minute:

* * * * * php /usr/local/src/onlytask.php

It works for a while. After a few day, when I do:

ps auxwww | grep onlytask

I found that there are two instances running! Not three or more, not one. I killed one of the instances. After a few days, there are two instances again.

What's wrong in the code? Are there other alternatives to limit only one instance of the onlytask.php is running?

p.s. my /tmp/ folder is not cleaned up. ls -al /tmp/*.lock show the lock file was created in day one:

-rw-r--r--  1 root root    0 Dec  4 04:03 onlyme.lock
ohho
  • 50,879
  • 75
  • 256
  • 383
  • Very interesting, but it seems like this should be correct... Is the skeleton of your code exactly that? I do wonder though what happens if flock is passed `false` rather than a resource. Surely it returns false. It would be very, very odd if it doesn't, but I've seen PHP make some pretty odd choices. Also, the internals of flock could be flawed (or the system calls used -- though that would've been noticed long before now). – Corbin Dec 12 '12 at 09:24
  • The code is almost 100% like the real code. The `// do something lengthy` actually `exec()` another .php which is the script which should run periodically, but not more than once. – ohho Dec 12 '12 at 09:34
  • Second process will wait for finish first process. See bug: http://stackoverflow.com/questions/5524073/lock-nb-ignored – Vitaly Fadeev Nov 20 '14 at 09:45
  • Possible duplicate of [How to prevent multiples instances of a script?](http://stackoverflow.com/questions/1861321/how-to-prevent-multiples-instances-of-a-script) –  Nov 29 '16 at 22:39

7 Answers7

13

You should use x flag when opening the lock file:

<?php

$lock = '/tmp/myscript.lock';
$f = fopen($lock, 'x');
if ($f === false) {
  die("\nCan't acquire lock\n");
} else {
  // Do processing
  while (true) {
    echo "Working\n";
    sleep(2);
  }
  fclose($f);
  unlink($lock);
}

Note from the PHP manual

'x' - Create and open for writing only; place the file pointer at the beginning of the file. If the file already exists, the fopen() call will fail by returning FALSE and generating an error of level E_WARNING. If the file does not exist, attempt to create it. This is equivalent to specifying O_EXCL|O_CREAT flags for the underlying open(2) system call.

And here is O_EXCL explanation from man page:

O_EXCL - If O_CREAT and O_EXCL are set, open() shall fail if the file exists. The check for the existence of the file and the creation of the file if it does not exist shall be atomic with respect to other threads executing open() naming the same filename in the same directory with O_EXCL and O_CREAT set. If O_EXCL and O_CREAT are set, and path names a symbolic link, open() shall fail and set errno to [EEXIST], regardless of the contents of the symbolic link. If O_EXCL is set and O_CREAT is not set, the result is undefined.

UPDATE:

More reliable approach - run main script, which acquires lock, runs worker script and releases the lock.

<?php
// File: main.php

$lock = '/tmp/myscript.lock';
$f = fopen($lock, 'x');
if ($f === false) {
  die("\nCan't acquire lock\n");
} else {
  // Spawn worker which does processing (redirect stderr to stdout)
  $worker = './worker 2>&1';
  $output = array();
  $retval = 0;
  exec($worker, $output, $retval);
  echo "Worker exited with code: $retval\n";
  echo "Output:\n";
  echo implode("\n", $output) . "\n";
  // Cleanup the lock
  fclose($f);
  unlink($lock);
}

Here goes the worker. Let's raise a fake fatal error in it:

#!/usr/bin/env php
<?php
// File: worker (must be executable +x)
for ($i = 0; $i < 3; $i++) {
  echo "Processing $i\n";
  if ($i == 2) {
    // Fake fatal error
    trigger_error("Oh, fatal error!", E_USER_ERROR);
  }
  sleep(1);
}

Here is the output I got:

galymzhan@atom:~$ php main.php 
Worker exited with code: 255
Output:
Processing 0
Processing 1
Processing 2
PHP Fatal error:  Oh, fatal error! in /home/galymzhan/worker on line 8
PHP Stack trace:
PHP   1. {main}() /home/galymzhan/worker:0
PHP   2. trigger_error() /home/galymzhan/worker:8

The main point is that the lock file is cleaned up properly so you can run main.php again without problems.

galymzhan
  • 5,505
  • 2
  • 29
  • 45
  • 2
    if the code dies at location `echo "Working\n";` as a result of a bug, the .lock file will remain on disk and not be cleaned up by anyone, thus no more processing will be done. – ohho Dec 12 '12 at 10:02
  • @ohho That's the case with any tool which relies on lock files. You can acquire lock inside main script, then run worker script as a child process and then remove the lock file in main script – galymzhan Dec 12 '12 at 11:18
  • @galymzhan `LOCK` will be released when the process die, file will not be deleted when the process die. Please check documentation. – ohho Dec 13 '12 at 01:34
  • what if the `main.php` is dead before `unlink`? nothing will be done after that. – ohho Dec 13 '12 at 06:13
  • @ohho How is that possible? You don't do anything complicated in `main.php`, so the only possibility to die is a bug in PHP itself (and there is nothing you can do about it). Unless you're writing software for a mars rover, the solution should be reliable enough. You can't be 100% protected from software crashes after all – galymzhan Dec 13 '12 at 07:16
  • 1
    for example, server reboots when `exec($worker, $output, $retval);` is running. – ohho Dec 13 '12 at 07:59
  • @ohho Add script on startup which deletes the lock file before cron – galymzhan Dec 13 '12 at 08:07
  • There seems to be a better way for when you want to avoid leftover locks, described at http://stackoverflow.com/questions/17708885/flock-removing-locked-file-without-race-condition – Josip Rodin Nov 17 '15 at 23:39
10

Now I check whether the process is running by ps and warp the php script by a bash script:

 #!/bin/bash

 PIDS=`ps aux | grep onlytask.php | grep -v grep`
 if [ -z "$PIDS" ]; then
     echo "Starting onlytask.php ..."
     php /usr/local/src/onlytask.php >> /var/log/onlytask.log &
 else
     echo "onlytask.php already running."
 fi

and run the bash script by cron every minute.

ohho
  • 50,879
  • 75
  • 256
  • 383
  • This is usually the way I (system admin) do it. But, I came across a situation where a php script executes without errors and correct return values when called from BASH script run by cron. However, the result was not as expected. In my case, php was supposed to create a dir on a remote server with an ssh call. When php scrpit is run directly, it creates the dir. But, through BASH script, it does not create it. @galymzhan answer solved the part where php should run itself without duplicate process. I am yet to test through cron. – anup Dec 28 '18 at 13:11
1
<?php

$sLock = '/tmp/yourScript.lock';

if( file_exist($sLock) ) {
 die( 'There is a lock file' );
}

file_put_content( $sLock, 1 );

// A lot of code

unlink( $sLock );

You can add an extra check by writing the pid and then check it within file_exist-statement. To secure it even more you can fetch all running applications by "ps fax" end check if this file is in the list.

Niclas Larsson
  • 1,317
  • 8
  • 13
  • It's possible for file_exists to be called while a script is just about to create the file, thus it being created/written to twice. This doesn't work. (Also, unsetting the file wouldn't mean anything as far as the actual file goes.) – Corbin Dec 12 '12 at 09:13
  • I suppose this application is not multithreaded as of php so that wont be a problem. Changed the unset to unlink (misspelled) – Niclas Larsson Dec 12 '12 at 09:18
  • If you remove the lock by hand (eg rm -rf /tmp/yourScript.lock) you have your self to blame. – Niclas Larsson Dec 12 '12 at 09:20
  • 2
    The problem is that file_exists and file_put_contents are not one atomic unit. You have two operations: x then y. Assume you have to processes. Ideally they would go as x1 y1 x1 y2 (though y2 wouldn't happen). What *can* happen though is x1 x2 y1 y2. That means the check just failed. – Corbin Dec 12 '12 at 09:23
  • if you run the script multiple times at the exact moment, yes. But this is a cron job. – Niclas Larsson Dec 12 '12 at 09:26
  • Oops! The cron job aspect of it slipped my mind. Sorry; you're correct. This would work. The `flock` strategy has the advantage of not requiring process spawning time to be significantly spaced though (but apparently that strategy isn't working). – Corbin Dec 12 '12 at 09:28
0

try using the presence of the file and not its flock flag :

$lockFile = "/tmp/"."onlyme.lock";
if (!file_exists($lockFile)) {

  touch($lockFile); 

  echo "task started\n";
  //
  // do something lengthy
  //

  unlink($lockFile); 

} else {
  echo "task already running\n";
}
k1dbl4ck
  • 176
  • 1
  • 7
  • it works in the context of the question since there will never be a race condition since the script creates its own lockfile. multiple scripts using same lockfile = different story. – k1dbl4ck Dec 12 '12 at 09:24
  • Nope, has nothing to do with the script file. Has to do with more than 1 process running at a time. The file_exists/touch/unlink chain isn't guaranteed to happen uninterrupted for each process without due to processor scheduling. (Although I must cede, Niclas just made a very valid point that the two processes will never spawn off quickly enough to matter, so yes, I suppose this would work. It depends the spawning being spaced apart though.) – Corbin Dec 12 '12 at 09:25
  • 2
    "check the presence of the file" does not work. if the process die in middle of a bug, the .lock file will remain in disk and prevents the cron job from starting another process. – ohho Dec 12 '12 at 09:47
0

You can use lock files, as some have suggested, but what you are really looking for is the PHP Semaphore functions. These are kind of like file locks, but designed specifically for what you are doing, restricting access to shared resources.

Jordan Mack
  • 8,223
  • 7
  • 30
  • 29
  • From the docs: "sem_acquire() is blocking, meaning that subsequent calls with the same semaphore will block indefinitely until the semaphore is released. This ensures serialization, but it is not very practical if all you want to do is check if you should proceed or not. Unfortunately, PHP does not yet support any method of querying the state of a semaphore in a non-blocking manner." – AlexMax May 29 '13 at 15:15
0

Never use unlink for lock files or other functions like rename. It's break your LOCK_EX on Linux. For example, after unlink or rename lock file, any other script always get true from flock().

Best way to detect previous valid end - write to lock file few bytes on the end lock, before LOCK_UN to handle. And after LOCK_EX read few bytes from lock files and ftruncate handle.

Important note: All tested on PHP 5.4.17 on Linux and 5.4.22 on Windows 7.

Example code:

set semaphore:

$handle = fopen($lockFile, 'c+');
if (!is_resource($handle) || !flock($handle, LOCK_EX | LOCK_NB)) {
    if (is_resource($handle)) {
        fclose($handle);
    }
    $handle = false;
    echo SEMAPHORE_DENY;
    exit;
} else {
    $data = fread($handle, 2);
    if ($data !== 'OK') {
        $timePreviousEnter = fileatime($lockFile);
        echo SEMAPHORE_ALLOW_AFTER_FAIL;
    } else {
        echo SEMAPHORE_ALLOW;
    }
    fseek($handle, 0);
    ftruncate($handle, 0);
}

leave semaphore (better call in shutdown handler):

if (is_resource($handle)) {
    fwrite($handle, 'OK');
    flock($handle, LOCK_UN);
    fclose($handle);
    $handle = false;
}
Enyby
  • 4,162
  • 2
  • 33
  • 42
  • I don't quite understand your lead statement - if you're unlinking the lock file, that means you're already done with your locked section in that process. Similarly, if the file descriptor is closed, for any reason, it means the locking is done. The other scripts should at that point be able to proceed. Why would that be broken? – Josip Rodin Nov 17 '15 at 22:19
  • @JosipRodin I thought so too, until I got a problem with that. You can try to test this behavior. Maybe it depends on the system and its implementation. On Gentoo I had a problem. I do not see any reason to look for the cause, if the above code works. With regard to the removal of the file - deleting a file does not cause the closure of the file descriptor. inode is still available. – Enyby Nov 18 '15 at 13:31
  • I suppose you're referring to the race condition described at http://stackoverflow.com/questions/17708885/flock-removing-locked-file-without-race-condition The reason the simpler methods should be better is that they cause less I/O, which can matter in systems that have a large amount of lock contention and a large amount of I/O on the same devices. – Josip Rodin Nov 18 '15 at 18:20
  • @JosipRodin May be you right. But `fstat` (or `stat`) on `Windows` always have `inode` == 0. I need solution worked in both systems. And I don't like `while(true)` - it cause of high load sometimes. For example on highloaded systems. – Enyby Nov 18 '15 at 18:34
0

Added a check for old stale locks to galimzhan's answer (not enough *s to comment), so that if the process dies, old lock files would be cleared after three minutes and let cron start the process again. That's what I use:

<?php
$lock = '/tmp/myscript.lock';
if(time()-filemtime($lock) > 180){
    // remove stale locks older than 180 seconds
    unlink($lock);
}
$f = fopen($lock, 'x');
if ($f === false) {
  die("\nCan't acquire lock\n");
} else {
    // Do processing
    while (true) {
    echo "Working\n";
        sleep(2);
    }
    fclose($f);
    unlink($lock);
}

You can also add a timeout to the cron job so that the php process will be killed after, let's say 60 seconds, with something like:

* * * * * user timeout -s 9 60 php /dir/process.php >/dev/null
RWaters
  • 1
  • 3