1

I have a PHP script located at /var/www/site/update.php full. The script is started automatically from cron:

/usr/bin/php /var/www/site/update.php full

But when I start the same script from my website:

<?php exec("/usr/bin/php /var/www/site/update.php full") ?>

It runs for about 20 minutes and then starts displaying a lot of error messages. At the same time the page stops loading and writes error 504 Gateway Time-out onto the screen.

I guess curl won't help either. Are there any other options? the question is how to run so it could work independently of the browser. the code already exists and worked out. now it runs through the cron every hour. but there is a need to run it unscheduled by pressing a button or link on the site.

romanown
  • 325
  • 2
  • 9

3 Answers3

0

you can modify set_time_limit(-1); in the PHP script to prevent PHP from dying but 504 sounds like you have nginx running, so you will also have to modify it's time limits for that particular resouce using location config node (or main ngix config) to set *_timeout settings

if you don't need the output to be refreshed every second, I'd do this:

  1. when button is pressed, store the flag "pressed" in some file

    if($_POST['schedule_script_button']){ touch(DIR."/script_is_scheduled.flag"); }

  2. use cron to run the script every minute

* * * * * php /var/www/example.com/my-long-script.php

  1. script would have to check if that flag is set, otherwise terminate

    if(!file_exists(DIR."/script_is_scheduled.flag")){ exit; } //full script logic executed here

  2. if flag is set, run the script, and save output into a file. if you are going with crontab, ou can use echo in the script, and the crontab should be modified as follows:

* * * * * php /var/www/example.com/my-long-script.php > /var/www/example.com/my-script-output.txt

  1. allow web interface to read the "output file" from another URL. I.e you can now read my-script-output.txt by using file_get_contents PHP function and simply echo its contents.

I'm against setting execution limit on web-urls to high values, this can be used by attackers to put your site down by taking its resources by long-scripts execution

Jan Myszkier
  • 2,714
  • 1
  • 16
  • 23
  • if I could run cron at the click of a button it would have already solved the problem. please show how you can run the event crown with a button on the website? – romanown Aug 22 '18 at 08:01
  • the script will run every minute and check the flag. if you need instant run - just implement rabbitMQ into that (which can trigger faster than cron) and redirect to "show output from file" page once button is pressed. If you don't want to use most optimal solution, just change the PHP and nginx timeouts. – Jan Myszkier Aug 22 '18 at 08:04
  • is there any way to simply launch the cron event? – romanown Aug 22 '18 at 08:06
  • no, you cannot "launch" cron from web site, cron is scheduled job. "flag" can be an empty file which can be later checked (flag is being set) with `file_exists` function. if file exists - launch scrip, if file does not exist yet, skip running the script ( simply `exit` or `die` ). – Jan Myszkier Aug 22 '18 at 08:32
  • Your answer is close to what is happening on the server. but I want to do so not to depend on the cron when you move the site to another server. – romanown Aug 22 '18 at 20:06
  • then go with the bad solution - alter the timeout settings as I suggested in my answer. While not a good solution, it will not make dependent on the crontab – Jan Myszkier Aug 23 '18 at 07:54
  • Your answer is the most realistic at the moment because the cron is already working. and there are no other real solutions that are clear how to implement. RabbitMQ should be additionally installed and the Redis in the description there is no information that it can run other scripts. – romanown Aug 23 '18 at 18:48
0

What you are trying to achieve cannot be done in the PHP Web paradigm: your script runs in the context of an HTTP request ; its execution will always be terminated the moment your visitor interrupts the request (closes the tab or browser, turns off WiFi, etc.).

In order to trigger the execution of a long background task from an HTTP request (either the click of a button, or a programmatic curl call), your server must be composed of more ingredients: typically a Message Queue (such as RabbitMQ).

The button click would only trigger a Job message: put a (string) message into the Message Queue which describes the type of execution to run, and the parameters (for instance in a PHP-serialized way, or JSON) — and then returns immediately. Your visitor receives a quick HTTP response which says "Your job has been queued successfully".

Then, in addition to the Message Queue ingredient, you must run a continuously running process (in PHP CLI with infinite time limit, or a daemon in any language of your choice). This infinitely looping script puts itself in listening mode (subscriber) against the Message Queue service, and every time a message comes in, it does whatever long processing you need.

The status of the long job may be consulted by your visitor on a dedicated page of their running jobs, where they might also obtain the outcome for download or view etc. (example: the various "Convert YouTube to Mp3" services around the Web)

This is the only robust way of handling long background tasks triggered online.

There are several alternatives to the middleware that you can use to achieve this: RabbitMQ, Redis, Kafka, are some example, depending on the specifics of your situation (according to considerations such as the number of daemon instances, the need for priority management of the pending messages in queue, or persistence of the queue in case of failure etc.)

You can find in the RabbitMQ How-To a detailed guide with PHP.

Gabriel
  • 1,401
  • 13
  • 21
  • thank You. How i can use the redis for this task? Please give link. Redis is installed on the server as cache and i can use it. – romanown Aug 22 '18 at 16:48
  • @romanown I think you should accept the answer, and post a new question with the Redis tag, on how to implement the publish/subscribe pattern with Redis in PHP :-) then paste the link to your question and I will see if I know the answer. – Gabriel Aug 22 '18 at 17:16
  • I read on the site of the redis and did not find that it can run scripts. is this a solution? – romanown Aug 22 '18 at 18:01
  • @romanown I indicated a proper design to achieve your requirement. You should mark my answer as the Accepted Answer, and post another question about Redis+PHP, so that other people can answer (including me). I cannot provide further explanations and code samples in a comment — and giving an example of Redis queue implementation goes beyond the initial question that you asked. Thanks – Gabriel Aug 22 '18 at 18:29
  • i am create new queston about Redis. https://stackoverflow.com/questions/51974188/ – romanown Aug 22 '18 at 19:54
  • how can I take that as an answer when it's not clear how it can work at all. when the working example and it will work, I certainly be sure to note that as the answer. so far, these are only assumptions that I do not know how to implement. otherwise, this answer will mislead others. but my new question about the redis was followed only by bad answers. – romanown Aug 23 '18 at 11:39
  • I accepted the answer. but it is strange to take as an answer something that is unknown how it will work and not checking it myself. – romanown Aug 23 '18 at 18:29
  • the previous answer I performed and it works. if Your option will work then I will gladly note Your answer as I use the Redis I like more than working with the cron. and Your solution is more universal. – romanown Aug 23 '18 at 18:35
  • There is a lot of misinformation and bad advice here. It's trivial to isolate the php execution from the request (but still a bad idea to launch a running script in the webserver process group). Using asynchronous messaging is overkill. Use 'at' or 'incron' : http://symcbean.blogspot.com/2010/02/php-and-long-running-processes.html – symcbean Aug 25 '18 at 21:39
  • @symcbean I don't think there is misinformation in my answer. What you are pointing at, is how to orphanize a *nix process from its parent, and make it a child of init instead. This won't prevent the Web server to terminate the initial PHP script when the socket is closed, even though the detached child could continue 'forever'. As for bad advice: unless one wants to remain stuck in year 2005 with non-distributable, non-scalable, non-transactional programs, using async messaging is definitely the way to go, especially in the context of a Web application. – Gabriel Aug 26 '18 at 08:02
0

I did as recommended in this discussion php execute a background process. page still not loaded but its result is. it's probably not the best solution but it works. I expect a better solution.

romanown
  • 325
  • 2
  • 9
  • this is because, trying to execute a PHP in a background process *is bad practice*. There are modern, robust, documented ways to achieve what you're trying to do. – Gabriel Aug 26 '18 at 08:04
  • Thanks. I understand that and will try to use the RabbitMQ later. but the use of the Redis I was told that this is impossible with his help. – romanown Aug 27 '18 at 06:48