0

I have a long PHP script (for scraping), It takes around 2 hours to execute, I want to run this on weekly basis, I have cron job option in Cpanel and i know how to use it, But i think it is not useful for that long scripts

I read some other answers about long PHP script, They say long PHP script should be executed via shell. I read this blog http://symcbean.blogspot.com/2010/02/php-and-long-running-processes.html But i couldn't understand anything,

I have never worked with shell, But i can spend time to learn it, But would it be possible to run the script via shell and also on weekly basis at same time automatically

johnDoe
  • 63
  • 2
  • 8
  • To me the best practice for you here is, run your php scrip via CLI. A PHP script executed from the command-line or from a shell script, cron job, etc. does not have a timeout.For CLI-invoked scripts, even if you set the PHP script's timeout dynamically with the set_time_limit() function, it has no effect. – Shahroze Nawaz Oct 31 '17 at 11:47
  • @ShahrozeNawaz I am trying to do this with cron job, I am not familiar with CLI, Is that better way ? – johnDoe Oct 31 '17 at 17:08

2 Answers2

0

When you run a cron job it basically runs a shell command of your choosing. That shell command can be php <your_script_name_here>.php I think anyway that cron job is the way to go (scheduled task on windows machine).

in need of help
  • 1,606
  • 14
  • 27
  • Thanks, So, should i use the cron job option in the cpanel, Would you please explain it in more details, I would really appreciate any help, I have wasted lots of time, But nothing helped – johnDoe Oct 31 '17 at 09:54
  • Thanks, i will give the cron job a try. – johnDoe Oct 31 '17 at 09:59
0

If you have a script fro scrapping I recommend you to use HHVM instead of php-fpm you gonna gain lots of performance.

And in second part you gonna need may be a worker or a job queue system to run your script via a cronjob.

If you insert data in your data-base don't insert row by row but create an array with all data and then insert all in one time. (there is lot of thing you can do to have best performance)

MEDALI
  • 55
  • 5