2

I have written a Zend Framework based cron service for parallel tasks, and ran into issues with the child threads sharing resources with the parent. I solved the database connection issue, but I'm now seeing periodic issues with Zend_Db_Table_Abstract failing to save table metadata to metadata cache.

Failed saving metadata to metadataCache

I initialize the metadata cache during bootstrap. Rather than copying my code from the bootstrap and executing after forking, I thought that it might be better to call the Bootstrap->_init[...] functions by using $application->bootstrap('[...]').


UPDATE

Because Zend_Controller_Front::getInstance() is a Singleton, using it to get the bootstrap instance and call the functions that way returns me to the same issue with shared resources that I had already resolved.

I want to somehow keep this DRY while avoiding the issues with shared resources after forking.

Community
  • 1
  • 1
Sonny
  • 8,204
  • 7
  • 63
  • 134

4 Answers4

3

The Zend_Controller_Front Singleton holds an instance of your bootstrap

$bootstrap = Zend_Controller_Front::getInstance()->getParam('bootstrap');
$bootstrap->bootstrap('db')
Benjamin Cremer
  • 4,842
  • 1
  • 24
  • 30
  • Thanks Benjamin! I gave this a whirl, but found that using `Zend_Controller_Front` Singleton continues to share with the parent thread and I get the "General error: 2006 MySQL server has gone away" again. – Sonny Jun 14 '11 at 13:08
  • Please open another question for that. The current question should be answered as it is.This comment in the PHP-Manual may help you with your forking issue: http://php.net/manual/en/function.pcntl-fork.php#70721 – Benjamin Cremer Jun 16 '11 at 17:09
  • The original thread that I linked to solved that issue. My current question is about code re-use. The Singleton behavior of Zend_Controller_Front::getInstance() prevents its use in a forking scenario. I will clarify this question. – Sonny Jun 16 '11 at 19:54
1

Zend_Controller_Front is singleton, but its constructor is protected, so you can extend that simply by creating a class called, App_Controller_Front

in that create a method for getNewInstance(), which can call constructor without checking for existence. This way you can override the singleton behavior.

Sandeep Manne
  • 6,030
  • 5
  • 39
  • 55
0

You are 'forking' these parallel processes not with pcntl_fork, but just by running them multiple times via your crobjob, correct? That would imply that they are running as separate processes from each other, and the only shared resources (and subsequent conflicts) are system resources (especially files).

The error you're seeing could be caused on any number of things related to the files that the cache is being written to. For instance, you might see that error if the metadata file is locked by another one of your processes, a problem which might not show up until the number of parallel processes you are running is quite high.

That error can also arise when the metadata cache file can't be written to for other reasons, such as the partition it is on is out of disk space or the file permissions aren't set correctly.

Perhaps you could you elaborate more on how you have configured your metadata cache and how your script parallelization works. Also, what version of Zend Framework are you running?

Nathan Bell
  • 2,364
  • 2
  • 18
  • 16
  • I am indeed truly forking with pcntl_fork. The thread I linked to about solving my db connection issue has all of that information. you may be correct about the cache writing error though. I haven't seen the error in a while. Regardless, I would like to keep my application DRY. – Sonny Jun 20 '11 at 12:27
0

Why each child thread reads config, caches db metadata, etc.? Use a master/worker hierarchy. Common idea:

$pid = pcntl_fork();

if ($pid == -1) {        
    // Fork failed            
    exit(1); 
} elseif ($pid) { 
    // We are the parent 
    // prepare common data, cache it, etc.
} else { 
    // We are the child 
    // create new db connection, use cached data
    // if there is no data in cache yet - sleep
    exit(0); 
}
Dmytro Zavalkin
  • 5,265
  • 1
  • 30
  • 34