Maugrim The Reaper's Blog - Zend Framework

PHP Framework Benchmarks: Entertaining But Ultimately Useless

nospam@example.com (Pádraic Brady) — Tue, 23 Feb 2010 23:27:11 +0000

Some recent attention in the PHP framework community has been focused on the recent publication of Symfony 2 Preview benchmarks showing that Symfony 2 outperforms Zend Framework by a factor of 3.5. It also outperforms every other benchmarked framework. This reminded me of the earlier Yii Framework benchmarks which allegedly proved that Yii outperforms Zend Framework by a factor of 7. At this point, the spirit of Glenn Beck would have me demonstrate that 3.5 multiplied by 2 (the number of eyes in Barrack Obama's skull) equals 7, thus proving the existence of a Liberal conspiracy led by a closet Muslim. That's probably bullshit though.

My fellow Zend Frameworkers, we cannot allow this to stand. We must put paid to these wild claims and prove, once and for all, that Zend Framework is the fastest. To present this undeniable truth rooted in solid statistics and Paul M. Jones (aka Benchmark God) inspired techniques, I have created the benchmark of benchmarks. Well, to be honest, I only really edited another benchmark. But still, it will prove Zend Framework is faster than everything else out there.

Before everyone gets too excited, the Symfony 2 Preview performance is quite impressive and it's consistent in my own rerun of the original benchmark. A lot of effort has obviously gone into speeding things up and it shows. My benchmark is not about busting some untruth, but about demonstrating how it is possible to outrace Symfony 2 and Yii with the Zend Framework. I'll dissect the methods after the results are reported .

To ensure the unchallenged fairness of my benchmark, I will publish its entire source code to Github. Unless there are any more sarcastic Irishmen out there, that should be relatively safe. I will also follow in the path of the Almighty Paul M. Jones, in whose shadow all PHP framework benchmarkers will find peace, flowing waters, milk and honey and, if not too much trouble, the extermination of all console noobs who insist that PC gaming is dying. Is that too much to ask, Paul? A little biblical gnashing of teeth would do them good. Paul (may he live forever) promotes using a standardised benchmarking platform, so I will be using an AWS instance reflecting Fabien Potencier's except it will be an m1.large instance instead (the other seeing a few issues in testing) and include some baseline HTML to make sure siege or ab are not misbehaving.

With all these preparations, the stage is set to demolish Symfony 2 and Yii and run them into the ground. Prior to running the benchmark, it's suggested to play the Battlestar Galactica theme tune in the background to set the right mood. Just think of Fabien Potencier dressed up as a Cylon (if only my Photoshop skills were better...sigh). It fits in well with the siege feedback. You can try some HipHop, but those Facebook guys must have been smoking something illegal because it had no impact during the Zend Framework runs. I also experimented with Metallica (surely that would work!) but no...nothing. Quite disappointing.

For those with attention deficit disorder, here are the results:

framework                |      rel |      avg |        1 |        2 |        3 |        4 |        5

------------------------ | -------- | -------- | -------- | -------- | -------- | -------- | --------

baseline-html            |   1.1972 |  5702.79 |  5527.29 |  5775.74 |  5779.59 |  5671.24 |  5760.10

baseline-php             |   1.0000 |  4763.50 |  4718.24 |  4751.51 |  4760.75 |  4763.68 |  4823.32

zend-1.10                |   0.1596 |   760.45 |   766.90 |   764.70 |   758.62 |   752.04 |   759.98

symfony-2.0.0alpha1      |   0.1366 |   650.61 |   641.98 |   655.41 |   653.86 |   656.68 |   645.12

solar-1.0.0beta3         |   0.1131 |   538.86 |   539.76 |   536.95 |   540.63 |   540.10 |   536.87

yii-1.1.1                |   0.0821 |   390.87 |   401.90 |   392.59 |   386.18 |   395.98 |   377.72

symfony-1.4.2            |   0.0441 |   210.22 |   211.20 |   209.78 |   210.72 |   210.49 |   208.92

cakephp-1.2.6            |   0.0406 |   193.56 |   193.84 |   193.35 |   193.27 |   192.57 |   194.75

As you can clearly see, Zend Framework is faster than all other frameworks. More enlightening, we beat both Symfony 2 and Yii. Lithium and Flow were omitted from the original benchmark because besides them being non-functional when forked from Fabien's benchmark (for whatever reason), they are both fairly young frameworks (Flow3 is alpha). Note: Lithium works fine outside of the benchmark (I've been working with it) and it is fairly fast.

Zend Framework wins. You can go now.

Still here? I suppose you're trying to figure out how I manipulated...er...improved the benchmark to reflect my reality of Zend Framework being superior at all levels. Well, I'm not sure I should tell you. It's hard enough to create and run these benchmarks without having to explain how to fuck them over so they show whatever you need them to. Alright then, I'll 'fess up.

To create a positive benchmark, you need to understand that all frameworks were born as festering piles of unoptimised stinking crap. They were all born bad and get worse with age. This sounds quite sad, but actually it's an inevitable compromise between performance and features. It's also a compromise between performance and ease-of-use. So you see, performance is unfairly faced by two opponents: features and ease-of-use. All performance is sacrificed in the name of serving the needs of rapid development, flexibility, prototyping, and making your source code look prettier than the other guy's. As if.

What happens if you move away from the enemies of performance and do some propping up behind the scenes? You get...wait for it...oodles of extra performance!

This is partly how both Symfony 2 and Yii manage to outperform Zend Framework. But hey, we can do it too!

To ascertain the perfect method of "propping up" to get the Zend Framework into speed demon territory without it costing me dozens of hours of in-depth analysis, I created four variants to play with: All-In, Optimised, More-Optimised and What-The-Fuck-Optimised. Here's the four final alternatives benchmarked against themselves (same run as the previous results).

framework                |      rel |      avg |        1 |        2 |        3 |        4 |        5

------------------------ | -------- | -------- | -------- | -------- | -------- | -------- | --------

baseline-html            |   1.1972 |  5702.79 |  5527.29 |  5775.74 |  5779.59 |  5671.24 |  5760.10

baseline-php             |   1.0000 |  4763.50 |  4718.24 |  4751.51 |  4760.75 |  4763.68 |  4823.32

zend-1.10-wtfoptimised   |   0.1748 |   832.63 |   830.64 |   837.20 |   836.20 |   824.90 |   834.23

zend-1.10-moreoptimised  |   0.1596 |   760.45 |   766.90 |   764.70 |   758.62 |   752.04 |   759.98

zend-1.10-optimised      |   0.0586 |   279.25 |   280.71 |   279.30 |   279.62 |   276.51 |   280.10

zend-1.10                |   0.0292 |   138.89 |   138.43 |   137.56 |   139.96 |   139.93 |   138.56

Pretty illuminating . Yes, the What-The-Fuck-Optimised variant is faster than the All-In variant by a factor of 6. The More-Optimised variant (fairer for reasons explained below) is 5.5 times faster than the base variant. In fact, the improvements are fairly smooth with each level of optimisation, at least doubling each time.

So just how does this miraculous optimisation work to such a degree that we can beat Symfony 2 and Yii at their speed games? There are four solid steps to being a speed demon. I only implemented these in the More-Optimised variant which was enough to destroy Symfony 2 (as configured) and Yii in the benchmarks. The Optimised version simply discarded some optional ease-of-use features. What-The-Fuck-Optimised is identical to More-Optimised except it exits in the middle of a controller (we only had to echo Hello, right? ).

1. Don't use Zend_Application. While Zend_App is great for creating consistent complex bootstraps within a standardised structure, it doesn't come without a significant performance hit to baseline performance. A more direct bootstrap (typical of ZF until Zend_App arrived) is far faster and can also be done without configuration files.

2. Skip using the ViewRenderer plugin. Without the ViewRenderer, you need to manually configure Zend_View and add render() calls to Controllers. This is actually very simple to do and is fairly fast - fast was never really part of the ViewRenderer's genetics.

3. Use autoloading. Strip require_once calls from the framework library so unneeded files are ignored. Replace uses of Zend_Loader_Autoloader with a not-so-crazy autoloader function. In fact, pray Zend_Loader is never used - it does a lot of file ops that, to date, have never been explained to me as having any value.

4. Preload everything (Symfony 2 Preview does!). It buys you some performance cookies and equalises the speed baseline. Using a simple preload script is not that hard.

With these four techniques, Symfony 2 and Yii are left in the dust.

It's time to ask the question on everybody's minds: what does it all mean? Give up. There is no meaning in benchmarks. They are designed to compare the relative performance of frameworks with different goals, design philosophies, development practices, and features. Might as well flip a coin, roll some dice or perhaps examine the entrails of a sacrificed goat for answers. The goat is your best bet. If all else fails at least you can have a nice goat-stew dinner while you think things over some more.

Benchmarks. Useless. Final words?

Know your framework! All this benchmarking nonsense does little good unless it's plastered with disclaimers. Symfony's preloading is an obvious example mainly because it's the only framework in Fabien's benchmark (that I know of) using it. Does it increase base performance out of the box? Yes. Can every other framework use it with little effort? Yes. So what's the point? Should everything else preload by default? Maybe, who can tell. Most developers simply don't work in scenarios where such a performance boost would make a difference. And if they do, it's easy to make the necessary changes.

What's interesting is that I just fiddled with the Zend Framework. You could probably find ways of making every other framework in there run a hell of a lot faster than currently presented. It's just a bit sweeter for Zend Framework since we're the self-proclaimed use-at-will framework. And its really fast when developers need it to be.

The Mysteries Of Asynchronous Processing With PHP - Part 3: Implementation With Spawned Child Processes Using Simple Scripts Or Zend Framework

nospam@example.com (Pádraic Brady) — Thu, 01 Oct 2009 19:12:04 +0000

Advantages Of Asynchronous Processing With Child Processes

Performing asynchronous processing using a child process is probably one of the simplest methods. It involves a parent process (usually the one serving the current application request) spawning an immediate child process which will continue running in the background even when the parent process has exited. This child process can be a simple PHP process called from the command line to execute a standalone script or initiate a CLI request to a fuller application. Typically you can avoid the web server (strongly recommended) which allows these child processes to be a bit more efficient than incurring the overhead of web server involvement. Because it is triggered by the application itself as part of a request, a child process is an excellent means of performing asynchronous processing on the spot, without incurring the delays sometimes associated with using a Job Queue coupled with a scheduled task manager (cron, for example, has a minimum interval of one minute).

To this advantage we can add its simplicity of operation. Since the child process is spawned from within the application itself, it requires very little surrounding code. Just add the task to a script and away you go. Creating the child process in the first place is easily done using pipes, process management or inline execution.

For security reasons, I prefer to avoid direct execution of a task, i.e. using the exec() function, since it can add a risk of code injection and in general is considered extremely dangerous - unfortunately you may not have another choice if in a shared host environment. Using process management (using proc_open() and proc_close()) or pipes (using popen() and pclose()) are much preferred, though the process management functions are typically not suitable for use in a web environment and are generally reserved for use in CLI application.

Using these functions within your applications is extremely simple, as we'll see later.

Disadvantages Of Asynchronous Processing With Child Processes

Of course, simplicity does not mean a method is perfect. Child processes, when triggered within requests, bind the asynchronous tasks to the time of the request. If you think of it, any request may also spawn additional child processes. This means extra resources are being used, CPU cycles are needed and memory is consumed. As the number of requests to the application increases, so too will the number of resource needy child processes. It should also be noted that all child processes are created on the same server as the parent. There is no opportunity to offload work to another server.

This is exacerbated if your script requires bootstrapping, i.e. the process of initialising and configuring any resources needed by the script. In many cases, there is an inclination to depend on the application's framework. You can imagine the resource needs of a script which, for example, is treated as a console call to a Zend Framework action. Now you not only need to contend with the resources needed by a simple script, but with the resources demanded by a complete application framework. This can push your server load to new and unnecessary heights. In fact you may easily find that up to 90%+ of a script's work is actually just bootstrapping. Note that I'm not saying to never use a framework - merely that it comes with a cost to be aware of.

This is all obviously a problem from a scaling perspective. It may even worsen a situation where the application already operates in a resource scarce environment. So while ideally it buys us extra responsiveness to serve clients and users, it does so at the cost of a difficult to scale method that requires additional resources. It is also difficult to load balance as the child processes are spawned on the same server as the parent.

Nevertheless, for certain tasks it may be an excellent fit. Not all tasks are resource intensive. Some simply take a while to complete for other reasons and may be time sensitive, requiring completion as soon as possible. You'll understand the trade offs in greater detail as the series continues to cover the alternative asynchronous strategies.

What About Forking?

In all our discussions, we've avoided mentioning an alternative to creating a brand new PHP process: forking. Using process forking (see the pcntl_fork() function) we are basically copying or cloning the parent process. This means the new process shares the parent's resources, including its open database connections and all variables used up to the forking point in the application. This, in theory, avoids the setup costs associated with a from-scratch process since a fork is simply copying the parent which is already bootstrapped.

While there's nothing fundamentally wrong with using forking, it is more difficult to manage than simple spawning due the fact it requires additional management to ensure shared resources (e.g. a database connection) are not prematurely closed by any forked process when the parent or sibling forks are using the same connection (the same can be said for other shared resources). Often this still requires some limited bootstrapping to replace shared resources with separate instances invulnerable to premature closing, most commonly database connections.

We also need to manage the source code of the application to determine whether it's being executed by the parent process or a forked process (both start at the exact same execution point - the point where the forking occured). Forking may also not be supported by the underlying system by default. For example, when PHP is used as an Apache module, pcntl support is usually disabled by default. Also, forking is not supported on Windows systems at all. Adding even more weight against forking, ideally the parent cannot exit while forked processes are still running since it would leave the children without a parent process and risk them becoming "zombie" processes.

Forking is a great means of achieving concurrency in an application where the parent remains in control of all forked processes. Concurrency refers to the ability to perform any number of simultaneous tasks which perhaps will interact with each other. This is somewhat related to asynchronous processing which is why I mention it. The interaction can occur at a few places, like shared memory, a common used file, or even a database.

The main reason I don't address forking in great detail, however, is stated in the PHP manual itself:

Process Control should not be enabled within a web server environment and unexpected results may happen if any Process Control functions are used within a web server environment.

Most of my own applications use Apache or FastCGI somehow and forking in these scenarios is not recommended. If, on the other hand, you're working from the CLI on Linux, forking can have some neat uses particularly for concurrency or when managing a daemon that is allowed to spawn and control worker processes. We'll be covering daemons later in this series.

Basic Implementation

Describing it may be complicated, but implementing child process spawning is very simple. All we are doing in effect is putting out a call to the command line to execute a script using PHP. We also add the notation needed to make the process operate in the background (i.e. the method used to initiate the process can itself be immediately closed without any consequences). We must close the resulting pointer to allow the parent continue it's own processing to it can deliver a response without waiting for the child process to finish.

We'll start with a basic example using pipes:

if (PHP_OS == 'WINNT' || PHP_OS == 'WIN32') {
$ppointer = popen('start /b php c:\\www\\myapp\\scripts\\deliver_registration_email.php', 'r');
} else {
$ppointer = popen('php /var/www/myapp/scripts/deliver_registration_email.php > /dev/null &', 'r');
}
pclose($ppointer);

In the above code example, we are using two functions to open a "process file pointer" (identical to a file pointer) to a new process we execute using a typical command line call. The command used differs between *nix and Windows for obvious reasons, but both are pushed to the background using "&" in Linux or "start /b" in Windows. The base command is simply "php" as this does not require a web server dispatch we can use the PHP CLI directly. You could call this code from anywhere in the application.

Just to be absolutely clear, there is nothing which says this MUST be a php call. You could use a batch file or bash script, or practically any other command line tool on your system - so long as its operable as a background process.

The second parameter to popen() is familiar from the basic file functions. It means that the pointer returned from the function is a file pointer, just as if we'd used fopen() with the same parameter. This also means we can use it with other file functions, however we intend cutting the child process loose as soon as possible so the parent process need not wait for it (and besides, it's being executed in the background).

The lack of parameters used in the command does raise the question of how the script finds out what email needs to be sent. There are a few ways of handling this, among them are passing a reference to a database user record to get the email and status from, passing the email address itself, or using something more deliberate like a queue implementation to keep it lightweight (perhaps using memcached, apc or a dedicated message queue for overkill ).

Another parameter concern is indicating an operation mode: production, development, staging or testing. Usually we'd communicate this in an application using an environmental value set perhaps in an application's .htaccess file or its virtual host configuration for Apache. However, a script operates from the command line so these would obviously not be available.

Solving the first is done easily enough:

$email = 'joe@example.com';

if (PHP_OS == 'WINNT' || PHP_OS == 'WIN32') {
$ppointer = popen('start /b php c:\\www\\myapp\\scripts\\deliver_registration_email.php --email ' . escapeshellarg($email), 'r');
} else {
$ppointer = popen('php /var/www/myapp/scripts/deliver_registration_email.php --email ' . escapeshellarg($email) . ' > /dev/null &', 'r');
}
pclose($ppointer);

We just add a parameter to the script containing the user's email address, allowing the script to parse this from the arguments (via $_SERVER['argv'] which contains a flat array of all space delimited terms in the command line arguments).

Solving the second can also be accomplished using another parameter which tells the script to manually set a matching environmental value during any bootstrapping. We can probably accomplish this to match the normal application mode using its current value. For example it may be set as $_ENV['APPLICATION_ENV'] from the app's .htaccess file or virtual host configuration.

$email = 'joe@example.com';

if (PHP_OS == 'WINNT' || PHP_OS == 'WIN32') {
$ppointer = popen('start /b php c:\\www\\myapp\\scripts\\deliver_registration_email.php --email ' . escapeshellarg($email) . ' --environment=' . escapeshellarg($_ENV['APPLICATION_ENV']), 'r');
} else {
$ppointer = popen('php /var/www/myapp/scripts/deliver_registration_email.php --email ' . escapeshellarg($email) . ' --environment=' . escapeshellarg($_ENV['APPLICATION_ENV']) . ' > /dev/null &', 'r');
}
pclose($ppointer);

So we now have the basics for calling a script. Let's look at a simple script example itself. I'm deliberately not complicating it beyond the absolute essentials, but I'll delve into a much better example in the next section where we apply what we've learned to the Zend Framework (a more realistic application structure).

<?php

// remove script path (first array element)
array_shift($_SERVER['argv']);

// compile options (assume all are key/value pairs)
$options = array();
while (count($_SERVER['argv']) > 1) {
$key = str_replace('--', '', array_shift($_SERVER['argv']));
$options[$key] = array_shift($_SERVER['argv']);
}

mail(
$options['email'],
'Thanks for registering!',
'Thank you for registering. You may now log into your new account.'
);

// In case manually run - echo feedback to command line
echo 'Email to ', $options['email'], ' sent.', "\n";

The script first parses out any options passed. Specifically it's looking for the --email option. The parsing logic is very simple since we assume it must be --email, not a shorter form, and all options are always in key/value pairs. Typically, it's recommended to using PEAR's Console_CommandLine or even ZF's Zend_Console_Getopt. I actually prefer the PEAR package where possible - it's a much more functional and feature rich option though slightly more complicated to setup.

Once we get the email address from the command line options, we simply fire out an email with a simple registration message using mail(). In the next example, we'll use an email library which is a better method.

Basic Implementation With Zend Framework

As covered in Part 2 of this series, using the Zend Framework from the command line allows you to run asynchronous tasks based from any Zend Framework application. All that is required is ensuring the relevant Controller and Action for the task are not accessible from the web so public access is impossible.

We can graduate the above task to a Zend Framework application by adding it using a Controller. Here I've created the controller MailController with the action "registration" in /application/controllers/MailController.php:

<?php

class MailController extends Zend_Controller_Action
{

public function init()
{
if (!$this->getRequest() instanceof ZFExt_Controller_Request_Cli) {
exit('MailController may only be accessed from the command line');
}
}

public function registrationAction()
{
$this->getInvokeArg('bootstrap')->addOptionRules(
array('email|p=s' => 'Email address for task (required)')
);
$options = $this->getInvokeArg('bootstrap')->getGetOpt();

$mail = new Zend_Mail();
$mail->setBodyText('Thank you for registering. You may now log into your new account.')
->setFrom('me@example.com', 'Padraic Brady')
->addTo($options->email)
->setSubject('Thanks for registering!');
try {
$mail->send();
} catch (Zend_Mail_Exception $e) {
// mail probably failed - at this point add to a Job Queue to try
// again later or analyse the failure.
}
}

}

You can, of course, manually call this from the command line using:

php /path/to/myapp/scripts/zfrun.php -c mail -a registration --email joe@example.com

Update (03 Oct): The generated command from the example will actually place single quotes around the argument values as part of the escaping mechanism I added. As Bruce Weirdan, quite rightly, points out in the comments, I really should have used escaping when this article was originally released.

The zfrun.php file was detailed in Part 2 of the series. It's basically what you would find in the index.php file of a ZF application using Zend_Application for bootstrapping.

The above action uses a bit of magic from the bootstrap class (from Part 2) which enables it to dynamically set additional valid arguments available from its command line call. Here, we supplement the default arguments (mainly those needed for setting the MVC values of controller, action, module and environment) with an additional --email argument (which can be shortened to -p, i.e. just a reference to it being a lone "p"arameter). The parameter is marked as being a required string (=s) argument when its added.

The rest is simply using the bootstrap's copy of Zend_Console_Getopt to parse the arguments with updated rules, so we can later grab the email address value for use with a typical example of Zend_Mail. The exception check should yield an exception if the email, for whatever reason, failed to be sent. To be perfectly honest, I don't use Zend_Mail (I'm a Swiftmailer user ), so if there's a better way of getting the context of a failure please let me know in the comments to this article.

If we assume we have a RegistrationController where registration takes place. We could trigger the performance of this new task asynchronously using the following:

<?php

class RegistrationController extends Zend_Controller_Action
{

public function indexAction()
{
// perform registration processing here but delegate emailing to
// an asynchronous process

// ...

$email = 'joe@example.com';
if (PHP_OS == 'WINNT' || PHP_OS == 'WIN32') {
$ppointer = popen('start /b php c:\\path\\to\\myapp\\scripts\\zfrun.php -c mail -a registration --email ' . escapeshellarg($email), 'r');
} else {
$ppointer = popen('php /path/to/myapp/scripts/zfrun.php -c mail -a registration --email ' . escapeshellarg($email) . ' > /dev/null &', 'r');
}
pclose($ppointer);

// ...
}

}

Now, repeating the same block of code with minor variations can be a pain. So, let's simplify things a bit.

Simplifying The Zend Framework Controller Calls

Since all these Zend Framework based asynchronous tasks may coexist with non-ZF based tasks (which exist when we really don't need the framework heavyweight for the task), we may end up with a lot of calls using the above method. In addition, there's no guarantee all tasks would remain as is, operated by child processes spawned from the parent.

It would be more flexible to delegate the calling of asynchronous processes to an Action Helper which implements a simple API more amenable to diverting asynchronous processing to specific implementations (process spawn, job queue, daemon, etc.). Here's an example of such a helper (simplified since we've only covered child process spawning so far!):

<?php

class ZFExt_Controller_Action_Helper_Spawn
extends Zend_Controller_Action_Helper_Abstract
{

protected $_scriptPath = null;

protected $_defaultScriptPath = null;

public function setScriptPath($script = null)
{
if (PHP_OS == 'WINNT' || PHP_OS == 'WIN32') {
$script = str_replace('/', '\\', $script);
}
$this->_scriptPath = $script;
return $this;
}

public function setDefaultScriptPath($script)
{
if (PHP_OS == 'WINNT' || PHP_OS == 'WIN32') {
$script = str_replace('/', '\\', $script);
}
$this->_defaultScriptPath = $script;
return $this;
}

public function direct(array $parameters = null, $controller = null,
$action = null, $module = null)
{
if (is_null($parameters)) {
$parameters = array();
} else {
foreach ($parameters as $key => $value) {
$parameters[$key] = escapeshellarg($value);
}
}
if ($module) {
$parameters['-m'] = escapeshellarg($module);
}
if ($controller) {
$parameters['-c'] = escapeshellarg($controller);
}
if ($action) {
$parameters['-a'] = escapeshellarg($action);
}
$this->_spawnProcess($parameters);
$this->_scriptPath = null; // reset
}

protected function _spawnProcess(array $args)
{
if (is_null($this->_scriptPath)) {
$script = $this->_defaultScriptPath;
} else {
$script = $this->_scriptPath;
}
$command = 'php ' . $script;
foreach ($args as $key => $value) {
$command .= ' ' . $key . ' ' . $value;
}
if (PHP_OS == 'WINNT' || PHP_OS == 'WIN32') {
$pcommand = 'start /b ' . $command;
} else {
$pcommand = $command . ' > /dev/null &';
}
pclose(popen($pcommand, 'r'));
}

}

The new helper makes spawning a new process that bit simpler. Besides setting parameters and specific arguments for MVC calls into the application, it allows the setting of a default script to execute (e.g. it could be our zfrun.php script for a Zend Application call). You can set a custom script before a new spawning, but it will revert back to the default on the next spawn attempt (simply to prevent the default being overridden by accident). Obviously the class could be improved a lot more, but let's leave it here for now.

Let's amend the RegistrationController example to use this helper, and then show how it can be initialised/configured during our bootstrapping process:

<?php

class RegistrationController extends Zend_Controller_Action
{

public function indexAction()
{
// perform registration processing here but delegate emailing to
// an asynchronous process

// ...

$this->_helper->getHelper('Spawn')->setScriptPath('/path/to/myapp/scripts/zfrun.php');
$this->_helper->spawn(array('--email' => 'joe@example.com'), 'mail', 'registration');

// ...
}

}

The new helper object's direct() method can be called directly from any action using $this->_helper->spawn(). Beforehand, we can set the script we intend using. This could also be configured by default during bootstrapping, using the setDefaultScript() method. Using spawn(), we set the "--email" option to pass onto the CLI, and set the controller/action/module separately. You could simply set the MVC options through the first array - just remember the full and short options they reserve.

After that, the helper does the rest of the work in creating the command and spawning the child correctly.

To enable this helper, you just need to make it accessible by registering it from your application.ini settings file (at /config/application.ini in the example application from Part 2). See the new line in "Standard Resource Options":

[production]
; PHP INI Settings
phpSettings.display_startup_errors = 0
phpSettings.display_errors = 0

; Bootstrap Location
bootstrap.path = APPLICATION_ROOT "/library/ZFExt/Bootstrap.php"
bootstrap.class = "ZFExt_Bootstrap"

; Standard Resource Options
resources.frontController.controllerDirectory = APPLICATION_PATH "/controllers"
resources.frontController.moduleDirectory = APPLICATION_PATH "/modules"
resources.frontController.plugins[] = "ZFExt_Controller_Plugin_ModuleConfigurator"
resources.view.encoding = "UTF-8"
resources.view.helperPath.ZFExt_View_Helper = "ZFExt/View/Helper/"
resources.view.helperPath.SpotSec_View_Helper = "SpotSec/View/Helper/"
resources.modifiedFrontController.contentType = "text/html;charset=utf-8"
resources.layout.layout = "default"
resources.layout.layoutPath = APPLICATION_PATH "/views/layouts"
resources.frontController.actionHelperPaths.ZFExt_Controller_Action_Helper = "ZFExt/Controller/Action/Helper"

;resources.layout.pluginClass = "ZFExt_Controller_Plugin_LayoutSwitcher"

; Module Options (Required For Mysterious Reasons)
resources.modules[] =

; Autoloader Options
autoloaderNamespaces[] = "ZFExt_"

; HTML Markup Options
resources.view.charset = "utf-8"
resources.view.doctype = "XHTML5"
resources.view.language = "en"

[staging : production]

[testing : production]
phpSettings.display_startup_errors = 1
phpSettings.display_errors = 1
resources.frontController.throwExceptions = 1

[development : production]
phpSettings.display_startup_errors = 1
phpSettings.display_errors = 1
resources.frontController.throwExceptions = 1

You could also register the same helper using cli.ini settings so it's accessible from there also.

Conclusion

In Part 3 of this series on Asynchronous Processing in PHP, we've covered two concepts: forking and spawning. Forking is not covered in detail (yet) since it is problematic to run within a web environment. However it is extremely useful in any command line application since it does not incur the startup costs associated with spawning a new process. Nevertheless, in a web environment spawning a new process from scratch remains the simplest option to implement.

I've done my best to at least outline in sufficient detail the advantages and disadvantages of spawning processes. With some luck, I've impressed upon you that asynchronous processing is not some weird and technically infeasible strategy for improving an application's responsiveness. It is, actually, very simple to implement - even in something as architected and involved as a Zend Framework application.

If you want to give the code here a trial run, you can simply change things for a few quick tests by having the asynchronous task do something like write its parameters to a temporary file. This "instant gratification" test will more than prove that the strategy above works in practice and with very little effort.

In Part 4, I'll be exploring an alternative to process spawning (at least as described above) by using the combination of a scheduled task and a Job/Message Queue. This offers a few improvements over process spawning, primarily that it allows spreading the performance of asynchronous tasks over time and, with a little nudging, over multiple servers (even to the point of being able to use a server dedicated to specific tasks).

The Mysteries Of Asynchronous Processing With PHP - Part 2: Making Zend Framework Applications CLI Accessible

nospam@example.com (Pádraic Brady) — Wed, 30 Sep 2009 01:19:00 +0000

In Part 1 of this series, we started an exploration of the concept of Asynchronous Processing as it applied to PHP. We covered the benefits it offers, the basic implementation directions often applied, and also discussed how to identify and separate tasks from the main application so they could be made subject to asynchronous processing. It is highly recommended that you read this before continuing with Part 2 so you can follow what I'm up to here .

UPDATE: Modified the bootstrap class and script based ZF runner to reflect some changes needed to support Part 3 of this series. These primarily allow for improved control over command line options.

With the theory heavy portion of the series out of the way, we can begin to explore the various implementation possibilities. In Part 3, we will examine implementing Asynchronous Processing using a child process, i.e. a separate PHP process we create from our application during a request. We'll analyse this implementation option before introducing some source code so we may understand its advantages and disadvantages.

While, technically, this series is not Zend Framework specific since the same principles can be applied to any PHP application, I'll be using the Zend Framework in examples of asynchronous processing from an application. As a result, Part 2 is a tangential detour into how to make a Zend Framework based application accessible from the command line before we delve into examples using this in future parts of the series. If you are not a Zend Framework user, I'm sure you can find relevant material online for your own preferred framework though the ZF pieces may still have some usefulness in understanding the approach from an MVC perspective.

Surprising to some, the Zend Framework is indeed usable from the command line...with some massaging. I've already noted that using a full application framework for a background task comes at a cost since you are using a lot of code, not all of which may be strictly necessary but unless you are willing to invest in a custom framework specifically for such uses, your framework of choice is probably the simplest option.

I'm not going to describe setting up a basic application with the Zend Framework, however you can do so by following the base application created in my book, Zend Framework: Surviving The Deep End (which is free, online in HTML form, and now duly advertised to you ). The relevant chapters are Chapter 5 and Chapter 6. If you want to get started quickly, you can download the example application for the book (in progress) from http://github.com/padraic/ZFBlog

Unfortunately, the ZF is not immediately accessible from the command line. Although it offers classes like Zend_Console_Getopt and Zend_Controller_Response_Cli, the remaining pieces are mysteriously (and conspicuously) missing. They are not difficult to add however, especially if you are using Zend_Application to fuel your bootstrapping.

Adding Custom CLI Support Classes

There are two very obvious problems calling a Zend Framework application from the command line. First, there is no Request class supporting CLI command line options (though there is a Zend_Controller_Request_Simple). Secondly, the Front Controller always attempts to route the request, and all of the standard Routers assume you will use a HTTP request. This HTTP focus results in an Exception when routing occurs.

To improve this situation, we will implement two very simple custom classes. ZFExt_Controller_Request_Cli and ZFExt_Controller_Router_Cli.

ZFExt_Controller_Request_Cli very simply accepts an instance Zend_Console_Getopt and attempts to locate a module, controller and action name from the command line options it exposes. If they exist, these are used to set the relevant module, controller and action names for the request (doing this manually negates the need for routing). Here's the class stored to (if using the example app) at /library/ZFExt/Controller/Request/Cli.php:

<?php

require_once 'Zend/Controller/Request/Abstract.php';

class ZFExt_Controller_Request_Cli extends Zend_Controller_Request_Abstract
{

protected $_getopt = null;

public function __construct(Zend_Console_Getopt $getopt)
{
$this->_getopt = $getopt;
$getopt->parse();
if ($getopt->{$this->getModuleKey()}) {
$this->setModuleName($getopt->{$this->getModuleKey()});
}
if ($getopt->{$this->getControllerKey()}) {
$this->setControllerName($getopt->{$this->getControllerKey()});
}
if ($getopt->{$this->getActionKey()}) {
$this->setActionName($getopt->{$this->getActionKey()});
}
}

public function getCliOptions()
{
return $this->_getopt;
}

}

ZFExt_Controller_Router_Cli is basically a "dumb" router. It implements Zend_Controller_Router_Interface but all of its methods are blank. Since our CLI access does not need to be routed, we're effectively just plugging the requirement for a Router object with something which is designed to do absolutely...nothing . Here's the class stored to (if using the example app) at /library/ZFExt/Controller/Router/Cli.php:

<?php

class ZFExt_Controller_Router_Cli implements Zend_Controller_Router_Interface
{

public function route(Zend_Controller_Request_Abstract $dispatcher){}
public function assemble($userParams, $name = null, $reset = false, $encode = true){}
public function getFrontController(){}
public function setFrontController(Zend_Controller_Front $controller){}
public function setParam($name, $value){}
public function setParams(array $params){}
public function getParam($name){}
public function getParams(){}
public function clearParams($name = null){}

}

Putting these new classes to use will require manually adding them to the Front Controller we'll use in our application bootstrap. For CLI use, I've elected to implement a new bootstrap which is very similar to the one implemented for the example app at /library/ZFExt/Bootstrap.php. The CLI bootstrap below is stored to /library/ZFExt/BootstrapCli.php:

<?php

class ZFExt_BootstrapCli extends Zend_Application_Bootstrap_Bootstrap
{

protected $_getopt = null;

protected $_getOptRules = array(
'environment|e-w' => 'Application environment switch (optional)',
'module|m-w' => 'Module name (optional)',
'controller|c=w' => 'Controller name (required)',
'action|a=w' => 'Action name (required)'
);

protected function _initView()
{
// displaces View Resource class to prevent execution
}

protected function _initCliFrontController()
{
$this->bootstrap('FrontController');
$front = $this->getResource('FrontController');
$getopt = new Zend_Console_Getopt($this->getOptionRules(),
$this->_isolateMvcArgs());
$request = new ZFExt_Controller_Request_Cli($getopt);
$front->setResponse(new Zend_Controller_Response_Cli)
->setRequest($request)
->setRouter(new ZFExt_Controller_Router_Cli)
->setParam('noViewRenderer', true);
}

// CLI specific methods for option management

public function setGetOpt(Zend_Console_Getopt $getopt)
{
$this->_getopt = $getopt;
}

public function getGetOpt()
{
if (is_null($this->_getopt)) {
$this->_getopt = new Zend_Console_Getopt($this->getOptionRules());
}
return $this->_getopt;
}

public function addOptionRules(array $rules)
{
$this->_getOptRules = $this->_getOptRules + $rules;
}

public function getOptionRules()
{
return $this->_getOptRules;
}

// get MVC related args only (allows later uses of Getopt class
// to be configured for cli arguments)
protected function _isolateMvcArgs()
{
$options = array($_SERVER['argv'][0]);
foreach ($_SERVER['argv'] as $key => $value) {
if (in_array($value, array(
'--action', '-a', '--controller', '-c', '--module', '-m', '--environment', '-e'
))) {
$options[] = $value;
$options[] = $_SERVER['argv'][$key+1];
}
}
return $options;
}

}

This new bootstrap class performs two important functions. First, it sets up the application's Front Controller to use our full set of CLI helper classes including the custom ones we added. Secondly, it allows for the setting of a command line option parser, an instance of Zend_Console_Getopt. The default used within the bootstrap class has a limited set of options, so we could set a replacement parser with an expanded set of command line options available. Unfortunately, we may not simply add new options and reparse due to the limitations of Zend_Console_Getopt but substitution will work just fine for most needs.

Adding A Calling Script

All that remains to enable CLI access is to add a calling script to run the application. We'll start by adding a php file at /scripts/zfrun.php. This will be very similar to how a Zend Framework index.php file would look like if using Zend_Application:

<?php

if (!defined('APPLICATION_PATH')) {
define('APPLICATION_PATH', realpath(dirname(<u>_FILE_</u>) . '/../application'));
}

if (!defined('APPLICATION_ROOT')) {
define('APPLICATION_ROOT', realpath(dirname(<u>_FILE_</u>) . '/..'));
}

set_include_path(
APPLICATION_ROOT . '/library' . PATH_SEPARATOR
. APPLICATION_ROOT . '/vendor' . PATH_SEPARATOR
. get_include_path()
);

require_once 'Zend/Loader/Autoloader.php';
$autoloader = Zend_Loader_Autoloader::getInstance();
$autoloader->setDefaultAutoloader(create_function('$class',
"include str_replace('_', '/', \$class) . '.php';"
));

// check for app environment setting
$i = array_search('-e', $_SERVER['argv']);
if (!$i) {
$i = array_search('--environment', $_SERVER['argv']);
}
if ($i) {
define('APPLICATION_ENV', $_SERVER['argv'][$i+1]);
}

if (!defined('APPLICATION_ENV')) {
if (getenv('APPLICATION_ENV')) {
$env = getenv('APPLICATION_ENV');
} else {
$env = 'production';
}
define('APPLICATION_ENV', $env);
}

$application = new Zend_Application(
APPLICATION_ENV,
APPLICATION_ROOT . '/config/cli.ini'
);

$application->bootstrap()->run();

That wasn't so bad . The script itself merely sets up the typical constants needed for Zend_Application. We also have a block defining the rules needed to parse any command line options. As the related comment suggests, we should in future iterations add a means of appending additional rules as needed by varying tasks. The resulting Zend_Console_Getopt instance is later passed to our bootstrap instance (ZFExt_BootstrapCli) before we bootstrap and run the application.

The final piece of this jigsaw is adding the configuration file, cli.ini, passed to Zend_Application. This is a cut down version of the original application.ini used by the example app stored to /config/cli.ini:

The main differences from the original application.ini is to remove any settings for a View. We won't be rendering any templates for our CLI access. Otherwise, you can retain any other settings for database access, etc. This could also be added as a separate section to application.ini, however I decided a separate CLI settings file made it a bit simpler to follow and allows setting the usual application environment based sections.

Adding CLI tasks to ZF Applications

We'll start by adding a TaskController to the application. The name is largely irrelevant so don't decide you must put all tasks into the same controller! You may also use controllers within a module should they require their own specific tasks or command line needs.

The new controller is added at /application/controllers/TaskController.php:

<?php

class TaskController extends Zend_Controller_Action
{

public function init()
{
if (!$this->getRequest() instanceof ZFExt_Controller_Request_Cli) {
exit('TaskController may only be accessed from the command line');
}
}

public function echoAction()
{
echo 'Hello, World!', "\n";
exit(0);
}

}

While this is a very simple example, echoing a message, the task itself could be as complicated as you wish. We've also added a quick check to ensure this controller cannot be accessed from a normal HTTP request - having publicly available tasks is not a good idea afterall .

Using the CLI access from the command line

Use of our newly added CLI access to this Zend Application is very simple. There are four command line options defined. Here's an example which calls the new task and sets the application environment (used in our configuration) to "development". Note that if absent, the environment defaults to "production".

php zfrun.php -c task -a echo -e development

Which is equivelant to:

php zfrun.php --controller=task --action=echo --environment=development

Using either, once you've navigated to the application's /script directory, should echo the message we added to the task.

Conclusion

In the second part of our look at Asynchronous Processing we've investigated how to enable CLI access to a Zend Framework application. In the future, this will allow us to delegate tasks asychronously using command line calls and using framework based tasks.

In Part 3, we'll return to the Asynchronous Processing topic and put this work to use in explaining a very common implementation strategy for asynchronous tasks.