(Photo credit: bertboerland)

In recent weeks, I consulted with the second most intelligent species on the planet: Dolphins. Dolphins are renowned across the known Universe for their awesome programming skills. After all, it was they who developed such insightful works as “Evolution By Example”, “Dude! We Wrote The Laws Of Physics!”, and “How Many Humans Does It Take To Screw Up A Planet?”. The answer to the last will be published on 01/01/2013 after the experiment is shut down and sent to a landfill site assuming the Supreme Spaghetti Monster signs off on the permit.

Dolphins think we are really dumb and theorise that this level of stupidity has one obvious cause: self-imposed ignorance. We are, after all, only the third most intelligent species on Earth and appear to have aspirations to lower our IQ just a bit more.

While it’s no harm poking fun at ourselves, in PHP we do have a serious problem. Cross-Site Scripting (XSS) remains one of the most significant classes of security problems afflicting PHP applications. Despite years of education, community awareness and the development of frameworks which can offer a huge boost in consistent practices – things are not getting any better.

So, I finally figured out what the core problem is: PHP programmers are completely clueless about XSS. It’s that simple. Instead of going out and studying the topic, we blindly follow some preferred herd of people offering advice with heartfelt conviction despite the fact that they are probably just as ignorant as the rest of us. Does that sound like the behaviour of something which allegedly evolved into an intelligent species? The result is a mix of ignorance and stagnant knowledge that leaves PHP in an unenviable position beset by wrongheaded zealots.

To get the ball rolling, this two-part article series is a tour of how NOT to use the htmlspecialchars() function that is typically pressed ganged into service as PHP’s universal output escaper. By offering an example based guide, I hope it will illustrate just how many ways a prospective attacker using XSS can exploit this function’s misuse to pull off a successful attack. The examples were written for PHP 5.3, so 5.4 users may need to imagine they still have 5.3 installed and/or lodge an official complaint with somebody who looks like they keep a complaints box handy (your local fast food restaurant is a good start).

This example led approach has another motive. Simple examples can be translated into unit tests. Ideally, many of the current crop of frameworks can use this article as a guide to what their unit tests should be looking for. This also makes it far easier for everyday programmers to consume the article and run around the place, drunk with ungodly power, identifying issues in the libraries, frameworks and other projects that they rely on.

To help us on the path of enlightenment before it’s too late (I’d lodge an appeal with the Supreme Spaghetti Monster but apparently the Mayans already tried and failed), I also invite other PHP programmers to blog about a security topic over the next month or two. Give programmers one last chance to get it right before the Planet is demolished by the Vogon destructor fleet. Just pick a topic that drives you up the walls in defiance of gravity and spend an hour writing something useful and (optionally) expletive filled. Every little bit helps.

What Is Htmlspecialchars()?

According to many programmers from Earth, htmlspecialchars() is a function used to escape output to prevent XSS. This is however a completely wrong definition. The function was actually co-opted by programmers to combat XSS because it was either that or create slow userland functions for which the internals developers might get around to creating, when the full moon coincided with the right planetary alignment in another 314 years, a speedier C alternative to. The actual definition (along with a half-hearted self-doubting nod to preventing XSS) is as follows:

Certain characters have special significance in HTML, and should be represented by HTML entities if they are to preserve their meanings. This function returns a string with some of these conversions made; the translations made are those most useful for everyday web programming. If you require all HTML character entities to be translated, use htmlentities() instead. This function is useful in preventing user-supplied text from containing HTML markup, such as in a message board or guest book application.

Note that this hints at, but does not explicitly use, the terms Cross-Site Scripting, XSS or even Security. Then again, it does refer to guest book applications so it was probably written in 1790 by the Dolphin who created PHP v86 and who then got around to backporting version 1.0 for Humans in the late 20th Century out of extreme pity for our reliance on CGI. No, not the let’s take an action movie and turn it into a plotless eyesore with computer generated fake stuff style CGI – though memories of both are comparably bad.

Does this make htmlspecialchars() terrible at preventing XSS? No. As part of a comprehensive well-understood strategy to prevent XSS, the function is very useful. However, in PHP it is frequently overused, misused, abused, confused and…. Darn it, ran out of rhyming words again. Suffice it to say that a good description of htmlspecialchars() is that it’s an unsuitable tool for preventing XSS that has slowly evolved into a better suited tool over the years. I keep telling myself that, at least.

The function, htmlspecialchars(), accepts four parameters. Here is its function prototype as of PHP 5.4:

string htmlspecialchars ( string $string [, int $flags = ENT_COMPAT | ENT_HTML401 [, string $encoding = 'UTF-8' [, bool $double_encode = true ]]] )

The first parameter accepts a string whose special HTML characters will be converted to HTML entities. The second accepts one or more flags which defaults to using ENT_COMPAT (does not convert single quotes to entities) but should be set to use ENT_QUOTES (does convert single quotes to entities). You can include another flag, in PHP 5.4, called ENT_SUBSTITUTE which is not a bad idea for UTF-8, i.e. ENT_QUOTES | ENT_SUBSTITUTE. You can pretend that all the other constants don’t exist. The third parameter accepts a string indicating the character encoding of the string being processed and defaults to ISO-8859-1 for PHP 5.3, and UTF-8 for PHP 5.4. Don’t ever set the fourth parameter to TRUE when escaping unless your filtering logic was written by an Über Dolphin – always keep filtering and escaping separate from each other to avoid confusing the two and then having to pointlessly argue why your way is better in defiance of all logic.

The function, if correctly configured using this super simple article for guidance, will now convert the following characters to entities: <, >, ‘, ” and &. These characters make sense to escape since they are used to construct HTML tags, delineate attribute values or reference HTML entities – none of which we want users to be able to do!

If you want some very good advice before your brain implodes from too much reading, a good way to potentially make yourself vulnerable to XSS is to not explicitly set the first two optional parameters ($flags and $encoding) to an appropriate value. In fact, if you see htmlspecialchars() missing any of those two parameters in someone’s source code, you should request that they fix it or, at the very least, curse their name and pray for the Supreme Spaghetti Monster to label them as biohazardous waste in need of emergency disposal.

Now, let’s get down to overloading your brain with information. I’m told that this part is like being sucked into the Total Perspective Vortex machine on Frogstar World B.

To Quote Or Not To Quote. How Is That A Question?

As it turns out, HTML is not simply a popular markup language, it is a popular markup language designed by a bureaucratic species of transdimensional beings seeking to drive Humanity insane by inventing the most impossible-to-secure markup language known in 172 Universes which is then interpreted by “browsers” written by Mice to test the patience of security professionals and keep the really intelligent Humans distracted from the truth of their soon-to-end existence as they search out ever more ludicrous examples of parsing weirdness. Excuse me, I held my breath writing that and need to fetch my Oxygen tank…

Consider the following example. If you want to see whether they work without copy pasting, you can clone all examples from my ominously titled xss repository on Github into a webroot somewhere to read or execute them.

Single Quoted Attributes

1. <?php header('Content-Type: text/html; charset=UTF-8'); ?>

2. <!DOCTYPE html>

3. <?php

4. $input = <<<INPUT

5. ' onmouseover='alert(/Meow!/);

6. INPUT;

7. /**

8.  * NOTE: This is equivalent to using htmlspecialchars($input, ENT_COMPAT)

9.  */

10. $output = htmlspecialchars($input);

11. ?>

12. <html>

13. <head>

14.     <title>Single Quoted Attribute</title>

15.     <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">

16. </head>

17. <body>

18.     <div>

19.         <span title='<?php echo $output ?>'>

20.             What's that latin placeholder text again?

21.         </span>

22.     </div>

23. </body>

24. </html>

If you run the example from a browser and pass your mouse pointer over the text, you will get a popup saying “/Meow!/”. Granted, this is hardly the most impressive XSS ever but remember that the Javascript executed could be a lot more ingenious and damaging. The reason you see alert() used everywhere in XSS examples is to prove that Javascript was executable – a real attacker will hardly advertise his success like this.

In this case, the htmlspecialchars() function call omits the second parameter which defaults to using the ENT_COMPAT flag. With this setting, the function does not convert single quotes to entities, allowing us to inject an unescaped single quote (to close the title attribute value) and another to start a new attribute and value which will be closed by the final single quote used in the template.

We can fix this problem in one of two ways:

1. Use double quotes which will prevent user input from breaking out of the HTML attribute value context using single quotes; or

2. Set the second parameter to htmlspecialchars() to use the ENT_QUOTES flag which will escape any single quotes a user tries to inject.

The moral of the story can be made even clearer by another example. In this case we use another perfectly validating means of delineating attribute values in HTML5 – we just don’t bother using quotes at all!

Quoteless Attributes

1. <?php header('Content-Type: text/html; charset=UTF-8'); ?>

2. <!DOCTYPE html>

3. <?php

4. $input = <<<INPUT

5. faketitle onmouseover=alert(/Meow!/);

6. INPUT;

7. $output = htmlspecialchars($input, ENT_QUOTES);

8. ?>

9. <html>

10. <head>

11.     <title>Quoteless Attribute</title>

12.     <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">

13. </head>

14. <body>

15.     <div>

16.         <span title=<?php echo $output ?>>

17.             What's that latin placeholder text again?

18.         </span>

19.     </div>

20. </body>

21. </html>

Without quotes delineating the attribute value, any space character (including any character a browser might interpret as a space – there are a lot!) allows the user to inject new attributes and values. As from the above, converting all quotes to entities is pointless if there are no quotes to start with! Our escaping doesn’t convert spaces or other space-interpreted characters into entities at all.

By now, you should see the obvious. All HTML attribute values MUST be quoted, and preferably DOUBLE quoted, in any scenario where you suspect untrusted input will be injected into an attribute value, or where htmlspecialchars() calls do not set the second parameter to use ENT_QUOTES. Believe it or not, using single quotes or no quotes remains popular and is perfectly valid under the new HTML5 spec. Some people even celebrate this new insanity. Keep an eye on any designers who look a bit wild eyed or spend too much time smiling while staring into empty space.

Excuse Me, Sir, But Someone Ate My Quotes

One of the great mysteries in escaping output is a common myth known as the Great ASCII Delusion (GAD). Those under the influence of this delusion, besides hearing voices in their head, have arrived at a belief that many character encodings are equivalent for the purposes of escaping those characters which have a special meaning for HTML, e.g ISO-8859-1 and UTF-8. Alas, this is untrue because the Mice created something called Internet Explorer 6 – a thoroughly shameful (but still commonly used) browser which corporations across the Planet continue to insist on using because buying new computers and upgrading operating systems just to use some fancy new Microsoft Office version is seen as a waste of shareholder funds.

Internet Explorer 6 is the bad boy of the XSS world since it’s vulnerable to ridiculous exploits no decent modern browser would dare associate with. Even Netscape would probably spit on it from beyond the grave. For example, have a go with this example using IE6 and PHP 5.3. If you need a testing version of all IE browsers since IE 5.5, you can download IETester from http://www.my-debugbar.com/ietester/index_all.php and use it from Windows. Try hard, I know Windows is bad and the new Tablet makeover for Windows 8 makes you feel ill, but it’s important to see these examples in action.

Source code

1. <?php header('Content-Type: text/html; charset=UTF-8'); ?>

2. <!DOCTYPE html>

3. <?php

4. /**

5.  * You could also subsititute \xC0 or any other impacted character

6.  * above ASCII number 192

7.  */

8. $input1 = 'fakeimage'.chr(192);

9. $input2 = <<<INPUT2

10. onerror=alert(/Meow!/)//

11. INPUT2;

12. $output1 = htmlspecialchars($input1, ENT_QUOTES);

13. $output2 = htmlspecialchars($input2, ENT_QUOTES);

14. ?>

15. <html>

16. <head>

17.     <title>Swallowed Quotes</title>

18.     <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">

19. </head>

20. <body>

21.     <div>

22.         <img src="http://example.com/images/<?php echo $output1 ?>"

23.         title="<?php echo $output2 ?>">

24.     </div>

25. </body>

26. </html>

With the above example, something very weird happens. Using ASCII character number 192 just before a double quote in a document being interpreted as UTF-8 results in the double quote…vanishing in IE6. Seriously, it’s there but not there. Obviously the Mice are behind it – no Human could possibly defy Physics like this!

This allows an attacker to once again break out of the HTML attribute they can inject values into. Using a coincidental opportunity to inject a second free text string nearby which a browser will concatenate to the broken out attribute value of the first, you get an effective XSS combo attack.

This IE6 quirk even bypasses the call to htmlspecialchars() which, as explained above, defaults to the ISO-8859-1 character encoding for PHP 5.3 or less. If the Great ASCII Delusion were not a fabrication of someone’s imaginative wishful thinking, this should not be possible. Not to be too harsh though, this weirdness is due primarily to a bug in IE6′s treatment of the various character encodings where you can trick the browser into thinking something like \xC0 (in hex) is the start of a multi-byte character thus swallowing the next ASCII character (the double quote).

To fix the above weirdness, you must make sure that escaping is done using the same character encoding that the document is being served as. The above HTML document is identifying itself as being UTF-8 but the default htmlspecialchars() encoding is ISO-8859-1 in PHP 5.3 – there’s obviously something not agreeing there! This brings us to the absolutely perfect use (well, almost) of htmlspecialchars(), the golden rule, the Word of The Supreme Spaghetti Monster, the bringer of frustration to XSS attackers:

Always set the third parameter to htmlspecialchars(), set it correctly, and make sure your document is never served with a mismatched or invalid character encoding! Don’t expect some theoretically perfect world to magically appear – browsers are filthily efficient at doing weird things you don’t expect.

I suppose I have to mention that most versions of IE have similar issues with other character encodings such as BIG5 and Shift-JIS. You can test your IE versions using http://ha.ckers.org/weird/variable-width-encoding.cgi to see what characters can be used across different character encodings. Believe it or not, these character encodings are actually still being used and, for some strange reason, people from China and Japan do use PHP.

If you want to be completely paranoid, you can either check the input for invalid UTF-8 (Drupal and HTMLPurifier have reusable functions/classes for this), and/or run it through a conversion function which should theoretically filter out the naughty bits:

$input = mb_convert_encoding($input, 'UTF-8', 'UTF-8');

This is probably a good idea for older PHP versions pre 2010 or earlier but recent PHP versions have specifically improved htmlspecialchars() to disallow invalid characters such as the above (if you set the right character encoding!). You should be aware, though, that htmlspecialchars() may still return blank strings on certain malformed input and, since PHP 5.4, will not issue any warnings about this.

I Broke It! I Broke It!

Before you think htmlspecialchars() is getting off lightly, there is one minor quibble. We’ll keep picking on Internet Explorer 6 for the rest of this article since it’s so easy to exploit.

Source code

1. <?php header('Content-Type: text/html; charset=UTF-8'); ?>

2. <!DOCTYPE html>

3. <?php

4. $input1 = 'fakeimage'."\xC0";

5. $input2 = <<<INPUT2

6. onerror=alert(/Meow!/)//

7. INPUT2;

8. /**

9.  * If you think PHP 5.4 will save you - empty strings make it guess the encoding

10.  * or use the default_charset value from php.ini. You sure everyone on the whole

11.  * planet uses UTF-8? Under 5.3 - empty strings === default encoding.

12.  */

13. $encoding = ''; // from outside source or unvalidated variable

14. $output1 = htmlspecialchars($input1, ENT_QUOTES, $encoding);

15. $output2 = htmlspecialchars($input2, ENT_QUOTES, $encoding);

16. ?>

17. <html>

18. <head>

19.     <title>Swallowed Quotes</title>

20.     <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">

21. </head>

22. <body>

23.     <div>

24.         <img src="http://example.com/images/<?php echo $output1 ?>"

25.         title="<?php echo $output2 ?>">

26.     </div>

27. </body>

28. </html>

Setting the third $encoding parameter of htmlspecialchars() to an empty string in PHP 5.4 will set the encoding to be auto-detected, grabbed from the php.ini value of default_charset, or guessed from the current locale (in that order). Be very careful under PHP 5.4 NEVER to let this happen. Don’t leave your escaping parameters to chance.

Use empty() or strlen(), for example, to spot this issue if accepting encodings from another source or variable that might allow for empty strings. Again, this behaviour is very secure and there’s nothing wrong with it whatsoever. Oh, who am I kidding… This is the dumbest parameter behaviour ever invented. NULL means use the default encoding; blank string means play a guessing game. Even Vogon poetry pales in comparison to such nonsense. One slip and an empty parameter string can rip apart this house of cards because who knows which character encoding will be used.

Oooh, I wonder what this does under PHP 5.3… Yes, er, don’t allow blank encoding parameter strings under PHP 5.3 either. Setting an empty string in PHP 5.3 is interpreted as setting the default character encoding, i.e. ISO-8859-1, instead of triggering the expected warning about an unsupported encoding.

So, be careful kids. When setting the encoding for htmlspecialchars() do a safety check to make sure it’s not an empty string you are passing in. Keep it predictable and consistent.

There’s also one other curious behaviour when using htmlspecialchars().

Source code

1. <?php header('Content-Type: text/html; charset=UTF-8'); ?>

2. <!DOCTYPE html>

3. <?php

4. error_reporting(E_ALL);

5. ini_set('display_errors', 1);

6. $input1 = 'fakeimage'."\xC0";

7. $input2 = <<<INPUT2

8. onerror=alert(/Meow!/)//

9. INPUT2;

10. /**

11.  * Invalid encoding makes htmlspecialchars() throw a warning but it continues

12.  * the current operation anyway using the default encoding even if the default

13.  * is an unsafe choice for the application. Don't allow invalid encodings!

14.  */

15. $encoding = 'invalid-encoding'; // from outside source or unvalidated variable

16. $output1 = htmlspecialchars($input1, ENT_QUOTES, $encoding);

17. $output2 = htmlspecialchars($input2, ENT_QUOTES, $encoding);

18. ?>

19. <html>

20. <head>

21.     <title>Swallowed Quotes</title>

22.     <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">

23. </head>

24. <body>

25.     <div>

26.         <img src="http://example.com/images/<?php echo $output1 ?>"

27.         title="<?php echo $output2 ?>">

28.     </div>

29. </body>

30. </html>

When you set an invalid character encoding, not the empty string of doom, htmlspecialchars() will issue a Warning level error…and continue merrily on its way by reinstating the default encoding. In a production scenario, you will likely have display_errors disabled and this warning will be logged and possibly ignored by some users. If this makes it through, setting an invalid character encoding whether by a deliberate user value or simple programmer error may create an exploitable scenario.

So, make sure you also validate the character encoding. Don’t just leave it up to htmlspecialchars() since it allows the continued execution of the application. Arguably this should be a fatal error since a bad encoding is itself a security problem.

Seriously, this function is like handing a box of matches to a Human and telling them there’s a rainforest nearby that’s essential to all life on Earth…

Internet Explorer: Master Of Supporting Stupid Character Encodings

Internet Explorer is unique in the Universe. Designed by Mice to be the dumbest, most frustrating, most stubbornly non-upgradeable piece of crap ever, it does things that make XSS far easier. The terrible part is that IE is popular with corporations and businesses using commodity hardware imported from whichever country currently has the lowest paid PC assemblers on Earth. One would think they’d like a more secure browser to protect their money making endeavours.

It’s no wonder that Dolphins had to think long and hard before deciding we were marginally smarter than the average cat. Cats, coincidentally, strenuously deny this claim having spent thousands of years demonstrating a lack of Human intelligence by showing how easy it is to make Humans cater to their every need…for free. Even their main rivals, Dogs, are expected to do useful work like herding sheep, chasing cars, digging holes, barking at strangers, and keeping bill bearing postal workers at bay.

All versions of Internet Explorer support a troublesome character encoding called UTF-7 which, oddly enough, is not supported by htmlspecialchars(). You can probably see where this is going. How do you escape a character encoding that your escaper doesn’t even support? Easy, you can’t. JUST DON’T USE UTF-7! EVER! UTF-7 has the distinction of definitely not being ASCII compatible – it encodes angle brackets (used to open and close HTML tags) very differently so they are never detected by filters or escapers relying on other character encodings.

Unfortunately, some applications do allow users to cherry pick an encoding. It’s not uncommon in international websites (e.g. Google which had this problem). Here’s an example of what not to do:

Source code

1. <?php

2. $input1 = 'UTF-7';

3. $input2 = <<<INPUT2

4. <script>alert(/Meow!/)//</script>

5. INPUT2;

6. $input2 = mb_convert_encoding($input2, 'UTF-7', 'UTF-8');

7. $output1 = htmlspecialchars($input1, ENT_QUOTES, 'UTF-8');

8. $output2 = htmlspecialchars($input2, ENT_QUOTES, 'UTF-8');

10. header('Content-Type: text/html; charset='.trim($output1));

11. ?>

12. <!DOCTYPE html>

13. <html>

14. <head>

15.     <title>Mismatched Encoding</title>

16.     <meta http-equiv="Content-Type"

17.     content="text/html; charset=<?php echo $output1 ?>">

18. </head>

19. <body>

20.     <div>

21.         <?php echo $output2 ?>

22.     </div>

23. </body>

24. </html>

This works in all IE versions. The problem here is that we’re letting the user set the character encoding without validating it against a safe whitelist of encodings that we can actually escape. This also works even when you plead with the Supreme Spaghetti Monster and try passing UTF-7 to htmlspecialchars() since the function simply issues a warning and reinstates its ISO-8859-1 or UTF-8 default before continuing on its merry way to making you vulnerable to XSS. Yes, very secure behaviour there…

Note: putting the @ symbol in front of htmlspecialchars() to hide these warning errors during development is not considered an act worthy of an intelligent species. Don’t let the cats win!

Now, you might think that this would be the end of it, but there’s one other problem afflicting older browsers (fixed as of Internet Explorer 9). In certain scenarios you can trick the browser into rendering pages as UTF-7 even when you can’t set the page’s character encoding. This is due to a bug in how some browser versions guess the character encoding when it’s absent (i.e. not set in a header or meta tag, or set incorrectly, e.g. UTF-8 is valid; UTF8 is NOT).

To pull off this exploit, you need to first set some UTF-7 text which is persisted across requests, e.g. a blog comment. Since we can’t escape UTF-7 in PHP, the persisted text will contain some UTF-7 encoded XSS code. Just in case, you’re smart and you’re thinking that mbstring functions might help detect UTF-7 – they won’t. mbstring will detect UTF-7 as UTF-8, and UTF-8 as UTF-7 depending on the detection order set in mb_detect_encoding(). After that it’s a long winded story of using iframes to trick a browser into rendering the innocent looking UTF-7 strings on your webpages as UTF-7.

Where escaping fails, some common sense should win out. Just make sure all the responses you serve have a header that sets the appropriate character encoding for the content (use a valid encoding string, not an invalid string form). In HTML, use the relevant meta tag to indicate the content’s character encoding as a backup should the header be somehow omitted.

Conclusion

Htmlspecialchars() as a function for escaping output has its limitations. If you’re unaware of these and wish to persist in using it incorrectly, you should expect to be burned. No, seriously, there really is an incinerator for those labelled as biohazardous waste over in Alpha Centauri.

I get the feeling I’ve written enough for you today. I’m very sorry for the 0.006% of you that Vogon studies indicate are now sitting at their desk drooling all over their keyboards from encroaching insanity. If you’re worried about joining the 0.006%, please submit the correct form in triplicate, completed in capitals using a blue ball-point pen, to your local Alpha Centauri Medical Facility where the friendly Vogon staff will give you a free brain scan and determine whether the incinerator next door needs more fuel.

So, what next? In Part 2, we continue our voyage into madness with more examples using htmlspecialchars() though in another direction this time. In the meantime, you have a lot of examples (aka ammunition) and there are a lot of applications/frameworks/libraries (targets). I figure the rest is obvious.

See you for Part 2!

Mockery is a simple yet flexible PHP mock object framework for use in unit testing with PHPUnit, PHPSpec or any other testing framework. Its core goal is to offer a framework for creating test doubles like mock objects through the use of a simple and succint API capable of clearly defining all possible object operations and interactions using a human readable Domain Specific Language (DSL). Designed as a drop in alternative to PHPUnit’s phpunit-mock-objects library, Mockery is easy to integrate with PHPUnit and can happily operate alongside phpunit-mock-objects.

Today, I am pleased to announce the release of Mockery 0.7.2, a maintenance release fixing a small number of bugs and annoyances. A special thanks to all those who forked the Github project at and submitted pull requests! Leaving a developer with hardly any work to do other than a quick test and merge is always appreciated! You can install or upgrade to the new version from the survivethedeepend.com PEAR channel.

Another piece of news is that Mockery is now available on Packagist.org for users of Composer. Composer is a tool to help you manage your own projects’ or librarys’ dependencies and it can handle and mix dependencies from Composer compatible repositories like Packagist.org, any git repository using tags, and any PEAR channel. I do this of my own free will and not because Luis Cordova and Benjamin Eberlei are standing behind me with pitchforks .

The more pertinant fixes include:

1. Fixed a problem in resolving methods chains which abuse the Law of Demeter (thanks to the wizardly Robert Basic).
2. Fixed unexpected static calls to an alias mock which were causing fatal errors (thanks to Luis Cordova).
3. Fixed a crash present since PHP 5.3.6 due to a referenced $this variable entering a closure (thanks to Martin Sadovy).
4. Added support for PHP_CodeCoverage 1.1 whose filter class is no longer a singleton (thanks to Matthew Vivian).
5. Added non-halting exception handling (for Mockery exceptions) to the PHPUnit TestListener (thanks to Adrian Slade).
6. Added boolean $prepend (defaults to FALSE) parameter to \Mockery\Loader::register() to allow for registering Mockery’s autoloader to the top of the autoloader stack even after other autoloaders have been registered (thanks to Hermann Kosselowski).
7. Updated documentation/tests for the release of Hamcrest 1.0.0 several days ago (thanks to me, me, me – who finally got to do something nobody else had a pull request for!).
8. Added new \Mockery::self() static method to make retrieving the current mock object simpler and more readable while setting expectations without the need to refer back to past variable assignments.

Users should also note that Hamcrest 1.0.0, which includes a small filename change (hamcrest.php was capitalised to Hamcrest.php), was released several days ago. If you use Hamcrest matchers with Mockery, you should ensure that both libraries are updated on your system.

As always, please report any bugs or potential improvements to the Github issue tracker using the relevant label or, even more appreciated, send me a pull request.

Image via Wikipedia

Back from my extended leave of absence, I’ll re-open the dusty cobwebbed depths of this blog to echo the sentiments of Paul Reinheimer in his recent article “Cookies don’t replace Sessions“. The topic is actually an old one since Ruby On Rails has adopted the strategy of storing application session data in cookies by default (take note, performance hounds). The purposes of storing sessions in userland cookies rather than the conventional “stick-it-on-the-filesystem/database” used by many apps is one of performance and a little obscuration. Cookie data can be accessed faster than hitting the filesystem/database plus it has the dubious ability to disguise the session-targeted programming language. Really though, PHP is assumed to be on all web servers so hiding its existence is a bit like trying to hide an elephant in a zoo. Hide it all you want – we still know there has to be one in there!

In exchange for speeding up session reading, storing session data in cookies has some fairly uncomfortable costs.

Now, developers are not unaware of the problems of storing potentially sensitive application data in plain text files on the user’s PC which users can manipulate, copy, and mangle to their (or the hacker’s currently fiddling with the user’s PC) heart’s content. It’s dangerous depending on just how much you rely on session data to drive other security rules or restrictions on business logic within the application. Technically, the reliance placed on sessions should be close to nothing – session data should drive the application towards other storage solutions for the really essential stuff and just stay around as a minimal identifier/stash of basic ID info. Such minimal information can be dumped, corrupted, or overwritten with the only cost being to perhaps require a user to login again when that happens. Stuffing a bank balance into a session, on the other hand, is one (very exaggerated!) example of the kind of data you should be shot for relying on a session for.

Programmers being programmers, it’s not rare to see sessions become a more intrinsically important storage location than it should be. In those cases, being able to manipulate the session data can become a problem and may give rise to exploitation scenarios where tampering with the stored data leads to some benefit for the manipulator. Obviously we want to make sure that that can’t happen even in scenarios where programmers may be a bit loose with where they store data. We don’t build frameworks and libraries for Gurus, we build them for all programmers – even the sometimes ignorant and under trained ones. This cookie stored session data is often coupled with the ability to encrypt that data. However…

As Paul Rainheimer remarks in his article, “Encryption is often viewed as a panacea for security problems, you sprinkle a little encryption dust around, and your problems dissolve”. This is an absolute truth in programming – programmers often view encryption as a solution without regard for one teeny tiny problem. If you encrypt a set of data for any purpose, even though it’s encrypted, the user (or the hacker hacking the user’s account) still has the data in some usable form!

With perfectly intact data, and even through it’s hidden by encryption, that data can be recycled simply by copying it to another machine. Depending on the data that is stored (which admittedly may require the hacker/user to figure out by doing actual work like finding your open source app on Github or breaking a developer’s fingers until they spill the beans), you can restore past data just by copying over a backup of a prior cookie or repeat a past transaction by continually reusing the original cookie it required. Paul offers a few trivial examples in his article.

Such reuse of data is known as a replay attack. A scenario where even encrypted data can be constantly reused to give rise to a positive result – all without any need whatsoever to break the encryption. The antidote to this vulnerability is to ensure that all data sets are unique and can be used only once, i.e. you include a single-use nonce (some generated set of characters or bits) in the data which is updated whenever that data is used. This continually forces the update of the relevant digital HMAC signature and/or encryption result (even for the exact same data otherwise) in order to prevent any reuse of old data in a replay attack. Once a nonce is used, it’s discarded, and the old data can no longer be accepted by your application. Of course, the downside is that since the nonce must be single-use, you need to keep track of all nonces to ensure they are not accidentally used again. You will need a database, possibly using a nonce-included timestamp as a time limit so your storage requirements aren’t completed insane, which obviously means that just using the traditional database storage for sessions in the first place would have been a much better and simpler choice.

So, in summary, encryption prevents the reading of data but it does not prevent the reuse of existing data. For that to be prevented you need a nonce implementation. And, due to the complexity of using and tracking nonces, practically no cookie stored session solutions will actually offer nonce support because it would eliminate their speed advantage. Which means they are susceptible to replay attacks, which means they are dangerous tools to be swinging around blindly, which means that the old local session storage strategies are still far superior from a security perspective, which all means that you should avoid cookie stores like the damned plague and stick to the old, traditional but secure session storage strategies we already have unless you a) are crazy or b) trust your colleagues (and yourself) not to screw it up.

Even without the security concerns, there is also another less critical downside to storing sessions in cookies which is that cookies have a storage limit of around 4KB. No other storage solution for session data should have that problem but you need to be aware of it anyway as using encryption may push you there sooner than the base data size might suggest (encrypted data size is usually larger than the original data). While noting this, you should never really hit that limit unless you are storing data there that you likely shouldn’t be anyway!

So, cookie based session storage: It’s very fast but lethally insecure if you store the wrong type of data. If you’re going to use it, make sure you keep a tight rein on what data is being stored.

Image via Wikipedia

If you’ve been watching the PHP weather vane (we call it Twitter for short), you may have noticed a shift in Symfony and Zend Framework. Version 2.0 of both web application frameworks feature Dependency Injection Containers (DICs) as the primary means of creating the objects (and even Controllers) your application will use. This is an interesting shift in a programming language that often stubbornly evaded adopting DICs to any great extent. In this mini-series of articles, I’ll take a look at the marvellous world of Dependency Injection as we run up to an examination of Zend Framework 2.0′s Zend\Di component in the next part.

What is Dependency Injection (DI)?

The short answer to this question is that Dependency Injection is a design pattern where, instead of dependent objects creating their dependencies internally, they instead define setters, constructor parameters or public properties which allow a user to “inject” dependencies from the outside into the dependent object and where such dependencies adhere to an expected interface.

If the definition sounds familiar, it’s because Dependency Injection is an obvious design pattern. As a programmer who knows how to use PHPUnit, you probably use the pattern every time you open an editor. So let’s quickly look at why the pattern is both obvious and ubiquitous.

Imagine a class implementation called Leprechaun. In writing the class, we realise we have a dependency on another class called PotOfGold. A naïve implementation would start out very simply with the Leprechaun object creating an instance of PotOfGold for use.

If you think this through, you may notice the problems. What if we want our Leprechaun to instead have a PotOfRareEarthElementsFromChina? What if we need to replace PotOfGold with a mock object during unit testing? What if another users locates a bug in PotOfGold and needs to replace it without editing the original class (since it’s under 3rd party version control)?

The answer to all these questions is to allow external parties to inject dependencies instead of relying on the object to create them internally. Based on our ridiculous example from above, we would define a setter called setPot(), and allow it to accept any object which implements a new Pot interface. Using an interface merely ensures the dependency that is set obeys some interface the dependent object is expecting.

That, in a nutshell, is why Dependency Injection is obvious. It’s a simple shuffling of creational responsibilities from within an object to some external agent which makes the dependent object more flexible, testable and amenable to the wisdom that Composition is preferred over Inheritance (i.e. injecting objects beats monkey patching!).

Some External Agent

In applying Dependency Injection, we eventually reach a state where all objects in a system are created by a mysterious external agent. What is this entity?

One possible candidate is whatever passes for a Controller in your framework based application. In Zend Framework, this would be an instance of Zend_Controller_Action. Our Controller, in this instance, would define an action method which would perform a necessary application task and create all the objects needed to perform that task. This makes a lot of immediate sense to programmers since allowing you to write Controllers with as little fuss as possible is a fundamental goal of any framework.

However, Controllers are objects! If you had a NewsletterController defining an emailAction method, you might expect that creating an instance of Zend_Mail inside that action is obvious (which it is). Think again! In Dependency Injection parlance, your Controller is a dependent object and an instance of Zend_Mail is one of its dependencies. This is no different from our Leprechaun example. If we create the Zend_Mail instance inside the Controller we get the same irritatingly stubborn question. How do we replace the Zend_Mail instance with an alternative, test double or monkey patched version containing an emergency bug fix?

Controllers, alas, are not the external agent we’re looking for to create objects. And yes, you really should be testing your Controllers .

The next entity a level above Controllers can be loosely termed the Bootstrap. In Zend Framework 1, this started out as a relatively simple script to do just enough that you could start the FrontController and dispatch a request. In other words, Zend Framework traditionally did not offer a final external agent as needed for Dependency Injection. It left it to individual users to create something of their own or, as became inevitable, to just create objects in the Controllers themselves.

More recent Zend Framework versions offer Zend_Application, a method of bootstrapping that allowed users to define Resources, i.e. using a method or class which created an object (and injected its dependencies) and returned it on demand when it was needed by a Controller. This was the first consistent approach to handling object creation in ZF which effectively involved defining any number of Factory classes or methods in one location and passing the managing object (the Bootstrap) around the application wherever specific objects needed to be retrieved. In effect, this was a Dependency Injection Container. So, surprise, users of Zend Framework already have a DIC. An even lesser surprise: Zend Framework 2.0 will be no different.

Dependency Injection Containers Are The Devil

The concept of a Dependency Injection Container (DIC) is to act as a programmable object assembler. You take your DIC, tell it how to construct objects (including how to construct and inject their dependencies), pass the DIC to wherever it’s needed, and eventually ask it to create an object it knows about. This is not rocket science. DICs are simple animals to understand, however the devilish suspicion that PHP developers have for DICs is not rooted in what they do but how they do it and whether they make a developer’s life easier.

There’s a widely known belief that the Ruby language doesn’t need a DIC. I’ll use Ruby as an example because it has a few features PHP programmers can salivate over (like how it uses a new method for classes vs PHP’s new keyword making class subsitutions stupidly easy). One investigator of Dependency Injection from the Ruby world is Jamis Buck. For Ruby he wrote two DICs: Copland (a port of Java’s HiveMind) and Needle (it’s like Pimple on steroids which…defeats the purpose). After fighting Ruby for a few years, he finally gave up on trying to write a Ruby DIC and documented his thoughts on his blog in “LEGOs, Play-Doh, and Programming“.

The core lesson from the article holds true even in PHP – by and large, complex DICs are a complete waste of time in most scenarios. Indeed, if you ever use a DIC and discover it requires just as many (if not more) lines of DIC code and configuration as it would to do the same thing in plain old PHP, you should start asking where the fabulous benefits have vanished to because it’s not delaying the onset of cramped finger muscles as advertised.

Most PHP developers understand this instinctively. Unlike Jamis, most PHP programmers probably won’t have a strong Java background. As a programming group, we’re less inclined to assume we need a special DIC blessed by the PHP Gods so we fall back to whatever strikes us as a simpler solution.

But here’s the rub – the simplest solution is itself a DIC.

In referring to Dependency Injection Containers as the devil, cursing their name, and blaming them as Java imports designed to make life more complex than needed, it’s easy to lose sight of the fact that such criticism is about the implementation of DICs and not their actual function. There is nothing wrong with having object assemblers – we use them all the time and call them Service Locators, or Factory Classes, or Zend_Application (Resources), or any of a dozen terms slightly different and probably not entirely accurate. Most of the time we’re trying to create a DIC without being aware of the term.

Needles and Pimples (It’s Not What You Imagine)

Jamis Buck hit the nail on the head back in 2004 with the creation of his Needle DIC Ruby. Instead of creating something inspired by Java that relied on static configuration and too many features, he realised that Ruby excelled (as does PHP to a growing degree – thank Closures) in expressing logic through a Domain Specific Language (DSL). The result was a DIC captured by a simple DSL – well, until he went and overcomplicated it (read his article).

You can see the exact same fundamental simplicity that a DIC is capable of in PHP. It’s a small so-tiny-you-won’t-believe-it DIC called Pimple. Try calling that complex, hard, stupid or any other adjective you might instinctively think of when faced with the term “Dependency Injection”.

The core of Pimple is that you define object creations as closures. This immediately resolves a few traditional DIC problems. There’s no static configuration, you hand code all creation logic exactly once, and objects are named services you can recall and inject into other objects from your closure bodies. It basically takes everything you’d do in creating objects by hand and captures it all in one container. Other than the fact I hate arrays (my version uses object properties instead – it’s 50 lines; nobody was killed during its 5 minute development period), Pimple is like Dependency Injection itself – so blindingly obvious you may kick yourself.

Pimple proves that DICs are not the devil – they can be incredibly simple and useful tools if you can tame the urge to complicate it’s implementation.

Then There Were Frameworks

As you can probably see, making a strong case for DICs is not hard. Dependency Injection is obvious and omnipresent in PHP. Dependency Injection Containers can be a simple 50 line class you can write over a coffee break. The going gets tough when the simple notions we desperately want to cling to meet the complexity of PHP’s now standard tool: the application framework.

Frameworks: Not Written By Monkeys

As we’ve already covered, Zend Framework 1.0 covered off the external agent problem in Dependency Injection by creating Zend_Application. As Zend Framework 2.0 moves towards beta, it also needs a Dependency Injection Container to do similar heavy lifting. This time around, we called a spade a spade and the O’Phinney/Schindler hive mind wrote Zend\Di\DependencyInjector.

The DICs used by Symfony and Zend Framework are not like Pimple. Symfony’s DIC is driven by static configuration (preferably YAML for brevity). Zend Framework 2.0′s DIC is driven by a PHP API (no static configuration). Both have their own set of performance boosting measures to minimise any overhead in using a more complex DIC.

In the next part this mini series, we’ll take a deeper look at Zend\Di and see how it fares compared to Pimple or Symfony 2. In the meantime, I hope I’ve busted a few apprehensions you might have about using a DIC .

Image via Wikipedia

It’s been a busy month in Zend Framework land which I’ll blog about shortly so, after a few weeks of delay, here’s the July 2011 Summary of the zf-contributor’s mailing list.

ZF2 Feedback

Late June kicked off with this topic from Robert Basic with a set of notes on his experiences in getting started with ZF2 by migrating a ZF1 application. Adam Lundrigan noted, correctly, that a lot of “bleeding edge” code is not included in the main repository at this time and is distributed across contributor Github forks. He also raised the suggestion for a ZF2 Status Page. Derek Miranda voiced his agreement with Adam. Robert also agreed noting the difficulty in assessing the state of components.

Summary: ZF2 is scattered across multiple forks – be prepared to rely on notes such as Robert’s if jumping in at the deep end.

Creating a 1.11.9 Hotfix Release

A short note from Matthew Weier O’Phinney announced that a 1.11.9 hotfix release would be made to fix a number of backwards compatibility breaks introduced in 1.11.8. Issue tickets involved were ZF-11548, ZF-11550, ZF-10991 and ZF-10725.

Summary: It’s a maintenance release. It fixes stuff.

Zend\Http and MVC Developments

Ralph Schindler presented a document outlining a requirement list and the overall architecture of classes and interfaces for Zend\Http, noting work would commence on a prototype once any outstanding items suggested were cleared. Rob Zienart commented that the document indicated interfaces for Zend\Http Client and Server components and mentioned they needed proposals. Matthew responded that Zend\Http’s Server would deal with classes extending Zend\Service\Abstract such as SOAP and AMF but would not comprise a HTTP Server given it was covered by PHP 5.4. Anthony Shireman asked whether there were any Zend\Http Server plans or whether it was a “time will tell” situation. Matthew confirmed that that was the case given PHP 5.4 would include a HTTP Server and ZF2 could piggy back that implementation in offering a development server environment.

Summary: HTTP work continues. We’ll need it to communicate with all those big tubes connecting PCs.

[Proposal] ActiveRecord Proposal

Artur Bodera raised the proposal and offered to implement an ActiveRecord solution noting its benefits compared to Zend\Db. The proposal was published at http://framework.zend.com/wiki/display/ZFDEV2/ActiveRecord+-+Arthur+Bodera with a working branch at https://github.com/Thinkscape/zf2/branches/ActiveRecord.

Nicolas Bérard-Nault asked why it was necessary to reinvent the wheel instead of integrating with other existing and mature implementations. Artur responded that other solutions did not integrate with Zend Framework noting his proposal is built on Zend\Db from ZF2 and he wondered what was the point of Zend\Db\Table otherwise in the face of Doctrine or Propel. Peter Kokx responded to note that Zend\Db\Table implements the Table Data and Row Data Gateway patterns as distinct from ActiveRecord and that users shouldn’t interpret MVC as referring solely to ActiveRecord. Artur conceded that this was a good point but pressed his point that ActiveRecord was one tool which did on impose on any others available to Zend Framework using Zend\Db. Tomáš Fejfar voiced his support for adding ActiveRecord noting its value in simple use cases to get things done fast.

Ralph Schindler leaped to the rescue by noting that ActiveRecord is indeed planned for ZF2 and noting the significant work done to date on Zend\Db in his own feature branch. Artur Bodera welcomed the progress stating he would migrate his ActiveRecord solution over to the improved Zend\Db once complete.

Summary: We’re getting an ActiveRecord implementation for ZF2.

ZF2 Docbook Sources Converted to DocBook 5

Another short note from Matthew Weier O’Phinney informed the community that ZF2′s docbook formatted manual files had been migrated to Docbook 5. The community silently admired the completion of this task (nobody responded but I assume they silently admired all the same!). Matthew noted the README for manual generation would be updated and that Docbook 5 made certain tasks a lot easier.

Summary: ZF2 Manual will be written in Docbook 5, those using a visual XML editor may celebrate.

ZF2 Zend\Mail: To strip/validate or not to strip/validate (email adresses)

Status of the Test Suite (ZF2)

Keith Pope asked after the status of the Test Suite mentioning that phpunit.xml was mostly commented out, Zend\Di was not using the @group annotation for the test runner, and TestConfiguration.php was nearing 800 lines. He suggested that the configuration be spread into a conf.d setup (i.e. each configuration segment split into a separate file and all combined at runtime). Matthew responded noting the ease with which ZF2 tests could be run by passing the necessary directory to phpunit from the main /tests directory, and noted configuration may be pushed into phpunit.xml instead of the current PHP file. While expressing an interest in a conf.d setup, Matthew noted this would depend on support in PHPUnit.

Summary: Ignore runtests.sh and just use the stock phpunit commands for ZF2.

Serious Question about Mcrypt

Artur Ejsmont observed that the mcrypt filter calls srand() with a limited range of potential seeds thus suggesting it would impact on the security of the filter. Enrico Zimuel replied that the srand() is only used in limited circumstances (where a better source of randomness is not available) and that it’s not a serious problem since the encryption security is not wholly based on the initialisation vector (IV) that uses srand() on some platforms. Nevertheless, he did note that some improvements could be made.

Artur responded with a general query on the efficacy of using srand() and rand() to avoid collisions. Pádraic Brady responded that rand() was particularly bad noting you could create collision in a matter of minutes. Pádraic also noted that mt_rand() was far more effective but also not entirely random (as a graph of its output would prove) suggesting that it was advisable to use better random sources such as /dev/random and /dev/urandom where feasible. Enrico also noted the availability of openssl_random_pseudo_bytes().

Summary: Getting random bytes is a tricky business.

ZF2 Zend\Code Bugfix

Nick Belhomme mentioned he had been looking at Zend\Code which is used heavily by Zend\Di. He noted his first impressions that it should work well by being token based but also referred to his opinion that it was quite error prone and the unit tests were not satisfactory.

To explain his case, he used an example of a method signature accepting four type hinted object parameters noting this could fail to be analysed correctly due to the whitespace in the parameter list (after each comma) not being handled correct by the ParameterScanner. Nick noted he’d committed a fix using a short trim function to his own git fork.

Regarding the unit tests, Nick explained why the current unit tests were insufficient in testing parameters and suggested rectifying the test doubles to account for whitespace.

Summary: Zend\Code needs to build up a fuller test suite accounting for different coding styles.

Image via Wikipedia

Some time ago, in between working on Zend Framework, I booted up a couple of libraries that I really wanted to integrate into my workflow. Recently, I’ve been being putting these through the grindmill so they can be properly released and supported for public consumption across PEAR. Just as Mockery fell out of older work on PHPMock, Mutagenesis will fall out of another project called MutateMe. This is a short introductory article as to what Mutagenesis will do and why. In other words, what the heck is Mutation Testing?

First, some background.

The most common means of measuring confidence in a test suite is the Code Coverage metric. Code Coverage essentially checks, on a per class basis, how many of the lines of code in the class are executed by a test suite and expresses this as a percentage. For example, a Code Coverage of 85% means 85% of the lines of code in a class was executed and 15% were not. The greater the number of lines of code executed, the more confidence one can presumably have that a test suite is doing its job, i.e. verifying class behaviour, preventing the introduction of bugs, supporting refactoring, and so on.

I have a huge and insurmountable problem with Code Coverage. For starters, my average Code Coverage is closer to 80% than the 90% expected of projects such as Zend Framework. The gap is explained by me not testing what I call “braindead” functions, i.e. methods which are either ridiculously simple, where a malfunction would quickly become self-evident, or which are marginalised (on the borders of deprecation). So Code Coverage actually increases the amount of work I need to do for very little gain and a lot of frustration.

Secondly, Code Coverage is easy to spoof or misinterpret. Since it’s a metric measuring the execution of source code, you need only…well…execute the source code. It’s a simple matter to construct a series of wonderfully useless tests to do just that and obtain a high Code Coverage result – it’s done all the time in my experience once someone’s patience in writing quality unit test runs out. It is particularly evident in cases where unit tests are written after the source code is completed – a still too common practice in PHP. The less villainous flipside is that certain nuggets of source code are fundamentally difficult to test. For example, a complex algorithm suffering from poor documentation may make composing a suitable unit test near impossible. The rollout of OAuth was filled with such examples.

This leads into my opinion of Code Coverage. I view the venerable Code Coverage metric as a near pointless exercise. While it may tell how much source code a test suite exercises, it tells you nothing about the actual quality of those unit tests. They could be good tests, sort-of-good tests or absolutely horrendous tests – Code Coverage will never tell you either way. I say near pointless because there are precious few alternatives. We need something to give us a reason to trust and have confidence in test suites and Code Coverage is easy to implement and has been a part of PHPUnit since forever. So, by and large, we make do. We measure Code Coverage just to make certain some kind of unit testing was performed.

Is there nothing better?

A good unit test serves a simple purpose. It verifies a behaviour of an object. In PHP, we’re more likely to verify umpteen million behaviours in a single test (count your assertions!) but we’ll let that slide. Since a test verifies behaviour, it follows that a test should fail when that behaviour is changed. If a test does not fail when class behaviour is changed, it also follows that the original behaviour was not fully tested, i.e. there is a gaping hole in our test suite whether due to a flawed or missing test that could allow bugs entry into our application. So, to really stick unit tests under a microscope to assess their quality and our confidence in them, we need to introduce changes into the source code under test and see if the unit test suite can or cannot detect them.

This process is known as Mutation Testing. Mutagenesis is a Mutation Testing framework for PHP 5.3+.

Mutation Testing, as you have probably surmised, is not a super-complex activity. You take a set of source code and compile a list of possible “mutations” that are likely to break the behaviour of the source code. Then, you apply one mutation to that source to create a “mutant”, i.e. a copy of the source code with the mutation change applied. Next, you run the source code’s test suite against the mutant and see if any tests fail. If a test fails, celebrate – the mutation was detected so your tests were, in this instance, adequate. If no test fails, curse the Gods – the mutation was not detected and you’ll need to figure out whether a new test is needed or an old one modified/corrected. Rinse and repeat the above for each mutation you’ve compiled.

Mutations are typically quite simple such as replacing operators, booleans, strings and other scalar values with either an opposing form or a random value. Expressions might also be reversed or driven to zero to give an opposing boolean or zero value. Making such minor changes seems like a minor irritation but behind every serious flaw in an application is one or more smaller contributing errors. If your test cases can detect the potentially contributing errors, then there’s an excellent chance it would detect the bigger ones anyway. This is known as the Coupling Effect in Mutation Testing.

Some of you will be vaguely aware of Mutation Testing. In terms of implementations, Ruby has heckler, Python has Pester, and Java has Jumbler, Jester and a couple of others. Those who prefer Microsoft’s technologies can use Nester. There’s a running ryhme apparent since so much is inspired by the original Jester framework for Java. To my knowledge, Mutagenesis will be the only Mutation Testing framework for PHP (though I sincerely wish I was wrong).

Examining those libraries, you eventually realize a few problems with Mutation Testing which explain its lack of popularity until relatively recently: performance is a concern and Mutation Testing requires a Human Brain to complete the process.

Performance is a concern because each mutation requires a test suite to be executed. Imagine a set of classes from which you extract 100 possible mutations, coupled with a test suite that takes 5 minutes to run. A basic Mutation Testing framework (e.g. Ruby’s heckler) would therefore take 500 minutes to complete a Mutation Testing session. That’s 8.3 hours of continuous Mutation Testing. Mutation Testing for Zend Framework would be very interesting .

Similar to Jumbler for Java, Mutagenesis will utilise a few heuristics (shortcuts) to significantly improve performance without compromising results. We only need one single test to fail in order to rule that a mutation was detected and killed, so we can do a few things to boost performance:

1. Terminate the test suite on first failure/error or exception.
2. Execute test cases in order of execution time ascending (fastest first; slowest last).
3. Prioritise execution of last test case to detect a mutant to take advantage of same-class detection.
4. Log which tests detect which mutations, and prioritise those associations in subsequent runs.

The effect of the above is to speed up Mutation Testing by a significant degree. The final heuristic ensures that for gradually changing source code and tests, the first Mutation Testing process might take a while but subsequent runs will be significantly faster making them far more usable in a Test-Driven Development setting. Mutation Testing is best served with a healthy dose of efficiency.

The second reason for its lack of popularity is that Mutation Testing can’t analyse the logic of the source code under test. For example, an expression might accept any integer less than 10 to evaluate to TRUE. If the input from another class were 7, and a mutation were generated to swap this for a 9, then the associated unit test would still pass (the mutation of switching 7 for 9 still allows the <10 expression evaluate to TRUE). If you recall, if a mutant passes a test suite than we assume either the presence of a flawed test or the lack of a suitable test. Obviously, as the above suggests, this isn’t always the case. Mutation Testing can and often will report false positives.

Ruling out false positives, coupled with the need to improve test suites to detect more mutations, makes Mutation Testing a source of extra work. Who likes extra work least? Programmers, especially the lazy kind .

Mutation Testing is not a far fetched idea. The principles are sound and it beats the pants off Code Coverage when it comes to measuring what confidence we can have in our testing suites. It is still hampered, as a methodology, by the lack of good implementations in other programming languages. Mutagenesis, by adopting implementation heuristics from Java’s Jumbler, should avoid that fate and offer a decent framework in PHP that performs as well as can be expected.

Once it’s released…of course . Mutagenesis is in development but should see a fresh release in a couple of weeks alongside Mockery. I’ll be looking forward to seeing how people perceive it. Mutation Testing has zero presence in PHP to date but having something to complement Code Coverage can’t do any harm!