PHP, Zend Framework and Other Crazy Stuff
Archive for February, 2009
Mutation Testing: MutateMe 0.2.0alpha Released
Feb 25th
I love git! After the initial alpha, the git repository was forked on github by Sebastian Bergmann and Benjamin Eberlei. The new release incorporates all changes pulled from their forks, and huge thanks to both for their feedback and fixes. Also, see the revised README (copied in this blog post) with much improved detailed installation and usage instructions included.
You can download the PEAR package at http://dev.phpspec.org/MutateMe-0.2.0alpha.tgz – the source code is maintained in git at http://www.github.com/padraic/mutateme
The other major changes included funtionality to ensure mutated methods retain their class visibility, i.e. whether public, protected or private. In addition, I also implemented optional static method support which is missing from the current Runkit extension CVS. The static support can be implemented in one of two ways.
The preferred option is to compile a new runkit extension using a slightly patched version of the extension from CVS, which I have forked to http://github.com/padraic/runkit. The only change from PECL CVS is to add a patch written by David Sklar a while ago to add static method support to runkit. As suggested on Twitter, I could simply request to maintain runkit but my C skills are rarely exercised so I wouldn’t be comfortable doing that. The small fork will do until a new runkit release appears on PECL.
The alternative option, currently applicable mainly to Windows users, is to disable E_STRICT error reporting when using MutateMe. I haven’t had time to compile a Windows DLL for the patched version of runkit but I’ll get around to it soon. E_STRICT kicks in, since mutated static methods only retain the public/protected/private visibility and don’t retain their static nature. Using a method statically, if not flagged as static, results in a deprecation error. This error will not impact anyone using the patched version of runkit.
The final major change was adding support for SimpleTest! It wouldn’t be much of a Mutation Testing framework if it only supported PHPUnit, now would it?
Here’s the updated README 
MutateMe is a Mutation Testing framework for PHP5, currently released as an alpha version for interested developers to assist in offering feedback. It is not yet fully functional, and stability is a relative term
PHP Mutation Testing With MutateMe
Feb 23rd
Writing a while ago about the problem of shallow testing (when tests appear to verify code is working on the surface but actually fail when you examine them more closely) dragged up some enthusiasm to revive my attempt to write a more publicly accessible library for Mutation Testing (other than the mish mash I concocted for my own use over a year ago).
Mutation Testing for PHP is currently almost non-existent though there are a few pieces of work buried in subversion repositories here and there. None of them however are capable of supporting ALL testing tools leaving many tools out in the cold. I never did like the constant parade of tightly integrated solutions in PHP, so I’ve been working on a standalone version that can operate with any testing tool once a suitable adapter is available for it.
I’ve dubbed it “MutateMe” and you can find the git repository at http://github.com/padraic/mutateme.
Update #1: The git repo now contains a preliminary SimpleTest adapter supporting both the TextReporter and HtmlReporter – simply pass “–adapter simpletest” on the command line and ensure –testfile contains the relevant group file, e.g. all_tests.php. If you want to do a manual install from git, the main thing is editing the files in /bin – they use similar conventions to PHPUnit so you can use the phpunit and phpunit.bat files for your system as rough examples of what PHP paths are needed to be edited.
Update #2: Added installation instructions for the required runkit extension. Added a Windows binary download, and some simple Linux compilation instructions. Note – do not install from PECL since the current release does not support PHP 5.2.
Update #3: I’ve discovered the runkit CVS does not yet support static methods. There is a patch by David Sklar so I may independently release a git fork and windows DLL with the necessary changes until the runkit CVS is updated in the future. In the meantime, be aware static methods will likely result in errors – oh, the fun of alpha releases . I’ll update git later with some changes contributed by Benjamin Eberlei in his MutateMe fork at http://github.com/beberlei/mutateme/tree.
Update #4: Those on Linux can now compile a slightly patched version of runkit which supports static methods using http://www.github.com/padraic/runkit. This is strictly a bridging fork until a similar patch finds its way into a new runkit release to PECL (though last release was in 2007, so…depends on Sara 
).
Update #5: Git repository now contains full static method support. I’ll roll out a new alpha release to reflect the changes during tomorrow. Thanks to everyone who gave feedback and even fixes since the initial 0.1.0 alpha!
What is Mutation Testing?
Mutation Testing is basically testing…for tests. It ensures that your tests are truly capable of detecting errors and problems with the source code. It does this by mutating the source code itself (using ext/runkit) in such a way that an error is created in the code. If your tests detect the error, all is well with the world. If your tests do not detect the error…well, you better add a new test that does .
These source code mutations are applied consistently against specific areas. For example, MutateMe can replace a plus sign with a minus sign, or substitute a variable value for the original (e.g. $state=0 becomes $state=123), or pretty much any other mutation or change you can imagine. These mutations are applied one by one, with the tests run after every change. At the end of the process you receive a report on which mutations did not cause a test to fail (in Mutation Testing, passing tests are a bad thing!) including a pretty diff of the mutation itself so you can replicate the errors that no test detected.
Obviously, this is a resource intensive operation since you run the test suite for every possible mutation. Unlike Unit Testing, Mutation Testing runs your tests for every mutation – and there can be a lot of mutations! It’s a form of testing best reserved for occasional use, and preferably on another machine so you can let it run in the background while you’re doing something else.
The other facet of Mutation Testing is that it is not definitive all the time. It may raise false positives because changing some code might not actually break anything. It may also raise very obvious errors which aren’t worth the time testing (otherwise we’d all have 100% code coverage ). Nevertheless, properly utilised it can point to areas where testing needs improvement.
Mutation Testing vs Code Coverage
In PHP, there is a persistent lack of focus on test quality. Code Coverage has become one of the primary and most celebrated measures of ensuring test quality even though it’s real purpose is to ensure tests execute all source code – i.e. it ensures a test is written which executes certain lines of code, but doesn’t inform you whether the test is poorly written and incapable of detecting the problems it’s suppossed to.
Mutation Testing is a complementary practice which does detect the existence of shallow or low quality tests since it deliberately introduces errors into the source code to see if tests will catch it. It’s a very simple notion .
Without Mutation Testing, the quality of a suite of tests is very difficult to assess without in-depth reading of the tests, a complete and godlike knowledge of the underlying source code, and the ability to see flaws in how the tests operate. In other words – you need a highly skilled human to do the task. Any human involvement is bound to be unreliable – and expensive. Just as with unit testing, automated tests are cheaper, faster and far more methodical when properly designed.
Introducing MutateMe
Rising from the ashes of my initial pass, then called PHPMutagen, MutateMe is the rewritten fully-tested version of my Mutation Testing tool for PHP. It is designed to hook into any testing framework (like PHPUnit or PHPT), mutate the underlying source code within the current PHP process (source code is mutated in memory using ext/runkit), and run the relevant tests against the mutated form to check whether they detect the deliberate error.
To install, download the current development release from http://dev.phpspec.org/MutateMe-0.1.0alpha.tgz.
You should install MutateMe using the PEAR installer, so from the location of the download run:
pear install MutateMe-0.1.0alpha.tgz
This will also download and install Text_Diff 1.1.0, and suggests you might consider installing ext/diff from PECL (optional). MutateMe does however require the ext/runkit extension which is the most problematic part – you must NOT install this from PECL! The current PECL release does not support PHP 5.2.
If you are on Windows, you can try a precompiled version that works with PHP 5.2, grab a copy from http://dev.phpspec.org/php_runkit.dll, then just copy it to your extensions directory for PHP5 and enable it in php.ini.
Under Linux, you need to compile ext/runkit (takes just a few minutes) from CVS:
1. Using your package manager, install autoconf, automake, libtool, bison, flex and re2c to compile PHP modules.
2. Log into the PHP CVS server anonymously using:
cvs -d :pserver:cvsread@cvs.php.net:/repository login
The password is “phpfi”.
3. Grab the latest runkit HEAD using:
cvs -d:pserver:cvsread@cvs.php.net:/repository co pecl/runkit
4. Compile the extension and copy it into your default PHP extensions directory using (assuming you are not already in the runkit source directory):
cd runkit phpize ./configure make make install
5. Edit whichever php.ini config file that governs the PHP CLI to include an enabling line like:
extension=runkit.so
6. Check the output of “php -i” or phpinfo() to ensure the extension is loaded.
MutateMe is now ready for action – well, the alpha version with it’s initial payload of 5 possible mutations and a dodgy PHPUnit adapter is . Unfortunately you’ll have to wait a bit longer for the SimpleTest and PHPT adapters.
The command line tool is accessed by calling “mutateme”. It accepts a few command line options:
–basedir: The base directory containing the source code and tests (see conventions later)
–srcdir: The directory where the source code to mutate is located
–testdir: The directory where unit tests are maintained
PHPUnit options (subject to improvement):
–testfile: The name of the test file to run (e.g. AllTests.php)
–test: The test class to run contained in the test file (e.g. My_AllTests – the actual class name in that file can differ from the file name)
–adapter: Select a specific adapter (currently phpunit or simpletest)
If you want to minimise options, MutateMe assumes the current directory is the base directory unless otherwise specified. On that assumption, it will attempt (unless options are passed) that source code is held in one of /src, /lib or /library subdirectories of the base directory. It also assumes tests are stored in either the /tests or /specs subdirectory of the base directory, where there exists an AllTests.php test file containing an AllTests class (assuming you’re using the PHPUnit adapter which is the default).
Unless you meet these conventions exactly – use the specific options! For example:
cd ~/projects/mathlib mutateme --srcdir ./libs --testdir ./tests --testfile AllTests.php --test Mathlib_AllTests
You should ensure, that before using MutateMe, your test suite is recording no Errors or Failures (Skips and Incompletes are fine though), since this a precondition before Mutation Testing can be performed. You’ll be told by MutateMe if it doesn’t anyway.
You can then look forward to the results…
MutateMe Alpha: Mutation Testing for PHP All initial checks successful! The mutagenic slime has been activated. PHPUnit 3.3.14 by Sebastian Bergmann. . Time: 0 seconds OK (1 test, 1 assertion) . 1 Mutant born out of the mutagenic slime! 1 Mutant exterminated! No Mutants survived! Muahahahaha!
If you see anything different, it’s time to see if the issue warrants an additional test to catch that defect if it appears in the future:
MutateMe Alpha: Mutation Testing for PHP All initial checks successful! The mutagenic slime has been activated. PHPUnit 3.3.14 by Sebastian Bergmann. . Time: 0 seconds OK (1 test, 1 assertion) . 1 Mutant born out of the mutagenic slime! 0 Mutants exterminated! 1 Mutant escaped; the integrity of your suite may be compromised by the following Mutants: 1) Index: ./Math.php =================================================================== @@ -1 +1 @@ -return$op1+$op2; +return$op1-$op2; PHPUnit 3.3.14 by Sebastian Bergmann. . Time: 0 seconds OK (1 test, 1 assertion) Happy Hunting! Remember that some Mutants may just be Ghosts (or if you want to be boring, false positives).
The mutations enabled are very limited for now. There are four primary mutations:
1. Replacing “+” with “-”
2. Replacing “-” with “+”
3. Replacing TRUE with FALSE
4. Replacing FALSE with TRUE
Before an actual beta release, I’ll be adding adapters for SimpleTest and PHPT, and obviously adding a lot more mutations to the mix. The current release is mainly there for those interested to take a look at.
Unit Testing: One Test, One Assertion – Why It Works
Feb 16th
In my last post I touched on the topic of multiple assertions in a unit test and linked it somewhat harshly to being a factor in “lazy” or shallow testing. I actually didn’t intend on linking them at all, but the truth is that unit testing isn’t a series of mutually exclusive practices – they are all linked to varying degrees. I’m sure I have at least a degree of laziness myself in testing .
The Basic Idea
As a recap, the idea is that every unit test should only have one single assertion. It’s a fairly well known method to combat a range of problems evident in assertion loaded tests where the numerous assertions obscure the meaning of the test, and where a failed test does not tell you the specific assertions which failed (since tests traditionally fail after any assertion failure, the remaining assertions are never executed thus leaving you blind to whether they would have passed or failed). There are other problems with the approach as well – which is why I do not like seeing multiple assertions in a single test. I’m not the only one – I think. And just as there are problems, there are also exceptions when multiple assertions might prove necessary – personally I think this scenario is pretty rare however. After my years of unit testing, I certainly have not encountered all that many.
These effects by themselves make me fairly comfortable in enforcing Single Assertions in tests as a rule, not a guideline. Guidelines are broken easily, but with rules you can at least strike back and make certain you don’t run aground on someone else’s inexperience or ill fated experiments in making testing “easier”. It’s also extremely easy to detect . PHPUnit, for example, reports the number of both tests and assertions executed – so you can calculate a quick ratio of assertions to tests in mere seconds. I’m going to start calling it the Obscurity Ratio! Obviously, a ratio of 1:1 would be perfect – but I’m sure we can accept some multiple assertions within a tolerance level. A ratio of more than 1.1:1 would indicate creeping obscurity (a signal it’s time for me to intervene, refactor, and do some mutation testing for extra assurance). A ratio of 2:1 indicates a lost cause. Anything higher and God help you all.
This Sounds Like A Bad Idea!
In watching the reactions to my original post (and not just the comments here), I’ve noticed that the excuses are generally similar. People have leaned towards justifying using multiple assertions for complex return values, or pointed out that my example was too simple (since that really really shows its a Bad Idea™ ), or that all the mini-tests would take too long to write, and even longer to execute, or that the world will end since Peter Petrelli thinks this is all a Bad Idea™ and everyone knows Sylar invented it anyway.
Not to completely oppose all the opinions (it really was a simple example), but people are spending too much time looking at the end result, and not enough at the process behind it all. And it’s by missing the process that you are missing the point.
Now let me disclaim a little – I’m really really sure there are times multiple assertions are needed. So my main point is not that this is an absolute rule with no possible exceptions so help me God, rather that it should be a rule unless you are literally forced to tackle an exception to it.
Why It’s A Good Idea…
Unit Testing has various interpretations. It’s original genesis promoted the verification of source code, a means of making sure code worked. It was not long before the obvious flaws in that thinking emerged – how do you verify the verified? You already know the code works before you write the test, so what value does it really bring to the table? Does it really promote code quality and rapid development – or just create a horrible task some low ranking developer will be landed with?
This sparked a revolution – Test Driven Design (TDD). In TDD, tests stepped back from verifying, and entered a role as specifications. Before you wrote the source code, you first wrote a test which specified the behaviour you intended on adding. Once you had the behaviour specified, you then wrote just enough source code to make that test pass. The result was a process of simple steps – and here’s where I diverge from others. A simple example works – because all half decent source code is nothing more than an amalgamation of simple examples – in fact, I dare anyone to prove otherwise. Then I can show you the PHP manual and you can explain how the hell you made PHP that complex .
In TDD we back away from testing complex highly involved code after the fact (too late then!), to preemptively writing tests to produce simpler straightforward code produced in steps. Simpler is better – always. The differences can be startling. Try writing two identical libraries both ways and you’ll see how.
It’s this notion of complexity that is often used to justify multiple assertions – if you break it down into the simple components that that complexity was built out of, multiple assertions don’t have a leg to stand on (just the rare instances out of your control because they absolutely must be done). The most common (just to preempt you all) scenarios for complexity in testing assertions boil down to BIG stuff – deeply nested stuff. Arrays and XML are the most common ones. Mathematical equations for relativity are simpler than those!
But let’s pass on through to the other benefits instead of obsessing over just one.
If your tests double as your specification to describe how a class should behave (and yes, I’m robbing the definition list in Behaviour-Driven Development, or BDD, blind) or documentation – you’re doing nothing wrong. Tests should document behaviour. This is why test method names are important – they summarise the behaviour the test is verifying.
Now add another little rule: every test only verifies one single behaviour. When we make that connection we realise something even more startling…
1 Behaviour == 1 Assertion
Now we’re into the meat of my madness, the gritty reality of the one assertion per test rule. Any behaviour is, by its very nature, both specific and unique. It does not require multiple assertions. Either it does one single thing, or it doesn’t do it at all. It’s black and white. The only danger is that you get so smart, you think in terms of big behaviours which are really just lots of little behaviours bundled together. Enjoy applying TDD in that case…it will sink you faster than you can say “Peter Petrelli Is Depressing” and you’ll end up back in the land of writing tests AFTER the code. Anyone going to admit to doing that? We all know who we are (yes, I do it too now and again).
That’s why Test-Driven Design is so effective. It enforces a habit (once you actually understand the damn practice which is almost designed to hide the behaviour facet!) of specifying class behaviour by behaviour, assertion by assertion. It breaks down the complex overall purpose of any class into discrete simple steps, a series of tiny goals you can easily achieve.
Simple, tiny, discrete – like individual assertions. It’s THAT simple.
That’s why the one assertion per test is a good idea – because it’s obvious in TDD. It’s what works, and it ensures your tests do exactly what they’re suppossed to be doing – verifying discrete behaviour, and documenting that behaviour to make other developer’s lives a bit easier (and saner).
The Spontaneous Test Population Explosion Myth
This is the one myth everyone knows targets the one assertion per test rule. The question is whether it’s a myth, or whether something else is. The idea is that by requiring one assertion per test, you end up with more tests, more code, higher execution times, etc. Which is to say…more behaviours. Which is to say – where the hell did the new behaviours come from?
Unwittingly, the excuse reveals itself to be the myth. If you have tests, and they can survive as multiple tests – you pretty much admitted your original tests were collections of behaviours. Welcome to the land of Obscure Tests – they test many things, and nobody can figure out just what.
Tests are documentation. Like any form of writing, you can block together ideas into a monologue or use properly formatted paragraphs and bullet points to break things down into a more digestable form for reading. It comes with a cost, but discrete tests are more helpful as documentation.
As for execution times, I really don’t understand why people find this a problem. PHPUnit let’s you run any test class in isolation, and you can group test suites in any shape or form. If your tests are running slow, treat them like any other code – use XDebug to locate the bottlenecks and slow tests and do something about them. Use more efficient code, refactor them to hell and back, isolate the slow tests – we can all do performance optimisation.
The other alternative is to let tests run in the background and use a notification system to report failures while you go away and write another test or two. I do this myself – see what notification apps you can hook into from PHP or some other test enviroment tool. I had working Snarl code for a PHP extension working under Windows somewhere if you can use a compiler and really need it.
Attitude Counts
Another compelling reason to adopt one assertion per test is your ego. Look at me – I have an ego, I write stuff because I enjoy doing it and am motivated by other people reading it and thinking it’s the best thing ever written . We all have egos. My ego has me writing free books (though the prospect of a new Macbook Pro is another motive…hehehe).
Unfortunately our ego is also a major enemy in unit testing. If you find yourself saying you prefer multiple assertions because it works for you, and you can understand them, and they make perfect sense to you, and hey, your code works with 200% code coverage – then count how many “you” words you just used (or “I” if writing first person!).
It’s not about you!
Other developers have to read your tests eventually, unless it’s a top secret project only you will ever work on. Don’t saddle the rest of the world with the product of your ego. Everyone finds simpler tests easier to work with. The general rule of thumb is pretty simple – if you give someone a copy of your tests, and nothing else, could they ever write the code to fit those tests in a reasonable time from scratch without begging you for assistance?
Conclusion
More food for thought . No pretty code to look at for this one so if you haven’t read the last post yet, go read it now.
Until next time…when I find something else to moan about.
Unit Testing: Multiple Assertions And Lazy/Shallow Testing Are Evil
Feb 12th
Unit Testing as a practice is like any other – there are good practices, and bad practices. Two of the worst practices are overloading tests with assertions, and writing lazy or shallow tests.
Before we recount the dire consequences of these practices, it’s worth knowing why they are so attractive and not immediately perceived as being bad. In short, every test you write requires that you setup the test environment, create a scenario for possible failure, add an assertion, and then ensure the source code makes that assertion pass. This requires code – sometimes a lot of code. So adding multiple assertions to each test minimises the work needed to write tests, since using multiple assertions takes advantage of existing code to avoid writing new stuff to clutter your test classes. It can also help to tackle multiple but related results in the same test.
So long as you know the assertions will pass – this makes writing unit tests quite a bit faster at times. Unfortunately, a preoccupation with minimising test code also encourages developers to keep tests overly simple to the point that they do not dig deep enough into whether a test actually accomplishes its objective – often because that objective has never previously been documented.
These considerations lead to tests which may be similar to (using PHPUnit):
[geshi lang=php]class GameTest extends PHPUnit_Framework_TestCase
{
public function testScoreIsZeroWithNoScoring()
{
$game = new Game;
$this->assertEquals(0, $game->score);
$this->assertEquals(0, $game->scoreTotal);
$game->score(1);
$this->assertEquals(1, $game->score);
$this->assertEquals(1, $game->scoreTotal);
}
}[/geshi]
Here we have a simple test with four assertions. All four test $score and $scoreTotal to make certain they remain at zero after the object is initialised and no score (or zero score) is assigned. They then revisit the situation after a score has occured. To the naked eye, the test is easy to understand. Here’s a class which will pass the above test (if you spot the problem after seeing the class, you should give yourself a treat ).
[geshi lang=php]class Game
{
public $score = 0;
public $scoreTotal = 0;
public function score($score)
{
$this->score = $score;
$this->scoreTotal += $score;
}
}[/geshi]
Back to those dire consequences. Consider the output of test results showing a failure I will now introduce by initially setting $score to 1 in the class.
PHPUnit 3.3.14 by Sebastian Bergmann. F Time: 0 seconds There was 1 failure: 1) testScoreIsZeroWithNoScoring(GameTest) Failed asserting that <integer:1> matches expected value <integer:0>. D:\projects\tinker\GameTest.php:11 FAILURES! Tests: 1, Assertions: 1, Failures: 1.
Multiple assertions do unfortunately have side effects. The most obvious one is that it only takes one assertion to fail, and the entire test will fail with it. It ignores whether other assertions in the same test would have actually passed (hence we see that the last line of the results show PHPUnit only executed the first assertion, and ignored all others), which leaves you blind as to whether the error is impacting other assertions. This creates a maintainance nightmare – for every test that fails, you’re never certain what should have failed! You only get one part of the puzzle to work with.
Unit Tests should be specific. In fact, as a general rule, there should only be one assertion per test method. If a failed test doesn’t immediately tell you where the problem is, and what assertions will fail, and offer at least some minimal description of the failed behaviour (typically the test title should be sufficiently descriptive) then its utility is severely reduced. You end up doing the same detective work needed in the absence of unit tests – which makes those unit tests less beneficial since you rob the maintainer of instantansous specific feedback and force them to edit tests, often rewriting them to be more specific, and/or employ typical debugging approaches to locate the problem in the source code.
Another impact, is that multiple assertions are often a sign that the tests were written post development, or without attention to the behaviour of the class. This increases the risk that the tests are not only confusing when they fail, but that the tests are not even complete. Truly paying attention to the role of behaviour discourages multiple assertions and promotes specificity.
There’s a side story here about the foolishness of believing that code coverage is an absolute measure of the effectiveness of a unit testing suite. It’s not – it’s only one metric to assist in that measurement, and not a very reliable one at that. It’s entirely possible to gain 90% or even 100% code coverage without writing tests that cover even a quarter of the expected behaviour of the class. Code coverage measures how much of the source code lines are actually executed – it doesn’t tell you if they were executed enough times, in the right order, or if the tests were even appropriate to start with.
This is the problem some people will have noted from before (give yourself a treat!). The class obviously has more behaviour than the original passing test seemed aware of. Despite this, guess what the original test had as a code coverage metric? 100%
Here’s how the test should have been written, code coverage and multiple assertions be damned. Your tests are only complete when you are absolutely certain they cover off on all class behaviour – and not a second sooner.
[geshi lang=php]class GameTest extends PHPUnit_Framework_TestCase
{
public function testStartingLastScoreIsZero()
{
$game = new Game;
$this->assertEquals(0, $game->score);
}
public function testStartingTotalScoreIsZero()
{
$game = new Game;
$this->assertEquals(0, $game->scoreTotal);
}
public function testLastScoreOnlyStoresLastScoreRewarded()
{
$game = new Game;
$game->score(2);
$game->score(5);
$this->assertEquals(5, $game->score);
}
public function testTotalScoreAccumulatesRewardedScores()
{
$game = new Game;
$game->score(1);
$game->score(2);
$this->assertEquals(3, $game->scoreTotal);
}
}[/geshi]
The first test class was very obviously smaller and simpler. Not only is it hampered by multiple confusing assertions, but its simplicity also indicates a lack of good test design – by reusing and ignoring specific test scenario setups (i.e. seeking a fail, before editing code to pass) it’s a test suite that passes, but doesn’t quite verify everything. Unfortunately, the two go hand in hand in my experience. To make the first test pass, and the second more specific tests fail, use the following version of the Game class.
[geshi lang=php]class Game
{
public $score = 0;
public $scoreTotal = 0;
public function score($score)
{
$this->score += $score;
$this->scoreTotal = $score;
}
}[/geshi]
Here we’ve simply assumed someone got confused, and mixed up the purpose of the two properties in the score() method, so the += sign has moved in error. Now how many times has that happened to you? If you run the original simpler and multiple assertion stuffed test – it will show everything is working as intended (as an aside, this is one example of where Mutation Testing could have picked up a problem which the original test couldn’t detect).
PHPUnit 3.3.14 by Sebastian Bergmann. . Time: 0 seconds OK (1 test, 4 assertions)
The later more detailed, specific tests show a different story:
PHPUnit 3.3.14 by Sebastian Bergmann. ..FF Time: 0 seconds There were 2 failures: 1) testScoreOnlyStoresLastScoreRewarded(GameTest) Failed asserting that <integer:7> matches expected value <integer:5>. D:\projects\tinker\GameTest.php:38 2) testTotalScoreAccumulatesRewardedScores(GameTest) Failed asserting that <integer:2> matches expected value <integer:3>. D:\projects\tinker\GameTest.php:46 FAILURES! Tests: 4, Assertions: 4, Failures: 2.
Now imagine someone has written hundreds of tests in the manner of my first example. Can you imagine the world of hurt anyone attempting to refactor and maintain the underlying source code is facing? The countless tests they’ll need to debug, rewrite, and expand? The tears of frustration? The hair pulling? The talking to your reflection because of a psychotic break?
Don’t do that the next time you write unit tests . Remember – be as specific as possible about what the class should do, and you will quickly realise that you only need one assertion per test. Sure, it means writing additional test code – but at least you’re now writing tests that truly work!
Zend Framework: Survive The Deep End – Still Kicking!
Feb 11th
Edit: Serendipity decided to password this entry originally. No idea why! 
It’s been a few weeks since the last chapter was released and no doubt the masses (all few hundred of you per day) are wondering when the next one is due. I’ve been busy in getting myself back on the job market hence the delay, but I expect to get to another two chapters during the next week onto Surviving The Deep End.
In the meantime I really need to fix the remaining few bugs in the application hosting the book! As you may have noticed, everyone likes commenting on 1 January. No idea when these individuals invented Time Travel . I also intend adding a few informative bits of information so readers know whether a chapter is draft/final or something in between (most are in between takes).
In experimenting with making money (so I can buy that Macbook Pro) I’ll be adding some changes to how donations work. One main change is that while the current PDF file for the existing chapters will remain available (including future updates), PDF files covering the entire book to date will only be available to those who donate. It wasn’t the easiest decision to make since I like free things (who doesn’t!), but the online HTML book will always remain free, will always be updated, and will never go offline. That’s one thing that will never change. Those who have previously donated will of course be included.
The second facet I’ve been considering is boosting the appeal and effectiveness of the book as a whole by entering the screencasting market. I figure that with my Irish accent, my pretty Ubuntu desktop, and a cheap mic (to replace the ancient OEM cheaper than cheap one I have at the moment) I can make the whole deal even more appealing to those who prefer visual/audio instruction (and get my Macbook a little faster…like in 2010 instead of 2011). Of course, if I end up creating something truly bad (I’m a writer, not a presenter) I may let that fade away silently with some embarrasement never ever to be mentioned again . Still, would be fantastic to give it a shot – even if its a total disaster waiting to happen.
In the meantime I have my pimped out gedit open on a chapter detailing a “Hello World” example. I keep thinking that if we shot Ralph Schindler up on speed, and repeated those words constantly he might use them in the default index.phtml for Zend_Tool when it reaches the standard library. Would be a pretty short Chapter then .
In any case, if readers are aware of any other niggling problems with the book site give me a shout in the comments to this post (we’ll do our best to ignore my lack of design skills – they are a feature, not a bug!).
