I love git! After the initial alpha, the git repository was forked on github by Sebastian Bergmann and Benjamin Eberlei. The new release incorporates all changes pulled from their forks, and huge thanks to both for their feedback and fixes. Also, see the revised README (copied in this blog post) with much improved detailed installation and usage instructions included.

You can download the PEAR package at http://dev.phpspec.org/MutateMe-0.2.0alpha.tgz – the source code is maintained in git at http://www.github.com/padraic/mutateme

The other major changes included funtionality to ensure mutated methods retain their class visibility, i.e. whether public, protected or private. In addition, I also implemented optional static method support which is missing from the current Runkit extension CVS. The static support can be implemented in one of two ways.

The preferred option is to compile a new runkit extension using a slightly patched version of the extension from CVS, which I have forked to http://github.com/padraic/runkit. The only change from PECL CVS is to add a patch written by David Sklar a while ago to add static method support to runkit. As suggested on Twitter, I could simply request to maintain runkit but my C skills are rarely exercised so I wouldn’t be comfortable doing that. The small fork will do until a new runkit release appears on PECL.

The alternative option, currently applicable mainly to Windows users, is to disable E_STRICT error reporting when using MutateMe. I haven’t had time to compile a Windows DLL for the patched version of runkit but I’ll get around to it soon. E_STRICT kicks in, since mutated static methods only retain the public/protected/private visibility and don’t retain their static nature. Using a method statically, if not flagged as static, results in a deprecation error. This error will not impact anyone using the patched version of runkit.

The final major change was adding support for SimpleTest! It wouldn’t be much of a Mutation Testing framework if it only supported PHPUnit, now would it?

Here’s the updated README :)

MutateMe is a Mutation Testing framework for PHP5, currently released as an
alpha version for interested developers to assist in offering feedback. It is
not yet fully functional, and stability is a relative term :) .

Requirements:

PHP: 5.2.4, probably less but I haven't checked
PEAR: Text_Diff 1.1.0
Extensions: ext/runkit, and optionally ext/xdiff

Note: Do not install runkit from PECL since the current release does not support
PHP 5.2. Follow the installation instructions below to grab a patched version of
runkit which supports both PHP 5.2 and static methods. Windows users should download
the DLL from the given link - a patched version of the DLL will be made available
soon. If you have installed from PECL delete the module and follow the instructions
below to replace it with a slightly improved patched version.

Installation (with PEAR):

1. Download the latest PEAR packaged release and run the following command:

   pear install MutateMe-0.2.0alpha.tgz

2. Follow the manual instructions below from Step 3 to install the PHP
    runkit extension.

Installation (Manual Installation without PEAR):

1. Copy the contents of /library to a location on the PHP include_path, e.g.
   /usr/share/php.

2. Copy /bin/mutateme and /bin/mutateme.bat to a location on PATH so they are
   accessible from the command line. Edit the contents of each to replace @php_bin@,
   @bin_dir@ with the path to the php binary (e.g. /usr/bin/php) and the path to the
   directory you're putting these scripts respectively.

3. Install the runkit extension but NOT from PECL. Windows users can grab the DLL
   from http://dev.phpspec.org/php_runkit.dll but should disable E_STRICT error
   reporting (I will compile a patched DLL for use with MutateMe soon). Linux users
   should compile runkit from source. I've put up a patched version supporting
   static methods (via a David Sklar patch) and PHP 5.2 (from CVS HEAD in PECL)
   at http://github.com/padraic/runkit

   Use the following commands to compile from source (install git first!):

   git clone git://github.com/padraic/runkit.git runkit
   cd runkit
   phpize
   ./configure
   make
   make install

   Finally, enable the extension in the php.ini file which is loaded for the cli
   environment using:

   extension=runkit.so

   Check the output of "php -i" or phpinfo() to ensure the extension is loaded.

Usage:

MutateMe has just a few necessary command line options:

   --adapter    Adapter to use: phpunit or simpletest. PHPUnit is the default.
   --basedir    Base directory containing source code and tests
   --srcdir     Directory containing source code to mutate
   --testdir    Directory containing the tests to run
   --testfile   Name of the test file to execute, e.g. AllTests.php or all_tests.php
   --test       (PHPUnit only) Name of the test class to run, e.g. MyLibrary_AllTests

If options are omitted, MutateMe will attempt to autodiscover the correct values
using the following conventions:

   1. The base directory is assumed to be the current working directory
   2. The source directory is assumed to be a subdirectory of the base directory
      called either "src", "lib" or "library"
   3. The test directory is assumed to be a subdirectory of the base directory
      called either "tests" or "specs"
   4. The test file is assumed to be either AllTests.php or all_tests.php depending
      on the adapter used
   5. (PHPUnit only) The test class is assumed to be AllTests

If you cannot use these conventions completely, please use the command line options appropriately.

Example:

$ mutateme --adapter phpunit --basedir ./ --srcdir ./src --testdir ./tests \
         --testfile AllTests.php --test MyLib_AllTests

In the example above, we could have omitted the --basedir, --srcdir, --testdir and
--testfile options since they meet the requirements of the autodetectable convention as
described above.

Output:

A successful Mutation Test would output something like:

MutateMe Alpha: Mutation Testing for PHP

All initial checks successful! The mutagenic slime has been activated.

PHPUnit 3.3.14 by Sebastian Bergmann.

.

Time: 0 seconds

OK (1 test, 1 assertion)

.

1 Mutant born out of the mutagenic slime!

1 Mutant exterminated!

No Mutants survived! Muahahahaha!

A failed Mutation Test would indicate a test failed to detect an introduced error in the
source code. This might be a completely spurious failed mutation, since not all
introduced errors actually cause problems, but often it will highlight a new condition
that you should a new test for so a future real error will be detected and not
completely missed by the test suite:

All initial checks successful! The mutagenic slime has been activated.

PHPUnit 3.3.14 by Sebastian Bergmann.

.

Time: 0 seconds

OK (1 test, 1 assertion)

.

1 Mutant born out of the mutagenic slime!

0 Mutants exterminated!

1 Mutant escaped; the integrity of your suite may be compromised by the
following Mutants:

1)
Index: ./Math.php
===================================================================
@@ -1 +1 @@
-return$op1+$op2;
+return$op1-$op2;

Happy Hunting! Remember that some Mutants may just be Ghosts (or if you want to
be boring, false positives).