Maugrim The Reaper's Blog - PHP Security

A Domain Specific Language for Behaviour Driven Development (BDD) in PHP

nospam@example.com (Pádraic Brady) — Fri, 18 Jul 2008 17:27:21 +0000

If you were expecting Part 19347 of the Zend Framework Blog Application Tutorial the next part is in writing . For now though, to another topic.

Back at the start of the year I started making early releases of PHPSpec a Behaviour-Driven Development (BDD) library for PHP. Regular readers are aware I pretty much left the scene for a few months, so it's about time I got back to it!

PHPSpec implements a domain specific language (DSL) in PHP for specifying the behaviour of functional units such as methods and objects. The purpose of a DSL was to move away from the xUnit style declaration of tests towards a specification language centred on describing behaviour. I've written about BDD previously, so I won't cover the same ground here in this haphazard entry. The current PHPSpec is sufficiently usable that I've covered most of the original goals. For example, a simple spec in PHPSpec could be:

require_once 'Bowling.php'; // contains Bowling class

class DescribeNewBowlingGame extends PHPSpec_Context
{

private $_bowling = null;

public function before() {
$this->_bowling = new Bowling;
}

public function itShouldScore0ForGutterGame() {
for ($i=1; $i<=20; $i++) {
$this->_bowling->hit(0);
}
$this->spec($this->_bowling->score)->should->equal(0);
}

}

I hate to be the one to admit it, but this sucks in many ways. The primary culprit is PHP itself. In BDD a specification of this order should be human readable, to the point, free of ambiguity and not require any programming experience if possible to understand. PHP syntax scores badly on all four. Now don't get up in arms - I love PHP to bits, warts and all . But once the warts demand compromises in what I want from a domain specific language I have to consider looking elsewhere.

Unless PHP miraculously adopts the syntax of Ruby, I'm therefore willing to concede defeat at some level. Ruby DSL's are sexy, PHP DSL's are verbose and clumsy.

My proposal therefore is to re-implement the current programming language DSL as a specification language - i.e. a new (extremely limited and narrow!) language capable of being parsed by PHPSpec into its PHP equivelant. This is certainly something of a drastic step but I'm close to being convinced it's a worthwhile endeavor that would still leave people free to fall back on the PHP DSL if they really dislike it .

To be clear on what I mean by a specification language - it's a language with it's own independently defined syntax which is designed to solve a specific problem as opposed to the one size fits all approach of general programming languages. Implementing such a limited language in PHP is easy - you just need to keep it tight, focused, and limited in scope. We're not out to implement Python in PHP are we?

Bearing in mind the original spec in PHP, here's one possible specification DSL example along the same lines:

require_once Bowling.php

 

describe "new bowling game" {

 

    before {

        bowling = new Bowling

    }

 

    it "should score 0 for gutter game" {

        bowling->hit(0) times(20)

        bowling->score should == 0

    }

 

}

The differences are obvious but subtle. The similarity to closures is deliberate - it's a simple construct to copy from (and has nice braces). PHP syntax elements that I consider unnecessary are removed. Some are retained rather than replaced - let's stick with PHP's conventions insofar as they make parsing, and learning, the DSL simpler. The individual space delimited terms on each line would each be individually parsed, and translated to a specific effect. times(20) replaces the for loop, == replaces an equal() call, and whatever follows an == is an obvious parameter. The flow of terms marks a new grammer which is deliberately ordered to resemble natural English.

I'm being very brief here but hopefully you're still following my train of thought.

There are of course the usual concerns. Is it performant? Won't IDEs be incapable of code completion? Obviously IDEs won't recognise this without help (surely there's enough TextMate/PDT/Other users to create solutions or suggestions though). Performance obviously suffers unless some caching is introduced or the parsing rules are really tight, and there is a final translated PHP form being executed afterall.

Aside from the obvious improvement to readability there are other benefits. In BDD behaviours can be thought of as being discrete sharable units. Four methods from three classes could each share the same behaviour in theory. Rather than duplicate the spec across three class descriptions - why not specify the shared behaviour in one spec, and tell other descriptions to integrate it as being shared. If you think that through (hint: composition) have a go at considering how to implement it in PHP. Then consider how a possible implementation would work within PHP. It gets complicated depending on how you figure it.

Using a specification DSL, we can simply ignore the existence of classes in PHP (well, the pretense is nice ). Instead the DSL would incorporate a syntax for denoting shared behaviours - and leave the parser free to implement nested behaviours behind the scenes in whatever manner best fits PHP. Another potentially confusing task a user can safely not worry about so long as they write the DSL.

I should stress I'm not aiming to completely replace the PHP DSL - it will still exist, but the specification language option would offer more features and benefits with far less overhead in terms of PHP code to write (and staheholders to possibly try reading!).

I'm throwing this all out for comment. I haven't seen much connection between DSLs and PHP so anyone's experience here would be really great to hear about too.

Google roll out OAuth Authorisation to all Google Data APIs

nospam@example.com (Pádraic Brady) — Mon, 30 Jun 2008 21:24:43 +0000

Last Thursday I saw this turn up on the OAuth mailing list, so I've spent a few hours over the weekend adding the final features to the OAuth For PHP library (proposed both to the Zend Framework and PEAR) to ensure it works. Thanks go out to David Koblas for his unprompted assistance to go test the library and uncover the remaining issues!

The actual announcement went out on one of the Google Blogs: http://googledataapis.blogspot.com/2008/06/oauth-for-google-data-apis.html

I'm currently rolling these changes into a new OAuth 0.0.3 release for the PEAR proposal (an arduous task which involves opening the command line and running "phing convert-to-pear" on a subversion export of the ZF base code ). Once that hits the proposal I'll switch into voting mode and see what comments the always insightful PEAR developers drop on top of me. Of course this means the changes are already in the matching Zend Framework Proposal respository at http://svn.astrumfutura.org/zendframework/trunk along with examples of its usage with the Ma.gnolia API and the Google Data APIs waiting for its future Zend review.

The OAuth Consumer is now hitting the point where I'd rate it as beta. Most of the absolutely neccessary features are finished, and could likely only do with a touch of the refactoring brush. The last major feature is implementing a backend storage solution for OAuth tokens.

The Google Data API support comes with a few gotchas. Its documentation show a clear preference for using GET instead of POST, though the Google OAuth server does appear to natively support POST requests for everything with the exception that it has a small bug which interprets an empty POST request body as a sort of phantom empty parameter which messes up the validation of a Client's RSA-SHA1 signature. Other then that it's ready to go. Might be time to add comments to my code.... The bug has been reported but it's still preferable to stick with GET requests unless the Google OAuth documentation ever specifies it as an option.

If all goes well, the dual proposal process across PEAR and the Zend Framework should make for a another good OAuth library in PHP. Next up in the near future when an OAuth Server is also implemented will be adding support for some common OAuth Extensions - talk of these is already accelerating on the mailing lists as discussions turn to the next iteration of the OAuth Core Specification which has taken on the name OAuth Core 2008.1 (a sort of dual date + revision number system).

For those watching the PEAR proposal, I'll roll out OAuth 0.0.3 to the proposal during tomorrow. The last two have been unstable at best while on the drawing board so this near-stable one should be a stark contrast since it works without any hitches.

OAuth For PHP Status Update...

nospam@example.com (Pádraic Brady) — Wed, 25 Jun 2008 00:21:00 +0000

Edit: Firefox 3 appeared to have auto filled in a password option on this entry previously - my apologies and here's the full entry!

Earlier today I finished up work on the initial OAuth library. It has a few rough edges and missing features, but the bulk of the work for an OAuth Consumer is there. The rest is refactoring, feature iteration, and testing.

The new library is specifically targeted as a dual-proposal. It has been proposed to both PEAR and the Zend Framework. Rather than maintaining two distinct versions, or one distinct version with a heavy dose of abstraction, both are derived from the Zend Framework version. In essence, I develop primarily on the Zend Framework version, iterate patches onto a simple Phing task-chain, and out comes the PEAR version. This extends itself into the test suite too. This allows me to use the same base code, with changes appropriate to the underlying system (e.g. using PEAR HTTP_Request instead of ZF Zend_Http_Request).

The Zend Framework Proposal
The PEAR Proposal

Comments to both are more than welcome. For anyone wishing to ask questions outside of these proposal processes, I have also established a mailing list on Google Groups for the "OAuth For PHP" project: http://groups.google.com/group/oauth-for-php.

Over the weekend I posted a quick example of using the Zend_Oauth component to retrieve data from Ma.gnolia's API, a similar example using the PEAR code is available at PEAR OAuth Ma.gnolia Example. If nothing else it highlights what remains to be done. Don't get me wrong - you could almost use the current code, but there is a lot of support that can be added to make use easier.

So what is remaining?

The initial library remained tightly focused on implementing OAuth across HTTP POST. Obviously this means GET is left languishing by the wayside for now. One of the reasons was simply that a narrow focus also has the benefit of simplifying testing. And what works with POST URL query strings, should apply similarly to GET requests.

Secondly, there is absolutely no backend storage medium available as yet. In the examples, I temporarily stored all tokens to the Session. Ideally I will implement both a File and DB based storage backend this week.

Thirdly, the workflow of an actual Web Service API interaction is externalised. There is no way to get the OAuth library to handle requests to web services internally, using a specialised OAuth client, automatically making authorised POST/GET requests with the relevant data. Rather you can extract the OAuth authorisation parameters as a Header or Query String value for use in any HTTP client you wish (perhaps not perfectly ideal but the flexibility won't tie you to any particular client implementation either ).

Fourthly, support for RSA signing is omitted. Technically there is a problem supporting PHP's openssl extension on two fronts. The first is that PEAR's Crypt_RSA is a slower native implementation of RSA which doesn't optionally offload work on ext/openssl. The second is that ext/openssl itself is a tricky extension - it's API is badly documented (one of the worst), and it tends to have BC issues across different PHP versions.

Lastly, there is no OAuth Server just yet! A Server is next on my agenda though, so expect it to commence development almost immediately.

The end goal however, is substantially closer than it was last week! We have code, tests and a working example. The community feedback channel is now open.

Another OAuth Library Is Born

nospam@example.com (Pádraic Brady) — Sat, 21 Jun 2008 20:40:56 +0000

I've spent some time over the last week, and I have completed an initial pass at writing an OAuth Consumer in PHP5. I'm biased, but it's a nice chunk of code capable of fairly routine POST based OAuth requests using either an Authorized header, or a raw url encoded POST request body. My main remaining task is final cleanup, included refactoring, rolling my final set of acceptance tests into PHPUnit from SimpleTest, and adding support for HTTP GET, RSA, and a storage API so tokens can be saved in the background rather than outside the API.

I wrote up a quick example script using the current source code - bear in mind the final API will be tweaked but this is a close match for what I'd expect to become final (I'll await community feedback before finalising anything!).

You can grab the code, which is geared up for integration into the Zend Framework as a Zend_Oauth component, from my proposals repository at http://svn.astrumfutura.org/zendframework/trunk/ (proposed components are in /trunk/library/Proposed). I fully expect to refactor a similar core library specifically for use with PEAR in the very near future.

Onwards with the example! I'm using the Ma.gnolia bookmarking service's API (version 2) here. If you intend actually running the example, you will need to create a Ma.gnolia account and visit http://ma.gnolia.com/applications/new to register an application to get hold of an OAuth Consumer Key. Registering an application is a bit confusing - but basically create an imaginary one (e.g. "Super Duper OAuth Test Application". Since you'll be the only user for now, you can use it across any OAuth testing regardless of application name. In the code, replace CONSUMER_KEY and CONSUMER_KEY_SECRET strings with the real thing.

The example isn't explained here - if you're unfamiliar with OAuth I'll explain it better in the future when this is all closer to completion. For now refer to the OAuth Core 1.0 Specification (the starting sections are pretty good at explaining things). The amount of code is indicative of features being wrapped around the existing API as they are not yet integrated directly (e.g. using Sessions to store temporary tokens instead of an internally referenced database). Also note that the example creates a new Access Key every single request (generally the annoyance of authorising yourself every time doesn't exist in a completed solution ).

What does the example do? Simply checks how many bookmarks members of the OAuth Ma.gnolia Group have saved. Think it was 126 or so. The printed response is XML.

<?php
session_start();

require_once 'Zend/Oauth/Consumer.php';

$options = array(
'requestScheme' => Zend_Oauth::REQUEST_SCHEME_HEADER,
'version' => '1.0', // there is only a 1.0 version for now <img src="/templates/default/img/emoticons/wink.png" alt=";-)" style="display: inline; vertical-align: bottom;" class="emoticon" />
'signatureMethod' => 'HMAC-SHA1',
'localUrl' => 'http://path/to/this/file.php', // change to this file's local URL
'requestTokenUrl' => 'http://ma.gnolia.com/oauth/get_request_token',
'userAuthorisationUrl' => 'http://ma.gnolia.com/oauth/authorize',
'accessTokenUrl' => 'http://ma.gnolia.com/oauth/get_access_token',
);

// replace with your own real application consumer key and key secret!
$consumer = new Zend_Oauth_Consumer('CONSUMER_KEY', 'CONSUMER_KEY_SECRET', $options);

if (!isset($_SESSION['ACCESS_TOKEN'])) {
if (!empty($_GET)) {
$token = $consumer->getAccessToken($_GET, unserialize($_SESSION['REQUEST_TOKEN']));
$_SESSION['ACCESS_TOKEN'] = serialize($token);
} else {
$token = $consumer->getRequestToken();
$_SESSION['REQUEST_TOKEN'] = serialize($token);
$consumer->redirect();
}
} else {
$token = unserialize($_SESSION['ACCESS_TOKEN']);
$_SESSION['ACCESS_TOKEN'] = null;
}

$methodUrl = 'http://ma.gnolia.com/api/rest/2/bookmarks_count';
$rawData = $token->toQueryString($methodUrl, $consumer, array('group'=>'oauth'));
$client = new Zend_Http_Client;
$client->setUri($methodUrl);
$client->setMethod(Zend_Http_Client::POST);
$client->setRawData($rawData);

$response = $client->request();
header('Content-Type: ' . $response->getHeader('Content-Type'));
echo $response->getBody();

You can comment on the formal Zend Framework proposal for a Zend_Oauth at http://framework.zend.com/wiki/pages/viewpage.action?pageId=37957.