Maugrim The Reaper's Blog - Openid and Yadis

Another OAuth Library Is Born

nospam@example.com (Pádraic Brady) — Sat, 21 Jun 2008 20:40:56 +0000

I've spent some time over the last week, and I have completed an initial pass at writing an OAuth Consumer in PHP5. I'm biased, but it's a nice chunk of code capable of fairly routine POST based OAuth requests using either an Authorized header, or a raw url encoded POST request body. My main remaining task is final cleanup, included refactoring, rolling my final set of acceptance tests into PHPUnit from SimpleTest, and adding support for HTTP GET, RSA, and a storage API so tokens can be saved in the background rather than outside the API.

I wrote up a quick example script using the current source code - bear in mind the final API will be tweaked but this is a close match for what I'd expect to become final (I'll await community feedback before finalising anything!).

You can grab the code, which is geared up for integration into the Zend Framework as a Zend_Oauth component, from my proposals repository at http://svn.astrumfutura.org/zendframework/trunk/ (proposed components are in /trunk/library/Proposed). I fully expect to refactor a similar core library specifically for use with PEAR in the very near future.

Onwards with the example! I'm using the Ma.gnolia bookmarking service's API (version 2) here. If you intend actually running the example, you will need to create a Ma.gnolia account and visit http://ma.gnolia.com/applications/new to register an application to get hold of an OAuth Consumer Key. Registering an application is a bit confusing - but basically create an imaginary one (e.g. "Super Duper OAuth Test Application". Since you'll be the only user for now, you can use it across any OAuth testing regardless of application name. In the code, replace CONSUMER_KEY and CONSUMER_KEY_SECRET strings with the real thing.

The example isn't explained here - if you're unfamiliar with OAuth I'll explain it better in the future when this is all closer to completion. For now refer to the OAuth Core 1.0 Specification (the starting sections are pretty good at explaining things). The amount of code is indicative of features being wrapped around the existing API as they are not yet integrated directly (e.g. using Sessions to store temporary tokens instead of an internally referenced database). Also note that the example creates a new Access Key every single request (generally the annoyance of authorising yourself every time doesn't exist in a completed solution ).

What does the example do? Simply checks how many bookmarks members of the OAuth Ma.gnolia Group have saved. Think it was 126 or so. The printed response is XML.

<?php
session_start();

require_once 'Zend/Oauth/Consumer.php';

$options = array(
'requestScheme' => Zend_Oauth::REQUEST_SCHEME_HEADER,
'version' => '1.0', // there is only a 1.0 version for now <img src="/templates/default/img/emoticons/wink.png" alt=";-)" style="display: inline; vertical-align: bottom;" class="emoticon" />
'signatureMethod' => 'HMAC-SHA1',
'localUrl' => 'http://path/to/this/file.php', // change to this file's local URL
'requestTokenUrl' => 'http://ma.gnolia.com/oauth/get_request_token',
'userAuthorisationUrl' => 'http://ma.gnolia.com/oauth/authorize',
'accessTokenUrl' => 'http://ma.gnolia.com/oauth/get_access_token',
);

// replace with your own real application consumer key and key secret!
$consumer = new Zend_Oauth_Consumer('CONSUMER_KEY', 'CONSUMER_KEY_SECRET', $options);

if (!isset($_SESSION['ACCESS_TOKEN'])) {
if (!empty($_GET)) {
$token = $consumer->getAccessToken($_GET, unserialize($_SESSION['REQUEST_TOKEN']));
$_SESSION['ACCESS_TOKEN'] = serialize($token);
} else {
$token = $consumer->getRequestToken();
$_SESSION['REQUEST_TOKEN'] = serialize($token);
$consumer->redirect();
}
} else {
$token = unserialize($_SESSION['ACCESS_TOKEN']);
$_SESSION['ACCESS_TOKEN'] = null;
}

$methodUrl = 'http://ma.gnolia.com/api/rest/2/bookmarks_count';
$rawData = $token->toQueryString($methodUrl, $consumer, array('group'=>'oauth'));
$client = new Zend_Http_Client;
$client->setUri($methodUrl);
$client->setMethod(Zend_Http_Client::POST);
$client->setRawData($rawData);

$response = $client->request();
header('Content-Type: ' . $response->getHeader('Content-Type'));
echo $response->getBody();

You can comment on the formal Zend Framework proposal for a Zend_Oauth at http://framework.zend.com/wiki/pages/viewpage.action?pageId=37957.

Services_Oauth and Zend_Oauth Revisited

nospam@example.com (Pádraic Brady) — Wed, 18 Jun 2008 11:48:25 +0000

With my cryptographic efforts exhausted (I'll wait for the weekend to figure out the intricacies of ext/openssl API changes across 15+ PHP versions), my attention has wandered back to OAuth.

What is OAuth? "An open protocol to allow secure API authentication in a simple and standard method from desktop and web applications" - http://oauth.net/. If you're not yet up to speed it's being adopted by both Yahoo and Google. Interested now?

Starting yesterday, I opened up my IDE, updated PHPUnit, and got cracking. At the current rate of development a Consumer is likely at the weekend. I've already started writing up a formal proposal for PEAR and, of course, the Zend Framework also. I'm thankful the OAuth specification is this simple - it's one of the easiest to read specifications I've had to pleasure to work with. The cost of that is a certain level of vagueness on some points, but nothing you can't overcome with a little reading of the OAuth mailing list.

Of course with any new Specification, there comes potential implementation trouble hard on its heels. Talk about an OAuth Core 1.1 Specification is well underway on the mailing list which drags up concerns over the future compatibility of an early API based on 1.0. I'm not too worried here since these types of APIs are often extremely simple and flexible, very much belying the enormous work and mountain of source code existing in the background.

The other concern are Extensions. OAuth 1.0 allows for Extensions, of which there are several Drafts already. One of the more interesting Extensions is OAuth Discovery 1.0 which will create a dependency (when implemented) on Zend_Service_Yadis (for the Yadis 1.0 Protocol). This Extension will be more formally integrated into OAuth Core 1.1 so it's optional only if you ignore it hard enough! The Yadis component is one of those semi-lost proposals - it's now hitting 15 months on the Zend Review list waiting for finalisation and comment (it has been released on PEAR as beta already).

The other reason this is a bit distracting is that a 15 month old proposal inevitably misses 15 months of clarifications, developments and the rapidly shifting sands of the specification community. For example, Yadis 1.0 Specification is making the move into the upcoming OASIS XRI Resolution 2.0 Specification (Section 6) and there is a new OAuth driven XRDS-Simple 1.0 Specification Draft. These all accumulate unless you regularly assess them all and make incremental changes.

I've had people beating the bushes trying to to flush me out into the open so they can request I get this started. They'll be happy to hear it's now in progress.

Edit: A few people were curious about why OAuth is worth paying attention to. The simplest convincing answer is to point out the alternative. Say you use GMail and a website asks you to import your contact data - how does it access that data?

Do you want every Tom, Dick and Harry website to potentially store your GMail email address and password for some future nefarious purpose? Yet people put their absolute trust in this model day after day on social networking sites.

Do yourself a favour - if you're a developer writing an API make it possible for web sites to access your users' data without having to beg them for their username and so-called secret private password.

PEAR::Encryption and Zend_Crypt Revisited

nospam@example.com (Pádraic Brady) — Tue, 17 Jun 2008 11:22:29 +0000

It's been a while since I did some active ZF/PEAR component development. It's been one of those 6 month periods where time to commit was a rarity for a few reasons. So now that I'm back on the road, where to? First of all I need to remember/reset my subversion password for the ZF repository! Whenever I don't use a password regularly I always forget it .

Zend_Crypt was accepted into the Incubator last Autumn with 3 initial broad implementable areas:

1. Hash Message Authentication Code Algorithm

HMAC is commonly used these days. You can find it in use in both OpenID and OAuth for creating secure authenticated signatures of exchanged data using a key known only by both parties to a message exchange. Obviously the benefit is to prove that any message you received does come from the expected party, and not an intercepting middle-man who's done some tinkering!

The Zend_Crypt version relies heavily on the available hashing support (i.e. the hash, mhash, openssl extensions, or the native md5/sha1 functions) which is a point of refactoring at the moment since I've added a new Zend_Crypt::hash() method which covers all the extensions and native functions for hashing.

2. Diffie-Hellman Key Agreement Protocol

One of the problems with connecting two random systems is ensuring the messages they exchange can be authenticated. A common solution is HMAC (as above) but HMAC requires that both parties have a shared secret with which to sign/verify message signatures. Diffie Hellman is a protocol for establishing such a common secret in such a way that a middle-man intercepting all setup messages between the systems cannot replicate the shared secret. The secret is in the math (which I won't bore you with!).

There are two major updates here. First of all the Zend_Crypt_Math library now fully supports the GMP extension. As a general rule - if you're doing big integer (i.e. > 32 bits) math then you should install gmp. Especially for Diffie Hellman use which is horribly slow on bcmath! It is incredibly faster than bcmath at certain types of calculations. My test suite for all of Zend_Crypt takes upwards of 2 minutes to run using bcmath, and only 6 seconds using gmp.

The second major change, since gmp is an uncertain extension to be installed on any system, is to add in support for Dmitry Stogov's ext/openssl changes coming in PHP 5.3. Once he no doubt upgrades the 5.3 extension to use openssl 0.9.8h (unless we like typos which narrow key ranges ) ext/openssl will add a more commonly available and extremely fast source of Diffie Hellman key generation.

3. Hashing Wrapper

Since hashing is an annoying part of some applications, there's a new static Zend_Crypt::hash() method (which accepts the same arguments as the hash() function in PHP 5.2 assuming you didn't disable it!). This offers a simple means to navigate the 4+ methods PHP can use to create hashes. Support extends to hash, mhash, openssl, and the two md5/sha1 native functions. It's particularly useful for hashing algorithms unsupported by native functions - such as sha256, ripemd, gost, etc. If it can't find anything to support your hashing method you get a neat catchable exception.

Why bother with this? If it doesn't matter to you - you can make your own if..else method. But it's handy to have a single all encompassing one built into the framework. And yes, it does support both hex and binary output.

4. (PROPOSED) RSA Public Key Cryptography

One of the more recent trends in web services has been the rapid adoption by large service providers of the OAuth open authentication standard. OAuth offers a single best practice standard for allowing users authenticate to web service APIs and has seen wide adoption from the likes of Yahoo and Google. The recent move towards OAuth by Google revealed they would require the use of RSA-SHA1 message signing.

The RSA proposal is intended as an OO wrapper around existing openssl functionality. It goes a bit further though since openssl has, shall we say, less than stellar documentation. My RSA test suite breaks at three points between PHP 5.2 versions, and 5.3. So poor documentation, and poor API compatibility across major versions. Besides solving this problem, having an OO wrapper allows for a more OO approach. Having a single API which works the same across all PHP versions from 5.1.4 to 5.3.0 can be quite valuable when you need a maintainable component relying on RSA in the ZF.

If accepted it's first use will be support RSA-SHA1 for my OAuth implementation. Some refactoring later I might look into a more general component which utilises RSA and other algorithms and offers broad based certificate support. HMAC will already cover other OAuth signing methods like HMAC-SHA1 and HMAC-SHA256.

Of course since a lot of this is already in beta for PEAR, you can expect the same updates rolled in there at a future date. That should level the playing field nicely for finishing up (finally) my open source OpenID For PHP library before a PEAR proposal for it is put up for voting.

RSA for PEAR I might look at - there's already a native implementation in PEAR for PHP4, but there's no PHP5 version, and it doesn't attempt to leverage off ext/openssl at all (which be a lot faster!).

Kicking The Bad Habit Of Being An Overworked Paddy

nospam@example.com (Pádraic Brady) — Thu, 24 Jan 2008 20:58:42 +0000

It's hard to believe we are already almost 1/12 of the distance into 2008. By now all of you have broken your new year resolutions. I know I've broken several at a minimum!

After some months of desperate oft-despairing struggling with work schedules I've finally once and for all conquered my lack of free time. It's an ingenious solution - I'm taking a small break from work before rekindling an interest in financial services in these doubtful times (ask Société Générale if you want to know how doubtful, or the US Federal Rserve).

The outcome of this reorganisation of my career direction is twofold. Firstly I get extra bags of cash. Secondly, I get slightly more vacation time. Thirdly, it won't require as much overtime. Fourthly, there's less chance of last-minute-scrambling which became exceptionally evident over the last few months as the Irish market continues to swell (in defiance of the laws of EU Economics). Of course added together this provides more of my most sought after commodity - personal time.

All that's left is how to use this new-found wealth. In between the extra pub-crawling exercises, engagements as the designated baggage mule on shopping excursions, and the other things an average 20-something is inclined to do, I want to enjoy some travel, take up writing again, and commit some completion time to the open source projects I contribute to.

I've been a very bad boy in that regard in the last six months and at one point I became an absolute nightmare for anyone who needed to contact me by email. It was not my finest hour, and I seriously doubt I escaped with a pristine reputation for being dependable. C'est la vie. A few of these "instances" shall we call them, have since been resolved to my satisfaction so I'm 95% back to nominal form as a powerhouse of innovation, inspiration and ingenuity (see, even my ego is back rockin' at full throttle!). Yep, you can always measure the normality of an Irishman by his level of self-directed sarcasm .

Anyways, enough self-critical analysis - it weakens the ego - since I'm back in fine form after two extremes (a two month vacation, and a four month chaotic period of non-stop work) I have the luxury of directing some of this time where it was always supposed to be: in supplementing my PHP experience with some open source doodling and manic self-promotion . The first target of my ire is a small project with Till Klampaeckel (Seek. Kill. Destroy.). After that is PHPSpec 0.3.0 (Exterminate! Exterminate! Exterminate!). After that is that frickin' promise-but-never-effing-do component for implementing a Yadis service (Off With His Head! Off With His Head!). I swear that thing has been sitting in a personal subversion repo begging for a few final hours of attention!

After that I'm taking a long breather, attending oodles of conferences, and finding something with a lot of words to write.