Archive for July, 2009

During yesterday, I finally got around to patching and finishing Zend_Oauth’s Consumer implementation for the OAuth Core 1.0 Revision A specification. Once I had it finished, I used it to write a quick and simple interface to post some Tweets on Twitter while I was testing it out. With some documentation and a few extra unit tests, the Consumer implementation should find its way into Zend Framework 1.10…along with the Server implementation I think. In this article I’ll explore how to writea quick Twitter client so you can post tweets (those short messages of less than 140 characters) once authorised across the OAuth protocol.

You can download all the necessary files (just be sure to edit them as described) or pull them from git from: http://github.com/padraic/Tweet-Lite/tree/master

What is OAuth?

If you’re not aware of what OAuth is, the OAuth specification puts it this way:

The OAuth protocol enables websites or applications (Consumers) to access Protected Resources from a web service (Service Provider) via an API, without requiring Users to disclose their Service Provider credentials to the Consumers. More generally, OAuth creates a freely-implementable and generic methodology for API authentication.

In other words, it’s a means of allowing websites to access your data on other services via a service API, like Twitter’s API or Google Gdata, without actually providing those websites with your username and password. Instead, OAuth allows you to authorise such websites to access your data so that they don’t need your username or password - they just use an Access Token supplied by your service provider - and you can easily deauthorise them if desired. The benefit is immediately obvious - your username and password are not shared or handed out to potentially untrustworthy sites. The glut of services using Twitter are a prime example - until recently they all needed your Twitter username and password and honestly, how would you know they wouldn’t misuse that? Because they said so? OAuth eliminates this problem.

The protocol works like this. The website (consumer) that wants to access your data from a service provider, contacts the provider using HTTP to retrieve a Unauthorised Request Token. The consumer will then redirect you, the user, back to your service provider so you can authorise the consumer’s access. The redirect URL will contain the Unauthorised Request Token as a parameter. If you approve the access, you are redirected back to the original website with a verification code attached to the URL. The website now knows you approved its access, so it contacts the service provider, including both the newly approved Request Token (once again) and the verification code in the URL. The response to this should be a fully authorised Access Token (associated with the User) which the consumer can use in all future requests when accessing your data (until either it times out or you deauthorise the access). The Request Access token can be discarded now - in OAuth parlance you exchanged an unauthorised Request Token for an authorised Access Token.

Preparations Are Always Inevitable!

With this understanding in hand, let’s get to writing this small Twitter client as an example! You will need to download the Zend Framework (whether the latest release or via subversion). You will also need to download/checkout the Zend Framework Incubator since Zend_Oauth is not yet part of the main trunk. Once you have stored both somewhere, make a note of the paths to their “library” directories so you can add them to the PHP include_path later.

On the Twitter side there are three steps.

First, get a Twitter account! Hopefully you already have one and are following @padraicb (i.e. me!).

Second, you need to configure your operating system for a new local domain. Under Linux, this is done by editing /etc/hosts. You’ll need root privileges here so use “sudo myeditor /etc/hosts”. Under Windows, the same file is located at C:\Windows\System32\drivers\etc\hosts and will require Administrator privileges to edit. Adding a local domain is dead simple, and it’s needed to use Twitter’s API on a local machine - Twitter refuse all requests from localhost or 127.0.0.1 but they won’t filter out a local domain name since it’s unfeasable and would make a lot of application developers extremely crazy. Edit the file to include a new entry like:

127.0.0.1 mytwitterclient.tld

Once saved, your browser should immediately respond to any address in the form of http://mytwitterclient.tld by attempting to call localhost/127.0.0.1 on your system. If you have a default web document root configured with a web server running, this is where the request will be mapped to and where you should store any files. If you are really troublesome and can’t take how easy that was, you can run off to play with Virtual Host configurations to use a different path.

Third! OAuth providers like Twitter don’t hand out access tokens to every Tom, Dick and Harry. Well, they do - but they like to know whether your name is Tom, Dick or Harry! You need to register all applications with them for this purpose and to get hold of a key to access their OAuth service. This is pretty much standard for all similar providers. Visiting http://twitter.com/oauth_clients and logging in using your Twitter account opens up a menu where you can register new applications/clients or delete them. Create a new client record here. Naming is not important - just make absolutely sure that a) you select “Browser” as the Application Type, b) set a callback URL of http://mytwitterclient.tld/callback.php (edit domain and path for your preferred location), and c) select “Read & Write” as the Default Access since we will be sending new Tweets and need that write access. We will not be using Twitter for logins. Once registered - it’s immediate - you’ll be faced with a page of details including URLs to various OAuth endpoints, and most importantly, the Consumer Key and Consumer Secret you should use for your application. Don’t worry! You can revisit this page at any time.

A Glimmer Of Actual Source Code

Since we’re sticking with the concept of “simple” here, this will be a scripted effort not an exercise in writing the next Zend Framework powered super app. Any permanent (these can be refreshed indefinitely so don’t worry about losing them) Access Token will be stored to the current session for reuse so we can skip a database.

Our mini application is comprised of six files (none of them big): config.php, common.php, index.php, tweet.php, callback.php and clear.php. Splitting this out across a few files makes it easier to follow, understand and edit. Let’s start with config.php which contains our OAuth configuration which you will need to edit for your own details.

[geshi lang=php]

define('URL_ROOT', 'http://mytwitterclient.tld');

$configuration = array(
'callbackUrl' => ‘http://mytwitterclient.tld/callback.php’,
‘siteUrl’ => ‘http://twitter.com/oauth’,
‘consumerKey’ => ‘xxxxxxxxxxxxxxxxxxxxxxx’,
‘consumerSecret’ => ‘xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx’
);[/geshi]

In a perfect world, this should be everything you need to add to your configuration. You should always be aware that a Service Provider is capable of setting their own requirements, service endpoints, and even preferred request method. A fuller configuration might look like this (still valid for Twitter since it mostly demonstrates internal defaults used by Zend_Oauth):

[geshi lang=php]

define('URL_ROOT', 'http://mytwitterclient.tld');

$configuration = array(
'version' => ’1.0′, // there is no other version…
‘requestScheme’ => Zend_Oauth::REQUEST_SCHEME_HEADER,
‘signatureMethod’ => ‘HMAC-SHA1′,
‘callbackUrl’ => ‘http://mytwitterclient.tld/callback.php’,
‘requestTokenUrl’ => ‘http://twitter.com/oauth/request_token’,
‘authorizeUrl’ => ‘http://twitter.com/oauth/authorize’,
‘accessTokenUrl’ => ‘http://twitter.com/oauth/access_token’,
‘consumerKey’ => ‘xxxxxxxxxxxxxxxxxxxxx’,
‘consumerSecret’ => ‘xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx’
);[/geshi]

We won’t worry about the details for now. The only two to bear in mind are “requestScheme” and “signatureMethod”. The request scheme is a means of sending the OAuth Protocol Paramaters to a service provider. By default (and it’s the preferred option per the spec) we send these parameters by way of an Authorization header. There is a fallback to two other methods - as raw data in a POST body, or as part of the query string (GET or POST). The default method should be supported as recommended by the OAuth Specification. The signature method has four supported options in Zend_Oauth: HMAC-SHA1, HMAC-SHA256, RSA-SHA1 and PLAINTEXT. HMAC-SHA1 is a reasonable default, but the others may be needed on some service providers. Finally, you will note there are three URL options and no “siteUrl” as in the short version. Setting a siteUrl is perfectly fine if the actual endpoints use the convention of /request_token, /authorize and /access_token as the paths in their endpoints.

Not let’s look at common.php (predictably the file all other files except config.php will be including):

[geshi lang=php]

// If you haven't edited php.ini to add the Zend Framework and the
// Zend Framework Incubator to the PHP include_path, then do so here.
// Don't use mine!

set_include_path(
'/home/padraic/projects/zf/trunk/library'
. PATH_SEPARATOR . '/home/padraic/projects/zf/incubator/library'
. PATH_SEPARATOR . get_include_path()
);

// Make sure Zend_Oauth's Consumer is loaded

require_once 'Zend/Oauth/Consumer.php';

// Start up the ol' session engine

session_start();

// Include the configuration data for our OAuth Client (array $configuration)

include_once './config.php';

// Instantiate an instance of the Consumer for use

$consumer = new Zend_Oauth_Consumer($configuration);[/geshi]

The common file is pretty simple stuff. You'll note we set the include path here (you can do it in php.ini either), start our session, and instantiate a new OAuth Consumer. Zend_Oauth currently offers a full Consumer, the Server/Provider implementation will follow in the very near future.

Time to look at the starting point to our little app, index.php:

[geshi lang=php]

// include some common code

include_once './common.php';

// Do we already have a valid Access Token or need to go get one?

if (!isset($_SESSION['TWITTER_ACCESS_TOKEN'])) {

// Guess we need to go get one!

$token = $consumer->getRequestToken();
$_SESSION['TWITTER_REQUEST_TOKEN'] = serialize($token);

// Now redirect user to Twitter site so they can log in and
// approve our access

$consumer->redirect();
}

// Got past that if block! Must have an Access Token. Let’s kick out a simple
// form for the user to submit their tweet with.

// echo the xml declaration in case shorty tags enabled - grrr. They’re
// a bloody nuisance at times.
echo ‘‘;
?> "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd">

You successfully sent your tweet!

Oops! Tweet wasn’t accepted by Twitter. Probable failure:

All that work on Zend_Oauth, and all you do is send Tweets with it?

What do you want to say to Twitterland using 140 characters?

Click below to delete the Access Token and force start another authorisation leg:

Clear Access Token

[/geshi]

In the top half of this script, we’re checking to see if we previously stored an Access Token (needed to use the Twitter API on behalf of the current user) as a serialized session variable. If it’s there, we’ll print the form, accept a status textfield, and send it to tweet.php. If we don’t have an Access Token, we need to get one. Calling “$consumer->getRequestToken();”, asks the Zend_Oauth_Consumer to fire a request to Twitter asking for a new Unauthorised Request Token, and we should get one back. With this in hand, we may then redirect the user to Twitter (the redirect URI will include the token) so they can approve our access. Before we redirect the user, we’ll serialise the token and store it as a session variable. Note that all tokens in Zend_Oauth are objects of type Zend_Oauth_Token.

The next step happens outside the application since we redirected the user back to the provider.

Here, a user is given the option to approve our access or deny it. As Twitter notes, users may also retract their authorisation at any time. Once the user clicks the big “Allow” button, Twitter will redirect the user to us. Within the redirect URI should be a verification code (think of it like a PIN number) with which we can now request an authorised Access Token. Now, as we configured, and as it was added during registration, a user should be redirected to callback.php:

[geshi lang=php]

// include some common code

include_once './common.php';

// Someone's knocking at the door using the Callback URL - if they have
// some GET data, it might mean that someone's just approved OAuth access
// to their account, so we better exchange our current Request Token
// for a newly authorised Access Token. There is an outstanding Request Token
// to exchange, right?

if (!empty($_GET) && isset($_SESSION['TWITTER_REQUEST_TOKEN'])) {
$token = $consumer->getAccessToken($_GET, unserialize($_SESSION['TWITTER_REQUEST_TOKEN']));
$_SESSION['TWITTER_ACCESS_TOKEN'] = serialize($token);

// Now that we have an Access Token, we can discard the Request Token
// Keep on eye on gathering RTs in real life which are never used.

$_SESSION['TWITTER_REQUEST_TOKEN'] = null;

// With Access Token in hand, let’s try accessing the client again

header(‘Location: ‘ . URL_ROOT . ‘/index.php’);
} else {

// Mistaken request? Some malfeasant trying something?

exit(‘Invalid callback request. Oops. Sorry.’);
}[/geshi]

In the callback file, we check the incoming request for a query string and if we also have an outstanding Request Token, we pass the $_GET array and the Request Token (unserialised from the session) to the Zend_Oauth_Consumer::getAccessToken() method. This sets up another request from the application to Twitter which includes both the Request Token and the verification (PIN) code from the user redirect. Twitter should now respond with an authorised Access Token, which we’ll serialise to a session variable for future use (in a serious app, this would be associated with the user account more permanently), and with an Access Token in hand we can discard the now useless Request Token. With this done, we can redirect the user back to index.php where they will be greeted by our application in all it’s glory (or not).

Now that we are here, our OAuth authorisation has been completed. But wait, there’s more!

Users may now input a status message of no more than 140 characters and submit it to Twitter. The form on the page is sent to tweet.php, so let’s see what it is doing:

[geshi lang=php]

// include some common code

include_once './common.php';

// Check for a POSTed status message to send to Twitter

if (!empty($_POST) && isset($_POST['status'])
&& isset($_SESSION['TWITTER_ACCESS_TOKEN'])) {

// Easiest way to use OAuth now that we have an Access Token is to use
// a preconfigured instance of Zend_Http_Client which automatically
// signs and encodes all our requests without additional work

$token = unserialize($_SESSION['TWITTER_ACCESS_TOKEN']);
$client = $token->getHttpClient($configuration);
$client->setUri(‘http://twitter.com/statuses/update.json’);
$client->setMethod(Zend_Http_Client::POST);
$client->setParameterPost(‘status’, $_POST['status']);
$response = $client->request();

// Check if the json response refers to our tweet details (assume it
// means it was successfully posted). API gurus can correct me after.

$data = json_decode($response->getBody());
$result = $response->getBody(); // report in error body (if any)
if (isset($data->text)) {
$result = ‘true’; // response includes the status text if a success
}

// Tweet sent (hopefully), redirect back home…

header(‘Location: ‘ . URL_ROOT . ‘?result=’ . $result);
} else {

// Mistaken request? Some malfeasant trying something?

exit(‘Invalid tweet request. Oops. Sorry.’);
}[/geshi]

When I said there’s more, I meant it. From now on, every single API request concerning our user must be authorised using our OAuth Access Token. This is done, handily enough, by signing all requests using HMAC-SHA1. Now, doing this manually can be a pain, so using the Access Token object you can simple retrieve an instance of Zend_Http_Client (subclasses by Zend_Oauth_Client) which will handle all the OAuth protocol parameters and request signing transparently. Just use as normal as you would Zend_Http_Client.

So using this new client, we merely implement part of the Twitter API. Sending status messages is done by sending a POST request to http://twitter.com/statuses/update.format (where “format” must be changed to one of “json”, “xml”, “atom” or “rss”) which includes a “status” parameter (in the POST body) containing our tweet. Once the response is received, we do a little checking to ascertain whether it was a success or failure, and redirect the user back to index.php once more.

The final file we haven’t mentioned is linked to from the bottom of index.php’s output. clear.php is a simple script to delete the current Access Token and trigger a new OAuth authorisation process back through Twitter (just in case you missed it the first time around):

[geshi lang=php]

// include some common code

include_once './common.php';

// Clear the Access Token to force the OAuth protocol to rerun

$_SESSION['TWITTER_ACCESS_TOKEN'] = null;

// Redirect back to index and the protocol legs should run once again

header('Location: ' . URL_ROOT . '/index.php');[/geshi]

Now go and do some Twittering! In any other Twitter client, your new tweets should also display nearby the name of the application you registered with Twitter.

Other Considerations on OAuth?

There are a few things I’d like to mention before closing. The first is that Zend_Oauth implements Revision A of the OAuth specification (1.0a) which accounts for a session fixation vulnerability in the original specification. There is no need to worry about that here, and 1.0a is being rolled out to other service providers as we speak, if not already. Twitter implemented the improvements back in June. The component is still compatible with the original specification for service providers who haven’t yet updated their implementation - it’s completely transparent. Zend_Oauth also makes use of Zend_Crypt (in the Incubator also) so we offer full support for HMAC and RSA hashing via its subcomponents.

Conclusion

Well, there you have it. A simple little app for making tweets using the Twitter API over OAuth. Obviously this is a very simple example to show off OAuth but I think I kept it neat enough to explore its operation without confusing everyone thoroughly . In reality, tokens will need to be managed with much more care since they are valid for extended periods (in fact many times the provider won’t expire them for the forseeable future). You also need to ensure they are associated with the correct user.

Have fun with OAuth!