Maugrim The Reaper's Blog

To keep folk informed, I'll be pushing Eve Online blog posts (and other topics) to a new blog separate from this one. Some quick polling showed people assume this is a near pure PHP medium so a new blog for the non-PHP topics is called for. This fits in with my target of 3 blog posts a week which now won't be seen as spamming Planet PHP .

When I left off my last post I was enjoying ownership of a new Caldari Kestrel Frigate in Uitra, in the game called Eve Online, and had just joined a Corporation to further my experience and have a support network available. You can find me in-game as "Maugrim McFiriba", the character I invented in the mid-90s with the now infamous "The Reaper" nickname, and later used as my internet handle all over Yahoo from 1998. Best of luck to other PHP developers trailblazing through Eve right now - I'll add you guys as buddies in-game . Email me if you want in on my chosen Corporation in the next week or two.

This week started off quietly with me outfitting my new Caldari Kestrel. After training the required skills I'm now running with 4 Standard Missile Launchers (1 is held in reserve). To add a little to my "tanking" (the ability of a ship to absorb damage for extended periods without exploding) I've fitted a Shield Hardener, a Small Armor Repairer and a Shield Booster. The first is passive, the final two eat Capacitor power. My naming is not exact - still getting the hang of having module names memorised . I figured out recently that I was off the mark about increasing my ship's capacitor max - I actually need to make modules I install require less by training relevant skills. In any case, I can now fit in those tanking modules and 3 Launchers without stressing my capacitor beyond it's design limits. If I disable the Armor Repairer I can put an extra Launcher online (at a safe distance from the enemy since it drains my Capacitor dry when activating!) and increase my fire power by 33%. Assuming my shields can take the return beating with the Shield Booster enabled for short periods, and I don't end up really really needing that Repair module, I can chew on level 1 Pirates from a distance of 20km or so and take out a ship with each grouped launch. I'm sure that won't last long as the Pirates get wind of my new death dealer .

More after the jump. Continue reading "Eve-Online: Review of days 3 to 7 progress"

It's a lesser known trait of mine that I enjoy playing computer games, specifically strategy games for the PC. The only console I own is a Nintendo Wii - which is saying something since it's the first console I've owned since the Sega Genesis! So this weekend, with all the free time I have, a new broadband connection courtesy of Eircom (after replacing the crappy Netopia router they give you for free; free as in scrap metal), and a little trepidation, I joined Eve Online.

Eve Online is a MMOG set in space. There are approximately 5000 star systems, 200,000+ subscribers, and perhaps 18,000 to 45,000 players online at any one time. Since I was playing at the weekend for extended periods, I noticed the numbers peaked on my GMT clock each evening. The idea behind Eve Online is to enter the vastness of space and make a name for yourself either through combat, mining, trading, production or research. These are not however true alternatives since any player can train any skill imaginable given enough time. So the name of success is called specialisation, not class leveling...

All star systems have a security rating from 1.0 to 0.0. I spent all my time in 1.0 and 0.9 star systems. The word is that going anywhere with a 0.8 rating or less is not something a Rookie should consider for at least a few weeks. Going to 0.5 or lower could charitably be called suicide. Check out YouTube for a few videos of what happens to players who get cocky and impatient and run off to a 0.4 system to mine. It only takes a high skill player with a few missiles...

As for strategic and tactical gameplay - Eve Online rocks. It's a thinking man's dream game. You need to select skills, compare weapons and ammunition types, review Market conditions for the best regional prices (some stations can charge double the average price for items), get used to how ships scale and how to assess which you can take on (which is pretty much nothing since 1.0 sec systems are heavily overwatched by the local race's police forces and only suicides would attack you, or you them ). Living in 1.0 space is quite safe and a more than a few corporations stay exclusively there. Even a few of the 0.0 Corporations maintain 1.0 sub-Corporations for you to join.

The game itself is beautifully rendered. Back in December CCP release the Trinity client was released which added an overall graphics update with high resolution textures. My PC was never taxed while running it. I have a pretty good gaming rig so I could easily run two clients at the same time (Eve also let's you play three characters on your single account). Th only niggle was collision detection on large objects like stations and planets. While it seems an odd flaw, I suspect it's a simple optimisation. The only annoyance it will serve is trying to reach anything on the opposite side of a station - my advice is to orbit around the station before making a straight-line approach to such objects.

My own experiences from a first weekend after the jump... Continue reading "My First Two Days Playing Eve Online"

One of the components I spent some time working with recently was Zend_Feed, which was interesting at first, then a little irritating, and eventually compliant. In this entry I explore Zend_Feed from the perspective of someone aggregating RSS and Atom feeds with a view to building a database of uniquely identified content for later presentation in a "Planet" style application.

My first overall assessment is that Zend_Feed needs work. It is a wonderful component that can simplify your life immeasurably, but it's up against competition from third party libraries like MagpieRSS which do a better job at the one malfeasant facet of blog feeds: invalid, malformed and non-standard RSS and Atom XML. What Zend_Feed needs to breach the threshold of usability is more ability to handle the various problems you meet parsing RSS and Atom to a common range of data. It also badly needs improved documentation with a focus on examples of real use - for example I can't find a single documentation snippet or blog tutorial showing how to get a entry's actual HTML content using Zend_Feed which is problematic, possibly symptomatic, and as you'll see unintuitive. Don't feel too disheartened - it's still a powerful package with sufficient utility to get you well on your way.

This tutorial is intended to cover some basics to get you started. In fact all we create here is a simple command line script to aggregate content frequently (e.g. just set up cron to run it every hour or so) into a database for later presentation.

Setting Up Database And Models

So what common data do I want for each entry in an Atom or RSS feed?

1. Unique Identifier
2. Author
3. Title
4. URL
5. Publish Date
6. Description
7. HTML Content

Based on this, we can predict a decent Model for an Entry table in a database. More on that later, but for now, how to get this data from a feed? The first thing is to have a database or other source to get the URL for each RSS or Atom feed. I'm using a MySQL table as follows:


You can insert into this table as you wish. One row per blog you intend aggregating. Here's some sample data:

url: http://blog.astrumfutura.org
feedurl: http://blog.astrumfutura.com/feeds/index.rss2
title: Maugrim The Reaper's Blog
author: Pádraic Brady

The other fields update without manual intervention.

A database table for entries may be something like:


Now that we have database tables to work with, we can use Zend_Db to create some Models. We'll store these in the usual ./application/models directory in keeping with the default Zend Framework directory structure as Blog.php and Entry.php:

Blog.php


Entry.php

The Aggregator Script Foundation

The aggregator script is a pretty simple one executed by php on the command line. Here's a basic foundation to start from stored to "./scripts/Zend/Aggregate.php":


The source code above is not difficult. We use a static main() method call at the end of the file to initiate the process. The main() method creates a new instance of Zend_Aggregate, set's up items like configuration, database connection, and Model instantiation, and finally calls aggregate() on the new object.

Inside, Zend_Aggregate::aggregate() we get a list of all blogs stored in the database. For each of these blogs, we fetch its relevent feed URL. The result of Zend_Feed::import() can be iterated across to get each individual entry or item. On each entry/item we pass it to the Zend_Aggregate::parseEntry() method - which bears overall responsibility for what to do next.

Using Zend_Feed to get common data for RSS/Atom entries

So what would parseEntry() do? Well, it's main role is to parse the feed and assemble a collection of common data we require - see the list above ;). So it is also responsibility for ensuring we get that data in a common format regardless of whether the source was RSS or Atom. Here's the first thing we put in parseEntry():


Yes, a lot of the resolution of source comparison and common format will likely require we use DOM. This gets interesting later since Zend_Feed appears to do two completely incoherent things. Firstly, it strips source of several XML namespaces (particularly "content" for RSS), but leaves in most others (e.g. "dc" for RSS). There's an alternative access method for namespaced elements which is plain weird, but which Nick Halstead reminded me of.

Let's hit our list though.

First item is getting some sort of unique id for each entry - usually it's a URL reference. RSS and Atom don't agree here, but both have some sort of id. For RSS it's the "guid" element, while Atom has an "id" element. So into Zend_Aggregate::parseEntry() we add:


So if we can't find one of those IDs, we'll just use the item's URL which is probably just about enough. In most cases though there will be an ID.

Next up we need a title. Thankfully, both RSS and Atom at least agree on this:


In case I was wrong, I'll just insert the blog title instead as a placeholder. Next up is a description. Atom rarely has something for this...in which case we'll just take the entry title.


If you're following this, note that all single elements can have their enclosed values retrieved by simple calling "$item->element()" as a method. Calling "$item->element" (as a property) actually returns another Zend_Feed_Entry_Abstract object which we can use to traverse into element children if required.

Now for some HTML content. The problem with the content is that it comes in two forms. Content as HTML in RSS is encoded (think something like htmlentities()) while the content in Atom is merely enclosed in a CDATA block. So firstly, if it's RSS we can decode it before use. The second part is ensuring the HTML is relatively safe from XSS, and finally that it all follows the same HTML standard. In this mind boggling decision making I've just delegated to the excellent HTMLPurifier library.

Now the interesting thing about content is that in RSS it's enclosed by a "content:encoded" element, i.e. it uses the "content" namespace with a URL of "http://purl.org/rss/1.0/modules/content/". Zend_Feed handles this by stripping out the namespace so we're left with an "encoded" element. Atom on the other hand just has a "content" element to start with (the HTML in a CDATA block).


We also retain the original content unchanged. Later we use a md5 hash of it to detect changes (e.g. maybe the author edits their entry) and update our entries on the database.

Next, we need a URL to the specific entry we're parsing here:


Since some RSS feeds like Planet-PHP's include the link in a href attribute, rather than the link element's nodeValue, we need to a few acrobatics with DOM just in case.

Getting the author or creator's name is another pain in the ass. RSS 2.0 often uses a dc:creator element, with a "dc" namespace with URL of "http://purl.org/dc/elements/1.1/". Unlike with Zend_Feed's previous RSS content:encoded stripping of the namespace - it doesn't do it here at all...:(. How naughty! So we have to parse out the dc elements ourselves assuming they exist, or otherwise search for an author or creator element ourselves, remembering that for Atom it's actually a name element child of author... I feel your confusion ;).


If all else fails, we'll just take the author name from the blog database entry. In the above I'm using the DOM to extract a dc:creator value. Here's another Zend_Feed snippet of a shortcut - if the DOM is not your thing, you can instead get the value of $creator using:


I love shortcuts - but here it's almost if not quite worse than using the DOM. You still need to strval() the resulting Zend_Feed_Element value returned, check if it's empty or not, and only then assign it. Still, using this without DOM can be an advantage if DOM is one of those less than familiar extensions. To be honest, I think Zend_Feed really desperately needs a simple get() method to centralise value fetching without all these gymnastics...

Let's not forget a published date!



Yeah, dates are worse than content sometimes... We'll need to normalise the date from the differing RSS and Atom forms.


Another method for you to append to the file...

Moving right along, we have two final parseEntry() parts:


You could return an array either, I just like objects. ;)

The rest of the tutorial after the jump! Continue reading "Zend_Feed: Getting Started With Aggregating RSS/Atom Content"