Archive for March, 2006

Where Bethesda fail, the Modders prevail…

After playing Oblivion and following the forums on how the game plays out I discovered (as did a lot of people) that the game comes with some serious flaws in later gameplay (level 15+). Now, being honest, this are not flaws per se in Bethesda’s perspective nor that of others - but being a long time Morrowind fan which I have played through a number of times, using different characters and joining different Guilds and Houses, it seems that way to me.

The problems are three fold:

1. Levelling seriously hurts unless you do some careful planning. It doesn’t bother me right now because my balanced Mage/Fighter class was designed to allow some choices in how I level attributes. But if you are not careful you can start skipping important attribute multiple-increases (up to 5x) by diluting your skills with common-use ones like Athletics, Security or Sneak.

2. Monsters and NPCs level with you - the infamous scaling system means a Level 5 character will meet Level 5 monsters, a Level 16 character will meet Level 16 monsters, etc. It’s not that clear cut of course, but imagine levelling to 16, having lower attributes for your combat/defence Major skills due to using non-character-specialisation Major skills, and having a lot of monsters running around with supposedly rare armour types… Not good IMO. Bethesda imposed a challenge, but left out a sense of achievement. At level 20, there is no achievement - everyone else is suddenly level 20 (or less) and often with the same great equipment you spent a fortune getting hold of. Again, Morrowind or main stay RPG fans will have issues with this.

3. Items are not rare. Need glass armour? Well, you and every other monster and NPC will be swimming in Glass armour once you hit a certain level. There are no statically placed, or rewarding rare weapons and armour to be found in dungeons and ruins. Instead MANY monsters will have “rare” items at later levels, right down to those scraggly looking Bandits who hijack you when travelling through the wilderness between towns. What’s the point of searching for those rare pieces of Glass armour when everyone has them? I don’t think there’ll be a challenging foray into any House treasuries in Oblivion looking those full sets of rare Ebony and Glass items…;-).

I call these the Three Stinkers of Oblivion. It’s such a cool game, but someone along the line dumbed down the RPG element and jumped on the hack’n'slash bandwagon. Now, this is a personal perspective. When I level I want to see some creatures that were horridly difficult to defeat now cowering in fear at my passing (or at least speared up against the nearest tree by either my sword). I don’t want to find that challenging dungeon now even more challenging JUST BECAUSE I LEVELED!. AT least make it a little less obvious, and allow me a sense of power - until the next big bad Daedra cuts me into ribbons…;-)

There are smaller annoyances. The Interface is clunky - sometimes I hate consoles for how they make developers forget about using mouses and keyboards. The maps contain zero detail defining elevation or terrain type. The maps are way too small. That horrible target reticle is way too big.

Other than the above I love Oblivion by the way. :-) It’s a great game - but its the later levels of your character that seem to go pear shaped.

Now to the Modders part. Since Bethesda changed things like the above, some Modders have quickly gotten around to changing them to something else more in keeping with the RPG spirit of things. I’m just going to list four Mods I’m currently using at the moment (the rest are small things that add convenience and hardly worth mentioning). The second is an absolute must have to fix the clunky UI in some limited way.

Auryn’s Levelling Mod:

This Mod imposes a new Levelling system which offers real time calculation of Attributes. If you needed before now to concentrate efforts to obtain 5x Attribute multipliers on level up, then stick this in and worry no more. In fact you can quit worrying about leveling at all - there are now no leveling screens to click through. It also slows down levelling rates a little bit (not much - other mods can slow it down far more by directly changing skill increase rates and such). Comes in two varieties - Level according to Skill increases within your Major Skillset, or Specialised Skillset.


This is a big improvement. It’s a UI mod which imposes a smaller Font on text in the Inventory, Skills, Magic, etc interfaces. It’s such a huge difference to be able to view three times the number of items on a single screen with this done. It also increases the map size (it’s stupidly small by default). Also contains optional Mod files to remove the display of certain subsets of Compass markers. For those with a real need for orienteering across Cyrodil.

Iyachtu’s Monster/NPC Scaling:

Simply put - it imposes a number of Level caps on NPC’s, Monsters, etc. The idea is that the more you progress in the game, the more challenging the monsters will become, but the less likely ALL monsters will match your Level. It’s pretty specific in what it does, and most caps will not emerge until a level of above 15-20. Some caps are static - meaning advanced NPC’s will actually be powerful, not Level 1 wimps you can rub out at level 1. It may sound doubtful, but it certainly sounds better than being able to finish main quests at Level 1 by becoming a weird Breton with severe insomnia…;-)Tom Servo’s Rare Items:

Items are the last problem - why should Ebony and Glass items be so hugely common at all? Where’s the fun in locating that full set of Glass Armour and Weaponry and stealing it? Where’s Morrowinds classic habit of hiding items in hard to reach, or even find, places? The Rare Items mod removes such items from common Monsters and NPC’s and leaves them (as a rarity) on higher class monsters such as the Minotaurs, Dremora Lords, etc. Since they are rare, monsters and NPC’s will be less likely to have them available - so it also ensures that as you level reaching for that Difficulty slider is not necessary, and there is some sense of achievement when you finally get a full set of rare items.

Auryn’s mod requires you to create a new Character (there’s are alternative player levelling mods but I like this one’s sense of balance and skill/attribute capping). The rest are simple drop ins.

ORM in Quantum Star SE Evolved

I’ve gotten (over my stint offline) a few enquiries about the Data Access model used in Quantum Star SE. Not everyone has come across the concept before, so I decided to take a stab at explaining it, and why I implemented it in the fashion used by QSE (or more accurately its backend, Partholan). This is just an overview - no real details. You can explore the concept better using Google and other sources.

ORM: Object Relational Mapping. ORM is (in this context) a method for mapping Objects to Database Rows. The simplest example is a db_user database table with three fields: user_name, user_id and email_address. At this point we have data fields and data - but how do we represent/use these in a PHP application?

The simplest approach is a direct SQL call, using the PHP functions to query the database and loading the resulting data into an array. The problem is that an array is still a variable - you need to somehow inject it into functions to be operated on, then you need more functions to change, update or delete the data - not to mention create new row records. At this point the inevitable conclusion is hard coded SQL, based on data arrays which are given a GLOBAL scope for use in functions (just might be all functions allow them as parameters). This is the de-facto standard practice in the majority of PHP applications. I’m not saying it’s a bad practice as such - it’s certainly fast and inherently insecure only for using GLOBALs - but it’s not re-useable nor easily maintained. Why? Because SQL queries may be replicated across many files - a bug in one necessitates fixes to many. There are other maintenance pitfalls with the approach - but that’s not the topic of this entry!

However if you progress to using Object Oriented Programming - you learn a valuable lesson. GLOBALs are a necessary evil, but an evil nonetheless… It’s at this point developers might start thinking in new directions. If we have user data, perhaps we should have User objects? Suddenly the OOP approach clicks home…;-). On a side note - I’m not a OOP “Purist”. If you are procedural minded, feel free to stay with that approach - OOP is however a very powerful approach to programming, and not half as complex as people think.

It’s here that ORM comes into play. If you intend populating an object with data from a database you have several options:

1. Build SQL queries into object methods (internal handling)
2. Pass data arrays into the object at construction or with a separate import() style method (external handling)
3. Give the object responsibility for managing data (hands-off handling)

The first two, again, are common. They are most intuitive when you have limited exposure to OOP (and even then it often just makes sense). The third has a bad reputation for being needlessly complex and over designed - untrue if all you look at are Propel style libraries in the context of simple web applications. Now all are just as valid as the other - but the third neatly sidesteps using manually coded SQL. It also “boils down” the functionality into a standard, predictable set of methods. A classic example common to nearly all ORM solutions.

We have a user (#3) for whom we need to reset an email address. Using the ORM model one could (in their PHP action) code this using:

$user = new User();
$user->setEmailAddress(‘[email protected]’);

Our User object has suddenly been transformed into a powerful unit for managing data. No SQL, no acrobatics, no blocks of code calling database functions (or database abstraction layers), no error management - nothing but 4 lines of code that do it all for you.

Now if that sounds magically fantastic I suggest reading up on ORM elsewhere (sorry, but I’m a truly bad technical writer and I’ll just ramble across the topic without digging into minutae). All this does come with a cost - which differs depending on the ORM approach taken. Since we rely on methods generated standardised SQL in place of custom optimised SQL, there is a performance hit. No JOINS, UNIONS, etc have that effect. So does not being able to limit the range of data from a row that is requested - this all can be solved by the way, but my home brewed ORM was built for simplicity, not speed. See ORM solutions like Propel for something far more complete (and therefore complex).

I think the above serves as a very non-detailed intro to the workings of ORM - if not a full technical debate over its merits. Now on to the implementation of the version in QSE.

To implement ORM you need a few building blocks as basics:

1. Database Table details and metainformation
2. An SQL Generator (and optionally an Optimiser - never added one)
3. A database abstraction layer (to enable rapid switching between database types)

Many fast and easy implementations of a dead simple ORM class like ActiveRecord omit all but number 2 - which is usually built in. This is a direction that personally I view as dangerous. One ActiveRecord implementation I saw did the following (most follow it closely):

1. Request Table details from MySQL
2. Generate a domain class from this metainfo specific to the Table structure
3. Instantiate the generated class to an object

That makes sense (it works for Ruby on Rails afterall ;-)) but it imposes a cost - all that extra querying and class generation is done at runtime, for every request a server handles. Now personally I prefer to take the pre-generate option. If my database tables are going to remain static (I don’t see many PHP applications changing table structure every few minutes) I will pre-generate all those domain objects and store them for use in the application before its released/published as Transfer Objects. Why not skip all three of those costly steps?

This still does not tell anyone where the SQL comes from. Since I aimed for simplicity, I designed a small class which generates SQL depending on the structure of the table - most of the SQL follow the simplest route for CRUD operations (Create, Read, Update, Delete). This Data Access object in combination with a Transfer Object enables a very simple ORM layer without the frills and thrills of something like Propel, but without the performance hit of RoR’s ActiveRecord. Is it feature full? No. Is it simple? Yes. Objectives met.

The last piece of the puzzle is using ADOdb-Lite as a database abstraction layer in place of where PHP’s native library functions (mysql_query() and such) would be. ADOdb-Lite simplifies other aspects of SQL generation a great deal - making even the all powerful Data Access class incredibly simple.

Basic operation: The user instantiates the selected Transfer Object (e.g. User). This object includes a set of CRUD methods which delegate SQL query operations to the Data Access object. The DAO generates the required SQL, and executes via the ADOdb-Lite database abstraction library. The sequence of method calls allows the return of any resulting data, usually a boolean false on failure, or some other value (an array for row data for example) back to the parent Transfer Object which stores such data internally. The stored data is accessible using a set of getters/setters which are based in individual names (e.g. setUserName() maps to the user_name field).

There ends my overview. Maybe it sheds a little light of what I was up to in QS, then again maybe not. If anyone needs further clarification, well, you all know my email address from the forums. :-)