File functions, Mods and the Patch Project
Thursday, 28 April 2011 Written by DOCa Cola
With Fleet Operations steadily growing over the years in complexity, we as developers and you as players more and more feel the age of Armada 2's underlying structure. This gets especially obvious if you want to play a multiplayer game.
Everyone who played Fleet Operations online or in LAN will know what he has to do first before he or she can do that - get a nice cup of coffee while the asset crc check is running.
This awful but necessary process takes nearly about a minute to see, if all your game files are valid to play error free with others online. This is not really Armada's fault. In stock Armada 2 this whole process took no longer than a few seconds. The blame lies with the growing complexity of Fleet Operations. Where stock Armada 2 had about 1000 object definition files (ODF) with only a few megabytes of size, Fleet Operations has over 9000 ODF files with about 120MB of size.
Now, one possibility to improve that is by removing ship ranks or the borg module system again, which i am sure you all would find fantastic The other involves packing up all the ODF files into one big file, similar to a Zip file. The main reason for the slugginess lies in reading 9000 small files. Reading one big file is a lot faster than reading many small files. While Armada 1 used that approach for it's stock files (though there was no possibility for us modders to use that system) and Armada 2's half brother Star Trek Legacy later too, Armada 2 itself natively has no option to pack up the files for optimized reading.
Such a archive system is exactly what we will introduce with one of the next versions of Fleet Operations.
Now the obvious choice for such a format would be using a Zip file. However we wanted a format that is highly optimized for reading speed and allows full extendibility for future features we might introduce. Unfortunately a file format fitting these requirements wasn't available, therefore we decided to create our own. We will provide tools to open, edit and create this new ODF archive format, so that all modders can profit from the very same enhancements we are using. Still, for modders usage of that format is still completely optional, ODF files can still be loaded regularly from the ODF directory.
For that new archive format to work in Armada 2, extensive changes to Armada's 'file system' were necessary. With file system i mean functions that index, read and cache files from the hard disk. This whole file system has been completely rewritten for this new ODF archive. Where the old system showed first signs of degrading performance with about 1000 files, the new file functions can now automatically adapt to any number of files without performance penalties. This was especially important for another feature this new 'file system' has.
But first some more numbers. The new file functions combined with the ODF archive format improves the performance of the Multiplayer Asset CRC checking 'benchmark' from about 70 seconds to roughly 2 to 5 seconds. Overall game load speed increases by about 25%.
Now about that other new feature the updates to the file functions give us - Though Fleet Operations is our main project, we always tried to contribute our achievements in terms of moddability of Star Trek Armada II back to the modding community. We promised that in the very early days of Fleet Operations and kept that promise with the later release of the so-called Patch Project. For those who haven't heard of it, the PP is basically an unofficial patch for Star Trek Armada II which fixes various bugs and provides interesting new options for modding. We released the first version of the Patch Project to the public way before we made use of it's features Fleet Operations. Later, after 5 versions it became increasingly harder to maintain new releases as Fleet Operations development intensified and many hardcoded features of Fleet Operations made it impossible to provide a unified code base for both the Patch Project and Fleet Operations.
From Fleet Operations version 3.0.0 and forward these hard coded features were step for step build back into options which can be configured through the config files. In combination with the rewritten file system functions, this provides us with a interesting new feature: Improved support for mods.
Modders will have the possibility to create modifications that players can install into Fleet Operations without breaking the installation's online compatibility or other mods installed. Each mod will be installed into it's own subdirectory. With this approach there is no need to have an extra installation for every single mod you want to play. Mods can be either stand alone or based on other mods. Stand alone modifications profit from all extensions and enhancements Fleet Operations has to offer to the Star Trek Armada II engine. Sub-Mods only contain the files different from the parent-mod. That means that if someone wants to create a sub-mod based on Fleet Operations where the Sovereign has changed weapon stats, it would be only necessary to include the changed weapon ODF file into the sub-mod. All other files like textures, sounds and all unmodified ODF files from Fleet Operations don't have to be in the directory of the sub-mod but are loaded from the existing installation. It is also possible to create sub-mods for other stand alone conversions but Fleet Operations.
All mods are loaded non-intrusively without modifying or moving a single file.
The mod system will be the successor of the Patch Project. We are thinking about providing an official "Classic" mod with stock Armada 2 gameplay and Fleet Operations graphics.
Now, the advantages of that system should be quite obvious for both, players and modders. Players can switch quickly between mods either to play a FO online match or to play their favourite singleplayer skirmish mod locally. No multiple installations with lots of redundant files are necessary anymore if you want to keep a mod on your system.
Modders aren't required to let the user follow complicated pre-installation instructions. They only need to deploy files important to their mod. Let that be a total conversion or sub-mod.
It should be quite easy to convert existing mods to that new system. I've had no problems to create a 'total conversion' out of my stock Armada II installation. Only a few changes were necessary.
You see, the new file system has a lot of benefits. Fleet Operations gets improved loading times and full mod support. When everything is running smooth, we will release these features soon too you.