A few publishers have figured out that the Wii market is different than the hardcore market from before, but that requires an understanding of a very different demographic than has bought Doom, Gears of War, and Call of Duty in the past.

Now that Microsoft’s Natal and Sony’s motion controller have been announced, the day for the joystick might seem to be limited.  However, not every player nor every gamer wants to jump around the room, regardless of the genre.

Press release

Interview at GamesIndustry.biz

One of the most obvious issues is knowing when the file has actually been loaded.  In an asynch environment, you issue a request, and some time later, the request is fulfilled.  Most systems return a handle which the game can check periodically to see if the data has been read.  This is fairly easy to implement, works well in a multithreaded environment, and is pretty easy to debug.  Other alternatives are to use a callback mechanism or put the calling thread to sleep until the data is loaded.  The sleep option is usually done in custom scripting languages, instead of trying to do it in C++.

A standard approach is to compress the data at build time and decompress at run time.  If you are reading from DVD or Blu-ray, then it is practically a requirement due to the slow read speeds of those media.  On the PS3 or XBox360, it is common to have your file system decompress on the fly, either on the SPUs or on another core.  The complication is that you are no longer loading the data directly where it will be used.  Instead, you need to load it a temporary buffer, decompress the data, then write it to the final memory location.

On some consoles, there is not unified memory.  This means that you need to decide either ahead of time or at load time where the data will be loaded.  On the PS3, half of the memory is “main RAM” while the other half in “graphics memory” (ignoring that data can be moved and read between the two systems).  A simplistic approach is to put the data in the RAM segment based on the type of data…geometry to RSX memory, AI parameters to system RAM.  Since some types of data can be in either segment, this approach limits how flexibility the memory arrangement is.

Most file systems use a Table of Contents (ToC) to manage where and how the data is loaded.  Having a ToC for the data which is loaded gives:

• Position on disc
• Memory segment to load into
• Compression parameters
• Data type
• Resource Id
• Human readable name

A ToC also gives you flexibility during development.  The ToC can either contain all of the data in a single packed file, or it can simply list the files which need to be loaded.  Assembling a packed file can take a lot of time during development, especially when you have just changed one file.  This arrangement also makes it easy to do single file overrides.  Your nightly build system creates packed files for the game from checked in assets, but individuals can modify single files which are loaded seperately from the main packed file.

If you can override individual files, then you should look into hot-reloading of data: reloading data in the middle of the game so that artists can see texture changes without restarting, etc.  On the prototype I’m working on, I have a PC app which monitors my processed file directory and sending a message over the network to the game to reload it.  I have the file change logic on the PC in a single app, while the hot reloading is based in the game itself.  Since I’m working on a multiplatform game, this makes it easy for any new platform to load in the file.  Even the PC version registers itself with the file monitor app instead of scanning the directories explicitly.

One minor point to mention is that DX9 is not fully multithreaded.  This means that if you are allocating vertex, index, or texture buffers, they all have to be done from the render thread.  Microsoft explains it fairly well in this presentation.  This restriction makes it a bit more complicated, but it mostly involves extra signaling between the loading and render thread.

As a note, these are only my views and do not represent my current employer, SCE, EA, or any past employers.  These views are also not based on any “inside” information, but instead are based on many years in the video games industry.

It is absolutely true that of the current consoles (PS3, XBox360, and Wii), the PS3 is the most complicated.  However, there are two aspects to developing on a console.  1) The hardware and 2) the development environment.  The thing that the author of the CNET article seemed to forget is that (excluding Nintendo), selling software is what makes the money, not selling the hardware.

The games industry makes most of its money from sequels and yearly franchises.  The Madden’s, the Guitar Hero’s, etc.  These publishers need to have consumers buy their games every year.  If there isn’t something which makes the game better each year, then why would consumers purchase it?  This concept extends to franchises such as Halo and God of War.  While I would love it if new stories and gameplay drove game sales, it is not not borne out by the sales numbers each month.

For the XBox360, there is a very easy to understand hardware and a great development environment.  This means that the games made today are not going to be much different from the XBox360 games of “tomorrow.”

For the PS3, it is a complicated hardware and historically is not a great development environment.  This means that games towards the end of the console will look much better than those at the start as developers learn how to take advantage of the hardware.  Keep in mind that currently, many hardcore consumers are looking for the better shiny object and realistically do not care about “story” and “gameplay innovation.”

Between the hardware and development environment, one of these can change over the course of a console’s lifecycle (and it isn’t the hardware…).  While the PS3 is still difficult to develop for, it has gotten much better over the past couple of years.  This means that developers are able to use more of the hardware and make their games better looking.  On the XBox360, how different is Gears of War 1 from Gears of War 2?  On the PS3, it will be interesting to compare Uncharted 1 to Uncharted 2 this next holiday season.

So put Kaz’s comments into perspective.   If the software looks the same every year, then why would consumers buy the next version of something?  If the software gets better every year, then consumers are likely to purchase it to be on the best and brightest.  This philosophy served Sony very well on the PS2.  The question is if it will work on the PS3…especially when it does not have the same market share.

Most game engines have gone away from trying to describe game objects with explicit C++ class hierarchies.  Very quickly, there is a base object called CObject which has everything but the kitchen sink in it.  This also creates an interconnected mess between the tick cycles for the object and all of the game systems.  It is very hard to parallelize the tick loop and impossible to move the objects off to extra processors such as the SPUs.

A number of years ago when I was refactoring an engine, I realized that game objects really do not fit into a hierarchy, but rather are a collection of behaviors or components.  This has become a fairly common way to deal with objects in game engines these days.

Possible components are:

• Render
• Animated
• Physics
• Tick
• Position
• Script
• AI
• HP and stats
• etc

A door object might consist of:

• Render -> Draw object
• Physics -> Collision calculations
• Position -> Location in world including rotation
• Script -> Interactions

An enemy character would have all of the components.  A non-interactive boulder might just have Render and Position.

The best situation is if the individual components are separable from other  component types and from the object itself.  For example, if all of the physics components are registered with a central manager, then the manager can “tick” all of the physics components without pulling in other data nor object dependencies.  This is an ideal situation for using the SPUs.

So far, this is all fairly standard engine design.

If an object is constructed from components, then how do you deal with temporary affects and behaviors?  For example, the player steps onto a burning floor and needs to take damage for some period of time while they are on fire.  This is not a component.  And you do not want to have the burning floor directly affect the player, especially since you do not know how many characters might walk through that floor space.

The additional part which I added was the concept of services.  Services tick every frame on an object and have a defined lifetime.  In the burning affect, the service would be a “take damage with fire pfx” which lasts for 10 seconds.  When the character walks through the burning floor, a burning service is attached to the character.  Every frame, the service is ticked.  It applies damage to the character and checks how long it has been in existence.  If its lifetime is up, then it is removed.  This does not require any involvement from the burning floor object once it has been attached to the character.

Another example is a buff.  A player casts a spell to increase their strength for a period of time.  The “spell” is a service which is applied to the player which increases the strength stat.  In the game, all of the spells are different variations on a service.  Ones which also change the rendering of the character are services which manipulate the rendering component in addition to the stats component.

Notice that in both the spell buff and the burning damage, the object does not have to have any special capabilities to use the service.  The service might depend on a component, but even that can be an optional (non-fatal) situation.  Adding new spells, new object affects, etc. can be done without needing to change the object hierarchy nor components.  Often, services can be defined in scripts and the C++ service is simply “run script service.”

Generally, services are very light weight computation or affects which do not require much processing time.  Because they are run directly with the object (and not batched up like the components and their managers), it is difficult to parallelize them.  Also, there tends to be more pointer chasing so DMA’ing them to the SPUs is problematic as well.

By creating a separation between components and services, the rule for deciding what something is becomes very simple.  If the behavior or affect starts when the object is created and only stops when the object is destroyed, it is a component.  If the behavior or affect only lasts for some period for time, then it is a service.

…but something came up…

I am starting a new game studio in San Francisco.

My work tends to be around making the SPUs easier for developers to use.  Software caching (like libswcache) are ways to deal with non-uniform memory systems and making the architectural complexities more transparent to the end programmer.  I have a lot of experience with C++, so I have been looking at how templates and auto_ptr’s can help.  Libraries like these do not give you the peak performance, but it makes it much easier for the general gameplay programmers to work on the SPUs.  I have found that game development is a constant trade off between easy to use architectures which any generalist programmer can code for and extremely high performance architectures which require a specialist.

I have used Lua on previous games and never been that happy with it.  It tends to run slowly and require more memory than you would expect.  Also, most people tend to use it during development by reading in text files and doing on demand parsing.  This scares me since I prefer doing type checking and variable checking at a compile time…one typo and suddenly your run-time can blow up.  In some games, I have made the memory allocator throw an assertion if someone tried to allocate a var, string, etc. that was not seen before.  This seems to stop the typo problem, assuming that the game (and testers) have coverage over every script.

There is always a trade off between writing your own scripting language and using a prebuilt one.  I think a lot has to do with how much time and customization you require.  At Stormfront Studios, level designers never wrote any scripts by hand, but used a visual editor which made machine generated scripts.  Since most of those games were level-based, adventure games, most of the game play was triggers.  This meant that if there was an error, we needed to know immediately.  It also meant that the games ran everything in script (back on the PS2) so performance and memory was serious issue.  On MySims, the “level designers” were engineers who wrote scripts all day.  They were comfortable with full blown programming languages and encoded the actual game play and mini-games into the scripts.

1. It is hard to debug the tasks when there are hundreds or thousands a frame
2. I want to emulate the PS3 SPU local store, which means segmenting my memory space

To make each of these easier to deal with, all of the task entry functions take an argument. This argument is a pointer to a ClTaskArg class. Tasks are expected to derive from this class to get their arguments into, and potentially out of, the task function.

Here is the basic flow chart for how a task gets executed in my app.

The decision points are mainly decided by the settings in the argument. This is different from how most PC or XBox360 apps would work, since they have unified memory. By creating a Local Store for the Task Function, it forces the app to think about how memory is handled and when to actually do a writeback. The nice thing about my system is that a fair amount of the memory handling is done automatically without having to explicitly do the memcpy() in the user’s code.

A simplified and incomplete version of my ClTaskArg follows. It mimics the PS3 by using EA – Effective Address. The implementation is missing, as well as a lot of the header itself, but it should give an idea of how I deal with the argument into the Task Entry Function.

class ClTaskArg
{
    public :

        //////////
        // Get the size of this structure
        int            getSizeOfStruct() const    { return(mcSizeOfStruct); }

        //////////
        // Get the write back address
        // [out] ClTaskArg * - the pointer in main RAM.  NULL if no write back
        ClTaskArg      *getThisEA() const         { return(mpThisEA); }

        //////////
        bool           breakOnEntry() const       { return(mBreakOnEntry); }

        //////////
        ClAlloc        *getLocalAllocator()       { return(mpLocalAlloc); }

    protected :

        //////////
        // Constructor
        // [in] sizeOfStruct - the full size of the structure to transfer
        // [in] writeBackStruct
        //        - should the structure be written back to main RAM on exit?
        ClTaskArg(int sizeOfStruct, bool writeBackStruct)
            : mpThisEA(0), mcSizeOfStruct(sizeOfStruct), mBreakOnEntry(false)
        {
            if (writeBackStruct)
            {
                mpThisEA = this;
            }
        }

    private :

        // Address of this structure in main RAM
        ClTaskArg    *mpThisEA;
        // The memory allocator for this core
        ClAlloc      *mpLocalAlloc;
        // The size of the structure
        const int    mcSizeOfStruct;
        // Should the thread break on entry?
        bool         mBreakOnEntry;
};

Going forward, it is critical that programmers understand how to deal with multiprocessor systems. While this book focuses on shared memory systems, the concepts are still applicable to the PS3 (which does not have a shared memory system due to the SPUs).