Mark Danks

In game development, a lot of time is spent processing assets to become game ready.  Often, the asset build system is a couple of python or perl scripts that are thrown together without thinking too much about how they will really be used.  This article isn’t going to show you how to build a scalable build system, but it might give you something to think about if you do make one.

Asset Build Systems

In many asset build systems, engineers will put in a lot of effort and code to decide if something should be rebuilt.  Since rebuilding an asset can take a fair amount of time, with a lot of assets, you want to minimize this.  A typical example is a character.  A character is usually a collection of:

• mesh
• texture
• shaders
• animations
• AI
• etc

When a content creator adds a character to a level or wants to see some change, they usually say something like “build the character.”  The build system then looks at some definition file to determine which Maya file is used…which is then opened to decide what textures and shaders are needed…which then looks at timestamps to decide if the textures need to be rebuilt.  For a character, this isn’t too bad.  For an entire level, doing all of these dependency checks can grind your iteration time to a halt for even the smallest changes.

References vs Dependencies

A character may be defined by all of those files, but they aren’t actual dependencies for the build system.  For example, if you change the mesh, there is no reason to reprocess the texture.  And if the texture changes, the shader doesn’t have to rebuilt either.  They are references.  The character just has a list of things which it needs at runtime to construct itself, but those things are not interdependent.  If artists (or the tools which content creators use) can say “built the texture” instead of “build the character and go find out what is really changed,” then all of your iteration time is limited only by the processing time for the single asset, not by the build system infrastructure.

Because users do not need all of the content on their desktop (only what they are working on), you can also avoid the morning ritual when everyone syncs to Perforce/Alienbrain/whatever and then builds everything which has changed.

There are obviously some things which are true dependencies – a rig and animation data is one example.  However, those are the exception, not the rule.  If you treat level data as just a list of references and do not bother checking the assets, then your iteration times drop to almost nothing.

Problems

There are some problems with treating most assets as references instead of dependencies.  The biggest is that get all changes which have occurred in the middle of the day is difficult.  This system works better when there is a nightly build which processes everything and then distributes it to the team.  Producers who constantly want to see the latest and greatest get upset with this limitation, but the wins that the entire content team gets is well worth it.