Plunging the Synch?

In the second part of his investigation into timescales and Machinima, Anthony Bailey delves into AV synchronisation issues in Machinima.

Previous article

Introduction

As investigated in the previous article, events in the world of each of the engines that we consider (Quake, Quake II, Quake III, Unreal, Lithtech 3D) are happen along a timeline. We saw that each engine allowed some control over the relationship between real playback time and this notion of world time. However, there could be limitations on both the content of this relationship and on its reliability, especially across different playback platforms.

The place that audio (such as sound effects and background music) occupies in this regime is the same in all the engines. All sounds are stored as audio samples and the start of a sound being played back is just one more event on the world's timeline. A machinima's soundtrack can thus be composed of individual sounds that are mixed by the game engine, although in practice often the producer may choose to provide a single (and separately mixed) audio soundtrack just as for a traditional film, which then plays back as a single audio event started at the beginning of the scene. Either way, once a sample starts playing back, it is to a large extent out of the jurisdiction of the game engine, and under the control of the platform's dedicated sound engine, which will be tied to the very precise system clock.

Thus the audio and video components of a machinima end up being played back by two separate engine subsystems which keep track of time independently, with the only points of guaranteed synchronisation being when a sound starts to be played. Since audio playback is normally under the control of the system clock, this is likely to be consistent across all platforms and circumstances, but as we saw previously the rate at which world time (and hence video footage) progresses is more malleable. It can be controlled by the creator (as examined in the previous article) or by the viewer (as we shall explore in this article) and under some circumstances, may not be consistent across all platforms (where as last time, we define a platform to be some particular instance of the engine; some version on some operating system, running on some hardware and playing back using a certain resolution and rendering options.)

Next: More...