Annotated scrawl from the wall of the audio hall...

Quotes from the industry's best with annotation and extended comments from the author...

Contributors:

Dave Albert, Alexander Brandon, Brian Coburn, Spencer Critchley, Donald S. Griffin, Chris Grigg, James Grunke, Tom Hays, Kurt Heiden, Rob Hubbard, David Javelosa, Kevin Russell, Andrew Schlessinger, Brain Schmidt, Michelle Sorger, Heather Sowards, Dave Sparks, Fxtracks@AOL.com,

Editorial Comments:

Mark Miller

Harmonix Music Systems

mark@groupprocess.com

This paper started out as a 'top ten things...' paper. As I got into it, I realized that I should really get some input from my colleagues in the industry. So, I sent out a request to the mailing lists where audio folks tend to congregate. As it turns out, I opened the flood gates. I received many responses that in addition to venting some major spleen, contained a great deal of wisdom and experience that should be shared.

I then was faced with the dilemma of having to add a narrative thread to all of this information. The more I thought about it, the clearer the solution became. What I had here was really more like an 'inside the locker room' peak at how audio people think. If I imposed too much of a continuous narrative on to the content, much would be lost. So, I decide to leave things generally as they were (as quotes), to organize the quotes, and to add some of my own comments where appropriate. I will leave you to draw your own conclusions at to what should be done.

I also decided that, while the above was certainly fun and contained some really valuable ideas, I should provide a more directly useful take away. Many of the less funny and interesting comments were on the topic of process. On how what gets done by whom and when. Rather than list these comments out individually, I chose to make a chart. Consider it as a road map to the process of creating excellent game audio with a minimum of waste and stress. Since every project is different, all of the times are relative. I hope that this will actually be useful….

 

A Sample Schedule to Help with Planning :

The following schedule outlines the order in which one might schedule audio development for better efficiency:

Milestone and Responsible Party Begins Lasts Comments

Identify Audio Lead /

Producer

Concept stage, as early as practical Remaining duration of project This is most likely an audio person, but could be a producer or designer from the team. All critical decisions will be made by this individual.

Define high concept of music and effects /

Audio Lead

End of Concept stage May change during design stage, but should be locked in by end of design The main task of the Audio Lead is to make this decision and carry it through as a single, coherent vision until the product ships.

Create the Technical Audio Design /

Audio Lead or more technically experienced person (if Audio Lead is primarily creative)

Midway through product design cycle Should be completed by end of product design cycle, but remain a living document (be updated rigorously until GM.)

The technical audio design should be created by the team member with the most experience in the creation and implementation of audio for video games

The TAD should include:

Complete lists of in-game audio files including:

File types

budgeted sizes

The naming convention

Overall playback methodology

Schedule

Assignments of responsibility to team members.

Degree of interactivity and methods/tools required to achieve same.

Audio requirements for cinematics

Obtain samples of the music and SFX from potential composers and sound effects creators. /

Audio Lead

Towards the end of the product design schedule Should be completed by the end of the product design schedule

It is critically important that the samples you receive are on the target platform and in the target format or at very least in some well constrained simulation of said.

If the music is to be interactive, make sure that your chosen composer can handle this (or at least understand it) or identify the person who will help them create assets in the required format

If there is voice over, obtain writing samples for the script. /

Producer or Designer

Towards the end of the product design schedule Should be completed by the end of the product design schedule

If the plan is for a producer or designer to be the writer, this is a sanity check. If it is an outside contractor, this will insure that they can cut it.

For outside contractors, check on union/contract issues. Professional writers are not always used to the cheap work for hire agreements that we favor….

Identify and qualify resources for audio post production of your cinematics /

Audio Lead and Producer

Beginning of production cycle Beginning of production cycle

These will often be different people from the in-game crew. The skill sets and esthetics can be quite different as can the equipment and facilities required to do this work.

The cost of such work is often different (usually more expensive) as well.

The mixing process is often quite different… More on this later.

Lock down your choice of both in-game and cinematic Composers, Sound Effects Designers, and Writers for both in-game and cinematic elements.

Identify the programmer responsible for implementation and tools development, if any. /

Audio Lead and Producer

Beginning of production cycle Beginning of production cycle

Get these choices made and make sure that the team is comfortable with the talent level and chemistry.

Get contracts in place and some up front money paid. This will secure your resources for when you need them.

If there is voice over, decide on whether you plan to go Union or Non-Union /

Producer

Early in product development A good three months before you plan to record…

This is a complex issue. Union talent is generally better, but the contracts are not well sync’ed with video game economics or production methodologies. They also bind you to use Union talent only for some number of years. If you plan to use even one ‘name’ talent, get going on this early….

If you are going non-Union, find a REALLY good NU casting director and give them ample time to locate the talent you need.

Get demo tapes from professional voice actors. /

Audio Lead or Producer

Once you have decided the U or NU issue One month before you plan to record

It is best to have the talent read from the script samples you have created to qualify the writer. Generic demo tapes rarely show you if someone can play the role required.

A casting director is worth ten times the money you pay them by virtue of helping you do things right the first time.

Make sure actors are available when you need them and one month later for call backs.

Produce a small, temporary but representative set of in-game sound assets in the expected playback format and implement them as soon as possible /

Audio Lead

As early in production as possible As early in production as possible

Having some representative sounds in the game early on will ensure that your technical sound design assumptions are going to ‘play’ out in the actual game. (Can you really keep your programmer’s hands off of the CD for Redbook audio playback)

It will also test the waters with the creative team and uncover and weird dynamics or psychosis when there is still time to adjust

Define the creative approval cycle. /

Producer and Audio Lead

Early in production Early in production

There needs to be one, clear, consistent vision form start to end.

It is VERY important to have a single source of feedback and creative input to the musician and sound designer. It is also VERY important to clearly define who has creative control and sign off and STICK TO IT!.

Define the process by which other team members get to have input. Make sure that it does not provide conflicting information. It is usually best to funnel this through the person with sign off and let them sort it out first.

BTW, don’t pretend to give the composer creative freedom ‘to see if something cool happens’ and then take it back and enforce you own creative vision if you don’t like the results. If you want to experiment with creative control, be clear that that is what you are doing….

Compose the in-game music /

Composer and Audio Lead

Early in production Roughly Beta Always listen to the work on a delivery system as close to the final payback system as possible. There is nothing worse that the disappointment of a highly resource limited conversion of a Redbook demo…

Implement the in-game music /

Audio Programmer, Audio Lead and Composer

As soon as it is practical, but no later than two weeks or so after it is delivered GM

The only way to know if something REALLY works is to get it in the game and have people test it out.

The longer you wait to make your decisions and comments the farther away from the original inspiration (and source data organization) you get. The longer you wait, the harder it will be to make changes.

Interactivity presents its own set of implementation challenges. Make sure that the programmers or designers are ready to implement whatever communication protocols that are required to make the interactivity work.

Budget for a few iterations of both the content and the implementation to make sure that everything works well in context.

Strongly consider a tool that allows the composer to test the interactive elements in a good simulation of what the game environment is going to look like. This will save programmer cycles and yield a highly refined result

Review the in-game sound effects design /

Audio Lead (or/and whomever created the TAD)

Once some levels begin to feel stable and reach at least 80% implementation GM

Shit happens. Enemies get cut, new ones are created, mechanics get tuned and redesigned.

Tune up this document before you go into SFX production.

Create the in-game SFX /

SFX Designer and Audio Lead

Once the level closest to completion hits about 90% stability Beta SFX should be created based upon animated and implemented elements. Anything else is simply guess work.

Implement SFX /

Audio Programmer, Audio Lead, and SFX Designer

As soon as humanly possible Beta

The only way to know if something REALLY works is to get it in the game and have people test it out.

Strongly consider creating a tool that allows the SFX designer to implement their own effects. It will save a massive amount of programming time and yield much more refined results

Finalize your script, casting choices, Union agreements, if any, contracts and availability of talent /

Producer, Designer and Audio Lead

As early as you can be sure that the script is not going to have massive changes made to it. No later than Alpha in any case. Alpha, at the latest

A good casting director is the best choice for assisting the producer in these tasks.

Insist on a sensible and consistent naming scheme for the lines of dialog. Trust me on this one.

Book a recording studio /

Audio Lead or Producer

As soon as the above item is complete Alpha at the latest (unless dialog is needed for lip sync’ing in the cinematics or in-game implementation)

Make sure that you schedule a second session in the SAME room for pick ups and revisions sometime around Beta.

I strongly recommend hiring a director to interface with the talent. The producer or designer can and should ‘produce’ the session at a high level, but let a professional deal with the talent. Everyone will benefit.

If there is lip sync’ing in the cinematics or in-game art. Make sure that the dialog is recorded in ample time for the animators to work with it. It is MUCH harder to record dialog to existing animation, that the other way around. (See any badly dubbed movie for examples…)

Record the dialog /

Audio Lead, Producer, Recording Engineer, Designer

On the scheduled date Alpha (See above note on lip sync’ing)

Go with a professional room with good mics and a talented engineer who understands the final output format.

KEEP THE SCRIPT UP TO DATE with any changes that you make in the session.

Log the good takes both on tape <slating> and electronically <ID>

Run a real time DAT backup. You'll thank me someday.

Don't look at the talent when they are performing. Looks can be deceiving. Look at a picture of the character instead.

Go back and listen to your keepers before you let the talent go.

Edit Dialog /

Audio Lead and Dialog Editor

As soon as the recording session is completed Depending on the amount of dialog…. Dialog recording sessions are like motion capture sessions: turning an actor’s output into usable elements can take a -lot- of cleanup work.

Create the audio assets for your cinematics /

Cinematics Composer and SFX designer, Audio Lead, Producer, Designer

As soon as the cinematics are complete Alpha

It is very important to have coordination and communication between the in-game and cinematic creative types. Where ever there are in-game elements that appear in the cinematics, the sounds should be very consistent. This is equally true of the music.

Process-wise, video tapes with SMPTE time code of the cinematics must be created and sent to the composers and SFX designers who are working on this part of the project.

They will generally work in their own environment and create ‘stems’ or sub mixes of their elements. These stems will then be brought together for a ‘mixing’ session where all elements are balanced and blended to the picture.

If you are planning on doing some kind of surround sound, this is when it will happen. Do your research. This is not a trivial or last minute process….

It is often necessary to have the various parties bring along their source assets (and the equipment required to re-assemble them) in order to resolve conflicts during the mix.

The end result of this session should be a mix in the required format for the product as well as ‘stems’ or separate mixes of the music, SFX and dialog (with time code or minimally sync pops) for re-synchronization. This is CRITICAL for localization and ports.

Review the implemented audio /

Audio Lead, Producer, Designer, Sound Programmer

Alpha Beta

In ideal circumstances, at least 80% of your audio should be implemented by Alpha.

Budget time and personnel to carefully review all of the audio that is implemented in the game.

Take careful note of how the different elements interact with each other. Make sure that all of the volume levels and intensity levels make sense when people actually play the game.

Make sure that the cinematics and in-game audio flow smoothly back and forth.

Test on a variety of playback systems like headphones, cheap PC speakers, and super high end home stereo speakers.

Have your ‘call back’ dialog recording session. Edit the new material. Re-integrate it into the product. /

Producer, Designer, Audio Lead, Recording Engineer, Dialog Editor, Sound Programmer

Just before Beta Beta

You should have scheduled this session for safety’s sake when you scheduled you original dialog recording session. There is almost always something that needs to be re-recorded, modified or improved. If you plan ahead, it will be much cheaper and less stressful.

Make sure that the Dialog editor and Audio programmer have reserved time to do their parts.

Integrate any revised or last minute changes to the audio design /

Sound Programmer, Audio Lead, Composer, SFX Designer

Beta or thereabouts…. As soon thereafter as possible.. This is where those tools that allow the audio folks to implement their own work really pay off. No one else will have the time and quality can really suffer if these refinements are not integrated.

Mix /

Audio Lead and required technical assistants

As soon as all final elements are in place As soon thereafter as possible.. This is a critical step in which the volumes of all audio elements are balanced and smoothed out. It is a bit of a balancing act to get this done on time, but it will add substantially to the perceived quality of the overall product. Think of the otherwise great game that had some really annoying sound effect that was too loud or played too often. Sucked, didn’t it? Could have been fixed…

Give all of the audio people appropriate recognition in the in-game and manual credits /

Producer

Whenever you get around to doing the credits… GM See the included list of Audio jobs and titles. This should make your job easier and make the audio people very happy. One global credit for ‘Sound’ is not appropriate or sufficient in this word of evolving craftsmanship.

Thank your audio team. /

Producer and anyone else ‘in charge’

    They worked as hard or harder that the rest of the team as most of their work had to come after someone else was finished. Also, audio people are often over looked or forgotten at the end of a title, especially if they are outside contractors. A small show of gratitude or simple thanks will earn you great respect and some capital to spend next time around….

Send me my consulting fee for helping you look really smart and make get everything audio done on time, on budget and with a minimum of stress. /

Anyone who uses this to further their own career…

    Just kidding….

General Graffiti:

On Managing Audio People:

"Anyone who has to constantly try to convince coworkers that his profession is challenging, worthwhile, and requires basic professional tools eventually gets burned out and depressed and either does a crap job (present company excluded) or changes fields. Oh, wait a second, that's not funny. Forget it."

- A discussion on the general psyche of the modern audio person in the field of video games....

On Planning and General Advice:

"Prior Planning Prevents Piss Poor Performance aka the 6 P's...."

- Bitter words to spit into the face of a crisis (or the unlucky person whose lack of foresight can be proven to be responsible for said)

On Timing:

"Begin the design (audio) more than 2 weeks before the project is due!"

- The classic mistake of questionable intentions where you get to pay huge amounts of money for highly compromised results....

"It's possible to involve the audio personnel into the development of a game too late, but it is also possible to get them involved too early. Having musicians & audio designers create tons and tons of sound effects and music before there is a clear vision of the game can be disastrous, not to mention expensive."

- The classic mistake of ambitious and overly good intentions where you can still salvage good results but waste a huge amount of money and good will.

"Don't schedule assets and integration to be completed on the same day." "Allow calendar time in the schedule to review the sounds in context (with real players) for a while, and then get one or two rounds of revisions in. A sound design concept takes calendar time to ripen, and it can take some hours to smooth rough edges in the first implementations."

- The French cheese approach which is closely related to...

"How does it feel have a bucket on your head an be pounded with a mallet for 3

hours?

- What happens if your sound effects or music are all too powerful, like when you only listen to them by themselves during the selection process."

"Avoid build processes and APIs that make the sound artist go through engineers to make sound-only changes."

- Put the power to refine into the hands of those who really, really care.

This is related to 'Miller's Law' which states:

"The likelihood of advanced audio features being actually implemented in a game is inversely related to the number of lines of code that anyone other than the audio person need to write...."

On Resource Allocation:

"What happens to 400 megabytes of audio when you discover you screwed up and there is only going to be 20 megabytes left on the CD? What happens to 400 megabytes redbook or wav music when you discover you CAN'T leave the CD alone after all and you have to change it all to MIDI in one week?"

and...

"Yes, that's the way it sounds when graphics interrupt the audio stream. But, if you blank/fade/cut out the graphics for a second , the audio will continue quite nicely."

- And other amusing stories of bad resource allocation planning...

On Who Does What and When:

A Story:

"For me, this calls to mind an experience I had once at when a game designer asked me to proofread his proposed credit roll, and "did I have any ideas?". So I proposed back to him with a credit sheet very much like the one Tom proposed. I had a fellow working for my dept at the time-he and I worked separate audio tasks for that same project; and I thought descriptive credits were in order.

Man, you should have seen the expression of bewilderment on his face ,..."What!?..." he said to me,"...You guys do SOUND!...the credits will have your names under the heading, 'Sound'; so all I'm really asking you to do is tell me if I have spelled your names correctly!..."

He was really put off that my ego was so large that I would request acknowledgements in any other way. So not only did my assistant and I fall under Sound in the credit roll, so did the programmer who was assigned the job of writing the various sound card drivers!"

On Roles and Definitions:

A list in progress from Tom Hays (with help from Chris Grigg and Kevin Russell):

Audio Lead: The person on a project responsible for making sure the soundtrack as a whole is great (see "sound designer"...)

Sound Effects Designer: creates, edits and otherwise deals with sound effects. This is distinct from...

Sound Effects Editor: person who edits and manages sound effects, using other peoples' design work and/or straight pulls from libraries.

Audio Director: head of an audio department; an administrative role.

Composer: a person who writes music.

Performer: a person who performs music.

Composer, performer, recordist: person who creates music from whole cloth and delivers it ready to go ("Music composed, performed, and recorded by...")

Mixer: person who does final audio balancing between different elements, especially for linear video segments (we could use more of these in the game business).

Dialog editor: Good dialog editing is an under-appreciated thing. This credit should imply some responsibility for whether the dialog sounds any good, not just the ability to cut 1,200 lines into individual files in one week.

Sound Programmer: The programmer who actually places the sounds (including music and dialog) into the game. "No matter how good your sound designer, without a good implementation (i.e. programmer), your game audio will s**k."

Also…

Assistant, intern, sound effects recordist, additional audio by, Foley performer, Foley recordist, ADR editor, ADR/dialog recording engineer, first assistant sound editor, sound interactivist, sound librarian, sound department tool developer…

 

On Technology:

"A great video game sound designer/composer has mastered the art of cheating."

- The trick to a great video game sound design is to fool people into thinking that they are hearing more than the machine is capable of creating....

"Stop saying, just add more bass, when the fx are playing back out of your tiny laptop speakers, because they are capable of reproducing nothing more bassy than a hamsters fart."

- Or the importance of auditioning sounds on appropriate hardware....

"MIDI is not a four letter word"

"MIDI is a control protocol, not a sound. ("What you hate is the sample set, not the MIDI.")"

" MIDI Sucks - that's why were are ONLY going to use DLS from now on!"

"Microsoft says that we all should stop using MIDI and only use Direct Music from now on."

- Don't kill the messenger

"The reality is you can not compromise the quality of the game, graphics or frame rate for audio, because if you do we all loose. Instead we fight very hard to make use of all the available resources we do get, and every year the quality does get better. We are making lots of progress, and in my view the audio is a lot closer to reality than the graphics, so why do we need more resources ?"

- See, we are not all just malcontents and complainers....

"Don't put too many eggs in the 3D audio basket, game-design-wise. (Y'know, with only two speakers, only the .helicopters. can go behind you...)"

"Latency .really. matters; a button click sfx heard 200 mS after you click the mouse isn't a button click anymore. (200 mS isn't a delay, it's a postponement!!)"

"When all games are using the same sample set you are, there's a limit to how much better you can make your game sound."

- The value of creating custom samples in addition to or instead of General MIDI

Happily Ever AFTRA (or some comments on voice overs...):

"Voice overs and Narration should be done by voice talent. Sending over "your buddy" in marketing because you think he has "a good voice" generally will not cut it."

"Just because the sound department owns a microphone does not mean they have all the equipment they need to do great voice overs"

"There's a good reason TV shows don't use cameramen for cartoon voices...why do you want to??????"

- Voice Over is a profession, not a hobby. Hire professionals and the results will be clear as day....

On Communication:

"After another good demonstration - I was asked why our sound set sounded so good. I then proceed to explain about our engine, and that our samples were all at 44k. One of the gents perked up and said "Wow, that's a lot of money" - when I told him that I didn't understand what he was talking about he said "Well, you, just said, your samples cost 44 thousand..." he stopped mid sentence, looked around and said "Ahhhh, never mind."

- It really helps to obtain a basic knowledge of the technology for which you are producing content....

"Don't forget to ask the sound people what would work best"

- You never know, people who have devoted their entire professional career to creating audio might add some value to the planning process

" Don't be an asshole"

- You might be surprised at how often this little gem is over looked....

"When the programmer says that what the sound guy wants is impossible, the programmer is frequently, er, lying. (How can you tell if the programmer's lying to the sound guy? His lips are moving!!!)"

- Most programmers HATE sound. They will lie, hide and pretend to be sick to avoid even the simplest audio related tasks. Keep this in mind when the audio folks or the designers want to do something cool....

"Yours is the only project I'm working on...really" is right up there with "I've got a bridge to sell you... So is, "my N64 music will sound just like this CD demo I did in my home studio"

- Speaking of lying....

"It sounds like it is in the key of C, and it should be in the key of G" A producer who I know for a fact knew less about music theory than he did about the latest developments in particle theory…

"Tell me the emotion you want to create, not how to write the drum part."

- Spouting nonsense instead of saying "I don't know why I don't like it" is not a big win....

"Listening to a lot of records isn't the same as knowing how to make one. "

- But it really couldn't hurt in developing some basic familiarity....

On Money:

"Quality costs money. If you're doing it cheap you're either not getting quality or you're ripping someone off."

"The composer with the lowest quote is probably worth every penny"

- I know of very few instances where you get more than what you pay for, although this is not true for the reverse...

"The more expensive the mixing console, the better the music."

- Don't be fooled by expensive studios....

"...we require a 'sound' (literal et. al.) budget that reflects not only the need for resources to get the job done but also included in that monies for continued education and attendance at conferences. We seem to somehow get lost in the shuffle unless we continually squeak. In the future I'd like to be able to concentrate more on good audio than appropriate squeaking!"

"If it's a new platform we'll need new equipment."

"If it's a new year we'll need new equipment. (Plan us into your budgets)"

"You'll get better sound for the same money if you amortize audio department infrastructure across multiple titles rather than expensing against individual projects."

"Budget realistically for dialog editing!!"

"Budget realistically for custom samples!!"

- Don't be pennywise and pound foolish....

"What do you call a musician that just broke up with his girlfriend? ....... Homeless!!!"

"Also, make sure their check arrives late because it will add that anxious edge to the gameplay feel of the music."

- In everything that is funny, there must be a bit of sad truth...

On Content:

"People don't really like music, they just expect it..."

"It's not the music that is important - just the rhythm!"

- wow...

"Don't think that just because you like White Zombie it should play incessantly in your game"

"Don't be influenced by flashes of marketing wisdom such as "let's have 'Wu Tang Clan' do the next "Final Fantasy" sequel!"

"What happens to a three minute radio-format song when you hear it looped for three hours?"

- The importance of recognizing the difference between what you like and what works....

"No, No, the music is too memorable..... Take out that melody bit.... Oh, hang on and the harmony bit..... Fuck it, just leave the bass and drums. We don't want the music getting in the way of the game now do we mister games musician"

- Or the opposite....

"Well, at the end of the day, their just games musicians, they just do those annoying bleepy bleep tunes, we need a proper musician"

"Who is going to hold the hand of the expensive rock star? (Who is going to do the work when you discover he can't or won't?)"

"Beware of "Guitar store clerk by day...game music composer by night"

Corollary to above: Beware those who do want to do game music "just until my

real music career takes off"

- Some incentives for working with professionals with good track records....


Adaptive Audio Now

The AAN website is the place to go for tips, tricks, and insights about creating adaptive audio scores for your game or application.
www.adaptiveaudio.org

Project Bar-B-Q

Project Bar-B-Q is a Texas-style think tank that strives to influence the state of audio technology for the next five years. Registration is now open for Project Bar-B-Q 2003, October 16-19.
www.projectbarbq.com

GANG

The Game Audio Network Guild empowers its members by establishing resources for education, business, technical issues, community, publicity and recognition.
www.audiogang.org