Diana Walczak and Jeff Kleiser are known for their early experimental films featuring solo performances by digital human characters. "Nestor Sextone for President" premiered at SIGGRAPH in 1988. A year later, Kleiser and Walczak presented their first female Synthespian, Dozo, in the music video "Don't Touch Me."
"Don't Touch Me" |
Jeff Kleiser recounts the story of how he and Diana Walczak created their first digital human characters - and how they came up with the term Synthespian - in the August 2000 issue of Film & Video.
SYNTHESPIAN HISTORY, TECHNOLOGY AND FUTURE
It was 20 years ago that testing began in an effort to create a computer-generated character for a feature film. Digital Effects in New York had been founded two years prior by seven computer graphics enthusiasts and this was our first crude attempt at bringing a character to life using digital technology - for the Lucasfilm project "Interface." We began by shooting 35mm film of our art director outfitted in a skintight suit with ping-pong balls as reference markers attached to his joints. This footage was projected down onto the platen of an Oxberry animation stand and the pathways of the reference markers were traced onto pegged animation paper. These cells were then digitized on a tablet and the collected data became the two-dimensional basis for the motion of the character.
Five years later I met Diana Walczak, and subsequently we formed a partnership called Kleiser-Walczak Construction Company which was dedicated to exploring the creative applications of computer animation to human figure representation for entertainment projects. Diana was a classically trained artist with strong sculpting skills and our first synthetic actor, Nestor Sextone, was composed of individually sculpted body parts and faces. It was while we were writing Nestor's speech to an assembled group of "synthetic thespians" that we coined the term "Synthespian." At the time, there was no software available to us that would allow for flexed joints, so the body parts were designed to inter-penetrate in approximation of a smooth surface. Our longtime associate Larry Weinberg (who later wrote POSER) wrote some software that allowed us to link together digitized facial expressions from multiple sculptures created by Diana to define phonemes of speech and emotional states.
Recently we have employed new technology to provide digital actors for the New Line Cinema production of "Mortal Kombat Annihilation" and the Universal Studios theme park project "The Amazing Adventures of Spider-Man." For the Spider-Man project, motion capture was tested, but dropped in favor of keyframe animation. The major advances in software techniques that have enabled an ever increasing level of realism in the creation of synthetic characters includes realistic hair and cloth (as in Alias|Wavefront's MAYA package), motion capture systems, facial animation systems and live action camera tracking systems. Conversion software such as Paraform allows us to convert polygonal character databases that we create using the haptic interface by SensAble Technologies into nurbs for production. The computing power that can be brought to bear on projects has continually multiplied, and with the recent advent of render farms like WamNet and Intel, bulk computing power can be purchased on a per project basis.
In this summer's film "X-Men," directed by Bryan Singer, the shape-shifting character Mystique played by Rebecca Romijn-Stamos is a Synthespian / human hybrid who transforms into various other characters featured in the film. Our work here involves a series of complex 3D procedural transformations from live actors to a photo-real fully CG character. Currently in production for Busch Entertainment is our first original content project, which will feature an all-Synthespian cast in a simulator attraction for Busch Gardens. Our goal is to create characters with whom the audience can form empathetic relationships. This is much more important than merely making an actor look indistinguishable from a human, and in many ways, much more challenging.
Looking toward future development, the march toward photo-realism will march inexorably forward, with a lot of development required to accurately model the structure and function of facial muscles to allow for convincing dramatic performances in close-up. This is extremely difficult to do technically. Most of us spend all day long watching the motion of the facial expressions of people with whom we interact and we are very good at discerning emotional content or sub-textual information from that process. As mentioned before, systems currently exist for rendering convincing hair and clothing and the facial textures can be simulated through photographic studies and sophisticated texture mapping, but the animator of the future will need to be a talented actor if he or she intends to create a dramatic performance in a synthetic actor. I don't believe that performances that include this level of dramatic impact can be routinized to the level that a programmer could use for an automated animation system.
Jeff Kleiser is a director, as well as cofounder and president of Kleiser-Walczak.