Team
Mansie Verma, MSc. Design+Digital Media
Role: Concept, Ideation, 3D visualization/object modeling in Maya, Basic coding in JAVA, Documentation, Promotions, Research, Installation mounting
Peter Reid, MSc. Digital Composition+Performance
Role: Concept, Sound Design, Building Interface in MaxMSP-Jitter, Writing Max external in JAVA, Documentation, Installation mounting
University of Edinburgh, 2005-06

Bubble n Squeak is an interface wherein people use movement and sound to mainipulate digitally animated 'bubbles'. A live set, whereby people are no longer passive receptors of information, but are engaged, connected and immersed in the performance that is driven by their own actions. It is located where members of the public could come across it unintentionally. People will be stimulated to explore their voice and physicality, providing recreation and entertainment for themselves and others.

Interaction within the installation makes people clap, shout, sing, make noise, wave their arms around and jump about.  When a visitor makes a loud sound it is caught in a coloured bubble that is blown up in the air then floats gently down, as though they were blowing a soap bubble. The louder and longer the sound, the larger the bubble becomes until a louder sound causes it to break away and float off. 
The visitor's movement creates a colourful aura on the screen and when they touch the bubble it pops and its sound is released.The person sees their reflection in the screen, standing amongst the bubbles they have blown.

Bubble and Squeak is made up of several modules that work together as a whole. This section summarizes what the modules do and how they communicate with eachother.

•Bubble and Squeak is coded in Max MSP and Jitter with bubble shapes created in Maya and code written in Max Java.  The setup requires a DV camera and a microphone to detect the visitor's movement and sound.

•When the amplitude of the sound signal from the microphone exceeds a background noise threshold, a bubble begins to grow at the bottom of the screen at a rate related to the amount by which it is exceeded.  The louder the sound, the quicker the bubble grows.  When a higher threshold is crossed it flies away at a speed relative to the amplitude and stores a short snippet of the sound.

•The video input is optionally reflected about the y axis depending on whether the camera is in front or behind the visitors as they face the screen.  Each frame, the pixels of the previous frame are subtracted from those of the current.  Any pixels' red green or blue values that exceed a certain threshold are set to 1 and the rest to zero and the image is displayed.  If any of these pixels coincide with a bubble, the bubble is burst.  Bubbles also burst when they reach the bottom of the screen.

•When a bubble is burst, it vanishes from the screen and the stored sound is played back along with a popping noise.

•When the bubble is released from the bottom of the screen it is given velocity, direction and rotation values.  The bubble then continues to move in that direction, bouncing off the sides and top of the screen and gradually slowing down to simulate air resistance and gaining downward momentum to simulate gravity.
•Each bubble is given a shape and colour chosen at random from a collection of predefined shapes and colours.

•Bubble Forms: The initial idea had started with 'puppets'. Puppets and puppet shows have been around for quite a while and have had power to express the reality in poetic and intutive forms. Puppeteers lend meaning to the characters, and not just mimic the action. Although the traditional form of puppetry has been on a decline, the implicit idea within puppetry is alive today, for example in the form of digital puppets that guide you through a computer program interface.
•After much thought over what the puppet subject could be, we came upon the idea of exploring 'bubbles' as animated puppet characters. Experimentation with bubble blowing was carried out to observe and determine both the tangible and intangible properties of movement, shape, response to touch, sound and wind.
•Bubble Shape: The shapes of each bubble are picked from a predefined set of shapes. The movie below shows various bubble shapes in relation to each other.

•Computer system: Macintosh with OS 10.3 or later((Windows XP will be supported soon) with G4 processor (> 1 GHz), 512 MB RAM, 1394/firewire socket and sound in
•Projector: about 6000lumens
•Camera: i Sight or DV camera
•Microphone
•Speakers
•Materials for projection surface:white fabric screen & inflatable white balloons

•The surface :
The SKIN of the interface is conceived to be as much living and breathing as the stuff that goes over it. A plain white fabric tends to obstruct and just remains as a static divide between the onlooker and the performing subjects.
Hence, to create some interest and complement the movement of bubbles, we experimented with an undulating and slightly translucent medium. The effects as seen below give an illusion of depth, without being too obvious and creates a dynamic yet subtle feel.

•The 'Total Kunst' gallery at Forest Cafe in the heart of the city proved to be an ideal location to mount the installation.
The busy cafeteria ambience, large glass facade facing the street, enough room for display and interaction all went in favour of the setting.
•Another interesting outcome of the location was that it enabled interaction with the outdoor. Placing the projection surface over the window made it viewable from the sidewalk as well, alongwith cutting out exterior light.

•A few captured moments of 'Bubbling Activity'

•and more..

•A computer with faster processing speeds could have given faster renders.  Lower level programming for instance in C++ would also help the program to run faster and on slower hardware.
•Positioning of the camera could be improved.  At the Total Kunst, the camera was positioned behind the participants and had to be pointed at the floor in front of the screen and not at the screen itself to avoid light feedback. People instinctively like to wave their arms in front of the screen however so a camera in the screen facing the participants would be better.
•The microphone input needs to be as sensitive as possible to capture sounds from as shy a participant as possible.  The sensitivity of the input must however be balanced with the volume of the output of the program so that the sound of a bubble popping doesn't release a bubble with a louder sound that will in turn set off a chain of increasingly loud bubbles.  Levels are set optimally when a loud bubble bursting can trigger off another bubble but of a quieter volume.
•Time constraint!...given more time, a more puppet like animation could be achieved, a further step to that can be a more involved design that makes user interaction more dynamic, hence offering more play & variation.

•Relaxing & stimulating activity - motivates people of all age groups to let loose and connect with eachother without inhibitions

•Community & social networking - brings people together in a shared activity, idea of their own reflection and movement captured on screen opens them up to interact more

•Education:
'Bubble and squeak' interface extends into the educational realm, wherein the dual benefit of learning and having fun at the same time can be achieved.

•Health:
The system could be very therapeutic for children suffering with a range of conditions including elective mutes, traumatized children and those with cerebral disabilities. Creating puppets that these kids can relate to, that respond to their inarticulate gestures, that move, sway, sing and hop in an imaginary world that is only of these special children could bring about an ‘ability in disability’.

•Live Performance:
Bubble and Squeak provides a new method of human-computer interaction and suggests new possibilities for musical and theatrical performance.  In an installation situation, the audience become the performers.  On stage however, an exciting new music and visual control interface is made available to twenty-first century performing artists.

•Entertainment:
Besides its entertaining role as a public installation, the interface could extend into Cellphone gaming as well. This simulated model can be further customized into a Gaming engine alongwith Game Technology within the cellphone interface.
Telephones come with a microphone by definition and increasingly have cameras and speakers too making a potential platform for a Bubble and Squeak like game!