Interactive Audio Technologies
* "Vector audio" is to audio what "vector graphics" is to animation (Macromedia Flash)
SSEYO Koan Interactive Audio Platform
SSEYO Koan software enables quality audio to be experienced by a user on any power digital device and over very low bandwidth connections, both major requirements for mobile music making, entertainment and personalization, including in the field of commerce.
SSEYO employs breakthrough text-based “vector audio” and “cooking” techniques to create quality, consistent sound from as little as 10 bytes of data and from a software footprint as small as 60 KB. SSEYO’s products deliver standardised, portable, interactive, rich audio content for digital devices and services within cost effective processing, memory and bandwidth constraints.
The SKIAP or individual components of the SKIAP are available to be licensed from SSEYO.
Overview of the SKIAP
The SKIAP provides full support for services normally required of an audio playback / rendering system. In addition, it provides many other features required for delivering enriched audio solutions over low bandwidth connections within cost effective processing and memory requirements.
The SKIAP includes:
The SKIAP offers the following key benefits:
The default configuration of Koan is as a full-featured interactive audio engine (the SKIAP), but Koan components may be deployed in any number of “mix and match” configurations.
The SSEYO Koan Synth Engine (SKSE)
Powerful, programmable small software synthesizer for vector audio and more
The SSEYO Koan Synthesizer Engine (SKSE) is a fully featured, real-time configurable software synth capable both of sample-based and analogue (oscillator)-based synthesis through arbitrary networks of effects such as reverberation, filtering, and modulation.
From an operational perspective, the SKSE generates an internal stream of audio data. This is rendered in a manner appropriate to the local system, for example through a low-level device driver interface, or within the context of a broader architecture such as the local operating system’s media streaming system. The SKSE brings considerable audio power to any embedded system which has audio capabilities.
The SKSE is a hybrid digital/analog synth with no inherent limit as to the number of voices. Each voice can be sourced separately from a combination of sample data (in formats such as MP3, WAV or DLS) and/or built-in oscillators. The SKSE supports a number of independently specified effects units per voice as well as global (piece-level) effects. Individual effect units in a Voice or piece-level effect chain (referred to as an "Effect Module") can be enabled, disabled, added and deleted in real time through the Koan APIs while the audio is playing. The SKSE allows multimedia developers to break beyond the limitations of MIDI.
The SKSE can be used with both sample-based and analogue-style approaches to sound generation. The SKSE can use samples (e.g. in MP3 format), and can also audio streams for voices from combinations of internal oscillators, in which case the voices being generated in that manner require no sample data at all. Voices generated in the latter way, are done so in the manner of a traditional analogue synth. Every voice can be subjected to a wide range of digital audio effects. Master effects modules may also be used, which effectively mix the outputs from each voice (the combination of mixed dry and effect-processed voice), in order to provide "global" audio treatments.
The network of voices and effects in the SKSE is dynamically re-configurable in real-time, unlike traditional software synthesizers. The SKSE is exposed to developers through all of the SSEYO Koan APIs, enabling developers to take full advantage of the power of the SKSE.
The SKSE is written using portable C++ and runs on all platforms to which the SKIAP is ported. It has been developed from the ground up with a particular view to suitability (in terms of both portability and high performance) for running on embedded systems.
The SKSE is capable of generating its audio at any audio sample rate supported by the local hardware / operating system (e.g. from 2KHz through to 48KHz and beyond), with either 8-bit or 16-bit sample data widths, and can be run in either mono or stereo : all of these features are software configurable.
The SSEYO Koan Ringtone Engine
Deterministic musical event generator
The SSEYO Koan Ringtone Engine is used for composing both monophonic and polyphonic Ringtones, such as with an application like the SSEYO Ringtone Remixer. The technology can be deployed either on a client or on a server, and is implemented in a very compact Java engine. This technology allows SSEYO to tap-in to developing trends in personalising audio for personal communication devices, whether or not a local Koan engine is embedded.
Koan Vector Audio (KVA)
Plain-text representations of audio and musical parameters
Koan audio vectors are very compact, plain text representations of the information required by Koan to create sounds, effects and music. They are in plain text, to make it easy to distribute them across a wide range of applications. They can also include links to audio samples located remotely or on the operating system or platform. For examples of audio vectors the size of only 1 SMS, click here.
In only a few hundred bytes, a stunning sound effect, ringtone or entire piece of music can be expressed, including not only the musical notes to play, but also complete parametric descriptions of the sounds (including sound effects e.g. reverb, chorus, tone generators, etc.) to be used in rendering the audio. A Koan audio vector can also be used to “re-direct” a piece : to “push” the audio experience in a new direction (i.e. applying a vector), in real-time, within only a small number of bytes of plain text.
Example : KVA for mobile data exchange, including custom email audio
Avoiding firewalls - Koan audio vectors and Internet usage To provide webpage audio would normally require the audio content to be supplied as a file. This has implications page loading times and firewalls.
However, use of text-based Koan audio vectors embedded within webpages means that webpages load very quickly and there are no firewall problems (e.g. blocking of MP3 files or JPG files with firewalls). This means that partners can use vector audio content confident that it will be delivered.
Note: The SKIAP is also available in the form of a Plugin for an Internet browser, with a comprehensive API that allows dynamic population of webpage content entirely through using Koan audio vectors.
Secure, fast content delivery to mobile devices with Flash and audio vectors
Portable music for creating, sharing and trading on networked devices
The SSEYO Koan Music Engine (SKME)
Playback system and generative music, real-time event generator
A powerful music engine is a key component of an interactive audio platform. It has specific application whenever there is a requirement to:
The SKME is fully real-time programmable rule-based musical event generator. It creates fully polyphonic harmonized audio output from a parametric description of music and sequence structures. It can be considered an “expansion” technology in that it can generate on-the-fly a wealth of music and audio content from settings of the relevant Koan parameters and rules. These parameters are supported within the Koan audio vector format.
With its tight coupling to the SKSE (which ensures that performances are consistent across all platforms that use the SKME), the SKME can provide even greater leverage to deliver music and audio over low-bandwidth connections. It also delivers enhanced creativity solutions for e.g. remixing or augmenting artist’s sample based audio tracks.
How it works
Whenever a rule-based Koan track is played, the SKME interprets the parameter settings contained within the Koan file or audio vector to "compose" in real-time the audio track. The audio ‘occurs’ as the SKME sends information, which is rendered either through the SKSE or through any supported MIDI device. The audio tracks created can span the complete range from being completely fixed (like a music score) through to being completely free-composed.
The SKME can be manipulated in real-time by an host application to accommodate or reflect changes in the state of the host application. This facility means that enriched audio environments can be built and customized on the fly, through manipulation of the rules and parameters on which the content is based. This is an essential feature when creating personalised (and potentially database driven) audio experiences. It would be simply impossible to pre-record the material required, where the outcome space is effectively infinite.
Additionally, as the Koan engine supports sequencing of patterns, it can be used effectively to deliver structured non-musical events. These can be "built" from audio fragments or triggered as discrete events - this has applications in the area of delivering speech-related effects.
The 4 key areas surrounding the creation of Koan music (Koan supports over 200 parameters) are the sound palette, the music rules, any 'seed' patterns used and the interrelationships between the various components in the Koan piece.
Portability, Performance and Porting
The SKIAP is written using portable C++, to run on a large number of platforms, with a particular view to suitability (in terms of both portability and high performance) for running on embedded systems. Supported Operating Systems currently include Windows 95/96/NT4/2000, MacOS 7.3+, Intent 1.1, Window CE, Symbian, Linux, etc.. Note that the Intent platform allows the SSEYO Koan engine to be used on a wide range of underlying operating systems, including (for example) EPOC, Linux and OS/9.
SSEYO's Koan technology has already been ported to (and is commercially available now for) a range of operating systems including Windows 95/98/2000/ME/NT4, Mac OS 8/9, Windows CE (ARM, MIPS etc. for Pocket PC and other platforms), Tao's intent™ (for a wide range of operating systems including Linux, EPOC, WindRiver, QNX etc. to run on hardware including ARM, MIPS, Hitachi, Intel, Motorola and other devices), and a native port to ARM. Future embedded system Koan ports expected to be completed in the near term include embedded Linux, EPOC, and QNX. Future desktop ports in the medium term include desktop Linux and MacOS X. Koan Server technology is available now in the Java language for any Java implementation for client - or server-side deployment, across a wide range of operating systems and hardware configurations.
Application Programming Interfaces (APIs)
The SKIAP is exposed to both Java and C++ applications via various programmatic interfaces:
Supported File Formats
The SKIAP can play Koan vector audio, MIDI files and Koan files. It supports the use within content of audio samples in MP3, WAV, DLS and other formats. Koan files can be created using an authoring tool such as Koan Pro (below). Koan files can also be generated in real-time, through any one of the Koan APIs. Note that for MP3 support, a 3rd party MP3 codec is required to be present, which is normally a component part of the Operating system (OS).
How is Koan content constructed?
Koan content may be designed using both on-line and off-line tools.
Creating Koan content involves editing and reviewing a number of parameters, either on-screen (using SSEYO Koan Pro, through 3rd party-created web-page based authoring tools or a 3rd party-created mobile applications), or interactively via one of the Koan APIs. All of the SKSE’s and SKME’s parameters are changeable in real-time, meaning that the input will take effect as soon as the SKIAP accommodate it (determined by how far ahead the SKME composes, which can be fine-tuned by the calling application). Some parameters can be fixed for the duration of the sound effect, ringtone or track and others, using ‘envelopes’ and internal waveform generators, can provide varying settings throughout the length of the sound effect, ringtone or track - useful for mixing purposes.
Koan content can be saved and distributed as either binary Koan files, or Koan audio vectors, depending on the system requirements.