Microsoft Corporation

October 1998

Summary: The convergence of desktop multimedia and the Internet presents tremendous opportunities for developers to reach broad audiences with compelling, media-rich content and applications. With the Microsoft® DirectX® 6.0 set of application programming interfaces (APIs), Microsoft delivers a unified, comprehensive solution for developers. This white paper details the application-level services of DirectX media that provide rich support for media interaction and integration.

Contents

Introduction
Direct3D Retained Mode
DirectAnimation
DirectShow
Conclusion
For More Information

Introduction

Application-level media services bring multimedia power to all developers.

The Microsoft DirectX set of application programming interfaces (APIs) offers a broad spectrum of services from chip level to cyberspace. In addition to low-level APIs that access hardware acceleration—called the DirectX foundation—DirectX now includes DirectX media, a layer of high-level services including streaming, animation, and behaviors.

DirectX media is a family of application-level APIs and controls for multimedia that provides rich support for interaction and integration of different media types for the development of online and digital media authoring applications.

DirectX media currently consists of four APIs:

DirectX Media Goals

DirectX media has been developed to meet a set of related goals:

• Provide application-level media services. DirectX media provides application-level services—device-independent services that take advantage of the system-level DirectX foundation services.
• Unify media services. With DirectX media, multimedia developers get the benefits of device-independent access to multiple media types without losing the benefits of direct access to the hardware. To accomplish this, DirectX media provides a consistent API interface, device independence through the HAL/HEL architecture, and APIs with high-level services based on DirectX foundation.
• Support multiple digital media users and environments. Through device and language-independent Component Object Model (COM) design and Web integration, DirectX media services can be easily used by many types of multimedia developers, from the casual Web page designer to the professional media developer.

DirectX Media Architecture

DirectX media is the result of the organization of DirectX 6.0 into two levels: a system-layer DirectX foundation and an application-layer DirectX media.

DirectX media services use DirectX foundation. These services include Direct3D Retained Mode, DirectAnimation, DirectShow, and DirectX Transform. Support for VRML is also provided in DirectX media.

DirectX foundation provides the basis for performance media on Windows-based computers through DirectDraw®, DirectInput®, DirectSound®, DirectSound3D, and Direct3D Immediate Mode.

Figure 1. DirectX 6.0 architecture

Direct3D Retained Mode

3-D scene graph API makes it easy to build and animate 3-D environments.

Direct3D Retained Mode is a high-level 3-D scene graph manager that simplifies the building and animation of 3-D worlds and data.

Note that Direct3D Retained Mode 6.0 does not work on the Windows NT® operating system, version 4.0, but will work on Windows NT 2000. If you want to use the Direct3D Retained Mode functionality with Windows NT 4.0, you need to use a version of DirectX 3.0.

Direct3D Retained Mode 6.0 introduces new interfaces with increased functionality. You can now:

• Access animation keys directly.
• Clip visuals against arbitrary planes.
• Select textures dynamically, based on specified criteria.
• Specify textures whose dimensions are not a power of two.
• Delete face, normal, and vertex objects from meshes.
• Retrieve submeshes and parent meshes.
• Specify whether a z-buffer should be created.
• Specify whether right-handed or left-handed geometry should be used.

APIs have been added to eliminate redundant state changes and optimize your programs, and to take advantage of new DirectX Immediate Mode functionality—such as better lights.

All access to Direct3D Retained Mode is through a small set of objects. The following table lists these objects and a brief description of each:

Eliminating Redundant State Changes

In earlier versions of Direct3D Retained Mode, Retained Mode could not assume that render and light states would remain the same from one call to the next. This led to many redundant state changes in a typical application because Retained Mode set the state to a value equal to its current value. The IDirect3DRMDevice3 interface introduces several methods that can be used to eliminate these redundant changes for a modest performance boost: GetStateChangeOptions, LightStateChange, RenderStateChange, and SetStateChangeOptions. To use these methods, follow these steps:

1. Turn on the redundant state change check by using the following two methods on the IDirect3DRMDevice3 interface. This indicates that you will not change any render or light state without notice.

   Dev3->SetStateChangeOptions(D3DRMSTATECHANGE_RENDER, 0, 
                               D3DRMSTATECHANGE_NONVOLATILE); 
   Dev3->SetStateChangeOptions(D3DRMSTATECHANGE_LIGHT, 0, 
                               D3DRMSTATECHANGE_NONVOLATILE); 
   

2. When you change a render or light state (either through an execute buffer or a draw primitive), you must inform Direct3D Retained Mode of the change so that the state cache is updated. You can do this in two ways: you can have Retained Mode change the state for you (and prevent redundant changes), or you can inform Retained Mode that you have changed the state to a new value. Both use the LightStateChange and RenderStateChange methods.

   For example, the following code shows you how to change the texture handle and update the state cache.

   Dev3->RenderStateChange(D3DRENDERSTATE_TEXTUREHANDLE, hTex, 0UL); 
   

   If you have changed the state and just want Retained Mode to update its state cache, use the UPDATEONLY flag, as shown in the following code:

   Dev3->RenderStateChange(D3DRENDERSTATE_TEXTUREHANDLE, hTex, 
                           D3DRMSTATECHANGE_UPDATEONLY); 
   

   If you are going to be changing a state value many times and do not want Retained Mode to keep track of current values, you can turn off redundant state checking for the specified state with the following code. However, this is not recommended because you will lose performance benefits.

   Dev3->SetStateChangeOptions(D3DRMSTATECHANGE_RENDER, 
                            D3DRENDERSTATE_TEXTUREHANDLE, 
                            D3DRMSTATECHANGE_VOLATILE); 
   

DirectAnimation

Multimedia API combines rich animation of multiple media types and strong integration with Dynamic HTML.

The DirectAnimation component of DirectX media provides an integrated, comprehensive API and run time with support for a diverse set of media types and a powerful time/event model for developing rich animation and interaction. Because DirectAnimation is integrated with Dynamic HTML, it is especially suited to adding compact animation effects to Web pages.

The DirectAnimation run-time library is part of the Microsoft Internet Explorer 4.0 minimal install. This means that Internet Explorer 4.0 contains all of the software necessary to view multimedia created with DirectAnimation.

The key features of the DirectAnimation API are:

• Multiple media types. Comprehensive support for different media types—2-D vector graphics, images and sprites, 3-D geometry, video, and sound (including MIDI)—is unified in a single animation API.
• Uniform time and event model. A uniform time and event model simplifies temporal coordination and interaction of the different media types. This model frees users from frame-level details and provides the foundation for supporting inherently time-based media types such as video and audio.
• Multiple access methods. DirectAnimation scales from the HTML author to the C++ programmer. You can access DirectAnimation from HTML, Visual Basic® Scripting Edition (VBScript) and JScript®, Java, and Visual C++®. DirectAnimation components include ActiveX® controls for HTML environments and a COM-based animation library accessible from any programming or scripting language.

DirectAnimation Architecture

The following diagram shows the DirectAnimation architecture:

Figure 2. DirectAnimation architecture

The DirectAnimation multimedia controls provide an interface to some of the DirectAnimation library, which is also accessible directly. The DirectAnimation library in turn uses the DirectShow API, the DirectX foundation, and certain operating system services. In the diagram, "SG" stands for Structured Graphics control and "Seq" stands for the Sequencer control.

How Do You Use DirectAnimation?

DirectAnimation is a COM API and an underlying run time, the functionality of which can be accessed in different ways by different user groups.

You can use DirectAnimation in the following ways:

• As an inline HTML Web animation through JScript, VBScript, or the DirectAnimation controls.
• As a Web animation in the form of a Java applet.
• As a stand-alone application or an ActiveX control implemented through Java, Visual Basic, or C++.
• To construct animated windowless controls.
• To animate properties of ActiveX controls.
• To load animations asynchronously and hide the load time.

The following table shows the typical ways different developers would access DirectAnimation.

You can access DirectAnimation from JScript, VBScript, Visual Basic, and C++ directly through the scripting (COM) interfaces. You can also add DirectAnimation content to your Web pages without programming at all by using the DirectAnimation controls and setting parameters on these controls. Using the DirectAnimation controls directly, or using JScript or VBScript, allows you to describe inline animations with HTML. Such animations can integrate with Dynamic HTML by being windowless on the page (overlaying other elements such as text) or by driving the properties of other entities on the page. It is also possible to import HTML-rendered text and use it as a texture in an animation.

There is a special Java binding for DirectAnimation provided on top of the COM API that takes advantage of specific Java features. For example, operations are overloaded so that several COM methods that perform similar functions but use different parameter types are given the same name in Java.

To create presentations with DirectAnimation, you need the following:

• The DirectAnimation run-time library, which is part of the Internet Explorer 4.0 minimal install.
• The DirectAnimation SDK, which includes samples in JScript, VBScript, Java, and Visual Basic.
• A Java compiler or Visual J++® development system, a C/C++ compiler (such as the Visual C++ development system version 5.x), or Microsoft Visual Basic programming system version 5.0 or later. You can also use the DirectAnimation API from VBScript and JScript without compiling.
• Visual Basic, Java, and C++ users need the Microsoft Visual Basic, Visual J++, or Visual C++ environments or their equivalents. Internet Explorer 4.x is all that JScript and VBScript users need to develop animations.

What's New in DirectAnimation 6.0?

The following features are new DirectAnimation 6.0:

• Third parties can extend or customize DirectAnimation functionality through DirectX Transform objects.
• Time substitution now can be used for sequenced behaviors, and time can be run backward.
• A new set of APIs, AddPickData and QueryHitPointEx, allows for arbitrary data to be attached to Image and Geometry behaviors and to be retrieved as a result of a pick operation.
• DirectAnimation now takes full advantage of Direct3D hardware acceleration and provides full support for textures in DirectX files (.bmp and .ppm are the only file formats supported).
• Additional 3-D attributes have been added, including ambient color, emissive color, specular color, and specular coefficient.
• Geometry behaviors now can cast shadows.
• Textures can be blended with the geometry's diffuse color.
• Geometry behaviors can be constructed from a Direct3D Retained Mode visual.
• You can specify wrap parameters for Geometry behaviors.
• MPEG movies have better performance, especially when played below their nominal rate.

Note that to use some of the new DirectAnimation APIs, you must have DirectX 6.0 or later installed. DirectX 6.x cannot be installed on Windows NT version 4.0. You will not get an error if you use these functions without DirectX 6.x; they will simply have no effect.

You must have DirectX 5.0 or later installed to use some of the Geometry shading and lighting functions. You will not get an error if you use these functions without DirectX 5.x or DirectX 6.x; they will simply have no effect.

To use 3-D DirectX Transform effects in DirectAnimation, you must have DirectX 6.0 or later installed. To use some 2-D DirectX Transform effects, you must have the optional DirectX Media DLLs installed. You can install these by choosing Optional Transforms when you install the DirectX Media SDK. Instantiation of the DirectX Transform COM object will fail if you attempt to use transforms without the proper software installed.

DirectAnimation Multimedia Controls

The DirectAnimation Multimedia Controls (formerly called Multimedia DHTML controls) supply a scripting interface to some of the DirectAnimation API functions and libraries. These controls allow you to deliver impressive animation, image, sound, and vector graphics content over the Web with low code overhead and without incurring long download times.

The DirectAnimation Multimedia Controls consist of:

• Path Control. Moves objects around the page over time in geometric or spline patterns, with control over speed of motion, forward or reverse playback, looping, and bouncing behavior.
• Sprite Control. Adds still and animated images on your Web page, with control over playback speed, forward or reverse playback, dynamic seeking to different positions in the sprite playback, and user interactivity on a frame-by-frame basis.
• Structured Graphics Control. Adds extremely lightweight vector graphics to your Web page. You can scale or rotate these graphics in 3-D, and you can easily apply solid colors, patterns, gradient fills, or bitmap textures.
• Sequencer Control. Creates complex sequences of action involving controls, scripts, or dynamic HTML elements.

A DirectAnimation Multimedia Control Example

In the following example, btnOval is a button that says, "This is a moving button."

A DirectAnimation path control named pthOval targets the button, telling it to move along an oval-shaped path on the page.

<HTML>

<INPUT NAME=btnOval TYPE=BUTTON VALUE="This is a moving button" 
    STYLE="position:absolute;LEFT: 20; TOP: 80">

<OBJECT ID="pthOval" 
CLASSID = "CLSID:D7A7D7C3-D47F-11D0-89D3-00A0C90833E6">
   <PARAM NAME="Target" VALUE="btnOval">
   <PARAM NAME="Shape" VALUE="Oval(50,50,400,200)">
   <PARAM NAME="AutoStart" VALUE="-1">
   <PARAM NAME="Repeat" VALUE="-1">
   <PARAM NAME="Duration" VALUE="10">
</OBJECT>

</HTML>

DirectShow

Professional, multiformat media playback, capture, and streaming make it easy to add multimedia streams to any application.

DirectShow (formerly the ActiveMovie SDK) is the media streaming architecture of DirectX media for controlling and processing streams of multimedia data.

DirectShow offers three core features:

• Media stream playback. Plays back multimedia streams from local files or Internet servers; these multimedia streams consist of video and audio content compressed in various formats.
• Capture. Captures video and audio for professional and multimedia purposes.
• Multimedia streaming. Loads and renders or converts multimedia streams from and to multiple media formats.

DirectShow is an architecture that controls and processes streams of multimedia data through custom or built-in filters. You can also use the set of media streaming interfaces to stream media data without creating filters.

In addition to the architecture and the set of classes and interfaces to support it, DirectShow is also a runtime that uses this architecture to enable users to play digital movies and sound encoded in various formats, including MPEG, AVI, MOV (Apple® QuickTime®) and WAV-formatted files. The DirectShow run time is a control (.ocx), called the Microsoft Windows® Media Player Control, and a set of dynamic-link libraries (DLLs) that enable you to play back supported media files.

DirectShow playback uses video and audio hardware cards that support the Microsoft DirectX set of APIs. The video and audio capture capability lets you programmatically control your system's video and audio capture hardware, as well as video and audio compressors and decompressors (codecs). The Plug and Play capability lets DirectShow automatically retrieve and use your filters once you register their properties.

What's New in DirectShow 6.0?

Features added to DirectShow since the DirectShow 5.2 SDK include:

• New Windows Media Player Control.
• Expanded MPEG-2 support.
• Support for playing video from within applications that are using DirectDraw exclusive mode.
• Support for dynamic audio format changes.
• Improved handling of noncontinuous audio data.
• DVD-ROM boilerplate video.
• New MPEG-2 Splitter filter and MultiFile Source filter.
• New samples demonstrating how to play video in DirectDraw exclusive full-screen mode and how to write audio streams to .Wav files.
• New DirectSound interfaces: IDirectSound3DBuffer, IDirectSound3Dlistener, and IAMDirectSound.
• New filter-level interfaces: IAMAnalogVideoDecoder, IAMCameraControl, IAMClockAdjust, and IAMFilterMiscFlags.
• New graph-level interface: IBasicVideo2.

DirectShow Filter Architecture

At the heart of the DirectShow services is a modular system of pluggable components called filters, arranged in a configuration called a filter graph. A component called the filter graph manager oversees the connection of these filters and controls the stream's data flow.

To use the filter graph manager from an application, you do not need to know much about the underlying filter graphs. It is useful, however, for you to understand the basic principles of filter graphs if you ever want to configure your own filter graph rather than letting the filter graph manager configure them for you.

A filter graph is composed of a collection of filters of different types. Most filters can be categorized into one of the following three types:

• Source filter. Takes data from a source, such as a disk file, camcorder, satellite feed, Internet server, or VCR, and introduces it into the filter graph.
• Transform filter. Processes data and passes it along.
• Rendering filter. Renders data to a hardware device or to any location that accepts media input (such as memory or a disk file).

In addition to these three types, there are other kinds of filters. Examples include effect filters, which add effects without changing the data type, and parser filters, which understand the format of the source data and know how to read the correct bytes, create times stamps, and perform seeks.

For example, a filter graph, the purpose of which is to play back an MPEG-compressed video from a file, would use the following filters:

• A source filter to read the data from the disk.
• An MPEG filter to parse the stream and split the MPEG audio and video data streams.
• A transform filter to decompress the video data.
• A transform filter to decompress the audio data.
• A video renderer filter to display the video data on the screen.
• An audio renderer filter to send the audio to the sound card.

The following illustration shows such a filter graph:

Figure 3. DirectShow filter graph

It is possible for some filters to represent a combination of types. For example, a filter might be an audio renderer that also acts as a transform filter by passing through the video data. But typically, filters fit only one of these three types.

Filter graphs stream multimedia data through filters. In the media stream, one filter passes the media downstream to the next filter. An upstream filter describes the filter that passes data to the filter; a downstream filter describes the next filter in line for the data. This distinction is important because media can only flow downstream, whereas other information can go upstream.

Using DirectShow

There are three ways to use DirectShow:

• Through the Windows Media Player Control. The Windows Media Player Control provides an easy-to-use interface that lets you manage multimedia using the control's properties, methods, and events. The control handles all video and audio rendering for you, simplifying your programming tasks and making it easy to add support for multimedia to your Web page or other application.
• With Prebuilt Filters. DirectShow provides a standard set of prebuilt binary filters as part of the DirectShow SDK.
• Writing Custom Filters. The SDK also enables you to create your own filters using the DirectShow class library. The base classes in the C/C++ library implement the required COM interfaces on the filters and provide the basic filter framework. You can write DirectShow filters in any language that can generate objects adhering to COM. The base classes for DirectShow are written in C++.

The amount you must know about an underlying or supported technology depends on your task. For example, you will need to understand COM programming when using C or C++ to control DirectShow playback or create a filter, but you do not need to understand COM programming to use the ActiveMovie Control.

Prebuilt Filters

DirectShow provides prebuilt filters as part of the DirectShow SDK. A prebuilt filter, supplied as binary code only, is one of the filters listed in the Filter Graph Editor when you choose Insert Filters from the Graph menu.

The DirectShow SDK provides the following filters:

Sample Source Filters

A sample filter includes source code; you must build and register it before it will appear in the Filter Graph Editor:

Source filters

• Asynchronous Reader Filter
• Audio Synthesizer Filter
• Bouncing Ball Source Filter
• MultiFile Source Filter
• Video Capture Filter

Transform filters

• Audio Stream to .Wav File Writer Filter (WavDest)
• Contrast Filter
• Image Effect Filter (EzRGB24)
• Infinite-Pin Tee Filter
• Minimal Null Filter
• MPEG Framework Audio Codec
• MPEG Framework Video Codec
• Null In-Place Filter
• Simple Audio Processing Filter (Gargle)
• VCR Device Control Filter

Renderer filters

• File Dump Filter
• Oscilloscope Filter
• Sample Video Renderer Filter
• Text Renderer Filter

Cutlist sample applications

• Simple Cutlist Application
• Text Cutlist Application

C/C++–based sample applications

• C/COM–based Media Player
• C++/COM–based Media Player
• DirectDraw Exclusive Mode Playback
• DirectShow Capture
• DVD Player
• Indeo Player
• MPEG Property Page Display
• Multimedia Streaming
• Simple Playback
• Video Editing Application
• Window Playback

Visual Basic–based sample applications

• Visual Basic–based ActiveX Player
• Visual Basic–based Filter Graph Builder
• Visual Basic–based Filter Graph Player
• Visual Basic–based Player

Plug-In Distributor Sample

• Plug-In Distributor Sample

IOStream Helper Library

• IOStream Helper Library

Multimedia Streaming with DirectShow

When developers use multimedia streaming in their applications, it greatly reduces the amount of format-specific programming needed. Typically, an application that must obtain media data from a file or hardware source must know everything about the data format and the hardware device. The application must handle the connection, transfer of data, necessary data conversion, and the actual data rendering or file storage. Because each format and device is slightly different, this process is often complex and cumbersome. Multimedia Streaming, on the other hand, automatically negotiates the transfer and conversion of data from the source to the application. The streaming interfaces provide a uniform and predictable method of data access and control, which makes it easy for an application to play back the data, regardless of its original source or format.

The following diagram shows the basic object hierarchy used in multimedia streaming:

Figure 4. Multimedia streaming object hierarchy

The following steps show how to implement streaming from hardware device to rendered playback:

1. A source of video data, such as DirectShow, exposes the streaming interfaces.
2. The application developer uses the Multimedia Streaming interfaces to handle data format conversion.
3. The application developer uses the DirectDraw interfaces to render the resulting data.

There are three basic object types defined in the multimedia streaming architecture:

• A multimedia stream, which supports the IMultiMediaStream interface.
• Media streams that support the IMediaStream interface and are data specific; every multimedia stream contains one or more of these media streams.
• Stream samples, which support the IStreamSample; these are created by a media stream and represent a basic unit of work for the stream.

Conclusion

Multimedia by definition requires integration—and multimedia over the Internet requires even more integration. DirectX provides the first unified solution to take advantage of the cross-platform flexibility of the Internet and the powerful multimedia capabilities of the personal computer. With all DirectX services designed to work together with a single programming model, DirectX makes it easy to develop innovation and ease of use into online and digital media-authoring applications with benefits that go far beyond present-day technologies.

Note that to use DirectX on Windows NT 4.0, you must use DirectX 3.0. DirectX 6.x will be available on Windows 2000. Although DirectX media 6.0 takes advantage of the improved features in DirectX 6.0, many of the Direct X media products can be used successfully with DirectX 3.0.

For More Information

For additional information on DirectX technologies, visit www.microsoft.com/directx/default.asp.

---------------------------------------------

© 1998 Microsoft Corporation. All rights reserved.

The information contained in this document represents the current view of Microsoft Corporation on the issues discussed as of the date of publication. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information presented after the date of publication.

This White Paper is for informational purposes only. MICROSOFT MAKES NO WARRANTIES, EXPRESS OR IMPLIED, IN THIS DOCUMENT.

Microsoft, ActiveMovie, ActiveX, the BackOffice logo, Direct3D, DirectAnimation, DirectDraw, DirectInput, DirectPlay, DirectShow, DirectSound, DirectX, JScript, Visual J++, Visual Basic, Visual C++, Win32, and Windows are either registered trademarks or trademarks of Microsoft Corporation in the United States and/or other countries.

Java is a trademark of Sun Microsystems, Inc.

Other product or company names mentioned herein may be the trademarks of their respective owners.