Application development managers repeatedly complain to me that middleware is complicating their lives. There is too much middleware. It lacks the right functionality. The functionality isnýt apparent. Or the benefits arenýt clear. And it costs too much. For something intended to simplify application development, middleware seems to be doing just the reverse.Middleware ý software that allows an application to interoperate with other software while eliminating the need to understand and code the low-level operations of the various parts ý isnýt new. IBMýs classic transaction processing middleware, CICS, has been around for decades in the mainframe environment, migrating off the mainframe only recently. BEA Systemsý Tuxedo has provided similar capabilities in the Unix world for well over a decade. Even database access middleware ý software that facilitates data requests between applications and database management systems ý has been around for at least a decade. And before the availability of commercial middleware, developers simply created their own by hand-coding connections to databases or building transaction management capabilities into the application itself. Many development organizations, in fact, continue to write their own middleware, often without even realizing it.
But we have noticed a proliferation of commercially available middleware in recent years, beginning with client/server computing and accelerating with the emergence of the Internet and Web-oriented development. Even Microsoft has recently plunged into the middleware market in a big way.
So confusion abounds. Middleware, as it is touted by many vendors, is a vague, poorly defined area. The middleware market appears chaotic, populated by many unknown products and low-profile vendors. The lack of any coherent, agreed-upon definitions for types of middleware make it difficult to compare one kind of middleware with another. Depending upon the needs of a given application, middleware of different types, say asynchronous remote procedure call (RPC) or message-oriented middleware, eventually will get you to roughly the same place, albeit by different routes.
While generally embracing the concept of middleware, development managers question why there has to be so much different middleware. A single, universal piece of middleware that could forge every type of integration and connection between every type of system would be wonderful, I agree. But such universal middleware is unrealistic today given the complexity of the current computing environment and the diversity of organizationsý needs.
We have different kinds of middleware to serve different purposes. No one kind of middleware is right for every situation. On the contrary, an organization is likely to use different kinds of middleware even within the same application. For example, a customer service application might include synchronous transaction processing middleware, object brokering middleware, and database access middleware.
With multiple types of middleware a fact of life, I have developed a taxonomy of middleware to help development managers sort through the myriad offerings and identify the appropriate type of middleware for a given problem. An organization can then focus on evaluating and selecting the various offerings within the middleware segment.
I divide the middleware market into six categories: asynchronous remote procedure call, publish/subscribe, message-oriented middleware, object request brokers (ORBs), SQL-oriented data access middleware, and synchronous RPC. Of these six, the first three are asynchronous and the last three are synchronous.
In looking at middleware, I focused on two areas that developers report are most important: scalability and recoverability. These two characteristics have an inverse relationship. The more recoverable the middleware is, the less scalable it is because of the high overhead associated with the journaling, handshaking, and confirmations required for recoverability.
Complementing middlewareýs scalability and recoverability is the nature of the communication, either synchronous or asynchronous. Synchronous systems handle each request while the requesting system waits for the response. Asynchronous systems, on the other hand, fire off requests without waiting for responses, which are eventually returned in whatever order they come. Meanwhile, the requesting system is free to perform other work. Synchronous middleware, often used with production transaction systems, doesnýt scale well but can be quite recoverable. Asynchronous middleware, such as we find on the Web, scales quite large but canýt recover very well, unless recoverability has been built into the application itself.
I classify the six types of middleware on the basis of scalability and recoverability (see Figure 1) from asynchronous, the most scalable and least recoverable, to synchronous RPC, the most recoverable but least scalable.
Asynchronous RPC allows a client to request services without waiting for a response. Typically, the client must know the explicit network address of the server and establish a point-to-point connection with the server. The client requests information from the server and continues doing other things while it waits for a response. If the connection is broken, the client starts all over again from the beginning. Asynchronous RPC is highly scalable because very little information about the connection and the session is maintained by either the client or the server. Asynchronous RPC is most appropriate when transaction integrity is not needed. NobleNet RPC is an example of asynchronous RPC.
Publish-and-subscribe middleware monitors activity and pushes information to subscribers. The publishers notify all subscribers of published messages containing information or datatypes that the subscribers have previously indicated that they want to receive. The subscriber, on receiving notification, can then choose to receive the material or not. Publish-andsubscribe is asynchronous in that the clients (subscribers) do other things while waiting for the next message from a publisher. Publish-and-subscribe middleware can be particularly effective for applications that monitor and respond to selective events. When an event occurs, it triggers the publishing of a message. The subscribers are thus alerted to the event and will respond as they have been programmed to do. Examples of publish-and-subscribe middleware include Active Softwareýs ActiveWeb and TIB/Rendezvous from TIBCO.
Message-oriented middleware (MOM) is another asynchronous technology. Here applications fire off messages that are collected and stored in queues until they are acted upon. Meanwhile, the client continues to go about its other business. MOM is often used for extended processes, such as are found in a variety of workflow applications. For example, a loan processing application will send messages through MOM to credit bureaus and other systems as it gathers information to complete the processing of the loan. MOM offers a measure of recoverability through the queuing mechanism, which can be journaled, and it allows queues to be restored in the event of a failure. IBMýs MQSeries, Talarian SmartSockets, Microsoftýs MSMQ, and BEAýs MessageQ are examples of MOM.
An ORB is middleware that allows applications to send objects and request services in an object-oriented system. The ORB keeps track of information about each object and the actual addresses of all the objects in the system, and it is able to route requests in the right form to the specific object. The application requests the object by name without knowing where the object resides. ORBs currently are synchronous although initiatives are underway to develop asynchronous ORBs. Examples of ORBs include Iona Orbix, Visigenic VisiBroker, and BEA ObjectBroker.
SQL-oriented database access middleware connects applications with databases over the network and translates generic SQL requests into the databaseýs native SQL or other database language. SQL-oriented middleware eliminates the need to code SQL-specific calls for each database and code the underlying communications. SQL-oriented middleware is a mature segment dominated by the database vendors themselves and established companies with multidatabase access middleware. The connection is synchronous, but scalability can be a serious issue with both the middleware and database becoming a performance choke point. Oracle Connect, Intersolvýs DataDirect, and JDBC are examples of SQL-oriented middleware.
Synchronous RPC is a message-passing facility that allows a distributed program to call services available on various computers in the network. Writing RPCs entails serious programming, but the middleware provides a highly abstracted interface that eliminates the need to do detailed coding. Synchronous RPC middleware offers the best recoverability. Examples of synchronous RPC include IBMýs CICS, BEA Tuxedo, NCR TOP END, and Microsoft Transaction Server.
We are witnessing some blurring of the distinctions between each category. Several types of middleware, such as SQL-oriented and RPC, are adding awareness of objects. ORBs, on the other hand, are adding services for transactions, queuing, and messaging. MOM and publish-and-subscribe middleware are taking on the qualities of each other and also adding transaction, Web, and object capabilities. At the same time, new types of middleware and middleware-oriented tools are emerging, such as middleware management tools and middleware security. Still, the basic differentiation around scalability, recoverability, synchronous, and asynchronous communication remain effective criteria on which to select middleware for a given application.
While the need for different types of middleware wonýt go away, I expect to see consolidation in the market over the next 12 to 18 months. This will reduce the number of options and simplify the purchase decision. I also expect the cost of middleware to drop in the short term because of competitive pressure.
Despite the confusion surrounding the proliferation of middleware, it does serve to simplify application development. Without middleware, the coding required to build distributed component-oriented enterprise applications in a heterogeneous systems environment would be so formidable as to be prohibitive.