DBMS - September 1997 - Next-Generation Middleware

Next-Generation Middleware - Multitiered distributed systems and the Web are changing the face of middleware as we know it. - By David S. Linthicum

In many respects middleware is the glue that holds client/server applications together, as well as a productivity tool that saves developers from having to hassle with the details. It's a layer that exists between the application and the underlying complexities of the network, the host operating system, and any resource servers (such as database servers). Middleware makes vastly different platforms appear the same to an application and, if used correctly, saves time and money in the development of client/server and Internet/intranet systems.

Middleware is changing once again, however. Where RPCs and native database middleware once ruled the land, the next generation of middleware products now ties distributed processing together and ties applications to resource servers. Where we once built applications that communicated intrasystem using interprocess communications, we now use middleware to link the same objects across many different systems. What's more, with the maturation of the multitiered client/server architectures, middleware is providing a location for application processing, placing the business logic on a centralized layer. Finally, the Web is bringing a new paradigm and new products to middleware. The new generation of Web-enabled intranet or Internet applications needs glue for database and application server connections.

Clearly, middleware is becoming more complex. The number of new product introductions is not exploding, although the variety of middleware is increasing, as are the capabilities. So how do you get a handle on this moving target? First, you need to understand the standards and forces at work. Second, you need to identify the categories of middleware available and what purposes each category serves. Finally, you need to understand the products and which product fits into which category.

Defining Middleware

Simply put, middleware places an easy-to-use API between the application and the resource that the application needs. For example, if you are building a Java applet and would like to gather data as part of the operation of the applet, you can use Java Database Connectivity (JDBC) classes to access information on any number of databases. The JDBC classes remove the developer from the complexities of the target database and allow the applets to use any number of databases without having to understand the native feature of each database. Use a common API, and the middleware does all of the work. The same concept applies to Open Database Connectivity (ODBC) for Windows-based client/server applications and even to proprietary middleware layers such as Borland International Inc.'s Borland Database Engine (BDE).

Middleware goes well beyond simple database access operations. It's also able to provide transparent API-level access to other systems and system services without requiring that the developer know where the systems are. (See Figure 1.) The middleware layer can find the system using some sort of naming service, invoke a remote process, and return a response to the calling process. Products such as the Open Software Foundation Research Institute's (OSF) Distributed Computing Environment (DCE), distributed object technology such as those based on Common Object Request Broker Architecture (CORBA), and most message-oriented middleware (MOM) products are examples of such middleware.

Base of Standards

One of the trends in today's middleware products is the movement to standards, including ad hoc standards driven by powerful vendors (as opposed to committee standards such as CORBA). In the past, middleware was a proprietary proposition without the possibility for interoperability. Today, however, we are learning to use standards such as CORBA and Distributed Component Object Model (DCOM) as a base-level model and thus the wire for middleware products.

DCOM serves as a standard base for homogenous Windows environments. DCOM allows COM-enabled applications such as those built using Visual Basic, Delphi, and PowerBuilder to link to and invoke services of other COM-enabled application across a network using a remote procedure call (RPC) mechanism. Thus, DCOM serves as a primitive middleware layer itself or as the infrastructure for other middleware products. Middleware products that use DCOM include Microsoft Corp.'s Transaction Server TP monitor and Microsoft's Message Queue Server (MSMQ). Transaction Server is a TP monitor (more on TP monitors later) that defines transactions using ActiveX controls, and it communicates with applications and resource servers using DCOM. Falcon is a DCOM-enabled MOM product. Other DCOM-enabled middleware products include Multitier Distributed Application Services Suite (Midas ) from Borland, a multitier middleware product that lets you build application services using DCOM-enabled object request brokers (ORBs). (I discussed MIDAS in my August DBMS column, page 22.)

But DCOM is not the only ORB in town, nor was it the first. CORBA also provides an infrastructure for middleware products, using CORBA's natural ability to communicate with other ORBs running on similar or dissimilar systems through a common interface and wire-level protocol. The idea here is to tie systems together using the plumbing of CORBA and thus provide a virtual system for application developers. Products that use CORBA for their infrastructure include VisiBroker from Visigenic Software Inc. and Orbix from Iona Technologies Inc. Both of these ORBs, by the way, provide bindings to Java, also making them Web-enabled middleware. Internet InterORB Protocol (IIOP) is the wire-level protocol that these CORBA ORBs use together. IIOP is also considered by Internet vendors such as Netscape Communications Corp. to be the next wave in Web connectivity technology.

Types of Middleware

Middleware comes in a few flavors. For our purposes, let's divide middleware into database-oriented, virtual system, middle-tier, gateways, and Web-enabled. Of course, the problem with categorizing middleware is that it's difficult to do so. For instance, some database-oriented middleware may use a MOM infrastructure; and TP monitors can use almost any sort of middleware even though TP monitors are themselves middleware.

Database-Oriented

Database-oriented middleware, as you may have guessed, includes those middleware products that enable applications to communicate with local or remote databases. The idea is to provide some sort of API access to a database using the middleware layer to traverse the operating system and network layers on behalf of the client.

In many cases, even the API is hidden from the developer behind the features of the development tool. For instance, in the world of client/server development, database-oriented middleware is built in. PowerBuilder, for example, comes with native links to most popular databases; or you can use ODBC. Borland provides its BDE with almost all the company's development tools. The BDE lets developers communicate with any number of databases using native or ODBC calls.

JDBC is the most significant new database middleware standard. JDBC defines a call-level interface (CLI) for Java development and is not a part of the latest Java Development Kid (JDK) from the JavaSoft division of Sun Microsystems Inc. JDBC is really just a set of Java classes for specific databases and borrows heavily from the ODBC architecture. I am beginning to see JDBC as part of Java development tools such as JBuilder from Borland.

OLE-DB provides a single point of access for multiple databases. While moving slowly, it is adding value to the aging ODBC standard. With OLE-DB, the goal is to provide OLE automation access to any number of databases or add a COM layer between the application and the database.

There are tool-independent database-oriented middleware products as well. Rogue Wave Software Inc.'s DBTools.h++, for example, connects most C++ applications to databases. DBTools.h++ is able to make relational tables and attributes appear as native C++ objects. If Java is more your speed, Rogue Wave also sells a Java-enabled version known as JDBTools, which provides database connectivity directly from Java applets and applications. A similar product includes Persistence from Persistence Software Inc. Persistence wraps relational databases to make them appear as objects for easy manipulation by an object-oriented application development tool.

If it's legacy data you're after -- or the ability to access data on multiple machines -- then you should look at high-end database-oriented middleware layers such as EDA/SQL from Information Builders Inc. (IBI). EDA/SQL's approach to database middleware is to support as many operating systems, networks, and databases as possible. For example, you can access information on an Rdb database running on a DEC platform, as well as a DB2 database running on an mainframe, using a simple ODBC driver from a client. High-end products such as this are helpful for organizations looking to migrate to client/server without having to turn off their business-critical legacy systems.

Virtual System

Virtual system middleware enables developers to deal with many different systems as if they were a single system, using a common middleware layer and API set. This is true plumbing. You install native versions of the virtual system middleware layer on each system, configure a directory service, and link to applications, thus exposing heterogeneous system-level services to other applications on the network. (See Figure 2.) By invoking a series of APIs, you may be launching a process on a mainframe, updating a database, and invoking an application service on a Unix system, all from the same client application.

DCE is a good example of virtual system middleware, providing RPC-based access over a variety of systems that DCE supports. DCE provides its own security service and directory service, among other things, and is able to communicate with any number of resource servers using gateways and interfaces. DCE, however, was not able to meet the performance expectations of client/server developers requiring a rather heavy-duty server and network connection. What's more, the synchronous RPCs required that all participating systems be active in order for an operation to take place. Typically, RPCs must complete their operations before the application can resume operation. In other words, they block the application. DCE is sold by vendors such as Transarc Corp. (a subsidiary of IBM) and IBM Corp.

MOM is able to get around some of the limitations of RPC-based middleware like DCE by providing an asynchronous communication mechanism using messages. This means that the target server need not be up and running, and the MOM API call does not block the application. What's more, MOM does not require a huge amount of network bandwidth and is able to track transactions using persistence or storing the transmission to disk, to recover from system and network failures. Examples of MOM products include MessageQ from Digital Equipment Corp. (DEC), IBM's MQSeries, Pipes from PeerLogic Inc., and Microsoft's MSMQ.

The MOM fight will be between IBM and Microsoft in late 1997. As of summer '97, both vendors were preparing for a fight with new versions of their MOM products. IBM will release a new version of its MQSeries, code-named "Armada." Armada will support ease-of-use features, including better configuration and management as well as better performance, support for Java, enhanced security, and integration with DCE. IBM is also going to reduce prices. MSMQ is leveraging COM and will run on Windows NT and Windows 95, using both TCP/IP and IPX/SPX. MSMQ will first be released as a part of the enterprise edition of Windows NT and as a part of Transaction Server. Level8, a Microsoft partner, will ship gateways to connect MSMQ with MQSeries and other non-Microsoft operating systems. Software AG of North America Inc. and DEC are expected to deliver cross-platform implementations of MSMQ later this year.

Although MOM seems like a superior solution to technologies such as DCE, MOM suffers from a lack of support from popular development tools and a lack of standards. To use MOM (or DCE for that matter) with traditional client/server development tools, you must use hooks from the MOM vendors such as DLLs, ActiveX controls, or integration with the class library of the tool. What's more, this extra step forces the developers away from the traditional repository-driven development paradigm behind tools such as PowerSoft's (a subsidiary of Sybase Inc.) PowerBuilder and Oracle Corp.'s Developer/2000. Although slow in coming, a few tools such as Intersolv's Allegris support MOM (Pipes) as a means of allowing their application objects to communicate intersystem.

TIBCO Inc. is a message-oriented, publish-and-subscribe middleware vendor that recently introduced TIB/Rendezvous 3.0. TIB/Rendezvous provides ActiveX and Java integration, certified messaging, and fault tolerance using publish-and-subscribe broadcast middleware features. Based on ORB technology, TIB/Rendezvous provides the most value for the finance community that requires that a large stream of information get out to many clients (namely traders). Using publish-and-subscribe technology, each message traverses the network only once; however, TIB/Rendezvous software carries it to all subscribers on the network. In contrast, point-to-point communications would send the same message individually to each subscriber, wasting network bandwidth.

TP monitors are complex middleware products that provide a location for application processing as well as a mechanism for communications. TP monitors enable developers to define transaction services within the TP monitor environment, such as recording a sale or deleting a customer. The TP monitor sits between the client and the database server and thus uses the three-tier client/server model. (See Figure 3.) The client invokes a transaction on the TP monitor through a transactional RPC (TRPC) mechanism, and the TP monitor runs the transaction linking to the database when needed. The response, if any, is sent back to the client. Popular TP monitors include BEA Systems Inc.'s Tuxedo, Transarc's DCE-based TP monitor Encina, and Microsoft's Transaction Server.

TP monitors are powerful because they allow developers to define portions of the application as transactions. A transaction has clear start and stop points; if a failure occurs during the transaction, the TP monitor is able to roll the transaction back, never leaving the system in an unstable or inconsistent state. TP monitors are also able to multiplex the database requests. Because the client is invoking transactions and not connecting directly to the database, the TP monitor is able to funnel the database requests. For instance, 100 or so clients may only need 10 active connects to the database server. The connection-per-client restriction that limits the ability for two-tier client/server to scale is thus removed. In addition, TP monitors can read and write from heterogeneous databases during any given transaction and even maintain heterogeneous database integrity. A transaction may delete a record out of an Oracle database running on Unix while updating a record on a DB2 database running on a mainframe. Therefore, TP monitors are helpful in tying various legacy systems and databases together as a common virtual system.

ORB-based middleware includes simple ORBs that exist on multiple machines, communicating through a common wire-level protocol such as IIOP. Developers can build ORBs or ORB access layers into their applications and communicate with ORBs (applications) that exist on other systems through the common interface. Examples of commercial CORBA ORBs include Orbix from Iona and VisiBroker from Visigenic. The strength of this middleware approach is strong cross-platform support. However, building such distributed applications is complex, and few tools provide direct support for ORB development. What's more, ORBs don't provide developers with load balancing capabilities, or failover services.

Middle Tier

In addition to using middleware for simple peer-to-peer or client-to-server communications, the trend is to provide middleware layers that offer a location for application processing. I've already discussed how a TP monitor provides such services as well as a virtual system middleware layer, but there are other players as well. ORBs, for example, provide a location for application logic as well as a common communication layer. Similar to the TP monitor model -- but a bit more distributed and heterogeneous -- ORBs can exist on any number of platforms and may be preprogrammed to perform some application functions when called upon. ORB-aware applications or other ORBs can locate and invoke a method on a local or remote ORB through IIOP. The idea is to create a single application with objects existing on multiple hosts. But ORBs and TP monitors don't have all of the middle-tier middleware market. There are many other proprietary solutions: Borland's Midas, for instance, will work with most Borland tools (it currently supports only Delphi 3.0) and allows developers to place the application object within the proprietary ORB mechanism of Midas. Midas uses DCOM for its infrastructure. IBM is fighting Microsoft with its new middleware called Business Object Server, which provides class libraries that add features to applications such as access to databases and common application services.

Older middle-tier middleware may not be considered middleware at all. Fort� Software Inc. and Dynasty Technologies Inc. provide other proprietary ORB solutions to support their application partitioning tools. Developers build the application on a single machine, then automatically migrate the objects to other servers. Each uses a messaging mechanism to allow the objects to communicate. Dynasty wraps its transaction services around Tuxedo, and both have links to open middleware products such as CORBA-compliant ORBs and MOM. IBI's Cactus uses a similar mechanism, riding on top of IBI's EDA/SQL.

Emerging Technologies

So what's next in the world of middleware? In short, it's the Web, distributed objects, Web, Java, and -- oh yeah -- more Web. The advent of Web technology, in both its Internet and intranet flavors, has given new life to the middleware business. As corporations begin linking their databases and other resources to the Web, middleware vendors are seeing a great opportunity to create products to make things easier. This trend is ironic because the use of a browser as a common client/server application platform, and thus HTTP as the common middleware, has driven down the interest in client/server middleware at the client level. When considering the Web, you should know that middleware adds the most value at the server.

You can divide many popular Web-based middleware products into server-side and client-side categories. Server-side products create the link between a Web server and a database server, providing not only the plumbing but the means of pushing data back down to the client using HTTP and HTML. (See Figure 4.) It's not unusual to find Web-based middleware that provides application-development features as well, such as mapping database attributes to Web pages. This category is the largest of Web-base middleware and includes products such as WebDBC from StormCloud Development Corp. and the Internet Database Connector (IDC), now a part of Microsoft's Internet Information Server, to name only a couple. These products are typically easy to install and use, and they provide webmasters and application developers with visual development environments for creating content and populating the content with data from a database. At the high end you'll find products such as ActiveWeb from Active Software Inc., a software communications system that gives developers access to resources such as applications, databases, and Java-enabled browsers to exchange information over the Web. Of course other server-side middleware products let developers connect to application servers -- even existing legacy systems -- with ease.

Web-aware client-side middleware products are the latest rage. These products let developers link Java applets and ActiveX controls running on clients to remote databases. Products such as JETConnect from XDB Systems Inc. were the first to appear on the scene. JETConnect enables Java developers to connect to remote database servers using native Java classes. With the advent of JDBC, Java developers found a standard mechanism to do the same thing as JETConnect, and today there are JDBC drivers for most popular databases as well as JDBC-enabled development tools such as Borland's JBuilder. JDBC is much like ODBC, providing a standard means of accessing many databases using a common set of interfaces (Java classes, actually). All those vendors who once created ODBC drivers will find a new market for JDBC drivers as well. In certain respects, the Java development world will look, feel, and function very much like the traditional two-tier client/server middleware world of today.

Middleware vendors are providing value to popular Web development technology as well. Open Horizon Inc., for example, just introduced Ambrosia. Ambrosia is an event-management system for Internet business applications. The Ambrosia system uses a native Java implementation to give Java developers guaranteed message delivery, comprehensive security, and transactional capabilities.

Where Now?

You have to admit that middleware is not sexy. You can't see it, and, if it's working correctly, you'll never even know it's there. As an industry, we tend to focus on advanced visual development tools and high-performance databases. Middleware, however bland, is critical to the success or failure of your distributed systems. It lets us define a common application layer that we can share across application, and even throughout an enterprise. Therefore, the selection of middleware should be a strategic decision rather than a tactical one. Middleware decisions should not be made just by the application developers.

Moreover, although vendors promise out-of-the-box transparent access to data and applications, I've found that solutions such as these are never shrink-wrapped. You're going to have to invest in training, and you need to determine quickly how the middleware will fit into your standard toolset. For now there are few standards, lots of solutions, and a lot of middleware failures deadly to client/server development projects. I can't stress pilot testing enough.

Clearly we are moving in two directions: the migration to multitiered distributed systems and the use of the Web as an extension of client/server. ORBs, TP monitors, and proprietary middle-tier solutions provide the foundation for the future of distributed computing. We are already seeing better tool support for such products and a better understanding of the capabilities of the now-mature products. The Web is running wild, but it provides a new platform for old middleware. While we are just beginning to learn how to attach our Web servers to our databases, we are also learning how to attach our databases to our Web clients. Remember, it's what you have in the middle that really counts.

A product chart listing middleware products accompanies this article.

David S. Linthicum is the author of David Linthicum's Guide to Client/Server and Intranet Development from John Wiley & Sons (1997). He's a widely published author, speaker, and senior manager with Ernst & Young's Center for Technology Enablement in Vienna, Virginia. You can email David at linthicum@worldnet.att.net.

Please send questions or comments to dbms@mfi.com
Updated Thursday, July 17, 1997.