DBMS Interview - June 1996
Building a departmental client/server database application for 20 users is challenging enough. But supporting 200 or 2000 users in an enterprise system is quite another challenge. Paul Butterworth knows this endeavor is not for the faint of heart. As the vice president and chief system architect for Forte Software Inc., Butterworth is responsible for designing the Forte Application Environment.
Since its initial release in August 1994, Forte has pioneered the shift from two-tier client/server applications to partitioned or multi-tier applications. Now in its second release, Forte is a leading development environment for the building of large and complex enterprise-scale client/server systems. In the process, Forte Software has learned many lessons about what makes these applications different and what they need to succeed. As more tools and more developers embrace distributed computing, and the Internet evolves into one huge distributed application, Forte Software's foresight into current industry trends is hard to deny.
Paul Butterworth brings a deep background to his role as Forte's chief system architect. He joined Forte after serving as the vice president of engineering at GemStone Systems Inc. (Beaverton, Ore.). (In mid-1995, GemStone changed its name from Servio Corp. when it repositioned its object DBMS as a Smalltalk-based application server.) Before that, Paul spent more than 10 years at Ingres where he was vice president, engineering and chief scientist.
DBMS Editor Maurice Frank spoke with Paul at the DBMS office in San Mateo in January. Forte executives were visiting DBMS to demonstrate Forte Express, a new visual application generator for the user interface and database access components of a Forte application.
DBMS: Forte is known for shooting at the high end of the client/server market. How do you define the high end of the market?
Butterworth: From the technological point of view, we think the high end comes in two dimensions. One is large systems in terms of volume. How many users are on the system? How many transactions are you running? That would be the traditional picture of the high-end system. But many high-end systems may not have extremely large numbers of users or transactions, they may just be very complex systems with extensive asynchronous processing, complex algorithms, and processes running in the background on a server.
When you were designing the architecture of Forte for the high end of the market, what did you do differently than if you were going for a broader part of the market?
Much of the engineering team building Forte had experience building tools for the low-end market -- the traditional query, update, and simple application builders. We were successful with those kinds of products, but there was always a large number of users and projects for whom those tools wouldn't work. Those users had to drop back to writing C++ code (actually it was C in those days).
We wanted to tackle the kinds of problems that users couldn't solve with the tools that we had built previously. For example, if you tried to build an application for large numbers of users, the resource requirements on the server were just too high. Another example is that many users wanted to do asynchronous processing or background processing for tasks that would take too long or when users couldn't wait for the response. None of the tools on the market could handle that. Reliability in distributed environments was another big issue, and none of the tools out there provided any facilities to make your application more reliable. If anything failed, you were stuck and had to restart.
Does Forte plan to stay in the high end of the market, or do you plan to expand into broader markets? And, if you do expand, how will you compete against the likes of Visual Basic and PowerBuilder?
Our picture at the moment is two-pronged. One is our focus on the high-end market because we think that there aren't any other good tools to support that market, and because there is a huge number of things we still want to do to cover all of the problems we see in that area. At the same time, Forte has all the facilities needed to build lighter-weight applications. And many of our users building big sophisticated systems also build smaller lightweight systems using Forte. This alleviates training problems and gains more leverage. That's our current focus.
The high-end market is huge. We're talking about multiple billions of dollars. Historically, this has been the mainframe CICS market that IBM and corporate IT have used to build critical systems. The low-end market is a reasonable size, it's on the same order of magnitude -- maybe more, maybe less -- than the high-end market. We view ourselves as alone in the high-end market now. There are other people targeting it, but we see ourselves as technologically ahead of everyone else. So to unfocus now would be foolish for us. I think we have established ourselves as the leader in this market technologically, and now we have to do it in terms of market share. Later on we can look at other options.
Take one beach at a time?
Yes, but having said that, if it ever comes down to having to win over the low end as well as the high end in order to become a de facto standard, then we'll get a lot closer to the low end. But right now the two markets are reasonably well-differentiated.
Many tools such as Visual Basic and PowerBuilder are now embracing partitioning and distributed applications. What would Forte Software say to developers using those tools? Are they missing anything that Forte has?
The bottom line is that our competitors still have a lot to learn. We've learned a lot over the past two years about what constitutes a full-partitioning solution. Moving code here or there doesn't really solve the problem. If you leave it up to the user to figure out where all this stuff goes, then you've made the user's life a lot harder. You need automated support to help users with those decisions. It may not be perfect the first time out, but at least you've given users some place to start, and you've relieved the burden of doing all the grunge work.
Another issue is reliability. If you're going to partition, you will get more moving parts, so things are going to be less reliable. That means you need features built into the system to bring levels of reliability back up. If you don't have that, it's not clear that partitioning is going to be a win.
Another requirement if you partition is support for a multi-threaded environment that makes the servers you create relatively efficient. Many of the newer partitioning systems don't have these infrastructure elements. Some just break up the system into two single user parts, so you don't really get any leverage, and the gains you expect to see just aren't going to be there when you need scalability. Those are some examples of obvious things. For big applications, some sort of transaction support is a big issue.
Partitioning is a nice buzzword. Many people are talking about it, and some people are starting to do some technology, but it's actually part of a larger concept: distributed applications. Yes, you can build in partitioning but then you have reliability issues, performance issues, management issues, and so on. How do you monitor the system to be sure you're getting adequate performance? Where are the bottlenecks, and how do you identify them? The technology is much more than application partitioning.
We've learned a lot about partitioning from release 1. We think we did a pretty good job of it, but we had to add a lot in release 2. For example, we had to expand our performance monitoring. In release 1 we monitored five or six parameters. In release 2 we monitor over 200 system variables. This is due to feedback from our users.
Forte was an early pioneer in distributed applications. You and your customers must have learned a lot. What surprised you and your customers the most about implementing large-scale partitioned applications?
The whole issue of how to manage large distributed applications is a big problem. If you're going to build a large application in terms of the number of users or the complexity of the application, the development effort is qualitatively different than it is if you're going to build a small lightweight inquiry application. You really have to think about designing the application. You really have to think about issues associated with doing some scalability testing. Even with all of Forte's facilities that make it easy for you to tune the application and rearrange the pieces for optimum performance, you actually have to do some testing to decide which arrangement is the best one. So the whole issue of how you develop applications is different from, "I'm just going to sit down and start typing some code." This isn't really a surprise, but if you think that way, you're going to have a tough time getting the application built.
Another issue is the importance of integration. When you're building larger systems, you're not just going to write some code. You're going to integrate other pieces such as mainframe parts. And integration is incredibly hard because any time you take two pieces of software and try to hook them together, both pieces may go on assumptions, and those assumptions never line up. One of the biggest values we add is that Forte provides "pre-wired" integration facilities that support other technologies such as CORBA and OLE. That's a huge plus because if you have to hook them up yourself, it's a major, major effort.
What kind of common mistakes have you seen people make when trying to build large partitioned applications?
Probably the biggest mistake is going into the development process assuming that you already understand everything you need to know. You might expect to use a standard waterfall development model in which you're going to do requirements analysis, then design, then you're going to write all the code, and then integrate and test it. But for many users a lot of the pieces of the problem are new. You may never have built a distributed multi-tier application. Many of our users have never used object-oriented technology before. A small fraction have never used a relational database before. So it's very important for you to build up your skill set as you go. And to do that you should definitely perform the development in an iterative or incremental fashion so that your skill set builds up over the life cycle of the application.
Have you seen customers go overboard and over-partition applications?
Yes, especially when users are first learning. When you get a new tool you really want to run it through its paces, so you get in there and create applications that have tons of partitions. Then you look at it and say, "Why did I create all those partitions? Did I gain anything by building the system that way?" Then after you get more familiar with the tool you get a little more practical. If you have five machines, and you want to take advantage of all of them, that doesn't mean you need a hundred partitions. But I don't think there's been a big problem with that. And because it's so easy to rearrange things, if you create too many partitions and you realize there's a lot of overhead, you can easily cut back and everything works fine.
The question is interesting because the default partitioning is relatively conservative. It creates what it thinks is the minimum number of partitions that will get you good performance and support the environment you describe. If you get a lot of partitions it means you're customizing the partitioning system and creating more partitions. You need enough complexity to get the job done, but you don't want to add any extra complexity because that makes building, testing, and maintaining the application more complex.
Are multi-tier applications more complex? Certainly! By definition they have more moving parts. But when you see applications moving from a two-tier to a multi-tier architecture, they're doing so for a reason. That reason is to get more scalability or more functionality or more reliability.
In terms of number of users, what are the largest Forte applications you've seen, and how do they compare to mainframe systems?
I'll give you a couple of numbers. If you look at the applications deployed with Forte today (January 1996), the largest ones in terms of the number of actual concurrent users are probably around 200. Some might be a little bigger; a few might be smaller. If you come back and ask me that question in two or three months, that number will be a lot bigger -- around 400 -- and the reason for that is that the product has been available for a relatively short time. And building these big systems takes time, so they're slowly rolling out.
Another way to look at it is to consider systems currently under construction. A number of those systems are in the 8000 to 15,000 user range. Those applications will start rolling out later this year and early next year. It takes a couple of years to roll out a system that large. Those systems rival mainframe systems, and many are actually replacing mainframe systems. We've talked about installed applications with 200 concurrent users. But one of our goals is to ensure that our testing goes well beyond the requirements of whatever systems are being built. So we do our own internal testing at approximately 1000 concurrent users.
Forte is able to wrapper existing mainframe applications. Approximately how many Forte users integrate mainframe applications into their Forte applications?
At this point in time, it's less than half of our users. If I had to take a guess, I'd probably say it's around 25 percent of our users. Getting connected to the mainframe is a part of what they're doing, but it's usually not the core of the application. There's a fair amount of connectivity to mainframes, mainly for data access, and some for application access.
How does Forte integrate with CORBA and OLE?
CORBA and OLE are standard interoperability interfaces that can be used across the industry, and we're focused on interoperability with those standards. If you look at trends in the market today, a lot of people are moving toward, or at least talking about, a more component-based application development environment. And a lot of those components will have OLE or CORBA interfaces, so being able to hook those into your application, however you're building it -- with Forte or something else -- is going to be critical. Similarly, if you're building components, you want them to be accessible from the rest of the world through OLE and CORBA interfaces.
Forte is one of the early pioneers in supporting CORBA. We had a CORBA interface in our first release, and that supported the first release of the CORBA specification. We've been very aggressive about supporting that kind of interoperability because that's what customers need to solve their application problems.
Can the current version of Forte work with OLE and OCX controls?
The current version works with OLE, but it does not support OCXs. Of course, we're working on that.
Can you talk about the major design goals for the next version of Forte?
We've just rolled out release 2, so our attention is now focusing on what we're going to do next. The Forte product has three major components: the development environment, the runtime infrastructure for distributed applications, and a collection of application management tools to distribute, install, and monitor the application. Customers building systems for 10,000 to 15,000 users need to have a collection of requirements in each of these areas. If you look at application development, a big thing for us is to make it easier and more productive for application development teams. You've seen a demo of Forte Express, and you'll see more advances in those areas in the next couple of releases.
In the infrastructure area, there are a couple of big things. Certainly performance is an issue, but we're actually in pretty good shape there. Another big issue with larger systems is reliability. If you think about an 8000 user system with maybe 1000 servers involved, all the pieces may not be up and running at the same time. We're building in more facilities to make it easier for users to manage reliability so they can support an environment in which pieces are failing all the time.
Managing the application is always the biggest problem when you build large distributed systems. We have a bunch of facilities to gracefully upgrade the application over time. Well, that's very nice, but when the applications get up to 8000, 10,000 or 15,000 users, we're talking about doing all that while the system is running. There are additional facilities you need beyond the obvious ones to make that work. That's another area we're doing a lot of work in.
How do you balance short- and long-term product plans, especially with respect to the product architecture?
We have what we think of as a long-term architecture. We've built Forte as a completely object-oriented system. There are a lot of interfaces, so we have the option of switching out any component of the system at any time to respond to new technology directions. We put a lot of effort into building this long-term technology architecture, then we build relatively short-term development plans for what we're going to do next. Our active development horizon is something like 12 months, but what we're thinking about is maybe two years. Anything beyond that is speculative. That gives us a lot of flexibility to respond to feedback we're getting. The best ideas for what to do next come from customers, so when we get that feedback, we can quickly factor it into the next release.
You've mentioned how important it is for an application to be able to recover from failures. What happens if an application server in a remote city goes down? How does Forte support that?
Applications can have fault tolerance built into them. Our partitioning system allows you take a partition, or what you can think of as an application server, and replicate it. You can replicate application servers to provide a higher level of scalability, or fault tolerance, or a combination of the two. So, if you're a client talking to a server and that server fails, and there's a replica of that server out there, then you'll transparently switch over. When we recognize that you can no longer send messages to the current server, we'll re-route you to another server and the conversation picks up there.
What types of modeling are available in the Forte environment?
In terms of design modeling, there isn't really anything built into the Forte system. We're very focused on object-oriented design because we feel that's very compatible with building an object-oriented application. We're partnering with half a dozen major vendors of object-oriented CASE tools including Select Software Tools, Protosoft, Ptech, IDE (Interactive Development Environments), Rational Software, and I think there are a couple more. We're working with all of them to have clean connections between their tools and our tool. Some vendors already have tools out; the others will be out in the first or second quarter of 1996.
How long does it take an experienced developer to become productive in the Forte environment?
If you're a developer with client/server experience, and some of that client/server experience is object-oriented, then you will pick up Forte very rapidly. Our standard introductory class that covers the Forte development tools, system management tools, partitioning, and the whole system is about a week long. Our users report that they're productive after taking that training plus another week of hands-on experience. That's for folks with client/server and object-oriented experience. As you take away chunks of experience, it takes longer to come up to speed, but that's not specific to Forte. We offer other classes, such as distributed design and advanced classes in tuning and system management, if people need them.
On average, how many developers are on a typical Forte team?
Typical is pretty hard to describe. Generally, the teams are smaller than the users expected them to be at the start of the project. We like to see application teams of four to eight developers. Some development teams are bigger than that -- as many as 20, or even up to 35 developers. At that point you need to break up the system into subsystems. We recommend relatively small teams.
On a project with a smaller team of up to eight people, how long does it take a company to bring a Forte application to a 1.0 deployed stage?
We have had a huge range. Some people fire these things up and have an initial application out in a month or six weeks. More typical is three to six months, but it depends on scale and complexity. There's always the issue of when to begin measuring: when you start analyzing the system, or when you begin coding. If you haven't done any analysis, I would guess you're in the six-month range. If you have already done the analysis and just need to code, three months is probably more practical.
How would a customer use Forte to build Web applications?
This is a good example of how short-term development plans pay off. A year ago there were no customers asking for Web support. Today that's not true. When you build a Forte multi-tier application, you're building all of these very useful services that collect all of this business information. Usually, day-to-day users spend hours working on Forte applications that have custom Forte interfaces. But a more casual user doesn't want to take the time to learn these applications. They're going to use their favorite browser, and they would like to be able to hook into these servers and access the information. It may not be as efficient an interface, but users understand it in a casual way. So we're building technology to let you build servers in Forte that talk to production applications, and they'll also have a Web interface so you can interact with them through a browser. We're building class libraries and development tools to make this interaction with Web browsers possible. These applications will accept HTTP requests and spit out HTML-formatted data to browsers. That's the initial piece of work we're doing, and it should be in customer hands by the time this interview is published.
Can you describe what the browser interface will be like? Will it use Forte-specific tags, or Java, or Microsoft VB Script, or something else?
The initial ones will be standard HTML. The next big question is how to interact with Java applets living in the browser or OCXs or VB equivalents. We're now starting to develop this. We'll be adding on to our integration packages so that's a pretty natural evolution for us.
What types of applications are customers building with Forte?
We have been talking to customers running transactions in the 1000- to 20,000-a-second range. When you talk about transaction loads that big, you can assume they're not being generated by folks sitting at a terminal. They're automated feeds of different kinds. They're looking at Forte because it provides the infrastructure and tools and good levels of scalability they need to do that. One user benchmarked 400 transactions a second from an automated feed.
We also have applications that require extremely high levels of reliability such as New York City's 911 system. These are systems that are limited to, at most, a few minutes of downtime a year, and even that downtime is undesirable. These are nonstop systems that need fault tolerance. These are much more realtime systems than the typical data-entry transaction-processing system. In one application there were five servers doing essentially the same work, so if any one failed, the other four could pick up the work.
We have several customers who are trying to put in place a distributed application architecture for the corporation as a whole, and they're using Forte as the foundation of that architecture. That's more than just a 10,000-user system. It's an entire corporate information infrastructure built on a collection of distributed services that get reused across all the applications -- and the infrastructure for that is Forte.
Forte Software, Inc., 1800 Harrison Street, Oakland, CA, 94612; 510-869-3400 or fax 510-834-1508; http://www.forte.com.