Book Excerpt

The 1997 Developer's Guide

Introduction

So what do you have to know before you start reading this book in order to make the most of your time? Who am I directing this book toward, what do you have know about me, and what assumptions am I making?

After I answer the first two questions, I'll discuss the two underlying assumptions of this book: that being able to manage the process of software development is critical for success in the next few years, and that fixed price software development projects are the optimal way to run a software development shop.

Who is This Book For?

This book is for the skilled software developer who is comfortable with the technical part of developing an application - the design and coding phases - and wants to improve how they go about the rest of the software development process.

You should find value if you're (1) an independent developer (the "one-man shop") who is either trying to polish their current process, or trying to grow into a multi-person shop, (2) the owner or a developer in a small (a few to a dozen people) shop, (3) a lone corporate developer, or (4) a developer or project manager in a small to medium size corporate MIS department trying to bring to some order to their group.

If you're in a large development shop or corporate environment, I'm not sure how well the procedures and processes here scale up. Our intent is to grow from a half-dozen people into a 20, 30 or 50 person shop sometime in the future, but until we get there, we won't know what we need. We're trying to put these processes in place now so that we're ready when that seven figure contract knocks on the door. How much of the material will scale to a 20 or 50 person shop? Don't know yet. But we'll find out, and I'm certain the groundwork we're laying now will stand us in good stead down the road.

Boring Stuff About My Shop

I've been writing custom database applications since 1982, first using dBASE and then FoxPro as my only tools. I had a computer training, consulting and programming firm with a half-dozen employees in Cincinnati throughout the '80's, but sold it in order to move back to my hometown of Milwaukee upon the birth of my first child. Until January, 1995, I worked out of my den as an independent. At that time, I hired my first employee and opened an office in downtown Milwaukee. At the time of this writing (Fall of 1996), we're up to six people - three developers, a technical support person who handles testing and other process-related tasks, a part-time Internet developer, and my administrative assistant who doubles as our webmaster and generally saves my butt on a daily basis.

Several years ago, I wrote a book about Rapid Application Development with FoxPro 2.6. Its purpose was to formalize the process of the development of desktop database applications. When a programmer starts out in the business of writing custom programs, they usually start from scratch and build everything themselves. After the first few (or many attempts), an idea dawns on some people: maybe not all of this has to be invented each time. That reusable code, repeatable techniques, and so on, can be an aid to building applications faster and with fewer problems.

Once the RAD process was in place, the work of building systems was a lot more fun, and a lot more profitable. The boring grunt work of setting up a new system and building the standard "we've done this screen a hundred times" interfaces was minimized, since core foundations and reusable routines that had been built and tested in the past could be plugged in easily, providing error-free functionality quickly and consistently.

As we've grown, one of the recurring themes in our staff meetings (OK, those impromptu chats in the halls) was how to improve the quality of our systems. The programming part was actually pretty easy, but writing the code is just part of the job. I stumbled upon Watts Humphrey's "Managing the Software Process," but it was one of those "10 equations per page" types of books. He managed a large organization and talks of "4% of your staff should be dedicated to..." For an independent developer, or a corporate guy who's one of three guys in the MIS department, this is fairly far removed from reality.

Programming is 90% of the job; the surrounding part - selling, writing specifications, design, estimating, coordinating the coding, finding employees, training them, testing (and re-testing), tracking employees activities, tracking bugs, shipping, handling issues after shipping, getting paid, and ongoing maintenance - these all take the other 90% of the time. And that second 90% of the time is the subject of this book.

The Process Is the Key!

In other words, this book is dedicated to the process of software development. Writing code is just one facet of that process. Yes, I know, I know, some people treat the choice of their programming language as if it were a matter of life and death. (And we all know that it's much more important than that.) But, throughout this book, I'm going to treat language as largely irrelevant. The crew in my shop use five to seven different languages, depending on how you want to define "different" - FoxPro 2.6 for DOS/Windows, Visual FoxPro 3.0 and 5.0, Visual Basic 3.0 and 4.0, Delphi 2.0 and a combination of Java and Javascript.

But the process of software development is the key to producing high quality custom applications profitably in order to grow your firm.

There are five reasons that I am as concerned about the process of software development as I am about the nuts and bolts of writing code and shipping EXEs to customers.

1. It's Deja Vu - All Over Again

The first big reason for me is that, as I grow my company, I find I'm seeing the same thing for the third time. I've been here before. Training people is time-intensive, and while I really like it, there are some parts I like more than others. I needed a way to get some of the routine stuff done better. So my training needed a set of common routines as well as the custom stuff.

2. Applications are Bigger and More Important

The second reason is that this process is becoming more and more important. When you were developing applications on a 286 with a local 20 MB hard disk and a network card to a server with a (gasp!) 250 MB main drive, well, the term "mission critical" didn't really pop up too often. These days, even small shops and individual developers have machines with the horsepower that can support large, important, sophisticated applications. The construction techniques you employ to panel your basement need not be nearly as rigorous as those you would use to build an entire house, and a sophisticated house-building process wouldn't be well suited if you tried to scale it to the erection of a 40 story skyscraper. And furthermore, given a bit of common sense, the crew that builds a multi-million dollar office building will benefit from their experience when they tackle smaller projects.

Of course, you may be able to get away with a casual, catch-as-catch-can approach to software development when you're writing smaller applications, but the first time you're asked to tackle a larger project, you'll need a more formal approach to the development process.

3. Revision Cycles are Accelerating

The third reason that process is important to me is that the rate of change is accelerating. (Well, I guess, technically, the rate of change is increasing. The fact that it's increasing means that change is accelerating, right? Nice to know something from college has still stuck around.)

Somewhere in the early '80's, dBASE II came out. I bought version 2.41 in 1982, and the main difference between it and version 2.40, 2.3, 2.2, and so on, were a reduction in the number of bugs and a few new features and commands. dBASE III and dBASE III Plus came out in 1985, and dBASE IV showed up, sort of, in 1988 and 1989. In 1990, FoxPro 1.0 came out, followed by FoxPro 2.0 in 1992 and Visual FoxPro 3.0 in 1995. Visual FoxPro 5.0 was released about 15 months after 3.0, and we'll probably have 6.0 arrive before most people get the plastic wrap off the box of 5.0.

So what's my point? Let's look at the amount of time between releases. It's shrinking rapidly. We had three to four years to get good - and to make a bunch of money - with each of the major releases. The next revision to 3.0 (which would be 5.0) came out 15 months later. Yikes! And this isn't the case just with FoxPro - it's happening with Delphi, Visual Basic, you name it. The product cycle is shortened. People aren't frantic about the rate of change anymore. We don't have time to think about it! It was truly frightening to hear an executive from Sun Microsystems repeatedly use the term "perpetual beta" when referring to their Internet products at the 1996 Borland Developers Conference.

4. Applications are More Complex

The knowledge requirements for developing applications now is significantly higher than it was a decade ago. Back then, you wrote a few hundred or a few thousand lines of procedural code, tried not to use the GOTO command too often, slapped in a few token comments, and shot the whole thing into a directory on the customer's machine. If you were sophisticated, you had a mechanism for separating data from the program, and you might even have a set of library routines that you shipped with every application.

In the big, bad world of development in the late 1990's (are we there already?), it's considerably more complex. Your application might have to run on Windows 3.1, Windows for Workgroups 3.11, Windows95, Windows NT, Netware, LANtastic, and maybe talk to some Macs as well. Your application has to install itself properly, talk nicely to other applications, be able to share data through OLE, and do a bunch of other nice Windows type things that aren't really that well documented and, frankly, don't always work that well.

You'll need to learn object-oriented programming, how to design classes, maintain encapsulation, avoid multiple inheritance, and handle performance problems that arise when your class hierarchy goes too deep. Your application must communicate with client-server back ends, switch between remote and local data, work with network drives with a variety of different mapping conventions, and deal with WANs as well as LANs. Of course, replication across three continents will have to be built in because some VP just read about this topic in Time magazine and wants their application to do it as well - by next Monday. And all of your applications will conform to the browser interface being fought over by Netscape and Microsoft these days, right?

Doesn't it just make you want to go back and bus tables at Burger Boy?

5. "I Hope You're Teaching Quality"

Finally, according to Dr. W. Edward Deming in his book "Out of Crisis," at least 90% of all defects are caused by process problems. My friend, Mac Rubel, has often stated "Programs don't come with bugs. You have to put them in there." To me, this means that a syntax error a programmer puts in their program isn't a defect. But when the program leaves their arena and goes to testing, the build lab, or the customer, it then becomes a defect. The proper process will prevent the syntax error in the program from becoming a defect.

Delivering high quality software has always been a particular issue with me. Of course, relatively few of you reading this have "We want to write crappy code and just shove it down the throats of our customers" as your company mission. But in the heat of the battle, it's easy to let substandard work get out the door. With the potential for errors increasing as described previously, it's important to buttress the process as well as we can.

This isn't just a good business decision, however. There was a landmark article in the September 1994 issue of Scientific American that bemoaned the crisis that software development is facing, and that it may well be the most critical issue facing the industrialized world over the next 10 years. Let's face it. The world revolves around software to an incomprehensible degree. Low-tech devices that have been around for hundreds of years now include microprocessors: a razor contains 2000 bytes of software code and a sneaker has twice that.

Just as pundits are arguing that the world is dividing into two groups - the "haves" being those that have access to the information, and the "have-nots" being those that don't. I would argue that the software development community is going to fracture into two groups. On the one side you have professional developers who understand the process of building applications, and on the other, hacks who go to bed each night and pray that none of their code breaks before morning. The fact that you picked up this book is a good indication that you belong to the first group (hey, even an author can suck up to his readers, can't he?) If you make it to the end of this book before dozing off, then the chances of you staying in the "haves" group is even stronger.

The Pricing Debate

One of the fundamental debates in the development of custom software applications is whether to charge a fixed price or charge on an hourly basis.

I believe that fixed price software development is a better business decision (if, of course, it's done properly) because it is a more profitable method. I also believe that it's a better decision for the customer, since they can evaluate a software project with a specific price and determine if it is a wise investment or not. And furthermore, many companies prefer a fixed price because it allows them to plan and budget for software just like most other "things" that they buy.

However, most developers charge on an hourly basis for one simple reason - because they do not know how to accurately price an application.

This book will show you how to accurately determine your cost of developing a custom application with a degree of repeatability that most developers can not do now.

Let's lay down some background on fixed price versus time and materials methods before we begin.

Why Do Developers Hate Fixed Price Work?

The fundamental reason is the way that developers create the price in a fixed price situation. We've all heard the jokes, you know, "take your best estimate, double it, and then tack on a bunch more." The trouble is these aren't jokes.

The bottom line is that developers are simply guessing. No wonder they hate fixed price work. They don't know how much to guess, and if they guess wrong, they're going to lose their shirt. Most programmers are incurable optimists. The ones who have been around for a while are knowledgeable incurable optimists. They always figure, despite being burned on twenty applications in the past five years, that the next one will work fine, that there won't be any problems, and they'll finally make a buck at it. It's in our nature to guess low to begin with. You never hear of a programmer guessing way too high and the customer going for it, do you?

Let's take an analogy. Suppose you were a grocery store clerk, and the customer walked up to you with a basket stuffed to overflowing. You could charge them one of two ways. The first is to guess how much they've got in the basket. You take a look, think real hard, try to remember what you guessed on another cart that looked like it, and then pull a number out of the air.

Obviously, this isn't a very good method. If it was, we'd all be playing grocery store lottery each week. Now, as bad as this method is, let's suppose that you never found out how good your guesses were? That's right, after a customer left the store, you were never able to find out what the actual value of that cart of stuff was. So you'd never be able to determine if you were guessing well, or if you were really lousy, or if, with just one small tweak to your guessing routine, you'd be able to guess right on target.

Of course, there is the ultimate measuring stick. If, at some point, the grocery store manager comes by and says that over the past month, the store's income was lower than the store's expenses and you're all fired - well, that's a pretty bad way to measure. If the manager would come over and say that today's guesses, in total, were lower than today's actual sales, then everyone would make random adjustments to their guesses, but it's still just a crapshoot. What if tomorrow all the customers start putting bottles of Dom Perignon in their basket?

This whole scenario sounds pretty silly, but isn't it what 9 out of 10 programmers do? They take a couple of long hard looks, perhaps shuffle around the contents of the top of the grocery basket a bit, and then - wham! - guess away. There is no allowance for something really expensive that's been hidden out of sight; like a routine that's going to be a bear to code, or a screen that's trivial at first blush, but contains some hidden requirements that are going to be extremely difficult to do.

The second part of this analogy is dead on accurate as well. Most programmers do not have any mechanism for determining how much they "spent" on a project, so they can't tell how good their guess on their last project was. Here we have a recipe for disaster. Wild, random guesses with virtually no logical reasoning behind them, and no mechanism for measuring the accuracy or providing feedback with the intent of improving the next guess.

Rather strange for a profession that relies on logic as its cornerstone, wouldn't you say?

You're probably thinking, "There are a lot of awfully smart people in this industry. If none of them have figured out how to, then it probably can't be done." But if you've lost all hope of ever figuring out how to do fixed price work, let me reassure you. It can be done. It just takes a few tricks and some discipline.

Before we get into the nuts and bolts, let me explain our reasoning for wanting to do fixed price work. This is more work than simply recording your hours and sending in a bill every couple of weeks. There has to be a corresponding benefit to it.

What's the Advantage to Fixed Price Work?

My firm, like most others, exists for one reason: to make money for the shareholders. The owners expect to make a return on the funds they invest in the company just as if they had invested the money in a mutual fund or precious metals. In my firm, there is one investor - me. These circumstances don't change the rules.

Given that a company makes money for the shareholders by making a profit, the mission of the company should be to maximize its profits, so as to maximize the return for the shareholders. I feel that software development firms can do so by doing work on a fixed price basis.

Doing hourly work has a built-in profit ceiling, because there are only so many hours in a day. This argument holds regardless if you're a one-person shop or if you have 50 folk in the office. There is still a maximum profit per "unit of production." With fixed price work, however, the maximum profit, while it still exists, can be significantly higher - to a factor of anywhere from two to ten times as much or more.

The advantage of fixed price development is being able to charge by value, not by cost. Suppose your hourly rate is $75/hour, and an application you're quoting will take you 200 hours. That's a price of $15,000. If, on the other hand, the application is worth $30,000 to the customer, your hourly rate has doubled, since you will spend 200 hours to bring in $30,000. The key is that the application is worth $30,000 to the customer, and it's totally irrelevant to them (in fact, it's none of their business) what your cost is.

Let me put this another way. You do not have to price the job at your cost. In fact, you'd be doing an injustice to your shareholders if you did so. Microsoft, at the time of this writing, is making approximately 20-30% net profit. This means that they could bring their prices down 10% and still be making a healthy profit. In fact, if it were not due to competitive market pressures, we'd probably still be paying $495 or $795 for Word, instead of $99.

The trick is to know what your work is worth to the customer, and to know what your costs are. Section One of this book will deal with assessing the value of the project to the customer, and Section Two will explain how to determine your cost to produce that project. When the value to the customer is larger than your cost, you both win. You'll make a profit, and the customer will get the project done. You wouldn't do the work if you were going to lose money.

I can't stress enough that your cost doesn't have to be your price. The only reason the cost number is so important is that you don't want to quote a price to the customer that is below your cost. Since most developers guess when putting together prices for applications, it's very easy for them to provide a price that will end up being below their cost. This is what has happened when you hear "I really took a bath on that one - I thought it would take 500 hours and I spent nearly 800!" They simply provided a price that was lower than their cost. Despite common wisdom, you can't make a profit through this method!

We're used to thinking that we should bill by the hour like the other professionals: lawyers, accountants, and so on. But with a properly drawn up specification, you can stop thinking about charging for time and instead build (and charge for) a product.

When Isn't Fixed Price Work a Good Idea?

The one requirement for fixed price work is that the deliverables be measurable. The application must be described to an extent that it is clear to both the developer and the customer what is included and what is not.

This is difficult to do. That's why an entire section of this book is dedicated simply to write a specification. When it's impossible to write a spec, the fixed price path is not a good idea.

Our company will go the fixed price route for new applications, modifications (upgrades, changes) to existing applications, and new modules for existing applications, because we can write detailed specifications from which we can determine our cost. We may or may not be able to provide fixed prices for modifications to another programmer's system. There are some types of work that we won't do on a fixed price basis - ever. Obviously, any situation where the specifications are so vague or incomplete that the deliverables can't be measured. Or for work that is time-dependent, but the amount of work is not quantifiable - such as design and analysis, general consulting on application requirements, and debugging or troubleshooting. Most of the time, working with someone else's system falls into this same category.

What About "Not To Exceed" Contracts?

Occasionally you have a request for a contract that specifies actual cost (time and materials spent) but with a top end "Not To Exceed" amount. These are the worst of all possible worlds. The customer is asking you to take the risk - since it's likely that a detailed spec doesn't exist - yet there is no reward for the corresponding risk. Should a customer ask you for a NTE contract, in the words of King Arthur in Monty Python and the Holy Grail, "Run away! Run away!"

The individual in question was a personal injury lawyer. The stereotypical "ambulance chaser" who worked on a contingency basis - netting fees way out of proportion to the time spent on the case.

There are all types in this world, aren't there?