|
TestBytes is a test data generation tool that connects to your database to create test data for your relational databases.
One of the least enjoyable things about building database applications is that you also have to test them. If only we could skip that phase, the whole process would be so much more fun. The artistic instincts in us could kick into gear, and we could crank out work at a frenzied pace.
But such is not the case. On the contrary, the testing phase of database application development often takes as long as the original construction phase did. I suppose this is one of the things that distinguishes artistic endeavors from craftsmanship -- craftsmanship must suit some purpose beyond the purely aesthetic. In our business, that means that the software we build must work. Thus the necessity of rigorous testing is inescapable.
With the acknowledgment that database apps require lots of testing comes the realization that they also require lots of test data. Often highly paid database architects or programmers sit around for hours on end, keying in banal customer names such as "John Doe" and insipid addresses such as "123 ABC Street." They do this because they know the database and the types of data it expects. Still, this just doesn't seem to be a very good use of anyone's time -- database architect or not. Who wants to sit around keying in data no one will actually use? If only someone would devise a way to generate this data for us automatically . . .
Enter Logic Works' TestBytes. It's designed to generate test data for a wide variety of DBMSs -- in fact, you can use it with just about anything for which you can get an ODBC driver. Even if you can't get an ODBC driver for your DBMS, you can still generate flat files, which you can later import. It generates random test data based on the datatypes of the your fields -- valid dates for date fields, numbers for numeric fields and characters strings for character fields. You tell it how many rows to generate and it takes care of the rest.
One of the things I liked the most about TestBytes is that it lets you get beyond generating purely random column values. Nonsensical data is of questionable value -- it can make determining whether your application is working correctly difficult. (Is the weird data my app or my test data?) TestBytes comes with a number of default "profiles" that let you specify that a column is a particular type of commonly stored data. For example, you could specify that the Name column actually stores first and last names, that the au_id column stores social security numbers (see Figure 1), or that Shipping_Dist is intended to store state abbreviations. Then, when TestBytes generates test data for your tables, you get valid first and last names, social security numbers, state abbreviations, and so on. The data will look as if it was entered by a human.
Each column in each table can have its own associate profile, and you can also create your own. For example, you can specify that the CustID column consists of the first four characters of the customer's name followed by a random number. You could specify that social security numbers must include hyphens and that phone numbers must include parentheses. You can also create completely new profiles, wherein you specify the actual values TestBytes should use when filling associated columns. A feature that's missing is the ability to use data from actual tables to fill columns with test data. It would be nice, for example, to be able to tell the tool to grab the customer numbers it supplies from the customer table. I don't consider this a huge problem, but I'd like to see the feature added in a future release. All things considered, TestBytes' intelligent data generation makes creating loads of plausible test data a snap.
A feature that would make a nice addition to TestBytes is a script-generation facility. Rather than having to choose between immediately adding rows to the database or saving them to a flat file, it would be nice to be able to generate a SQL script containing the new rows. Then you could run the same script over and over without having to regenerate the test data. This would be particularly helpful if the generated data turned up a unique problem with the database design. You could make adjustments to the database, then run the script again to see if your changes worked.
Another feature I'd like to see in the product is report generation. I'd like to be able to generate a report listing important statistical information such as the number of rows created and the types of data generated. I think this would help the tester manage the testing process a little more easily. Hopefully this will be added in a future release of the product.
I was disappointed with TestBytes' inability to reverse engineer referential and domain integrity. I assumed it would look at the constraints in my databases and respect them, but this was not so. Not only were RI constraints not respected, check constraints were also ignored. This caused data to be generated that was rejected by the database. I had hoped that the program would use the schema information at its disposal to generate not only plausible data but valid plausible data, but that wasn't the case. Though TestBytes obviously interrogates the system catalog to find out what columns a table has and what their data types are, it doesn't go far enough. All the information it needs can be found in the system tables -- it just doesn't use it.
I wouldn't mind having the ability to generate invalid data if it were available as an option. I think this feature could be useful, particularly with boundary testing. However, generating random data that may or may not be valid is a serious flaw in the product.
Logic Works' answer to this is to define table relationships inside TestBytes itself -- to basically redefine your database schema within the TestBytes application. This works but is unnecessary. A tool that generates data for relational DBMSs (especially one from the makers of ERwin) should certainly respect relational and entity integrity constraints. If nothing else, I would at least expect the product to be able to import ERwin models, but that's not the case. Logic Works assures me that the next version of TestBytes will indeed reverse engineer column and table constraints.
I also had some sporadic problems with TestBytes and certain ODBC drivers, but I attribute these more to problems with the drivers and with the ODBC spec in general than to TestBytes itself. For example, try as I may, I couldn't get the default profiles required for sensible data generation to be recognized or used when connected to Borland's InterBase. I got a bevy of strange errors when I connected and also when I later disconnected from InterBase. The same was true when, connected to DB2 via ODBC, I received a number of strange error messages when connecting and disconnecting, but the software still appeared to work. Again, I blame these on the inconsistencies among different ODBC implementations rather than on TestBytes itself. I was successfully able to generate test data for Sybase, Microsoft SQL Server, and Oracle, so I suspect the problems with the other platforms are ODBC-related.
All in all, I'd say TestBytes is a worthy addition to your toolbox. The current version offers a data generation facility that could prove invaluable to many database developers. I found performance acceptably fast and the product itself reasonably robust. Future versions promise to add additional key features, making the solution a must-have for most of us. Especially if you usually key in all your own test data, TestBytes could save you a huge amount of time during the testing phase. And you never know -- it might even make the whole process a little more fun.
What did you think of this article? Send a letter to the editor.