The inside scoop on testing database-driven Web sites, and how new tools can help.
Although Web site testing has much in common with the testing of standard client/server applications, there are unique considerations that can affect the focus of your testing strategy. In this article I discuss some of the factors that determine what to test in a Web site, the special considerations of database-driven Web sites, and how new Web test tools can help.
Validation or functional testing. This is typically a core aspect of testing to determine if the Web site functions correctly as per the requirements specifications. Sites utilizing CGI-based dynamic page generation or database-driven page generation will often require more extensive validation testing than static-page Web sites.
Load testing. Will there be a large number of interactions per unit time on the Web site? If so, you may want to perform testing under a range of loads to determine at what point your system's response time degrades or fails. The Web server software and configuration settings, CGI scripts, database design, and other factors can all have an impact. You will probably want to test the entire system under various conditions to get realistic results, but you may also want to consider separate testing of database response, server response, applet responsiveness, and other areas if the application is especially complex.
Stress testing. This refers to testing system functionality while the system is under unusually heavy or peak load; it's similar to the validation testing mentioned previously but is carried out in a "high-stress" environment. This requires that you make some predictions about expected load levels of your Web site.
Usability testing. Is your intended audience the general public? In-house Intranet users? Computer experts? Schoolchildren? The intended audience will determine the "usability" testing needs of your Web site. Additionally, such testing should take into account the current state of the Web and Web culture, because these will influence user expectations (for example, Web site navigation is expected to be extremely intuitive -- Web users do not expect to read manuals or help files).
Security testing. If your site requires firewalls, encryption, user authentication, financial transactions, or access to databases with sensitive data, you may need to test these and also test your site's overall protection against unauthorized internal or external access.
Unit and integration testing. Unit testing of code modules, objects, or discreet application functions is a standard part of testing any client/server or distributed application; integration testing may be needed to determine if various modules, other applications, and other parts of your site work together properly.
Regression testing. If the project is large and complex, you may need to continuously retest everything as the site is initially developed and code is reworked to accommodate changes and bug fixes. Smaller, less complex projects may have minimal regression testing needs. As the site is upgraded and modified over the years, will there be significant changes requiring continuous retesting?
Link testing. This type of testing determines if your site's links to internal and external Web pages are working. A Web site with many links to outside sites will need regularly scheduled link testing, because Web sites come and go and URLs change. Sites with many internal links (such as an enterprisewide Intranet, which may have thousands of internal links) may also require frequent link testing.
HTML validation. The need for this type of testing will be determined by the intended audience, the type of browser(s) expected to be used, whether your site delivers pages based on browser type or targets a common denominator, and how strictly you want to adhere to HTML standards and/or Netscape, Microsoft, or other extensions.
Reliability and recovery testing. Do you require uninterrupted 24 3 7 availability? Redundant database servers? Scalable Web servers? Depending on how critical your Web site is to your business, you may want to simulate various "emergency" scenarios (such as failure of a hard drive on the Web or database server, or communication link failures) in a test system to be sure that your production system will handle them successfully.
Server log testing/report testing. Web sites that use advertising and track site usage for marketing needs may need extensive testing to ensure the accuracy of logging and reporting capabilities. (For more information on log analysis, see Dan Rahmel's article on page S12.)
The primary factors in performance that are under your control are your Web server software and hardware, the database, and the Web server connection capacity. Use of scalable systems can mitigate performance problems that aren't initially apparent, but the true scalability of such systems should be tested if it's a critical factor. Also, Web servers, database servers, and other system servers may need to be tested separately to find any bottlenecks that would negate improvements in other system servers.
Load Test Tools. The use of new test tools can significantly improve the efficiency of Web site load testing and scalability testing. Many of the leading test-tools, including Mercury Interactive Corp.'s LoadRunner/WebTest, Rational Software Corp.'s recently purchased preVue-Web product (from Performance Awareness Corp.), Pure Atria's PurePerformix/Web (PureAtria is soon to be bought by Rational Software as well), and Platinum Technology Inc.'s WebLoad, have come out with these types of products during the past year. A load test tool can emulate hundreds or even thousands of users accessing a Web site during short periods of time. Typically, such software includes capture/playback capabilities, a programming or scripting language, monitoring and control functions, and test logging and reporting tools. The capture/playback functions can be used to "record" representative Web-user activities during interaction with your Web test site, save them as scripts, and then play back multiple instances of these scripts to simulate real loads (in the form of HTTP requests) on your site. Because these tools are relatively new, they can vary considerably in price, reliability, and capabilities. You'll want to consider the product's limitations and capabilities regarding OS platforms, compatible browsers, secure HTTP capabilities, and the quality of the scripting language. Cost is often based on the number of simulated users. Some tools have a demo version available.
Regression Test Tools. These are most useful for the automation of tests that will be repeated at least several times. The effort that may go into scripting and debugging these tests is only justified if the same test scripts can be reused without major modifications. For example, a set of tests might need to be carried out each time a new build of your site's CGI software is modified or each time there are modifications to your Web site database stored procedures. As with the load test tools, regression test tools are designed to work efficiently with Web-based applications and have capture/playback, scripting, and logging/reporting capabilities. Load test tools create direct HTTP requests; however, regression test tools instead simulate a user interacting with a browser and let the browser create the HTTP requests. Such tools can also be used in tandem with load test tools for stress testing by repeating a suite of tests under varying load conditions. Segue Software Inc.'s SilkTest and Platinum's FinalExam InternetTest are two examples of such tools.
Java Test Tools. Java applets on the client side and Java applications on the server side can be tested separately, if necessary, with new test tools specifically geared to Java, such as those from the SunTest division of Sun Microsystems Inc. These tools test APIs and code coverage, and they perform capture/playback.
Other Test Tools. A wide variety of other tools are available, including link-checking tools such as the Electronic Software Publishing Corp.'s LinkScan and Tetranet Software Inc.'s Linkbot.
Database-driven Web sites are taking an increasingly critical role in the day-to-day operations of many organizations. You can minimize your risk of site problems, errors, and failures through careful analysis of requirements and risks, well-planned site test strategies, and appropriate use of new Web test tools.
Web Resources on Software Testing and Quality Assurance
|
DBMS and Internet Systems Buyer's Guide | www.dbmsmag.com/pctest.html The Testing and Software Quality category describes almost 30 testing products, including many products that test Internet and Intranet applications. Web server load testing products are also listed. Each product listing links to the vendor's Web site for more information.
Software QA Magazine
Software QA and Testing Resource Center | www.charm.net/~dmg/qatest Rick Hower's (the author of this article) site includes a comprehensive FAQ (Frequently Asked Questions); descriptions of and links to testing tools, configuration-management tools, and problem-tracking tools; and links to other Internet resources.
Software Quality Engineering
Software Testing Newsgroup | comp.software.testing A newsgroup for discussing software testing and related issues.
STORM: Software Testing Online Resources | ||
|---|---|---|---|---|---|
What did you think of this article? Send a letter to the editor.