Itıs January 1998. Youıve just celebrated the New Year, and as a part of this ritual, youıve also made your annual resolutions. 1998 is a significant year. It is the last year that you have to test a standard year-end processing. Why? Because year-end processing at the end of 1999 will cross over into the new century. Thatıs the day the earth will stand still, at least for those who havenıt taken the Year 2000 (Y2K) problem seriously.
Before rendering yourself utterly helpless, ask yourself the following questions: This New Yearıs Day, did you celebrate the successful Year 2000 conversion and certification of your companyıs applications and databases? Or did you make it a New Yearıs resolution to see that your companyıs applications and databases would be Year 2000-compliant during 1998?
If you celebrated, youıre in the minority, and you should turn back the clock to New Yearıs Day and celebrate all over again. You deserve it. Otherwise, further introspection is in order. If you resolved to make it happen in 1998, did you resolve this as well for 1997, 1996, or earlier? Unlike most software development projects, this one must be completed by a specific calendar date, which is not only unchangeable, but also quickly approaching.
As you realize what has not been tested and how much testing is still to come, you may ask yourself, "What do I need to do to ensure that my applications will be Y2K-compliant, and how am I going to test differently to achieve this goal?" So read on. Understanding what should be tested and how to perform these tests will go a long way toward defining what can be done by you and your organization in the months ahead.
What is Y2K testing? It is the best and worst investment your company will ever make. How is this possible? Itıs the worst investment because after all of the money and time spent on it, your applications and databases will provide no more capabilities than they did before the conversion and testing began. With resource and time constraints, not to mention competition and decreasing margins, this "investment" overshadows any real business functionality or enhancement requirements, which are deferred.
Itıs the best investment because although you will not see a short-term payback for the effort expended, it helps to guarantee your companyıs continued operation into the next century.
How does Y2K testing differ from traditional testing? Y2K testing is all about how dates affect your applications. If your applications use dates that are not used for calculations, then it is not important to test these dates. However, dates that are directly or indirectly used for calculations and sorting should be tested.
Y2K testing is not intended to be the once-in-a-lifetime opportunity to create a complete set of system regression tests for all your applicationsı undocumented functionality. There is not enough time for this, and your efforts should focus first and foremost on Y2K testing activity.
The issues covered here focus on the application software component of Y2K testing. Networks, operating systems, and data feeds are all affected and susceptible to the Y2K problem, but they are beyond the scope of this article.
Certification is the process of ensuring that date-dependent functionality of your application has been tested to work within, across, and into the next century. Certification is a self-governing activity. No external organization will audit this process and validate its effects. This responsibility lies within the organization. For organizations that depend on external data feeds, the onus increases in that it is no longer sufficient to validate your internal processes. Likewise, for any commercial software used, certification must come from the vendor.
Certification tests are those tests that focus on dates and date processing. Many of these tests can be based on existing regression tests, but the key is to go beyond existing regression tests and ensure that dates found in databases and used across application screens are thoroughly tested. Using any available regression tests will simplify and speed the process.
Think about a date field as a bug. Your mission is to locate all dates within your application, understand how they are used, and perform a series of tests to ensure that they work properly. Clearly, many of the tests you currently perform on your application already benefit from using dates. Certification focuses exclusively on the dates and functionality that dates affect (for example, payments, renewals, and so on).
There are four basic steps leading to and incorporating testing for Y2K. These include:
Conversion: Where application code and databases are updated so that date processing can occur without error. The most common techniques include using date expansion, a fixed window, or a variable window.
Regression tests: Running standard tests to ensure that conversion did not upset standard application functionality. These tests are standard functionality tests that you would typically run after any code maintenance or enhancement. These are not specifically designed to test Y2K problems but will most likely include tests that involve dates.
Certification tests: Tests specifically created to show that date processing in an application functions properly into the new century. These are built by using a subset of regression tests and supplementing these with additional date-specific scenarios.
Production: Placing applications into production once it has been determined ı through testing ı that the functionality works properly.
There are two basic approaches to testing applications for Y2K compliance. One is to convert the code, run a standard regression test, certify that it is Y2K compliant, and put it back into production. The other is to convert the code, run a standard regression test, place it back into production, and then test for Y2K compliance. There are subtle differences between these two approaches, but how your organization is set up and what testing tools you use will vary with each approach. (See Figure 1.)
The advantage of the first method is that all testing occurs together, allowing for quicker feedback to development should any Y2K certification test not pass, while the advantage of the second method is that the amount of time the application is out of production is shorter ı an important concern for organizations that cannot afford to "freeze" applications for too long.
What are the pieces of the Y2K testing puzzle? You need your converted applications ı the applications where program logic on dates has been identified and corrected ı so that these may work into the year 2000. You need to define input test data as data entered into the application via an interface (for example, through a Windows- or terminal-based interface front-end program). This data includes validation criteria that defines data used to validate the correct results of an application, including specific values, which need to be verified on screens, in reports, or in databases. Databases are defined by preexisting data resident on a system and updated or otherwise changed by input data and/
or system calculations. Date scenarios define the specific dates against which applications should be tested. Some of the more common examples include:
Defining appropriate date scenarios is essential to ensuring proper date processing. However, the execution-time requirements by some applications may be so great that multiple date scenarios may not be possible in the testing timeframe allowed. Mitigate this by prioritizing the most critical test dates for a given application.
Running test cases based on future dates requires the following: First, you need to create input data for future date scenarios. Next, application databases need to be aged to match the future input data. Finally, the operating system date on which the application/database resides must be advanced to match the input and database future dates.
Aging data and moving the system clock ahead may provide erroneous results if the application has not performed the calculations from a day-, month-, or quarter-end process. The potential problem faced here is that everything may appear to be aged correctly though none of the intermediate calculations have occurred.
The solution to these problems requires shortening the proposed number of test cases and simulated date scenarios to be tested. Itıs a trade-off between breadth and depth of testing.
From the client-centric perspective, most testing can be performed from a workstation. Testing tools work best from a workstation perspective because they emulate a userıs interaction with an application. Itıs common today to find mainframe users with PC workstations and terminal emulation and PCs running X-Server software to access Unix systems. Itıs easier to standardize one user interface when it comes to automating tests.
To simplify the complex process of Y2K testing, it is advisable to create a single test case that can be reused for different date scenarios. The relationship among the dates within this test case should remain even as the dates are aged. This generic test case keeps all fields constant, with the exception of date fields that can be changed dynamically. Doing so requires redefining both input and expected results into the existing test case.
Many organizations will look to automation to streamline the testing process. Not all applications and functions are conducive to automation. There are varying levels of analysis that must occur to determine the feasibility of automation.
The following are sample questions you should ask to validate the feasibility of using automation for your testing:
While a business user gathers information to create test cases, an automator should begin to prototype scripts for the target application/environment. As test data becomes available it is processed by the automation prototype, allowing its use and capability to be adapted.
A common misconception about test automation tools is that they can only be used once an application is available for testing. In fact, quite a bit of prototyping can ı and should ı be done before an application is ready to test. Prototyping should be divided across the various environments where applications run. A prototype running against a mainframe, Windows, DOS, or X-Windows environment will create capability for applications running within these environments.
There are a number of tools to assist in the Y2K testing process. Some of these have specific functionality critical for Y2K testing, while others can benefit the overall testing process, leaving capability in place well beyond the year 2000.
Automated Test Execution. Many of the popular test automation tools support testing from the client point of view. Some products have specific Year 2000 functionality that simplifies date-format checking for old and newly renovated applications and automating the process of aging input and validation data. Checking for date format changes is especially useful when certification occurs immediately after regression testing of converted applications.
Because automation is complex, it makes sense to buy a toolset that supports the greatest number of environments. Executing from a common platform simplifies script creation, promotes reuse, and minimizes training.
Multiplatform test synchronization. Not all Y2K testing will be limited to one desktop. Many organizations have complex applications with Windows, X-Windows, and mainframe interfaces. Test automation across each platform becomes as important as automation within each platform. Tools to support these complex heterogeneous environments facilitate this process. Although many of these products are primarily thought of as load- and stress-testing tools, their ability to communicate and synchronize among various hosts makes them ideal for Y2K testing.
Operating system date simulation. Changing the current system date on a computer to a future date might be a simple task on a desktop PC, but it is a time-consuming process on a mainframe. There are products that can simulate this activity without requiring a shutdown and restart. Changing a system date to the future can have side effects that need to be understood and anticipated. These include, but are not limited to: premature software license expiration and unintended batch runs that may archive, move, or delete files.
Database aging. To model an applicationıs behavior at a future point in time requires aging all components of the application. Databases need to be aged as well. Most tools do this by converting database files to a common format, aging those fields that have been preselected as dates, and returning the data to its native format.
Test management. Iım not aware of any test-management tool geared specifically to the Y2K problem, but many vendors of test automation tools have complementary offerings in this area. These tools allow for multiple users to plan, execute, track, and monitor a wealth of metrics on progress and defects.
Configuration management. There are four areas that necessitate configuration management: application programs, databases, test datasets, and automated scripts. Here the platform differences will dictate which tool to use. Many automated scripts create a series of files and directories, which require a tool to support these complex nested structures.
Project management. Keeping track of activities, deliverables, and resources can be facilitated with project-management software products. These tools will allow you to track activities as well as delays. By linking activities that are dependent to one another, the software will automatically adjust schedules should one activity take longer than expected.
Staff requirements. Any successful project requires the correct mix of staff. First and foremost, business users who are familiar with the functionality and use of the applications are key to ensuring the proper definition of test cases necessary for testing ı Y2K or otherwise.
Next, automation experts, those staff members with a strong technical background who can develop robust, generic test automation capability, are needed to develop and support the test cases created by the business users.
Estimating timeframes can only be done after defining activities. Activities should be grouped by each of the following phases:
Planning: Defining the scope of what is to be tested
Analysis and design: Defining specific functional requirements to be tested
Construction: Developing the data (input and expected results) required for each test case
Execution: Running the test via automated or manual means
Verification and validation: Ensuring that the test ran correctly and that results matched expectations.
These generic phases can be applied equally at the macro and micro levels and for both manual and automated test case creation.
Convert first, certify later. Itıs far simpler and quicker to correct code, perform a standard regression test, and place it back into production first, then separately test the production code to certify that it is Y2K compliant. This will also shorten the amount of time that code needs to be "frozen" while it is being tested.
Pilot an application. Before spending too much time estimating the Y2K effort for every application within the organization, select a small- to medium-size application and make note of the time and resource requirements expended throughout the Y2K phases described earlier. This will provide a reasonable estimate of the resources that other applications will also require, and it will provide an opportunity to modify and adjust implemented procedures.
Track delays. Most organizations pride themselves on detailed project plans describing activities at inception. These documents quickly become obsolete. A more realistic approach is to track activities that add delays ı examples include applications not available, down database servers, rerunning tests to ensure problem is corrected, and so on. Identifying and tracking causes of delays will go a long way toward more accurate estimation.
Formulate and replicate. Find an effective approach to project and process management and test automation, and use it consistently across all applications. Although specific implementations may vary across platforms or business users, a generic "cookie-cutter" approach facilitates acceptance and is easier to implement organizationwide.
Stick with what you know. If your organization has not yet embraced any test automation, the Y2K project with its fixed and rapidly approaching deadline may not be a good candidate for embarking on new initiatives. Using current best practices and adapting them to support the Y2K process will yield the best returns.
Use content experts. The greatest knowledge and insight into the operation and use of your applications will come from the business users within the organization. Use the business knowledge of these users and the technical knowledge of application developers and database administrators to create test cases representing real-world business life-cycle processing of an application. Using eager, but unknowledgeable resources for this area is less effective given the application and business life-cycle complexity issues.
Use process experts. Process-related activity, including project management, test automation, and configuration management, can be effectively executed by individuals who routinely are engaged in process work and who donıt require the intimate content perspective of the business users. Bringing together content and process experts is an ideal way to leverage the expertise within an organization and to produce complementary, productive Y2K teams.
Although Y2K testing efforts may appear to offer little beyond ensuring that programs will continue to function into the next century, there is a wealth of process-related activity that can provide benefits to the continued operations of the organization. Automation that was developed for Y2K can be expanded and used to create additional regression tests. New procedures and automation capabilities provide momentum to an organization and an incentive to continue to promote the effective use of technology, standards, team building, and overall greater organizational efficiency. This, in turn, will simplify operations, lower costs, and ultimately increase product quality.
Now imagine for a moment that its New Yearıs Day, 2000. Youıve had a great celebration, and you return to work to observe that all systems operate as they normally would any other year. Youıre pleased by the proactive efforts of years past to ensure this day would come ı and pass ı without incident. Not everyone, though, took the task to heart. Too bad the ball over New York Cityıs Times Square never fell that night ı certainly a disappointment to those who count on this tradition to mark the New Year. I guess they didnıt take the Year 2000 issue seriously, and for them that year marked the day the earth stood still.
The two basic approaches to testing applications for Y2K compliance.
| The following is a list of testing-tool vendors. In addition to these vendors, another source of information is the Year 2000 Web site at www.year2000.com | |
|---|---|
|
What did you think of this article? Send a letter to the editor.