DBMS, June 1996
If you've been following these pages, you might have recognized that the theme of my column this year is my renewed acquaintance with application development tools that I last looked at in 1987. So far, I have caught up with 4th Dimension and what's going on in the 4th Dimension development community, and with R:Base and its recent introduction of version 5.5. For this month's column, I checked out Clarion for Windows.I opened the box at 10:00 on a Saturday morning. By 4:00 that afternoon, I had worked my way through the tutorial and had a functional sales order-entry application with customer and item maintenance, and had added a few of my own twists to the script. That got my attention. I have since spoken to several developers who confirmed my first impressions about Clarion's ability to quickly create applications that have a great deal of functionality, as well as my concerns about the application model that Clarion employs. I'll try to describe that model in some detail, but first, a little history.
Clarion Software introduced the first version of Clarion in April 1986, and followed that in May 1988 with a 2.0 release. During the late 1980s, Clarion was considered one of the leading PC DOS programmable database managers, alongside dBASE, R:Base, and Paradox. In the early 1990s, products such as Clarion and R:Base fell into the database black hole, as data users bought query and reporting Windows products, and developers assessed database-independent tools that would support developing Windows database applications. Clarion managed to retain a strong core of loyal, mostly independent developers who continued to deliver new applications to their clients. During 1991 and 1992, Clarion teamed up with Jensen and Partners International (JPI), a firm launched by Neils Jensen, one of the founders of Borland International and an architect of Turbo Pascal. Clarion and JPI worked together on integrating JPI's TopSpeed compiler engine into Clarion, and on building an optimizing code generator and a project system.
Clarion then released a version of its product dubbed Clarion Database Developer 3.0 (CDD), which incorporated the compiler technology and extracted Clarion's code-generation drivers into a template language. CDD also introduced plug-and-play database drivers, which let Clarion programmers access many popular PC database file formats such as Btrieve and dBASE simply by choosing a driver from a list. The Windows version of Clarion was not completed until late 1994. By then, Clarion and JPI had merged to create TopSpeed Corp. TopSpeed touted Clarion for Windows as the first optimized 4GL compiler for Microsoft Windows. The 1995 release of version 1.5 improved the product's GUI form design capabilities. Again, this was similar to the path R:Base took in which its initial Windows version enabled developers to begin migrating their applications from DOS to Windows, and the subsequent release started to enable development of native Windows applications. At press time, TopSpeed is scheduled to release Clarion for Windows 2.0 this spring. Version 2.0 adds support for OLE and object-oriented programming constructs, as well as tools for deploying Internet-enabled database applications.
The primary enabling technology in Clarion is a powerful visual template language that controls a high-speed source-code-generating engine. To build an application in Clarion, you start by defining the data model in Clarion's data dictionary. Once the data model is defined, you can define application components. The primary means for doing so is to choose a template for the application component and then fill out a series of dialogs that specify how your application requirements should be addressed by the component. After you complete the dialogs, you can customize any generated user interface components and embed custom source code at a variety of embed points, which are junctions similar to predefined events in other tools at which you can insert code or calls to external routines. When you are done with a component template, Clarion generates a procedure for that component. Finally, you compile and run your application.
Because everything in Clarion ultimately resolves to source code, it's necessary to understand the language in order to become really proficient at using the generation tools. With only a few days of experience, I'm clearly not yet proficient, but I think I'm beginning to see some patterns emerging. Clarion is a comprehensive, general-purpose language optimized for writing business applications; it has been developed by a relatively small team during the past dozen years. Although it borrows some concepts and features from other languages, it's both unique and extensive, encompassing data management, memory management, thread management, queue management, and event management -- in short it's a complete programming language.
Clarion's extensive data handling services, which include table creation commands, navigation, indexing, encryption, view definition, and transaction processing, give it a heavy database orientation. The language supports more than a dozen data types, including two packed decimal types for highly accurate numeric computations, and a variety of scoping methods. With "safe typing," Clarion permits mixing different data types in numeric and string expressions, which the compiler resolves based on its mapping rules. Clarion treats record structures based on database files the same as other memory structures. This leverages language features such as support for memory queues and bulk assignment statements, making them available for process-oriented applications and interactive programs as well as data-oriented applications.
Clarion's emphasis on data handling does not diminish the extent of Clarion's functionality in other areas. Its user interface definition and management components support a wide range of user interface styles and models. At the core of its event management is a simple accept ... end loop, which cycles through a case structure when an event occurs. The event( ), field( ), and focus( ) functions called within the case structure catch the events that apply to them and let you initiate appropriate responses. As a good business language, Clarion includes an integrated and comprehensive report grammar that supports defining graphic reports with elements based on data fields, variables and expressions, headers and footers, and multiple group breaks with subtotals and totals.
The specifications that you provide in the Clarion data dictionary really drive the generation of your Clarion application. The extent of Clarion's data dictionary is evident when you are maintaining fields. (See Figure 1.) The tabbed Field Properties dialog has eight tabs. The General properties tab lets you specify the field name and description, data type and size, an entry mask, a default prompt for use on entry and edit windows, and a default heading for display on reports. The Attributes tab starts to reveal the extent to which the dictionary will drive the application generator. Here you assign formatting rules for justifying contents of the field, the default value for the field, and whether to display text in upper, lower, or mixed case. A radio button group controls setting the cursor to insert or overwrite mode when a user enters the field. Check boxes control whether changes to the value of the field should immediately trigger an update of dependent controls, whether the field should be read-only, and whether characters entered into the field should echo on the screen.
The Options tab offers additional control over generation time treatment of the field. You can suppress generation of the field from forms, reports, and browses. The "Add Extra Vertical Space" check box permits tuning the display of the field while the population order drop-down controls the relative position of the field in generated forms, reports, and browses. Hooks for using external and utility templates are provided by the user options list box. The Validity Checks tab presents controls for establishing that the field must have a non-null, non-zero value, that a numeric field's value must be within a specified range, or that the field represent a Boolean value. Here, too, you can set a string of valid list choices or the name of a field in another file that the field must match. To further customize application generation, you can specify what type of control should represent the field on forms and on reports. Finally, there is a tab on which you can define a help ID for context-sensitive help, a message to display in the status bar, and a tool tip.
Interestingly, you use the same tabbed dialog to define memory variables as you use to define fields. As a result, you can assign the same sets of rules and behaviors to memory variables as you assign to fields, with a couple of additions. For instance, you can specify that a memory variable reference another variable's memory location. You can also specify a variable's storage class, which controls whether your program will allocate stack memory (the default), static memory, or static memory in a thread, or whether it is defined in an external library or DLL. Variables assigned to static memory are declared once and shared by multiple instances of the procedure that declared them. Variables assigned static memory in a thread will be assigned one value per execution thread. These capabilities amply demonstrate the Clarion language's flexibility and enable the creation of procedures that easily manage multiple instances of a form or process.
You define keys and indexes on the Keys tab of the Field/Key definition dialog. Clarion maintains an interesting distinction between keys and indexes: A key is automatically updated as records are added, changed, and deleted; an index is not maintained, but needs to be rebuilt whenever an application needs to use it. The key/index builder supports optionally unique keys, assigning primary keys that must be unique, and generating code to manage auto-incrementing keys. The Key Options dialog permits specification of whether the application should permit users to use the key for ordering browses, forms, and reports, and the order in which available keys should be presented in pick lists and tabbed dialogs, as well as developer supplied drivers.
After defining keys and indexes, the next step in building a dictionary is establishing relationships. Completing this process correctly is critical to successfully implementing an application because the application generator depends on the relationship information to create lookups and dependent entry forms. The relationship editor only supports one-to-many relationships, which you can initiate from either the one or the many table. For each relationship, you must identify the primary key to base the relationship on in the parent file and the foreign key to connect with in the related child file. Based on the keys that define the relationships, Clarion will map the required fields. Finally, you specify referential update and delete rules. Here I should clarify that none of the rules or relationships that you define in the dictionary are enforced by the database engine; rather, they become directives to the code generation process that will be performed by the application generator.
It pays to spend a lot of time on data dictionary definitions before you start building application components. I found that when I tried to change table and index structures after creating application components I had difficulties getting Clarion to apply my change specifications to the underlying physical tables, which in one case were in FoxPro 2.x format. Another shortcoming of the development environment is its failure to provide data dictionary reporting and inquiry tools. For instance, it's impossible to print the dictionary definitions from Clarion or to open a dictionary while you are working on an application that uses that dictionary. Developers get around this restriction by making a duplicate of the dictionary that's not assigned to any application, or by using one of the several dictionary viewer utilities that are found in the TopSpeed forum on CompuServe. Because the development tool is so dependent on dictionary specifications, I expected it to be much easier to get information stored in the dictionary. As it is, I would suggest doing a lot of thinking about table structures before starting to develop an application (business application development 101, really!), then defining the tables in the Clarion dictionary, and then sleeping on the table structures before starting to develop an application.
Once you are comfortable with your data dictionary, building application components with the application generator is straightforward. You start by completing the Application Property dialog in which you associate the data dictionary that you worked so hard on with the application. You also specify the name of the initial procedure to run, the name of a help file to associate with the application, whether to build an executable, DLL, or library file, and which set of generation templates to use. You can also use an application wizard to walk through the steps of creating a basic application.
The objective of the application development process is to build a collection of related source-code procedures that define an application. Each procedure is the result of running a "procedure type" template against a series of specifications that you provide by completing template-specific dialogs. You access those dialogs via the Procedure Properties dialog (see Figure 2), a control panel-style interface for maintaining the data elements, user interface, local variables, and code that the procedure will manage. Next you can define calculation formulas with an Expression Builder dialog, specify parameters to be passed to the procedure and return values for functions, set the value of any .ini files that will impact the procedure's behavior, and add custom code at embed points provided by each procedure template.
Most business applications will be built around a core MDI application window using the Frame template. This template provides an MDI parent frame that contains a shell menu with File, Edit, Window, and Help drop-down menus predefined. The easiest way to start is by opening the procedure's window design surface, editing the menu, and adding a new menu with choices pertinent to your application. For each choice, you will specify a procedure or program to run when the menu choice is selected. When you assign a procedure name, Clarion offers the option of initiating a separate execution thread for it, making it possible to create systems in which the user can launch a series of independent tasks and let them run simultaneously. When your menu system is complete you can return to the Clarion Application Tree dialog (see Figure 3), which displays branches with "to do" indicators for each of the procedures that are to be called from the main menu. The Application Tree is the top-level control panel for the entire project. It lets you view the project hierarchically by procedures, generated source modules, or templates.
For each procedure on your "to do" list, you need to select a template. Clarion predefines a group of templates that represent generic application tasks: browse a table, view/edit a record, process records sequentially, generate a report, or view an ASCII file. If none of these represents the model for the application component on which you are working, more generic templates provide a framework for working with forms you design from scratch, for writing procedures entirely in Clarion code, and for calling external procedures from a DLL or library file. Because the application components I created were fairly basic, I was able to use the prebuilt templates and specify the tables and fields on which they should act using the Procedure Properties dialog. In many cases, I was able to use the prebuilt control templates, such as the BrowseUpdateButtons control, the RelationTree control, and the FieldLookupButton control, to extend the general-purpose forms that the templates created. I especially liked the RelationTree's ability to automatically create a hierarchical tree view that would drill down through a parent to child and grandchild relationships.
Clarion for Windows achieves database independence by incorporating driver technology that enables the creation of drivers for any file system. The product ships with drivers for data files created with older versions of the Clarion product; a proprietary format that is new to Clarion for Windows called TopSpeed, which stores multiple tables and key files in a single DOS file; all the popular Xbase formats; Btrieve; and comma-delimited ASCII files specified by the original Basic programming language. It also supports ODBC connections for developing applications that manage client/server databases. Since shipping Clarion for Windows 1.5, TopSpeed has announced that it is producing drivers that communicate directly with popular RDBMS products, the first of which it dubbed Oracle Connect.
All of the Clarion developers with whom I have corresponded are developing some, if not all of their applications for client/server databases. This represents a paradigm shift for Clarion developers, who, like most PC database developers, are used to working with one record at a time, executing a statement at a time, and using assignment statements of one form or another to update our tables. While you can work with server-based data using the same record-oriented commands that you use against local files, you will get better performance and reduce network traffic if you leverage the client/server architecture. To support this, Clarion has taken an interesting approach to embedding SQL support into the language. You declare a data file structure using the file statement, in which you can declare the structure of the table that you need to connect to and assign it a local name. Like any of Clarion's memory structures, files have a list of properties they support, including a SQL property. Setting the SQL property to a query string and issuing SQL statements via a send message, which is a general-purpose "pipe" for sending a message to a file structure, are the primary mechanisms for embedding SQL. Clarion buffers the result set and permits sequential processing using standard Clarion language commands.
Clarion's development model is very different than the model presented by visual tools that maintain window layouts in a private repository that they interpret at runtime. Its model of merging the application repository with the data dictionary to create code that can be submitted to a compiler yields high-performance applications with consistent appearance and behavior, irrespective of the developer. This is an ideal model for the world of flat files in which Clarion originated. To its credit, the language's designers have done a graceful job of extending the language to support event-driven graphical applications and multiple execution threads. Now Clarion needs to adapt to a world in which application and business rules are increasingly applied by the database engine rather than, or in addition to, application code. I envision that future versions of the generator will need to create database structures, triggers, and procedural logic in server-side languages, as well as in client applications. Such enhancements will complement the support for OLE and object-oriented programming technology that are slated to appear in the imminent 2.0 release.
With elements borrowed from such diverse language sources as PL/1, Modula-2, Basic, and Cobol, Clarion is a powerful, but idiosyncratic language. Its steep learning curve would deter many Xbase developers whose strengths are in business analysis rather than in programming, but who do like having source code available to customize their applications. Its idiosyncrasy discourages organizations that want to leverage existing programming expertise or prefer to adopt products based on real or de facto standards. Another issue for organizations trying to choose a standard development tool is Clarion's lack of automated team development and source management tools. Lastly, the maintenance of one language for template programs and another language for generated code troubles me. I've always resisted environments in which I had to learn two languages to consider myself sufficiently expert to have complete control over the behavior of the applications that I was creating. These concerns make Clarion a tool best suited for independent developers with a strong programming background, looking for power and high-productivity development of mid-range business applications.
TopSpeed Corp., 150 East Sample Rd., Pompano Beach, FL 33064; 305-785-4555 or fax 305-946-16500; http://www.topspeed.com.
