Defsystem

$Revision: 5.0.2.4 $

The document introduction.htm provides an overview of the Allegro CL documentation with links to all major documents. The document index.htm is an index with pointers to every documented object (operators, variables, etc.) The revision number of this document is below the title. These documents may be revised from time to time between releases.

1.0 Defsystem introduction
     1.1 Defsystem terminology
2.0 Using Defsystem
3.0 Module-specifications
     3.1 Short form module-specifications
     3.2 Long form module-specifications
4.0 Redefinition of systems
5.0 Predefined operations on systems and defsystem variables
6.0 Extending Defsystem
     6.1 Class hierarchy
     6.2 Defining new classes
7.0 Extending defsystem syntax through shared-initialize methods
     7.1 Defsystem methods that can be specialized
8.0 Defsystem extension examples
    8.1 Example of defining a new system class
     8.2 Example of defining new system operations
     8.3 Example of master and development directories
     8.4 Example of one system definition referencing another

Note that there are many classes defined as part of the defsystem facility and named by exported symbols in the defsystem package. These classes are discussed in this document but do not have individual page descriptions.

1.0 Defsystem introduction

Defsystem is a tool designed to help developers deal with large programs contained in multiple files. The defsystem facility improves the development environment by providing quick and easy ways to perform complicated tasks on a set of files.

For example, let's say a developer has a program with 20 different files and has just edited 3 of the files. Those 3 files will need to be recompiled. There may also be other files that will need to be recompiled because the code in them depends in some way on the code in the files that were just edited. Without defsystem, the developer would have to separately issue commands to compile and load each of the files in the correct order. With defsystem, the developer can set up a system by indicating which files should be included in the system and in what order they need to be compiled. Then after editing some of the files, the developer can just issue one function call and the defsystem will figure out which files need to be recompiled and will do the compilation in the correct order.

The defsystem facility is strongly based on CLOS, which means that the defsystem can be easily extended and customized to meet individual needs. The second part of this chapter describes the ways in which the defsystem facility can be extended and customized.

Many symbols associated with the defsystem facility are in the defsystem package. It has nicknames defsys and ds. However, most operators (functions, macros, etc.) are named by symbols in the excl package.

1.1 Defsystem terminology

This chapter uses a hierarchy of terms. We define those terms below:

• source file: a file which can be compiled to produce a product file. In Allegro CL, a source file name usually ends in .cl (i.e. foo.cl).
• product file: the result of compiling a source file. In Allegro CL, a product file name usually ends in .fasl (i.e. foo.fasl).
• module: a module represents one source and product file pair. Modules contain data concerning dependencies between it and other modules and attributes of the file. The filename without the extension names the module. Module names are strings. For example, if foo.cl is the source file and foo.fasl is the product file, "foo" is the module name.
• module-specification: a module-specification describes one or more modules and dependencies between modules. There are two types of module-specifications: short form and long form.
• short form module-specification: A short form module-specification provides a simple syntax to describe one or more modules.
• long form module-specification: A long form module-specification provides a more complex syntax to describe one or more modules.
• module-group: a module-group contains one or more module-groups and/or one or more modules. A module group is created from a defsystem module specification.
• system: a system contains one or more module-groups.
• system name: A system name is a symbol that refers to a system. We recommend that one use a keyword so the system-name is not in any specific package.
• system-object: A system-object is a data construct that contains information about a system.
• system-specifier: A system-specifier is either a system-name or a system-object.

2.0 Using Defsystem

Systems are defined with the defsystem macro. There are two parts to a system definition: the system options and the module-specification. System options specify attributes of the system or define default values for attributes of modules in the system. Module-specifications describe the module-groups of the system and dependencies between modules and module-groups.

An item in a module-specification can be specified with a file name, the name of a named module group defined earlier in the same system, or the name of another system. Supplying the name of another system in a module-specification causes the named system to be a component system of the system being defined. This means that system operations will be performed recursively on the component system (if the :include-components keyword to the system operation method has the value t).

The defsystem macro is defined on its own page. It takes a module name (a symbol) and a possible empty list of system-options as required arguments followed by list of module-specifications.

The allowable system-option keywords are shown in the table below. After the table, we describe module specifications.

3.0 Module-specifications

More than one module-specification may be specified when defining a system. Module-groups can be specified using a short form module-specification or a long form module-specification. We describe short and long form module-specifications below. Note that we use the terms short form specification and short form module-specification interchangeably. They mean the same thing. We do the same thing with long form specification and long form module-specification.

(:module ...) and (:module-group...) create module groups, as do all other defsystem module-specifications. Module-specifications which begin with the keywords :module and :module-group provide ways of associating a name with a module-group.

Note that a module-specification which begins with :module really creates what we call a module-group.

The (:module ...) syntax for long-form module-specifications is used to maintain syntax compatibility with the defsystem facility available on Symbolics machines. Note that our term module-group is similar to what is called a module in the Symbolics' documentation of their defsystem.

3.1 Short form module-specifications

Short form module-specifications provide a simple syntax for describing systems that:

• have only serial and parallel dependencies
• have only one package (given by the :default-package option to defsystem)
• have only modules of the default type

More complex needs can be met by using long form module-specifications.

A short form module-specification is a list beginning with one of :serial, :parallel, :definitions, or :module-group. The table below gives a description of each of the short forms. In the table, module-spec-element may:

1. name a module
2. specify another module-group using a short form specification
3. name a module-group defined earlier in the system definition
4. name another system to be treated as a component system.

Most simple needs can be met by use of the :parallel and :definitions short forms. The following are examples of defining short form systems.

(defsystem :foo ()
           (:parallel "file1"
           (:serial "filea" "fileb") "file2"))

In this example, filea would be compiled before fileb, but files filea and fileb are not dependent on file1 and file2. Files file1 and file2 are not dependent on each other.

(defsystem :my-sys ()
           (:serial "file1"
                    "file2"
           (:parallel "filea" my-other-system)))

In this example, file2 is dependent on file1. Also, filea and the system my-other-system are dependent on file2.

The :definitions module-specification keyword is different from the :serial module-specifications keyword in that :serial does not force recompilation of modules farther down the list. As an example, suppose we have:

(defsystem :bar ()
   (:definitions "macro-def-file" "file-that-uses-macros"))

In this example, file-that-uses-macros is dependent on macro-def-file. If macro-def-file is compiled, then file-that-uses-macros will be compiled too.

(defsystem :named-system ()
   (:module-group my-group (:serial "file1" "file2")))

This example associates the name my-group with file2 being dependent on file1.

3.2 Long form module-specifications

Long form module-specifications should be used when more control over some module-group or module-groups is desired. With the long form it is possible to override the system default values for various parameters and to specify more complex dependencies.

There are two types of long form module-specifications, one allows a group of modules to be given a name which can be referenced by other module-specifications in the system and the other type provides a simple way to specify non-default options for just one module.

A long form specification is a list of one of the following forms:

(:module name system-or-files {long-form-option}*)

or

("filename" {long-form-option}*)

Here name is a symbol that names the module-group. name can be used by other module-groups in the system to refer to the module-group it names. system-or-files can be a string or (non-empty) list of strings representing the one or more source files of the module-group, or a symbol representing the name of another system.

Note that :module-group is used to name short form module-specifications, while :module is used to name long form module-specifications. In both cases, the name is associated with the resulting module-group.

("filename" {long-form-option}+)

is functionally equivalent to

(:module foo "filename" {long-form-option}+)

except

("filename" {long-form-option}+)

does not have the added name foo associated with it.

Zero or more keyword options can be specified in the long-form-option position. The allowable keyword options are given in the following table. Note that some options have more than one or a variable number of arguments. All have at least one argument. The `+' after an argument indicates at least one value must appear but as many as you like can appear.

4.0 Redefinition of systems

When a system is redefined the following events occur:

• old system options, if any, are removed
• new system options, if supplied, take effect
• modules which represent the same source files in the old and new systems maintain whatever state they had before the redefinition, which means, for example, that a module which has been compiled and loaded will not be recompiled or reloaded after the redefinition.

5.0 Predefined operations on systems and defsystem variables

The full description of each function described in this table is given in the page description of the function. :simulate means to print the actions that would be taken but do not actually perform them.

In this example using map-system, we pretty print each module in a system.

Example using find-system:

Example using show-system:

Example using undefsystem:

6.0 Extending Defsystem

Because defsystem is CLOS-based, it can be easily extended in many ways in order to add functionality or meet special needs. It is possible to extend existing system operations or to define totally new operations. The module actions that implement the operations on the modules of a system can also be customized. New classes of system, module group and module can be created to have specialized behavior and new slots. It is also possible to extend the syntax of the excl:defsystem macro that defines systems. Each of these extension methods is described below.

6.1 Class hierarchy

The following is a description of the defsystem class hierarchy. Users can build new classes based on these classes. The following can be used as a guide for choosing which classes to build on.

defsys::defsystem-base-class

All defsystem classes inherit from this class.

defsys:module-container

This is a class of objects that contain modules. Objects of this class have a list of modules and a default-module-class which indicates the default class to be used when modules are created for this module-container.

defsys:default-module-group and defsys:default-system both inherit from defsys:module-container.

Exported accessors for defsys:module-container:

defsys:modules returns a list of modules and module-groups contained by this container
defsys:default-module-class returns the default class for modules created under this container.

defsys:default-system:

Exported accessors for defsys:default-system:

defsys:pretty-name returns the pretty name of the system
defsys:system-name returns the name of the system
defsys:default-package returns the default package of the system
defsys:default-pathname returns the default pathname for the files of the system
defsys:property-list returns the property list of the system
defsys:parent-object returns the object which contains this system, or nil if there is no such object

defsys::default-module-group

defsys::default-module-group has the following exported accessors:

defsys:pretty-name returns the pretty name of the module-group
defsys:default-package returns the default package of the module-group
defsys:default-pathname returns the default pathname for modules of the module-group
defsys:property-list returns the property list of the module-group
defsys:parent-object returns the object which contains this module-group

defsys:default-module

defsys:default-module has the following exported accessors:

defsys:pretty-name returns the pretty name of the module
defsys:default-package returns the default package of the module
defsys:default-pathname returns the default pathname for modules of the module
defsys:property-list returns the property list of the module
defsys:parent-object returns the object which contains this module
defsys:module-file returns a string representing the file name (without file type) associated with this module
defsys:lisp-module, defsys:text-module, and defsys:foreign-module inherit from defsys:default-module.

defsys:lisp-module

The following methods are specialized for defsys:lisp-module:

defsys:load-module
defsys:compile-module
defsys:source-pathname
defsys:product-pathname

defsys:text-module

The following methods are specialized for defsys:text-module:

defsys:source-pathname
defsys:product-pathname

defsys:foreign-module

The following methods are specialized for defsys:foreign-module:

defsys:load-module
defsys:product-pathname

defsys:c-module

inherits from defsys:foreign-module and should be used for C code modules. The following methods are specialized for defsys:c-module:

defsys:compile-module
defsys:source-pathname

6.2 Defining new classes

By defining new subclasses of existing defsystem classes, it is possible to add new slots to defsystem objects and to write specialized methods on defsystem generic functions which will affect only objects of the new class.

If new classes are defined, defsystem needs to be told when to use them for creating defsystem objects. The following variables determine which classes the excl:defsystem macro will use when creating defsystem objects.

Another way to alter the module class used by the excl:defsystem macro is with the :default-module-class option, as described in table 1 above.

7.0 Extending defsystem syntax through shared-initialize methods

The syntax of the excl:defsystem macro can be extended through the use of shared-initialize methods. In order to understand how to extend the defsystem syntax, first it is necessary to explain how a defsystem form is processed to create the systems, module-groups and modules.

When excl:defsystem is called, the system name is looked up in a table of systems to determine if this is a new system definition or the system already exists. In the former case, a system object of class defsys:*default-system-class* is created for the system and the options list is passed to make-instance as a list of initialization arguments. In the case of reinstalling an existing system, the existing system object is reinitialized with the new options list. In either case, shared-initialize is called to perform initialization of the object. The options are handled either by the shared-initialize method for the system class which handles all slots that have initargs or by keyword arguments to an after-method of shared-initialize defined for the system class. So new system options can be added in either of two ways:

1. by defining a new system class with a new slot where the initarg of the slot will be a valid option.
2. by defining an after-method of shared-initialize for the new system class which has a keyword argument for the new option.

The list of module-specifications in the defsystem form is processed in a similar manner by shared-initialize methods and after-methods handling the options for module-groups and modules.

7.1 Defsystem methods that can be specialized

The following defsystem methods that can be specialized to obtain customized behavior.

8.0 Defsystem extension examples

In this section, we provide a number of examples of extending defsystem. The example are:

8.1 Example of defining a new system class
8.2 Example of defining new system operations
8.3 Example of master and development directories
8.4 Example of one system definition referencing another

8.1 Example of defining a new system class

As an example of defining and using a new system class, let's say we want a form to be evaluated after a system is loaded. We will need to create a subclass of defsys:default-system which will contain a new slot, eval-after-load. Then we will need a way to initialize this slot at system creation time, and a way to evaluate the form at system load time. For initialization we will want to add a new option to the excl:defsystem macro, which can be done through the use of a shared-initialize method for our new system class. Evaluation of the form can be implemented with an around-method on the load-system generic function.

First we will need to create the new system class. The following code defines a subclass of defsys:default-system with the new slot.

Now to enable initialization of the eval-after-load slot, we will need to extend the excl:defsystem macro syntax for this class by creating a shared-initialize after-method that takes an :eval-after-load keyword.

Next, we need to define an around-method on load-system so that the contents of the eval-after-load slot can be evaluated at system load time. In this method, we will first call the other methods to load the system, then check the return value to see if the system was loaded or not. If it was, then we evaluate the eval-after-load method.

The following macro binds defsys:*default-system-class* to our new system class to ensure that excl:defsystem creates systems of this class.

Finally, we can define our system, which has an eval-after-load form to set a variable when the system is successfully loaded.

8.2 Example of defining new system operations

The following is a description of how system operations and module actions work together.

System operations recursively invoke methods on each of the components of the system. The methods that are invoked are named foo-module-action, where foo-system is the name of the system operation. The foo-module-action method for a component system invokes foo-module-action on its components if ds::.include-components. is non-nil. The foo-module-action method for a module-group invokes foo-module-action on each of its components. The foo-module-action for a module decides if the foo-module method needs to be called and calls it if necessary. The foo-module-action methods also print messages about the actions that are being performed. In simulation mode, the foo-module method is not called by the foo-module-action method.

For example, load-module-action checks if the module has been loaded and if not, calls the load-module method to perform the actual load. compile-module-action and load-module-action are the only actions that do checking on the modules. The other module-actions unconditionally invoke the method to perform the action unless ds::.simulate-mode. is non-nil.

The following example demonstrates how to define a new system operation and the module-action methods that go along with the operation. This example implements a grep-system operation which searches each source file in the system for a given string. Note that this could be implemented more easily using map-system, but this is just an example of defining new system operations and module actions.

8.3 Example of master and development directories

This example involves creating new system, module-group and module classes and specializing defsystem methods for the new classes.

If a developer wants to edit a few files of a large file tree, but doesn't want to make a complete copy of the tree, he/she can define a system that looks for files first in the developer's copy of the tree, then in the master tree. If some modification is done that requires the recompilation of a source file not yet copied to the developer's tree, then defsystem will compile using the source in the master directory and put the fasl (product) file in the developer directory.

The default system and module classes will by subclassed with the new slots master-directory and developer-directory, that will contain the pathnames of the master and developer directories.

The source-pathname method will first look for the source file in the developer directory and then in the master directory.

The product-pathname method will always look in the developer directory.

8.4 Example of one system definition referencing another

Given a system A and a second system B, how can one get system B to reference system A?

Copyright (C) 1998-1999, Franz Inc., Berkeley, CA. All Rights Reserved.