Scanalyzer

The scanalyzer module is a tool for analyzing PSL functions, expressions, and source files. It makes it easy to analyze PSL code and to write preprocessors for PSL. It supports preprocessing of PSL code in much the way that macroexpand does, but is more flexible and extensible. The scanalyzer understands the various PSL function types, including macros, and has some understanding of each of the PSL special forms. Scanalyzer lives in SCANALYZER.SL, some support functions (including scanalyze-file) live in XREF-SUPPORT.SL.

Apologies to John Brunner, author of ”Stand on Zanzibar”, for the name of this facility.

17.2.2 Philosophy

The meaning of a piece of LISP code can depend a great deal on the context in which it is compiled or evaluated. Virtually everything in a LISP system can change dynamically, and analyzing the meaning of code in the right context is a significant problem.

This facility focuses on analysis of code that is compiled, and it does so for two reasons. One is that most large systems written in PSL run compiled. The other is that once LISP code is compiled, the meaning of the object code is much more static than the meaning of the corresponding interpretive code.

Reliable system operation depends on modules being compiled in an environment that is known and fixed as much as possible. To do this, we compile each module with a compiler that is started afresh. After one module has been compiled we do not compile other modules with the same compiler.

To precisely analyze a module, we can start a compiler. To it we add the analysis facility, which should not change the compilation context. We then analyze the module, just as we would compile it. The analysis facility must be sensitive to the same contextual information that the compiler is.

Many modules have some side-effects on the context of compilation – defining a macro is just one of the things that affects compilation context. To be sure that a module is analyzed in the right context, the analysis should have the same side-effects on compilation context as the compilation would have.

17.2.3 Functions

(scanalyze-file INPUT-FILE:string, pathname
OUTPUT-FILE:string, pathname): string expr

Applies the function scanalyze-form (defined in section 2.3.3) to every form in input-file. Scanalyze-file assumes that information accumulates in the list xref-assertions. After all the forms have been analyzed, scanalyze-file dumps all the assertions into output-file as a list (reversing the list first). The assertion (< input-file > IS-SOURCE-FILE) is added to the front of the assertion list.

File name defaulting: As its arguments scanalyze-file accepts both filename strings and pathnames. In any case the output-file argument is defaulted from the input-file argument. Any missing components of the name in output-file are defaulted using the function merge-pathname-defaults to be the same as in input-file, except for the file type (suffix). If that is not specified in output-file it is set to ”.XD”, which is the tentative standard for Xref-Databases.

For example, in the call (scanalyze-file ”pk:load.sl” ”mydir:”), the output filename generated is ”mydir:load.xd”. In the call (scanalyze-file ”#5:/pslroot/foo/test.sl” ”mydir:test-out.dat”, the output filename generated is ”mydir:test-out.dat”.

(scanalyze-form FORM:form): form expr

This expects an s-expression as its argument and analyzes it as a top-level form in a source file. By recognizing which forms would be evaluated at compile-time, load-time, or at both times, this keeps the context for analysis the same as the context would be for compilation. Those forms that would be compiled are passed to the function Scanalyze.

Scanalyze-Form recognizes forms such as compiletime bothtimes, and loadtime. It evaluates each at compiletime if the compiler would and analyzes each if the compiler would compile it. Actually, it recognizes the IGNORE and EVAL flags used internally by the compiler. This means that if analysis is started in the same context as a compilation, during analysis macros are defined in the same way in both cases, as are wconsts, if systems expand the same way, the same ”compiletime-loaded” modules are loaded, etc.. In PSL this means that the analysis context very closely matches the compilation context.

(scanalyze FORM:form ENV:form): form expr

Analyzes the given form (s-expression) in the given environment. The value returned depends on whether the caller has set up values for functional variables or properties that this function is sensitive to. If the caller has set up no special actions, the value returned should be the same that the function MACROEXPAND would return.

17.2.4 Environment Arguments

The caller can set up special actions that are in two categories. These are ”analysis hooks” and ”preprocessing hooks”.

The ENV arguments to various things are environments. An environment is currently a list of ”frames”. Each frame is a list of 2 elements. The first identifies the type of frame, currently always ’LOCALS. The second is the ”contents”, for locals a list of the local variable names.

17.2.5 Analysis Hooks

The scanalyzer defines a number of variables that the user may give functional values to. Some support the user of this module in analyzing code. Others provide the means to preprocess or transform code. The hooks are functional variables of 2 args (expression and environment) to attach an analysis function to. Each of these must either be NIL or a list of functions. The first three of these hooks are called before any preprocessing or expansion of the form is done; the last is called after all preprocessing and expansion of a form is complete. Scan-Macro-Hooks Called before each macro is expanded. Scan-Fn-Hooks Called before descending into any expr to analyze it. Scan-Non-Pair-Hooks Called with each non-pair examined.

After-Expansion-Hooks Called with each fully expanded expression. Not applied to forms such as macro calls, because they are expanded further before being returned.

17.2.6 Properties

These are checked respectively before and after an expression has been expanded. After expansion means that the call has been expanded until it is not a macro call, and all subexpressions have been expanded also. If the car of an expression is an id that has the appropriate property, each member of the list that is the property value will be funcalled. These properties are checked at the same points where the scan-xxx hooks are checked. Thus it is of no use to give a macro a post-expand-scanners property. The properties are:

17.2.7 Information Made Available By Scanalyzer

Current-Function This has the name of the current function where that can be determined. Forms such as DE and DEFUN that expand into calls on PUTD with constant function names cause this to be rebound inside analysis of the function body argument.

Current-Top-Level-Form Contains the argument that was passed to the call on scanalyze-form currently being executed.

Top-Level-Code? Is T for expressions not nested in any invocation of the function FUNCTION. Expressions for which this variable is T are known to be performed at initialization time.

17.2.8 Expansion And Preprocessing Hooks

Special-Expander Should be a functional variable of 2 arguments when set to a non-NIL value. If it is, and if the similar functional value of expand-specially? is non-NIL, the special-expander function will be used to expand the form.

Expand-Specially? If non-NIL, we use special-expander to expand the form. Non-Pair-Expander Either NIL or a functional value of 2 arguments used to expand expressions that are ”atomic”.

If any function or macro, etc.has an Expr-Expander property, its value must be a 2 argument function that scans and expands expressions whose first element is that id. This module provides several such functions so that built-in special forms can be analyzed.

Special-Expander and Non-Pair-Expander have first priority. Then comes any Expr-Expander property. Then normal processing by type of function.

17.2.9 Cross Reference Support

The fluid xref-assertions holds the assertions generated by xref-assert and xref-assert-list, dumped out to a file by scanalyze-file.

(xref-assert A:assertion): assertion expr

(xref-assert-list L:list): assertion-list expr

(record-usage EXPR:form ENV:form): nil expr

Suitable as a function to be applied to each (non-atomic) expression and subexpression being analyzed. It is especially suitable as an after-expansion hook. Except for various special cases described below, record-usage makes an assertion for each expression that is a function call. The assertion is of the form, (<fn1> CALLS <fn2>) where <fn2> is the function being called (car of the expression). <Fn1> is the current-function, or if that is NIL, it is the current-file.

Record-usage ignores non-pair expressions. Expressions that are pairs (function calls) are checked for an openfn or opencode property, which would imply that they are compiled in-line. No assertion is made for these or for expressions where the function part (car) is not an id.

If the function being called has a usage-asserter property, the value of that property is funcalled to make whatever assertions it will. Otherwise if the function is flagged dont-record-usage, no assertion is made.

(record-macro-usage EXPR:form ENV:form): nil expr

record-macro-usage is like record-usage, but it is for macros and cmacros. This function is suitable as a scan-macro hook. The assertions generated by this function in the usual case are (<fn1> USES-MACRO <macro>) or for cmacros, (<fn1> USES-CMACRO <cmacro>).

(setup-some-xref-actions): t expr

The programmer who likes some of the facilities in this module, but doesn’t want everything here can just load the module. If you want to use the facilities and defaults available in the module, call this function. In addition to setting up record-usage and record-macro-usage as analysis hooks, it flags some sets of functions as not having their usage recorded. It also sets up some special usage asserters for certain functions.

The value of each of these variables is a list of functions (or macros or both). The ”setup” function, above, flags each function on each of these lists as dont- record-usage.

17.2 Scanalyzer

17.2.1 Introduction