The scanalyzer module is a tool for analyzing PSL functions, expressions, and source files. It makes it easy to analyze PSL code and to write preprocessors for PSL. It supports preprocessing of PSL code in much the way that macroexpand does, but is more flexible and extensible. The scanalyzer understands the various PSL function types, including macros, and has some understanding of each of the PSL special forms. Scanalyzer lives in SCANALYZER.SL, some support functions (including scanalyze-file) live in XREF-SUPPORT.SL.
Apologies to John Brunner, author of ”Stand on Zanzibar”, for the name of this facility.
The meaning of a piece of LISP code can depend a great deal on the context in which it is compiled or evaluated. Virtually everything in a LISP system can change dynamically, and analyzing the meaning of code in the right context is a significant problem.
This facility focuses on analysis of code that is compiled, and it does so for two reasons. One is that most large systems written in PSL run compiled. The other is that once LISP code is compiled, the meaning of the object code is much more static than the meaning of the corresponding interpretive code.
Reliable system operation depends on modules being compiled in an environment that is known and fixed as much as possible. To do this, we compile each module with a compiler that is started afresh. After one module has been compiled we do not compile other modules with the same compiler.
To precisely analyze a module, we can start a compiler. To it we add the analysis facility, which should not change the compilation context. We then analyze the module, just as we would compile it. The analysis facility must be sensitive to the same contextual information that the compiler is.
Many modules have some side-effects on the context of compilation – defining a macro is just one of the things that affects compilation context. To be sure that a module is analyzed in the right context, the analysis should have the same side-effects on compilation context as the compilation would have.
File name defaulting: As its arguments scanalyze-file accepts both filename strings and pathnames. In any case the output-file argument is defaulted from the input-file argument. Any missing components of the name in output-file are defaulted using the function merge-pathname-defaults to be the same as in input-file, except for the file type (suffix). If that is not specified in output-file it is set to ”.XD”, which is the tentative standard for Xref-Databases.
For example, in the call (scanalyze-file ”pk:load.sl” ”mydir:”), the output filename generated is ”mydir:load.xd”. In the call (scanalyze-file ”#5:/pslroot/foo/test.sl” ”mydir:test-out.dat”, the output filename generated is ”mydir:test-out.dat”.
Scanalyze-Form recognizes forms such as compiletime bothtimes, and loadtime. It evaluates each at compiletime if the compiler would and analyzes each if the compiler would compile it. Actually, it recognizes the IGNORE and EVAL flags used internally by the compiler. This means that if analysis is started in the same context as a compilation, during analysis macros are defined in the same way in both cases, as are wconsts, if systems expand the same way, the same ”compiletime-loaded” modules are loaded, etc.. In PSL this means that the analysis context very closely matches the compilation context.
The caller can set up special actions that are in two categories. These are ”analysis hooks” and ”preprocessing hooks”.
The ENV arguments to various things are environments. An environment is currently a list of ”frames”. Each frame is a list of 2 elements. The first identifies the type of frame, currently always ’LOCALS. The second is the ”contents”, for locals a list of the local variable names.
The scanalyzer defines a number of variables that the user may give functional values to. Some support the user of this module in analyzing code. Others provide the means to preprocess or transform code. The hooks are functional variables of 2 args (expression and environment) to attach an analysis function to. Each of these must either be NIL or a list of functions. The first three of these hooks are called before any preprocessing or expansion of the form is done; the last is called after all preprocessing and expansion of a form is complete. Scan-Macro-Hooks Called before each macro is expanded. Scan-Fn-Hooks Called before descending into any expr to analyze it. Scan-Non-Pair-Hooks Called with each non-pair examined.
After-Expansion-Hooks Called with each fully expanded expression. Not applied to forms such as macro calls, because they are expanded further before being returned.
These are checked respectively before and after an expression has been expanded. After expansion means that the call has been expanded until it is not a macro call, and all subexpressions have been expanded also. If the car of an expression is an id that has the appropriate property, each member of the list that is the property value will be funcalled. These properties are checked at the same points where the scan-xxx hooks are checked. Thus it is of no use to give a macro a post-expand-scanners property. The properties are:
Pre-expand-scanners take precedence over scan-macro-hooks and scan-fn-hooks.
Current-Function This has the name of the current function where that can be determined. Forms such as DE and DEFUN that expand into calls on PUTD with constant function names cause this to be rebound inside analysis of the function body argument.
Current-Top-Level-Form Contains the argument that was passed to the call on scanalyze-form currently being executed.
Top-Level-Code? Is T for expressions not nested in any invocation of the function FUNCTION. Expressions for which this variable is T are known to be performed at initialization time.
Special-Expander Should be a functional variable of 2 arguments when set to a non-NIL value. If it is, and if the similar functional value of expand-specially? is non-NIL, the special-expander function will be used to expand the form.
Expand-Specially? If non-NIL, we use special-expander to expand the form. Non-Pair-Expander Either NIL or a functional value of 2 arguments used to expand expressions that are ”atomic”.
If any function or macro, etc.has an Expr-Expander property, its value must be a 2 argument function that scans and expands expressions whose first element is that id. This module provides several such functions so that built-in special forms can be analyzed.
Special-Expander and Non-Pair-Expander have first priority. Then comes any Expr-Expander property. Then normal processing by type of function.
The fluid xref-assertions holds the assertions generated by xref-assert and xref-assert-list, dumped out to a file by scanalyze-file.
Record-usage ignores non-pair expressions. Expressions that are pairs (function calls) are checked for an openfn or opencode property, which would imply that they are compiled in-line. No assertion is made for these or for expressions where the function part (car) is not an id.
If the function being called has a usage-asserter property, the value of that property is funcalled to make whatever assertions it will. Otherwise if the function is flagged dont-record-usage, no assertion is made.
The value of each of these variables is a list of functions (or macros or both). The ”setup” function, above, flags each function on each of these lists as dont- record-usage.