Skip to content

Structure editors, IDEs, and another Lisp flashback

Tree structure editors were once all the rage in the Lisp community (and elsewhere). The idea is compelling – work directly on the structure of your code, rather than a text representation. In practice tree structure editors turned out to be rather tedious to use. This of course was back in the day of terminals with 4800 baud serial connections (or slower), so perhaps faster computers with drag and drop GUI interfaces would change the game enough. The subtext demo linked to from Jon Udell’s article on Exploring the deep structure of code offers a GUI version of the old tree structure editor.

The example and the accompanying narrative is somewhat interesting. Oddly enough it is not the tree-structure-editor aspect that I find compelling. Rather it is the use of sample values, live computation, and highlighting of “dead” (not executed) code.

None of this requires a new visual representation – by the way.

One of the gifts earned from early heavy exposure to Lisp was the notion of surface representation. Lisp code could be automatically formatted into MLISP – an Algol-like text, edited as MLISP, and the MLISP text could be parsed directly back into Lisp. In another project, Ada source code could be parsed into a Lisp structure representing the Ada parse tree, the parse tree could be interpreted, compiled, or pretty-printed as text for editing.

Put differently, your Java source code (or your language of choice) is simply a surface representation for a parse tree. You can map from the parse tree into Java source text. In fact you could do most everything in the demo without having to use a different surface representation. So I do not see (for example) Java source as “hiding” the deep structure of the program — rather this is simply a compact textual surface representation.

Note that there are difficulties in preserving comments (you would likely have to restrict comment placement somewhat). Note also that the use of preprocessor macros in C/C++ makes this all a lot harder (a good argument restricting or eliminating use of the preprocessor in other languages – too late for C/C++).

One of the strong, subtle advantages in the Lisp environment was in having the parser and pretty printer as simply another set of functions in the library. In the C/C++ world the compiler is a large monlithic seperate program, and offers only a one-way path from source text to compiled code. You cannot reuse parser or gain access to the parse tree. In the Java world the compiler is once again part of the environment (with only slightly awkward access). This is one prime reason Java IDEs can support refactoring, and “deep” operations on your Java programs.

In my opinion at least, the notion of tree-structure editing is still only of limited value, and not a replacement for simple text editing. Back in college, after we had all played with structure editors for a time, one member of our group identified the source of our discomfort by noting that many simple common operations in a text editor were much more complicated in a structure editor.

What I take away from all this is that most of the time when programming, what I want is a text editor. A smart text editor with insight into the parse tree generated by the text is useful. Alternate representations (tree views, etc.) for navigation and some sorts of tree-oriented operations are useful. But for the 90+% of the time when I’m hammering out code, the main interface should be a text editor.

Oddly enough this sounds an awful lot like a present day IDE :).

What we can do is go further down the path – finding new ways to use the knowledge of the underlying parse tree. We can highlight and animate the program text. We can offer alternate views (surface representations) of the underlying structure.