7.3.7.5. Tangling

There's no support for 'chunking' (noweb) or 'macros' (FunnelWeb): i.e. the ability to define code fragments to be included inside others, building the code heirarchical. This is an absolutely fundamental requirement, and it is not omitted by accident. There is some primitive macro support: you can tangle a file and read it in on the next pass, or you can tangle a memory sink and then use it as a source. It's easy to write commands that simplify use of these mechanisms, and I had some, but they've been removed.

The reason there's no chunking is that I'm not happy with any of the mechanisms yet, nor certain of the requirements. FunnelWeb, for example, tangles code after scanning the whole document, and so can include a macro A in a macro B, even if B is defined _before_ A. This isn't possible with a simple one pass dynamic system. On the other hand, it seems obvious that tanglers that generate chunks should perform language specific processing, such as formatting comments and introducing the #line directives, while the generated output should be included raw (since it has already been processed).

Unfortunately, this is a gross oversimplification of the requirements of a meta-programmable system. For example, a lot of programming would benefit from chunks with arguments, which FunnelWeb supports (and which is why they're called macros). As another example, FunnelWeb tracks usage of chunks, and requires every chunk be used exactly once, unless otherwise declared in chunk definition. Yet another example: templates, or skeleton. This is conceptually the same as a macro with arguments, but that is a lousy way to actually have to write the call. The correct method is to use co-routines (interleaving), diversions, continuations (etc).

So we see that generating code by expanding with substitutions, even the simplest kind of parameterless chunk substitutions, seems to be the thin end of the wedge leading to the development of a complete programming language --- and Interscript is already equipped with at least one of those.

Here's a final 'killer' example. One of the languages we are sure to want to tangle is ... Interscript.

If a tangler supports source cross references such as the #line directive, then there will always be such a directive as the first line of a code file. Unfortunately, some languages and systems have special requirements on what's in the first line. Cases I know of include:

the Unix #! convention, which could be used by Python and Tcl programs,
vi: set tabwidth type lines in text files (including programs),
and quote a lot of typesetting other systems (including compilers) which require global flags to be set by pragmas and the like, right on the first line.

The solution is to have a special command to generate the first lines of a code file, which doesn't trigger generating the #line directive.

There's no tangler for many languages: Modula, Pascal, Cobol, Scheme, etc etc.

There's no support for bolding keywords.

There's no Tcl string tangler. The Perl string tangler is bogus.