Cameron Laird's personal notes on Tcl compilers

[re-write along functional dimensions]

The most current document on this subject might be the Wiki page.

[I know a lot of people end up here just wanting to compile work into a deliverable they can give customers. I'll address that in the re-write.]

[Some of the information here is stale--that is, not pertinent to recent Tcl releases. I need to rewrite soon.]

Table of contents

Compilers

Available products

Flash! One conclusion of the February 2000 Tcl Conference is an initiative to consolidate at least four of these approaches into one. More on this, later.

Jan Nijtman's PlusPatch was, for a long time, the correct starting point for work with Tcl compilation. Jan has moved all his current development to his new Wrap project. Bill Schongar's April 1998 description description of how to build (obfuscated!) Windows executables with PlusPatch still makes for good reading.

I exercised the ICE compiler before commercial release. I'm confident it has been done by knowledgeable engineers, but I haven't had occasion to use it myself for a significant project. I've considered ET as a development tool, because I certainly combine work in C and Tcl. I know several happy ET customers, but I have yet to give it a serious trial; my impression is that its event-driven orientation is a bit skew to my needs. Its author has been responsive and thoughtful in all my dealings with him. Also, note that, as of fall 1998, Richard is extending his approach with his new Mktclapp. As he has written,

"For new development, please use mktclapp instead of ET".
Peter N. Schweitzer, incidentally, testifies to mktclapp's utility in a newsgroup posting.

[Explain David Ashley's IJU.]

I'm a fan of SWIG, a general-purpose tool which extends and embeds not only Tcl, but also Perl and Python. Larry Virden calls it "a _very_ nice extension writing aid". The documentation is literate and infectiously enthusiastic.

jWrap does much of what SWIG does, except it's far more intimate with Tcl internals, and correspondingly swifter in performance. Unlike SWIG (at least SWIG in July 1998) it permits overloading of C++ classes from within Tcl, and correctly handles structures of structures and polymorphism.

D. Richard Hipp writes

SWIG takes an existing C library and makes a Tcl interface for it. The cool thing about SWIG is that it will also make interfaces for other languages besides Tcl. But SWIG does not (I believe) help you to build a stand-alone executable, or to intermix Tcl/Tk code with C code.

ET, on the other hand, is designed to let you write a mixture of C, C++ and/or Tcl/Tk and compile the result into a stand-alone executable. Using ET, you can put bits of Tcl/Tk code in the middle of your C routines. And there is a facility in ET that makes it very easy to create new C routines that implement Tcl/Tk commands -- thus allowing you to put C code in the middle of your Tcl/Tk.

Paul Doyle reports
SWIG is designed for wrapping existing C/C++ code and, unlike ET, it does not facilitate adding Tcl/Tk code interpretation in C routines. I have to say that I found it quite easy to build a standlaone executable on both Unix and NT using SWIG - it looks after all the gory details for you. . . . I have used SWIG to provide a Tcl interface to C++ routines and ET to add scripting functionality to C++ classes.

Dennis LaBelle's freeWrap is a very exciting work-in-progress which, like TclPro Wrapper, converts Tcl scripts into stand-alone executables. Of complementary interest is Dennis's freeDelivery packaging utility [...]. Jan Nijtmans joined the competition with his October 1999 announcement that his Wrap for Tcl/Tk 8.2.1, which replaces tcl2c, does the same (except more compactly).

David Cuthbert started an interesting little bytecode-to-C compiler he calls kt2c in summer 2000.

Andreas Otto is quite proud ("The technology is a revolution in computer science and boost TCL in front of programming languages for the new millennium.") of his "Token Stream" compiler. As of May 2000, I'm unconvinced, to the point that I haven't yet tried it for myself. George Petasis and Frank Pilhofer have, without ever achieving any useful results. On the other hand, Mr. Otto's comp.lang.tcl postings make it evident that he's grappling with real issues of the sort a performance tuner must solve, and I have a vague memory of someone (who?) saying that his creation did indeed accelerate a significant application. I confess that the bad taste I perceive in several of his c.l.t postings rather prejudices me.

I know less about Koski's compiler, tclc [STALE LINK], protoTcl, and tcl cruncher; to this point, I've only collected references to them and confirmed that they exist. Tom Tromey's compiler is clever work, but dated, and "heavy-weight", in the sense that it relies on C++ templates, and perhaps specifically on an installation of g++ at 2.6.3 or later. Finally, note that tcl8.0 offers a byte-code compiler, although programmatic access to it won't be exposed for at least a couple more minor releases.

"TclPro Compiler is primarily intended to protect source code," writes John Ousterhout. TclPro Wrapper packages an application into a single executable for distribution.

Why compile?

There are at least a couple quite different reasons to use a compiler:

Think carefully about what it is you want to achieve with a compiler. In regard to the usual security concerns, Larry Virden has acutely declaimed that the best alternatives are these:

Don't give it to them.

Don't write it at all.

Create a legal license.

Prosecute them - as soon as you write the code, it's copywritten - nothing else really is needed to prevent them from using it. The license allows you however to specify who is and isn't allowed and what speciifcally can be done and what you plan on doing if they violate it.

Joe Moss explains many of the same points.

In a response to a question about how Tcl applications are installed, I give context for a fraction of the applicable "use cases": developing on your own desktop; distributing to other developers; distributing to end-users; ....

Static analyzers

Of interest in some of the same contexts are Lindsay F. Marshall's tclCheck package for formatting, grouping validation, ..., the ICEM CFD Tcl Lint (tcllint, tcl_lint) program, and Stefan Schreyjak's ambitious tclparse. Clif Flynt has written a delightful preliminary comparison of these three. Scriptics' TclPro includes a static analyzer, the Checker. Finally, recall Tcl's built-in "info complete", which is a quick answer to many programmatic needs.

A related theme is that of editors which are more or less aware of brace-matching, indentation styles, and so on:

Allied technologies

Grant Reaber uses his interesting EC to inline C within Tcl codings.

[Explain how crucial TclKit is.]


Cameron Laird's personal notes on Tcl compilers/claird@phaseit.net