Programming: Lua versus TeX

As LuaTeX progresses, the desire to use it in real documents increases. ConTeXt Mark IV is clearly the leading exponent of this, although it is still experimental and so perhaps not the best choice for day-to-day work. Karl Berry has said to me that he hopes that the inclusion of a “proper” programming language will bring new talent to (La)TeX. After all, some of the methods used to program TeX are complex to say the least.

This raises the question of how much to do in Lua and how much in TeX. Clearly, LuaTeX brings things to TeX that are impossible without it. So there it is simple: use Lua. However, what about things that can be done with either Lua or TeX? Part of this probably depends on the complexity of the different solutions (if it is easy in TeX, why do it in Lua, and vice versa). I’d also imagine that experiences TeX programmers will tend to stick to what they know, and favour TeX. On the other hand, new programmers coming to TeX because of Lua will favour the Lua route. I imagine that there will still be a lot more TeX than Lua code.

10 thoughts on “Programming: Lua versus TeX

  1. I have never been comfortable with low level TeX programming (mostly because I don’t fully understand the internals yet). If I have a choice between doing something in TeX or using a high level scripting language, I will choose the latter.

    One of the things I look forward to when LuaTeX becomes mainstream, is faster and easier number crunching. Till Tantau has mentioned a couple of times on the pgf-users mailing list that reimplementing PGF’s mathematical engine in Lua will speed up things by several orders of magnitude. It will also allow drawing more complex drawings like for instance 3D-graphics.

  2. I’d agree about the number cruching. I’d imagine my first steps with LuaTeX will be writing an add-on for siunitx to “farm out” the rounding and so on to Lua.

    As to the TeX versus Lua side, I suspect that things will depend on what you want to do. I’d see abstract programming as more Lua, but more typeset-related things in TeX.

  3. In MKIV, a lot of bookkeeping has moved to Lua. Things like crossreferences, bookmarks, are other things that were traditionally done using a two pass mechanism are now done in Lua. Soon it will be possible to get rid of the auxilary files altogether for ConTeXt.

    Also, a lot of font related stuff is now handled by Lua. You can create a virtual font on the fly, combine multiple fonts, borrow a character from another font, all sorts of things for whic .vf and .enc files were used, are now handled by lua.

    Another advatange is that lua in luatex comes with the lpeg parsing library, so you can write parsers. ConTeXt XML processings is now done by lua. There is also a proof of concept calcmath module which converts “T1-calc math syntax” into “TeX markup” on the fly. So you could write “calcmath{f(x) = sin (x) + sqrt(2*x)” and it will be converted to “$f(x) = sin x + sqrt {2x}” behind the scenes.

    Also, part of the mplib parser is written in lua, which makes including mp graphics in MKIV really fast (Most of the speedup is due to the availability of MP as a library, but still lua parsing is much easier than parsing done in TeX).

    So, what has happened in ConTeXt is that a lot of backend has moved to lua. The user interface is still almost the same, so normal users do not notice any difference.

  4. Related to what Aditya has written: Although this probably wont happen immediately, I see the possibility to get rid of external programs like bibtex or makeindex as a potential advantage of LuaTeX. No need for external programs, no need for multiple runs, just include everything in one package which takes care of everything.

    IME explaining why you need an external program like bibtex is always difficult with people new to LaTeX. Of course, it’s not a huge hurdle, but it would reduce complexity.

  5. Thanks for the comments Aditya and Simon. The low-level stuff in ConTeXt Mark IV is very clever, and I’d say falls into the “not possible in TeX” category. The same really applies to BibTeX or MakeIndex implemented in LuaTeX. I’m wondering about things that can be done in TeX, but also in Lua.

    On the BibTeX issue, my understanding (from various biblatex-related discussions) is that the biggest issue with BibTeX is not the program but the data format. The .bib format has some fundamental problems that need a good solution.

  6. Re: BibTeX. What I said wasn’t really about BibTeX per se. BibTeX just served me as a widely used example for an external program which could be implemented in LuaTeX and would reduce complexity for the user. That BibTeX (the program and the format) needs improvement should be obvious to anyone who has actually used it (and with biblatex, many of the problems have already been overcome); but even if only the current BibTeX was re-implemented with LuaTeX and you could just compile a bibliography in one run, this would already make LaTeX more user-friendly IMO.

  7. Yes of course the principal is a good idea, and should also be (relatively) easy. I was only pointing out that there are some “hard problems” with BibTeX which need good ideas, independent of LuaTeX per se.

  8. Hi,
    I know that this is really off topic, but every time someone says “well, I have a problem with bibtex”, I feel obliged to point out that the amsrefs package made bibtex almost entirely obsolete, with much clearer syntax and styling done in pure (La)TeX.

  9. amsrefs is very clever, but I think that there are a few issues (at least for me). First, there are already a LOT of BibTeX styles about. From designing a couple, the major pain is not BibTeX but the way publishers make subtle changes between different types of reference. So implementing styles a second time is not attractive. Second, I don’t see that amsrefs lets you use a BibTeX database directly as the source of your refs. This is a problem as many LaTeX users have lots of references in that format. Finally, like Simon I use biblatex. It does lots of clever stuff which I’m not sure amsrefs can do.

Leave a Reply