Some TeX Developments

As LuaTeX progresses, the desire to use it in real documents increases. ConTeXt Mark IV is clearly the leading exponent of this, although it is still experimental and so perhaps not the best choice for day-to-day work. Karl Berry has said to me that he hopes that the inclusion of a “proper” programming language will bring new talent to (La)TeX. After all, some of the methods used to program TeX are complex to say the least.

This raises the question of how much to do in Lua and how much in TeX. Clearly, LuaTeX brings things to TeX that are impossible without it. So there it is simple: use Lua. However, what about things that can be done with either Lua or TeX? Part of this probably depends on the complexity of the different solutions (if it is easy in TeX, why do it in Lua, and vice versa). I’d also imagine that experiences TeX programmers will tend to stick to what they know, and favour TeX. On the other hand, new programmers coming to TeX because of Lua will favour the Lua route. I imagine that there will still be a lot more TeX than Lua code.

I’ve been discussing some points about the future of (La)TeX with various people, and some key issues come to mind. Most LaTeX users do not want to meddle with the internal parts of LaTeX or TeX. In an ideal world, I suspect most users would like to need little beyond the correct document class to get things “just right” in their layout. Perhaps a few simple settings, but really little more than that.

With the correct packages and class loaded, you can do many things in LaTeX. However, you really shouldn’t need to load specific support for hyperlinks, T1 encoding, basic font changing, creating new float types and so on in 2009 (let alone 2010, 2011, etc.). The efforts of the LaTeX3 team have to date focussed on programming and to a lesser extent document design. How things will work at the user level is much less clear.

I’d suggest that a real focus on getting something for users would be the best way forward. This might mean less improvement internally, but I’d think that a LaTeX kernel which could do everything in The LaTeX Companion would be pretty successful, even with few changes “under the hood”. This would mainly be a re-coding excercise from existing packages, which in a way is similar to what I’ve tried to do with siunitx. Much of the basics in siunitx are taken from other packages (at least in terms of user interface), but it brings several ideas together in one place. The same idea could easily be applied to the kernel. Of course, this might leave some of the clever ideas for LaTeX3 out of the code at this stage, but I’d hope would get momentum behind a more regularly updated system.

One particular area to think about is fonts. With both XeTeX and LuaTeX able to handle system fonts directly, the basic LaTeX system seems very antiquated. At present, LaTeX3 only requires e-TeX, not LuaTeX (in contrast to ConTeXt Mark IV). Should the LaTeX team say something like:

For current testing purposes, only the e-TeX extensions are needed, but this is likely to change. XeTeX or LuaTeX will be required to run the release version of LaTeX3 with full functionality.

I’d say yes, as I think that it’s time to move on from complex font installation and usage restrictions. I’d also be very tempted to say that LaTeX3 will assume UTF-8 input unless otherwise specified (as both XeTeX and LuaTeX are native UTF-8 systems).

This type of approach will make LaTeX easier to use, and I’d hope to see it arrive! After all, users are the TeX community.

The XeTeX and LuaTeX engines both offer exciting ideas to the TeX user and the (La)TeX programmer. Using XeTeX, UTF-8 input is easy: no more awkward escape sequences. Font handling in XeTeX also makes it trivial to use any font installed on your system. On the other side of the equation, LuaTeX is going to bring a lot of useful programming tools to the TeX world. This should make some things a lot easier (handling floating point manipulation seems an obvious one), and make other things possible that are not currently.

The issue for me is the gaps between the two. XeTeX doesn’t have things like microtypography (try loading the microtype package and doing a XeLaTeX run), and it’s not clear to me how LuaTeX will work out with font handling. There’s also the DVI versus PDF issue: will we see a system where EPS, PNG, JPEG and PDF graphics can all be included without worrying about the engine in use? In many ways, I suppose these questions are more for the LuaTeX team to address: XeTeX has essentially delivered what it set out to do (a Unicode TeX which can use system fonts natively).