Some TeX Developments

A 'new' primitive: \expanded

In recent years, development of pdfTeX has been very limited, with the v1.40 branch now being around for over 10 years. However, in the past there were plans for a v1.50 branch, and some code was actually written. One primitive that was fully coded-up at that time was \expanded. The idea of this is pretty simple: it carries out full expansion like \message (and almost) like \edef), but it is still expandable. For example, try

\def\a{\b}\def\b{c}
\message{Hello \a\space #}
\detokenize\expandafter{\expanded{Hello \a\space #}}
\bye

using LuaTeX.

Why is the example for LuaTeX? When LuaTeX development started, the team behind it used the development code from pdfTeX as a starting point, and that included \expanded. However, release pdfTeX itself didn’t incorporate this code, and so it’s not been more widely available.

Enter the LaTeX Team

For some time, the LaTeX Team have been thinking about asking for \expanded to be made more widely available. Unlike the \romannumeral ‘trick’, \expanded does not require any hard work to get ‘past’ any output, so it is very useful for creating macros that work like functions. It’s also fast and clear in intention.

In the past, making requests for changes to the pdfTeX codebase was hard as building and testing is non-trivial. However, nowadays there is a GitHub repo which is also set up for Travis-CI. That means that there is an easy way to test: set up an Ubuntu virtual machine, clone the repo there, and run the tests in the same way Travis-CI does.

With that handy set up available, I sat down (on behalf of the team) and did the hard work: a bit of copy-pasting! As well as pdfTeX, I worked out how to add \expanded to XeTeX and the Japanese TeX engines pTeX and upTeX. After a bit of discussion, this code has been accepted by TeX Live, and will be there in the 2019 release.

Get it now

For those people who want to test now, LuaTeX of course has \expanded, so it is easy to try out. For MiKTeX users, Christian Schenk has already updated all of the binaries, so a quick update will give pdfTeX and XeTeX with \expanded. For TeX Live users, binary updates only happen once a year. But if you want to grab something now, you could look for example at W32TeX (which is the source for Windows binaries in TeX Live): you’ll have to manually rebuild your formats, but if you know enough to want to test, you probably understand that instruction!

Using \expanded

The team have already started planning to use \expanded, and recently added a new expansion type to expl3: e-type. We have some emulation code that allows this to work (slowly) even with older binaries. I’d expect us to make heavy use of this in new functions: it’s a lot easier than the \romannumeral approach.