Some TeX Developments

There was recent question on TeX.SX about expansion, where I suggested using \romannumeral for full expansion. This idea comes up quite often, so I thought it would be useful to look at how it works.

The \romannumeral primitive was intended to turn integers into roman numerals. As such, the input to \romannumeral should be an integer, in the same way that the input to \number is an integer. However, while \number produces output for both positive and negative integer input, \romannumeral produces no output at all for valid negative integer input. That is pretty clear with a simple case such as

\romannumeral -15

but becomes more powerful when used along with the ` syntax which TeX allows for including an integer

\romannumber -`0

Here, the `0 is converted into a integer which TeX treats as complete: `0 is 48, but `01 is the (terminated) integer 48 followed by a separate 1 and not the integer 481. The important thing for expansion, however, is that TeX always looks for an optional space to gobble after an integer, even in a case like `0 where the integer is automatically terminated.

How does this help with expansion? It’s all to do with how TeX terminate numbers. If we have the demonstration macro

\def\demo#1{%
  \detokenize\expandafter{\romannumeral-`0#1}%
}

the \romannumeral will produce no output (as the number it finds is -48). However, it will expand #1 until it finds an unexpandable token. That means that something like

\def\testa{\testb}
\def\testb{\testc}
\def\testc{abc}
\demo{\testa}

will be expanded to abc without needing to know in advance how many expansions are needed. In a real case, this is a great way to fully-expand \if... tests and so on, leaving only the result, but without needing to know how many expansions are needed (and having long \expandafter runs).

What you should notice here is that TeX will reinsert the expanded result, and will not complain if something non-expandable appears

\def\testa{\relax}
\demo{\testa}

will print \relax, as this is inserted with no errors or loss of unexpandable tokens.

The only proviso here is that TeX stops at the first non-expandable item. So something like

\def\testa{ \testb}
\def\testb{123}
\demo{\testa}

will stop at the space in \testa and not expand \testb. For real applications, that is not usually an issue as we are usually aiming to expand the beginning of the input.

My siunitx package continues to attract feature requests, even though for me and many people it is essentially ‘feature complete’. Many of these requests are as a result quite complex, and probably somewhat esoteric, but I do aim to be as accommodating as I can. At the same time, I want to avoid breaking things for the majority. So my approach is to take a few issues at a time and to work on them for each 2.x release, plus of course the regular bug fixes. At the same time, I’m keen to review my own work and to tighten up on some parts of the code. That’s particularly important in adding new features, as there are places where it turns out a bit more structure is needed.

For version 2.3, I’m focussing on the mechanics of tabular alignment. This entire part (around 1500 lines of code) is going to be rewritten, hopefully making things more reliable, improving performance and also ease of maintenance. For version 2.4, I’ll probably look again at how tablular material is collected up by siunitx,  and also at the parser for numbers: I’ve had some ideas to improve performance. Beyond that, I have a few general thoughts, for example the idea of ‘multiple error‘ parsing: that looks tricky!

Anther area to work on is the option interface. Some of these are not perfect, and v2.3 will see some revisions. I have further revisions already in mind, but don’t want to do too much at once so again have some ideas for v2.4. At the same time, I think there are some performance enhancements available by recoding parts of the option system. I’ll be doing that as part of the general review, and so again we should see evolution in that area. I’ll return to this general issue in another post.

That is quite a list, and so I’ll certainly be kept busy. I have other things to do as well, both in the TeX world and outside, so at the moment my thinking is v2.3 in July, with v2.4 in the autumn (perhaps October or November) and v2.5 vaguely pencilled in for ‘first half of 2012’. Hopefully that will keep everyone happy.

Now that TeX Live 2010 is frozen, work has started on test builds for TeX Live 2011.  This is really part of a continuing evolution, and so there are (so far) no big surprises for me. Everything installed smoothly on my Mac, and I’ll be updating my PC tomorrow if I have a chance. I do hope other people are getting on with downloading the test code!

(For those people who, like me, are TeXworks fans, I notice that TeXworks v0.4.1 is included in the Mac version of TeX Live 2011 and so presumably in the Windows version too. Great work by everyone who helped getting it to build successfully on the Mac, and of course everywhere else!)

Work on test TeX Live 2011 for release is now well under-way. That means that TeX Live 2010 is now frozen: there will be no more updates added to TeX Live 2010 from now on. In a few days, testing builds for TeX Live 2011 should appear. I’ll report on those once I’ve grabbed one!

A quick post which follows on from my earlier mention of QuickLaTeX. What I did not say was that as well as letting me use LaTeX in my posts, it also lets everyone use it in their comments. All you need to do is to put ![latexpage] at the start of your comment, and QuickLaTeX will then parse it and insert the relevant graphics. See for example my own use of this feature in a comment earlier today.

The yearly TUG conference for 2011 was originally scheduled to be held in Cairo, Egypt. Recent events there mean that there has been a rethink, and that this year’s event will now take place in Kerala, India, for the 19th to the 21st of October. This is going to be hosted by River Valley, a typesetting company with a very strong interest in TeX.

The {TeX} Q&A site is holding it’s first elections for community moderators. The idea is that anyone with enough involvement in the community can vote: there are 8 candidates (including me). In contrast to some of the other Stack Exchange sites, the election is a pretty calm affair, with a broad agreement on the nature of the job between the candidates. So quite how to make a decision on how to vote is not so easy. There are 8 days to think about it, and it’s worth noting that there will be a chat session on Thursday, so that people can put questions to the potential moderators. It will be interesting to see what gets asked.

The increasingly-popular {TeX} Q&A site is holding it’s first moderator election.Anyone with a reputation of 300 or greater can stand, which means a good portion of the community is eligible. The nomination stage runs for 7 days, so there is no immediate panic to decide to stand!

Detexify has rapidly become a standard tool for LaTeX users, allowing you to draw a symbol and be pointed in the direction of how to enter it in LaTeX. People who’ve followed Detexify from the start will know that it began as a ‘bit of fun’ project, but it’s clearly grown since then! There is now a fundraising call to pay for hosting the site: these things stack up. Hopefully, the community can find a good solution to this: it’s a great service, and I think is something that we should all look to encourage if we can.

On the {TeX} Q&A site, there was a question recently about splitting the first token off a list, with the requirement that spaces are not skipped. In my answer, I’ve used \let to remove one space. The question is how to do this. Normally, if you want to use \let you do

\let\TokenA\TokenB

In this case, TeX will skip spaces after \let and \TokenA, so we can’t use it to \let to a space. However, what we can do is notice that TeX allows us to have an optional = followed by one space in the syntax for \let. We also need to make sure that TeX does not discard two spaces in the early stage of parsing, so can use \@firstonone:

\@firstofone{\let\TokenA= }

This will \let \TokenA to the next token in the input, even if it is a space. I’ve used this to remove the next token from some input in combination with \afterassignment:

\long\def\firstofone#1{#1}
\def\GobbleExactlyOne{%
  \afterassignment\NextThing
  \firstofone{\let\TokenA= }%
}

Not something you need every day, but worth knowing about I think.