Testing TeX: Lua and TeX, and not just for LuaTeX

I wrote recently about the LaTeX3 build scripts, and that we are moving them to Lua for cross-platform work. One particular area the team are interested in is ‘unit testing’, something that’s common in ‘real’ programming but not so widespread for (La)TeX. The main reason for that is obvious: most people programming TeX aren’t professionals but instead do it as an add-on to their ‘real’ jobs (in my case, chemistry).

The LaTeX3 team have had unit tests for LaTeX for many years. The way these work is quite simple. First, set up a TeX run which does whatever tests are needed and write the output to the log file. The raw logs tend to be rather long, and have information in that varies from system to system, so the second stage is to run a script to extract out just the ‘important’ parts of the log (those get marked up by the TeX set up). The ‘processed’ log can then be compared to one prepared in a ‘reference’ run (where someone has checked the results by hand): if the two results match, the test passes.

Up to now, we’ve used Perl for that ‘log processing’ and have only run tests using pdfTeX. Moving to Lua for our scripting, we can drop Perl and do the post-processing in the build scripts themselves. That makes almost everything self-contained: other than needing Info-ZIP for making CTAN release zip files, all that is needed is a TeX system (featuring LuaTeX for texlua) and the operating system itself (basic file operations but also diff/fc). At the same time, we’re expanding our tests to run them with pdfTeX, XeTeX and LuaTeX. That’s already thrown up several bugs in LuaTeX: nothing most people will notice most of the time, but reported to the developers and to be fixed. (Most of these are about formatting in the log: if you test based on log changes, these are important!)

While the scripts aren’t fully ‘portable’ as they are designed around our development set up, the structures should be pretty clear. The LaTeX test script is also quite general. So we’d like to hope that other people can adopt and adapt them: feedback always very welcome!

Lua for LaTeX3 build scripts

Anyone who follows the LaTeX3 GitHub pages will have seen a lot of checkins recently, not only for the code itself but also for a new set of build scripts. The work should hopefully make us more efficient but also impact on others, directly or indirectly.

Once of the first things I did when I joined the LaTeX3 Project was to write a load of Windows batch scripts for building the code. At the time, most of the rest of the team were using Unix (Linux, Macs, …) but I was using Windows, so this was quite important. Since then, the team’s IT set up has varied a bit but so have the requirements for our scripts, so I’ve ended up working quite a bit with both the batch files and the Unix Makefiles.

The ‘two sets of scripts’ approach has a few problems. The most obvious is that we’ve had to keep the two in synch: not always entirely successful, and not always remembered! At the same time, the ‘standard’ facilities available on the two systems are different: we’ve had to require Perl on all platforms, and work out how best to make everything ‘looks the same’.

Some recent discussion prompted me to consider some new work on the scripts, but with Lua now available anywhere that has a TeX system (as texlua for script work), it seemed natural to move to an entirely new set up. The plan has been to have one ‘master’ script for the entire LaTeX3 repository, rather than the ‘copy-paste’ arrangement we’ve had to date, and to use very simple ‘configuration’ files for each separate module. That’s worked really well: we’ve now got one big file covering almost everything, no longer need Perl and have also addressed some of the things that prompted this in the first place! (More on those in another post soon.)

Even if you are not using LuaTeX for day-to-day TeX work, the scripting ability for supporting development is very handy. Lua doesn’t cover the OS-dependent stuff, but using our existing script code and a small amount of detection we’ve been able to work with that. The new scripts are hopefully nice and clear, more flexible than the old ones and most importantly only need changing in one place for each issue.

So how does this impact on others? First, it makes it easier for the team to work on LaTeX3, which should be a good thing all round. The scripts should also mean that we don’t have the odd strange change depending on which operating system is used to do a release. Second, we’d like to hope that other TeX developers can take ideas from our scripts for their own work. We’re particularly interested in testing TeX code, and I’ll come back to that in my next post. Last, and linked to testing, we are picking up some ‘interesting’ engine-based issues. Those are getting reported, and even if you never use LaTeX we’d hope that will be a benefit!