Fixed point calculations in TeX

It’s well-known that TeX is good at integer arithmetic, and does not provide any primitive functions for real number calculations. Experienced TeX programmers will know that you can make use of dimen registers to do real number calculations at speed, at the cost of accuracy (as TeX truncates dimensions at five decimal places). However, this is not exactly ideal and can be a bit awkward to use.

An alternative for LaTeX users is to use the fp package (which has been around since LaTeX 2.09). This is a powerful package, but can be rather slow. Part of the reason is that it allows 18 digits either side of the decimal point: a wide range of numbers, but almost certainly overkill in most cases. For applications that really need large numbers (such as pgfplots) input using floating points (such as 1.23e20) is probably better. Floating point calculations make life a bit more complex again, but can be done in TeX (see for example pgfmath).

For LaTeX3 I’ve been working on what would be a sensible approach: real number functions are needed in various places. At the moment, the plan is to support fixed-point numbers with nine digits either side of the decimal place, so up to 999999999.999999999. That should be enough for most application, and at some stage supporting floating-point numbers might well be added. To date there is only basic arithmetic (add, subtract, multiply an divide) in the new code, but the plan is to add trigonometric and logarithmic functions. I’m sure that there will be other functions to add: I’ll be interested to see what is asked for.

One area that I’ve been working is overall performance compared to the fp package. The biggest single gain is of course moving from 18 plus 18 digits to 9 plus 9, which makes quite a big difference on it’s own. However, there are various places inside the code where there are opportunities to save time. First, as LaTeX3 requires ε-TeX I’ve exploited the \numexpr primitive where it makes a difference (mainly where it cuts down on the number of assignments needed). At the same time, there are places where using delimited macros is faster than actually doing mathematics! The exact performance gains depend on what exactly you are doing, but it is possible to draw some comparisons. Doing lots of repeated calculations it’s possible to get some feel for the difference between fp and the new LaTeX3 module. On my system 100 000 additions take 31.6 s with fp and 6.0 s with l3fp, while for 20 000 divisions it takes 64.8 s with fp and 4.1 s using l3fp. Quite some speed enhancement, and I think enough to justify using the new code!

6 thoughts on “Fixed point calculations in TeX

  1. Do you mean with TeX or breaking out into Lua? With TeX, things are slower than pdfTeX: my simple test file takes twice as long with LuaTeX as with pdfTeX. (XeTeX is also slower than pdfTeX, but not by as much.) Breaking out into Lua should be much faster: I’ve not tried that yet.

  2. I mean breaking out into Lua. Ttake division as an example. pgfmath has a 80 line function for it (I am assuming that LaTeX3 divide function will be of similar size). In LuaTeX that can simply be


    The same goes for trigonometric functions, etc.

  3. I suspected you meant that, but as I’d not tested it couldn’t say much. pgfmath uses dimens to do calculations, so l3fp works differently (much more like the fp package). Doing the raw calculation there is really no contest: Lua is faster than doing things in TeX by a large margin (addition seems to be about 4 times as fast, division about twice as fast, but I’ve not tested extensively).

    However, there is an additional issue to worry about. In l3fp I’ve defined a pretty rigid format for storing numbers as it makes sense when doing calculations in TeX:

    fp_set:Nn l_my_fp { 1.234 }
    fp_show:N l_my_fp % => '+1.234000000000'
    fp_add:Nn l_my_fp { 5.678 }
    fp_show:N l_my_fp % => '+6.912000000000'

    If you define things differently for Lua the outcome should still be the same (expl3 is meant to be a well-defined toolkit, after all). Doing some tests which still use the TeX code to reformat the numbers after Lua has done the calculations, addition and subtraction end up slower with Lua than with TeX! Obviously I’ll need to work out a Lua-based reformatting routine.

  4. well… that will just be something like
    string.format(“%0.9f”, x). Of course, you will need to worry about the idiosyncrasies of the IEEE floating point calculations, but then fixed precision calculations are not straight forward either.

    I am wondering what is the target for these functions. Are you thinking of a faster replacement for fp, or will this support be added to the latex3 kernel?

  5. There are a few more wrinkles to thing about (for example, dealing nicely with division by zero), but I’m not arguing that Lua won’t be much faster than TeX for f.p. calculations.

    The target is as follows. expl3 is supposed to provide a proper programming framework for LaTeX3 (both now in development and later as part of a LaTeX3 kernel). We and others need some real arithmetic functions. The immediate driver is working out intersections for doing rotation of boxes (work that Frank Mittelbach is undertaking), but longer term you can see that a proper “toolkit” for TeX must include some real number functions. The l3fp functions will be part of expl3: the basic arithmetic functions are on the SVN version of expl3, and will appear on CTAN next time I do an expl3 upload.

Leave a Reply