Pretesting TeX Live 2010

The first testing builds of TeX Live 2010 are now available, which you can also read about in the TeXblog entry. I downloaded it a few days ago, currently just to my Mac (Windows testing on my system at work starts next week). There are a few changes, some of which were planned for TeX Live 2009 and did not make it. The highlights for me

  • Restricted \write18 support is back. I’ve written about the issues with this before, but as I understand it these are now solved. The idea of this support is that EPS graphics can be turned into PDF graphics automatically, meaning that pdfLaTeX is much easier to use for end users with mainly EPS graphics available.
  • The default PDF output is level 1.5, which means that more compression of the output is available. The amount of compression depends on the type of output (files with lots of hyperlinks seem to show the most dramatic results). I’ve been using PDF 1.5 for a while with no issues, so I hope that this is applicable to most users.
  • The is a Unicode version of BibTeX included: BibTeXU. I can’t see any details of where this is coming from or the exact nature of the support: I hope to gain enlightenment at some stage. I’ll certainly be testing it.

As I’m currently testing on my Mac, I’ve installed the 64-bit binaries (these still have to be installed in addition to MacTeX at the moment). I’m seeing slightly better performance with the 64 bit binaries than the 32 bit ones, but not by much. On Windows I’m currently limited to 32 bit, so there I’ll have nothing to worry about!

So far, I’ve not had any major issues. TeX Live is very much evolution, not revolution, so that is not too much of a surprise. The team have done a good job, as usual, and I hope that others will brave the testing status of this release to help find any bugs before it’s unleashed on the TeX world at large.

siunitx version 2 released

After many months of work, I’m pleased to announce that I’ve just sent version 2 of siunitx to CTAN. Many readers will be familiar with the package and some of the development process. Here, I’ve put together a summary as ‘release notes’ for the new version.

A comprehensive (SI) units package

Typesetting values with units requires care to ensure that the combined mathematical meaning of the value plus unit combination is clear. In particular, the SI units system lays down a consistent set of units with rules on how these are to be used. However, different countries and publishers have differing
conventions on the exact appearance of numbers (and units).

The siunitx package provides a set of tools for authors to typeset numbers and units in a consistent way. The package has an extended set of configuration options which make it possible to follow varying typographic conventions with the same input syntax. The package includes automated processing of numbers and
units, and the ability to control tabular alignment of numbers.

Version 2

Over the past two years siunitx has developed to include many features not originally foreseen when development began. While it has been possible to add a range of new features, some of the underlying limitation of the version 1 code have made this difficult. At the same time, renewed effort by the LaTeX Team on the development of LaTeX3, and in particular the expl3 programming system, has offered a more robust method to create the internal structure of siunitx. As a result, version 2 of siunitx has been almost completely re-written internally.

As well as fixing a number of bugs and limitations in the original release, version 2 is also much better written to work quickly. As a result, most users should see performance enhancements with this new release of siunitx.

As part of the revision of siunitx, the option system and user macros have been completely re-thought. The options now have longer, descriptive names and also a much clearer range of input values. The options which in version 1 took either a key word or a literal value have been replaced by ones which take literals only: in some cases this means that advice has been added to the documentation on how to get particular output effects.

Moving from version 1 to version 2

Depending on how you use siunitx, there may be very little to do to move to version 2. The new version includes a compatibility support file, meaning that loading siunitx using:

\usepackage[load-configurations = version-1]{siunitx}

should mean that existing documents compile with very few changes.

There are some changes to standard settings between version 1 and version 2, which may lead to some alterations in documents. At the same time, a small number of the features of siunitx version 1 which I feel did not work cleanly have been dropped. At present, some of these are scheduled to be re-examined for
inclusion in later releases of siunitx.

While there is a back-compatibility layer for users upgrading, it is strongly recommended that documents are updated to use the new option names and functions. The new approach has been chosen as it is an improvement on the previous version, and in the longer term this layer may be removed.

Installation

Most users will obtain siunitx as part of their TeX distribution. MiKTeX 2.8 should include siunitx version 2 after a short delay (a few days after CTAN upload). For TeX Live users, there will be a slight delay as the package will appear in updated form in TeX Live 2010 but not TeX Live 2009 (which is frozen).

For users who wish to install siunitx themselves, the package is available as a pre-extracted zip file, siunitx.tds.zip. Simply unzip this in your local texmf directory and run ‘texhash’ to update the database of file locations. Version 2 of siunitx requires up to date versions of the LaTeX3 packages expl3 and xpackages. These are also available from CTAN in ready to install format (as expl3.tds.zip and xpackages.tds.zip), and can be installed in the same way if necessary.

If you want to unpack the dtx yourself, running ‘tex siunitx.dtx’ will extract the package whereas ‘latex siunitx.dtx’ will extract it and also typeset the documentation. Typesetting the documentation requires a number of packages in addition to those needed to use siunitx. These should all be available in a complete TeX Live 2010 or MiKTeX 2.8 installation.

Development code and bug database

In order to help users see what is happening, and also to allow me to work efficiently, the development code for siunitx is available on the code hosting site BitBucket.

You can download the very latest code from there: of course, this may or may not work properly depending on exactly what I have added to the code.

The BitBucket site includes an issue tracker, where you can report bugs or make feature requests. I also add bugs to the database from e-mails I get from users. Filling in the bug database helps to make sure that I do not forget things, and also helps other users see what issues are known.

If you want to contribute code to siunitx, you can of course send patches directly to me. Alternatively, the code is hosted using the revision control system Mercurial, which was chosen as it is decentralised and is easy to install on a range of operating systems (I use MacOS X, Windows XP, Windows 7 and Ubuntu!). I’m happy to explain to potential contributors how Mercurial works for developing siunitx.

Roadmap for future releases

The bug database already includes a number of feature requests which are marked to be looked at for version 2.1. The current intention is that the next few months will be devoted to bug fixes in this release (v2.0), with moves to add features for v2.1 beginning in the late summer. I anticipate that v2.1 will be released toward the end of 2010.

It is likely that not all of the features currently marked as to be looked at for v2.1 will be fully working by the time it is released. At the same time, there are some longer term areas which may also need attention. Version 2.2 of siunitx is therefore planned, but with no current list of features marked for inclusion. This version is likely to appear in Spring 2011.

One longer term aim is to include LuaTeX support in siunitx, so that the entire package can work much more rapidly with LuaTeX than when using TeX macros alone. This is not likely to happen until next year (2011), but is in the bug database and is part of the longer term development plan for siunitx.

The internals of siunitx

Currently, the only documented interface to any of the functionality of siunitx is via the key-value control system and functions described in the manual. The internal code of the package is not documented, and there is therefore no guarantee of stability of internal functions. While it is common for users to have to modify the internals of LaTeX2e packages as part of their documents, this is not good programming practice and is not encouraged for siunitx, or indeed in general.

If there is a user function that you require that is not available using the documented tools, please either e-mail or report a bug in the database. One of the general aims of siunitx is to provide a proper documented interface for all of the
functions of the package. I am therefore very happy to add interfaces to internal processes as necessary.

Programmers should note that siunitx is coded using the LaTeX3 ‘expl3’ programming system. This looks somewhat different to traditional TeX or LaTeX programming. Details of the programming environment are documented as part of the expl3 bundle. Currently, none of the internal functions or interfaces are documented, and so are not meant for use outside of siunitx. Other programmers wanting to make use of internal siunitx functions are encouraged to get in contact with me. This will enable me to ensure that the parts of siunitx which are needed by others are documented and are not changed without consultation.

From \newcommand to \NewDocumentCommand

Following on from my last post, I thought it might be useful to give some simple example of how the xparse package works and why it’s useful. I want to do this to show end users of LaTeX how it can replace \newcommand, so the example will not involve anything too complex, code-wise.

First, why would you want to use xparse’s \NewDocumentCommand in place of \newcommand? First, \NewDocumentCommand can make macros that take a mixture of arguments that \newcommand cannot. With \newcommand, you can make a macro that takes a number of mandatory arguments, or ones where the first argument is option and in square brackets, but that is it. Anything else then needs the use of TeX programming or internal LaTeX macros: not really helpful for end users. The second thing is that \newcommand macros are not ‘robust’. This shows up where you need to \protect things, which can be very confusing. Macros created with \NewDocumentCommand are robust, and this means that they work more reliably.

I’m going to illustrate moving from \newcommand to \NewDocumentCommand with a series of simple examples. For all of them, you need to load the xparse package:

\usepackage{xparse}

Macros with no arguments

The simplest type of macro is one with no arguments at all. This isn’t going to show off xparse very much but is is a starting point. The traditional method to do this is

\newcommand\NoArgs{Text to insert}

which becomes

\NewDocumentCommand\NoArgs{}{Text to insert}

That does not look too bad, I hope. Notice that I’ve got an empty argument in the xparse case: this is where the arguments are listed, and with \NewDocumentCommand there always has to be a list of arguments, even if it is empty. That’s a contrast with the \newcommand approach, where we only need to mention arguments when there are any.

One or more mandatory arguments

The most common type of argument for a macro is a mandatory one. With \newcommand, we’d give a number of arguments to use:

\newcommand\OneArg[1]{Text using #1}
\newcommand\TwoArgs[2]{Text using #1 and #2}

\NewDocumentCommand is a bit different. Since it can work with different types of argument, each one is give separately as a letter. A mandatory argument is ‘m’, so we’d need

\NewDocumentCommand\OneArg{m}{Text using #1}
\NewDocumentCommand\TwoArgs{mm}{Text using #1 and #2}

This is still pretty similar to \newcommand: the useful stuff starts when life gets a little more complicated.

One of more optional (square brackets) arguments

To really get something clever out of xparse, the arguments need to be a little more varied than I’ve show so far. Let’s look at optional arguments, which LaTeX puts in square brackets. If I want the first argument to be optional, then LaTeX can help me

\newcomand\OneOptOfTwo[2][]{Text with #2 and perhaps #1}
\newcomand\OneOptOfThree[3][]{Text with #2, #3 and perhaps #1}

If I want anything else, I’m on my own (so no more \newcommand examples!). First, let’s do the two example using xparse. An optional argument in square brackets, which works like a \newcommand one, is ‘O’ followed by {}:

\NewDocumentCommand\OneOptOfTwo{O{}m}%
  {Text with #2 and perhaps #1}
\NewDocumentCommand\OneOptOfTwo{O{}mm}%
{Text with #2, #3 and perhaps #1}

How about two optional arguments? It’s pretty obvious:

\NewDocumentCommand\TwoOptOfThree{O{}O{}m}%
  {Text with #3 and perhaps #1 and #2}

What if we want something as a default value for the optional argument? With \newcommand, that would be

\newcommand\OneOptWithDefault[2][default]%
  {Text using #1 (could be the default) and #2}

which would become

\NewDocumentCommand\OneOptWithDefault{O{default}m}%
  {Text using #1 (could be the default) and #2}

The same idea applies to each optional argument: whatever is in the braces after the O is the default value.

More complex optional arguments

You might be wondering why we need the ‘{}c after ‘O’ when there is no default value: why not just ‘o’? Well, there is ‘o’ as well. Unlike \newcommand, \NewDocumentCommand can tell the difference between an option argument that is not given and one that is empty. To do that, it provides a test to see if the argument is empty:

\NewDocumentCommand\OneOptOfTwoWithTest{om}{%
  \IfNoValueTF{#1}
    {Do stuff with #2 only}
    {Do stuff with #1 and #2}%
}

Don’t worry if you forget to do the test: the special marker that is used here will simply print ‘-NoValue-’ as a reminder!

Two types of optional argument

Sometimes you might want two different optional arguments, and be able to tell which is which. This can be done by using something other than square brackets, often using angle brackets (‘<’ and ‘>’). We can do that using the letter ‘d’ (or ‘D’ if we give a default).

\NewDocumentCommand\TwoTypesOfOpt{D<>{}O{}m}%
  {Text using #1, #2 and #3}

What input syntax does this make? Let’s look at some examples

\TwoTypesOfOpt{text}             % One mandatory
\TwoTypesOfOpt[text]{text}       % A normal optional
\TwoTypesOfOpt<text>{text}       % A special optional
\TwoTypesOfOpt<text>[text]{text} % Both optionals

How did that work? The first two characters after the ‘D’ are used to find the optional argument, so in this case ‘<’ and ‘>’.

Finding stars or other special markers

Another common idea in LaTeX is to use a star to indicate some special version of a macro. Creating those with \newcommand is difficult, but it is easy with \NewDocumentCommand

\NewDocumentCommand\StarThenArg{sm}{%
  \IfBooleanTF#1
    {Use #2 with a star}
    {Use #2 without a star}%
}

Here, ‘s’ represents a star argument. You’ll see that it ends up as #1, while the mandatory argument is #2. You’ll also see that there needs to be a test to see if there is a star (\IfBooleanTF). This doesn’t mention stars as the test can be used for other things.

Summing up

There is more to xparse than I’ve mentioned here, but I hope that this is a useful flavour of what it can be used for. To get more flexibility there is a bit more to think about compared to \newcommand, but the overall consistency is hopefully worth it.

Promoting xparse

I’ve had a few chances in recent weeks to promote the xparse package. Regular readers will know that xparse is part of the efforts of the LaTeX3 Project, and is meant to provide a replacement for \newcommand, etc., for creating document macros. The promotions have come up where there are things that are hard to do with \newcommand but easy with xparse. One example was creating an optional argument for a macro but giving it in normal braces. With xparse, this is pretty straight forward

\documentclass{article}
\usepackage{xparse}
\NewDocumentCommand\en{g}{%
  \IfNoValueTF{#1}{\epsilon}{\epsilon_{#1}}%
}
\begin{document}
\( \en \) and \( \en{stuff} \).
\end{document}

This is clear, and does not need any knowledge of the internals of the LaTeX3 approach. So I’m going to keep promoting xparse for day to day LaTeX users, even if they aren’t using much code. Hopefully this will make life easier for them, and for me, and will be a real benefit now from the LaTeX3 work.

Building biber on Ubuntu 10.04 (Lucid Lynx)

As I’ve just installed Ubuntu 10.04 (‘Lucid Lynx’) on my test system, I thought I should check that I can get biber working. As in my earlier posts, this is not too hard, but it’s nice to have some instructions. As usual, first you need to download biber from the homepage and unpack the files. Using the Terminal, move the directory where the source is and do

sudo cpan Config::AutoConf
perl Build.PL
sudo ./Build installdeps
./Build
./Build test
sudo ./Build install

The cpan line adds one module to Perl which for some reason biber’s installdeps routine doesn’t find automatically: if you miss this out then the build will fail. There are a lot of Perl questions while the additional modules are installed: I just say yes to all to them. The build itself is pretty quite, and it’s almost at the point of being trivial (the above instructions now seem to work on all the platforms I use).

TeX Live 2009 on Ubuntu 10.04 (Lucid Lynx)

TeX users on Linux tend to use the packaged TeX installation provided by there distro, rather than using the standard TeX Live installer and managing things themselves. Ubuntu is one of the more popular Linux distros, and there has been an issue for a while that they had stuck with TeX Live 2007, despite TeX Live 2008 and then TeX Live 2009 being released.

The latest Ubuntu release, 10.04 ‘Lucid Lynx’, finally moves to TeX Live 2009. This finally makes it easy to get TeX Live in a reasonable up to date version:

sudo apt-get install texlive

This grabs a subset of the complete TeX Live 2009, but seems to include quite a good selection (a bit like the MiKTeX basic installation on Windows, but I think a bit bigger). TeXworks is also on the list (at least the stable version), so if you don’t want to compile it from the source you can do

sudo apt-get install texworks

and be ready to go. Of course, you might still need to grab a few more bits of TeX Live (for example, XeTeX is not included in the standard selection). However, it’s definitely an improvement of the earlier situation.