Dependencies

There’s been some recent discussion on the TeX Live mailing list about recording dependencies for (La)TeX packages. This is a good idea but means that package authors need to think about their dependency situation. So I thought a few words on this would be helpful, at least from the point of view of the most common case: LaTeX packages.

It’s pretty easy to accumulate \RequirePackage lines in your source, but if you are serious about giving a useful set of dependencies you need to know what each one is for. In many ways the rule is easy: require each package you use. What makes that more complicated is that you might use features which are available when you load package X but are actually provided by package Y. For example, if you load my siunitx package, it loads array so means that you can do for example

\begin{tabular}{>{$}l<{$}}

So how do you tell what your ‘real’ dependencies are? The usual rule is that you check the documentation: does it say that package X itself provides the features you use? In the case above, siunitx doesn’t document that syntax extension for tabular: it’s documented by array. So if you wrote a package that uses siunitx but also needs to use features from array you should

\RequirePackage{array}
\RequirePackage{siunitx}

This means that even if at some future stage there’s a change in the internals of a package you load, things should still all work.

If you want to track down where stuff might be coming from, you can always \listfiles to get a full overview of your current package use (starting from a small example).

There are a few places were packages are so closely linked you might not have to list them both. The most obvious is TikZ/pgf: the two are different ‘layers’ of the same set up but are documented together, so if you load TikZ you can assume pgf. Of course, there is no harm in listing both!

Beamer moves to GitHub

There are lots of places one can host development of open source code. I’ve used a few over the years, but in recent times have mainly focussed on GitHub. That’s true not least because the LaTeX3 development code is held there. The one package I’m involved in that’s to-date been elsewhere has been beamer: there are lots of issues in the tracker which I didn’t want to lose. So for some time it’s been slightly orphaned on BitBucket. I’ve now (finally) been able to migrate everything using a very handy script maintained by Jeff Widman. I’m making final arrangements on the move, but the key is that new issues should to to GitHub.

LaTeX2e and e-TeX

LaTeX2e was released in 1994 and since then the LaTeX3 Project have been committed to keeping it working smoothly for users. That means balancing up keeping the code stable with fixing bugs and adding new features.

Back in 2003 the team announced that the e-TeX extensions would be used by the kernel when they were available. The new primitives offered by e-TeX make many parts of TeX programming easier and  often there’s no way in ‘classical’ TeX to get the same effect. As e-TeX was finalised in 1999, starting to use it seriously in around 2004 meant most people had access to them.

Since then, the availability and use of e-TeX has spread, and almost all users have them available. Indeed, the standard format-building routines for LaTeX have included them for many years. There are also a lot of packages on CTAN that use e-TeX, most obviously any using the expl3 programming language that the LaTeX3 Project have created.

The team had always meant to say at some stage that e-TeX was now required, and indeed thought we had until I checked over the official newsletters! So as of the next LaTeX2e release, scheduled for the start of 2017, the kernel will only build if e-TeX is enabled. For this release, we are likely to add a test for e-TeX but no actual use directly in the kernel, though in the future there will probably be more use of the extensions.

Biblatex back-end update

I wrote recently abut the need to manage biblatex back-ends, and thoughts the maintenance team had on which way to proceed.

After a bit of thought, we’ve gone for a solution we hope works for users and for us. Reversing what seemed like a ‘good idea at the time’, we’ve re-integrated (almost) all of the TeX code into one pathway. This is focussed on Biber, but we have a small stub that converts the relevant parts to work with BibTeX. So users how don’t need Biber or can’t use it can still use BibTeX and get mainly the same results.

There are a few changes, mainly related to places where BibTeX and Biber had ended up with different interfaces and where BibTeX can’t (reasonably) support the Biber one. In those places we’ve dropped the ability to use a ‘BibTeX alternative pathway’ as it makes the code complex and makes switching between back-ends tricky.

Hopefully the balance we’ve gone for will work for everyone: you can still use BibTeX, it still does almost everything it always has with biblatex, but we’ve got a more sustainable code base for the future.

biblatex feedback

Getting an idea of what users actually use can sometimes be tricky. To help understand how people are using biblatex, there’s a short survey set up. In particular, we are looking for some idea of how people use back-ends: as I’ve mentioned before, having two of them means more ‘interesting’ development so it’s important to get some insight into real life use.

Managing biblatex backends

For the past few years, biblatex has been looked after by a small team led by Philip Kime. When we first took this up we were mindful of the need to think carefully about back-ends: how reference data is extracted from a .bib file or similar sources.

There are currently two back-ends for biblatex: BibTeX and Biber. Biber is where the development is taking place and offers Unicode support, whilst BibTeX itself is frozen but also does a lot less ‘stuff’. So there are several features in biblatex that are Biber-only. When the current team took over the maintained, there was consideration of dropping BibTeX entirely. Philip and I have discussed this quite a bit, as the original biblatex developer (Philippe Lehman) picked BibTeX as the original back-end from necessity. (Biber was developed after biblatex, and for extracting and sorting data there was only originally BibTeX.) We decided against it some time ago: what the BibTeX back-end offers is stability but also speed, precisely as it’s more limited that Biber. At least for people like me, in the physical sciences and writing in western European scripts, the BibTeX back-end is perfectly usable.

The way we originally decided to allow continued Biber improvement but keep BibTeX use stable was to split the LaTeX code into two paths. That made sense with the proviso that new Biber features were essentially extensions to the code rather than any changes to existing ideas. However, over time that’s not quite worked out, particularly recently with the new data model driven approach that Philip has developed for Biber. As I’ve detailed elsewhere, that’s led to a new (breaking) syntax for \DeclareNameFormat, as well as various other changes that could be covered in BibTeX but to-date haven’t been. We’ve therefore decided we need to look again at this.

The current plan is for me to work on re-integrating the two back-end code paths, which I’m doing in a fork of biblatex as it’s non-trivial and I don’t want to mess the main development line up. I’ll also look to extend the BibTeX back-end code as appropriate such that we get back to the differences being about the differing capabilities of the back-ends rather than anything in the LaTeX code. I need a little while to do that, probably a couple of months. However, if I get it right we should be in a much stronger position for the future.

biblatex: A new syntax for \DeclareNameFormat

The ‘traditional’ BibTeX model for dividing up names is based around four parts:

  • First name(s)
  • Last name(s)
  • Prefix(es) (the ‘von part’)
  • Suffix(es) (the ‘junior part’)

This works well for many western European names, but falls down for many cases.

As part of Biber/biblatex developments, Philippe Kime has been working on moving beyond this rigid model for names to allow true flexibility. However, this comes with a caveat: a breaking change to \DeclareNameFormat in biblatex. The older syntax takes hard-wired arguments for each name part, but that obviously can’t be extended. The new format only deals with one argument (the name as a whole), but this requires changes to (non-standard) styles.

At the moment, the change is only true for Biber, which means some conditional code is needed. The best way to do that is to test for the older (BibTeX) back-end. For example, in the latest release of biblatex-chem I have in chem-acs.bbx:

% Modify the name format
\@ifpackageloaded{biblatex_legacy}
  {
    % Original syntax for BibTeX model
    \DeclareNameFormat{default}{%
      \renewcommand*{\multinamedelim}{\addsemicolon\addspace}%
      \usebibmacro{name:last-first}{#1}{#4}{#5}{#7}%
      \usebibmacro{name:andothers}%
    }

    \DeclareNameFormat{editor}{%
      \renewcommand*{\multinamedelim}{\addcomma\addspace}%
      \usebibmacro{name:last-first}{#1}{#4}{#5}{#7}%
      \usebibmacro{name:andothers}%
    }
  }
  {
   % New syntax for flexible back end
    \DeclareNameFormat{default}{%
      \renewcommand*{\multinamedelim}{\addsemicolon\addspace}%
      \nameparts{#1}%
      \usebibmacro{name:family-given}
        {\namepartfamily}
        {\namepartgiveni}
        {\namepartprefix}
        {\namepartsuffix}%
      \usebibmacro{name:andothers}%
    }

    \DeclareNameFormat{editor}{%
      \renewcommand*{\multinamedelim}{\addcomma\addspace}%
      \nameparts{#1}%
      \usebibmacro{name:family-given}
        {\namepartfamily}
        {\namepartgiveni}
        {\namepartprefix}
        {\namepartsuffix}%
      \usebibmacro{name:andothers}%
    }
  }

I’ll deal with the differences in back-ends in another post, but for the present this formulation will keep styles working for everyone.

TeXworks developments

Over recent weeks, Stefan Löffler has provided new builds for TeXworks
featuring a re-implementation of the PDF view. This offers
new modes for previewing the output, most notably continuous scrolling.
It’s also intended to be much faster than the older viewer.

Stefan’s also been setting up Travis-CI testing for TeXworks, and this
has the added benefit that he’s now able to provide Mac binaries in addition
to Windows and Linux ones. Stefan himself doesn’t have a Mac for testing
them, but Travis-CI can run automated tests. Moreover, individual users
can grab the burning edge code and use it themselves.

Making custom loaders expl3-aware

The expl3 syntax used by the developing programming layer for LaTeX3 is rather different from ‘traditional’ TeX syntax, and therefore needs to be turned on and off using the command pair \ExplSyntaxOn/\ExplSyntaxOff. In package code making use of expl3, the structure

\ExplSyntaxOn % Or implicit from \ProvidesExplPackage
....
\usepackage{somepackage}
....
\ExplSyntaxOff % Or the end of the package

will switch off expl3 syntax for the loading of somepackage and so will work whether this dependency uses expl3 or not.

This is achieved by using the LaTeX2e kernel mechanism \@pushfilename/@popfilename, which exists to deal with the status of @ but which is extended by expl3 to cover the new syntax too. However, this only applies as standard to code loaded using \usepackage (or the lower-level kernel command \@onefilewithoptions). Some bundles, most notable TikZ, provide their own loader commands for specialised files. These can be made ‘expl3-aware’ by including the necessary kernel commands

\def\myloader#1{%
  \@pushfilename
  \xdef\@currname{#1}%
  % Main loader, including \input or similar
  \@popfilename
} 

For packages which also work with formats other than LaTeX, the push and pop steps can be set up using \csname

\def\myloader#1{%
  \csname @pushfilename\endcsname
  \expandafter\xdef\csname @currname\endcsname{#1}%
  % Main loader, including \input or similar
  \csname @popfilename\endcsname
}

Of course, that will only work with LaTeX (the stack is not present in plain TeX or ConTeXt), but as the entire package idea is essentially a LaTeX one that should be a small problem.