Checking over the beamer codebase

Anyone who is watching the beamer GitHub repository will notice quite a lot of checkins, starting from the beginning beamer.cls and working forward. Most of these are ‘internal’ changes, so readers might wonder what is going on.

I’ve been involved with beamer for a while, but have not really taken a good look over the code yet. Several of the entries in the issue list are quite subtle, and it’s clear that a proper tidy-up of the code is needed to sort them out. That means addressing several internal inconsistencies, but it’s also helpful for me to get the code into a form I’m used to. So there are a mix of cosmetic changes in with some real improvements in the code.

Hopefully I’ll not introduce and new issues doing this work. However, it’s not easy to avoid changing behaviour in LaTeX, especially as beamer is somewhat delicate in the first place. So if anyway is keen to help with the work, simply grabbing the development code from time to time and making sure it doesn’t break existing presentations would be very helpful!

Beamer moves to GitHub

There are lots of places one can host development of open source code. I’ve used a few over the years, but in recent times have mainly focussed on GitHub. That’s true not least because the LaTeX3 development code is held there. The one package I’m involved in that’s to-date been elsewhere has been beamer: there are lots of issues in the tracker which I didn’t want to lose. So for some time it’s been slightly orphaned on BitBucket. I’ve now (finally) been able to migrate everything using a very handy script maintained by Jeff Widman. I’m making final arrangements on the move, but the key is that new issues should to to GitHub.

Biblatex back-end update

I wrote recently abut the need to manage biblatex back-ends, and thoughts the maintenance team had on which way to proceed.

After a bit of thought, we’ve gone for a solution we hope works for users and for us. Reversing what seemed like a ‘good idea at the time’, we’ve re-integrated (almost) all of the TeX code into one pathway. This is focussed on Biber, but we have a small stub that converts the relevant parts to work with BibTeX. So users how don’t need Biber or can’t use it can still use BibTeX and get mainly the same results.

There are a few changes, mainly related to places where BibTeX and Biber had ended up with different interfaces and where BibTeX can’t (reasonably) support the Biber one. In those places we’ve dropped the ability to use a ‘BibTeX alternative pathway’ as it makes the code complex and makes switching between back-ends tricky.

Hopefully the balance we’ve gone for will work for everyone: you can still use BibTeX, it still does almost everything it always has with biblatex, but we’ve got a more sustainable code base for the future.

biblatex feedback

Getting an idea of what users actually use can sometimes be tricky. To help understand how people are using biblatex, there’s a short survey set up. In particular, we are looking for some idea of how people use back-ends: as I’ve mentioned before, having two of them means more ‘interesting’ development so it’s important to get some insight into real life use.

Managing biblatex backends

For the past few years, biblatex has been looked after by a small team led by Philip Kime. When we first took this up we were mindful of the need to think carefully about back-ends: how reference data is extracted from a .bib file or similar sources.

There are currently two back-ends for biblatex: BibTeX and Biber. Biber is where the development is taking place and offers Unicode support, whilst BibTeX itself is frozen but also does a lot less ‘stuff’. So there are several features in biblatex that are Biber-only. When the current team took over the maintained, there was consideration of dropping BibTeX entirely. Philip and I have discussed this quite a bit, as the original biblatex developer (Philippe Lehman) picked BibTeX as the original back-end from necessity. (Biber was developed after biblatex, and for extracting and sorting data there was only originally BibTeX.) We decided against it some time ago: what the BibTeX back-end offers is stability but also speed, precisely as it’s more limited that Biber. At least for people like me, in the physical sciences and writing in western European scripts, the BibTeX back-end is perfectly usable.

The way we originally decided to allow continued Biber improvement but keep BibTeX use stable was to split the LaTeX code into two paths. That made sense with the proviso that new Biber features were essentially extensions to the code rather than any changes to existing ideas. However, over time that’s not quite worked out, particularly recently with the new data model driven approach that Philip has developed for Biber. As I’ve detailed elsewhere, that’s led to a new (breaking) syntax for \DeclareNameFormat, as well as various other changes that could be covered in BibTeX but to-date haven’t been. We’ve therefore decided we need to look again at this.

The current plan is for me to work on re-integrating the two back-end code paths, which I’m doing in a fork of biblatex as it’s non-trivial and I don’t want to mess the main development line up. I’ll also look to extend the BibTeX back-end code as appropriate such that we get back to the differences being about the differing capabilities of the back-ends rather than anything in the LaTeX code. I need a little while to do that, probably a couple of months. However, if I get it right we should be in a much stronger position for the future.

biblatex: A new syntax for \DeclareNameFormat

The ‘traditional’ BibTeX model for dividing up names is based around four parts:

  • First name(s)
  • Last name(s)
  • Prefix(es) (the ‘von part’)
  • Suffix(es) (the ‘junior part’)

This works well for many western European names, but falls down for many cases.

As part of Biber/biblatex developments, Philippe Kime has been working on moving beyond this rigid model for names to allow true flexibility. However, this comes with a caveat: a breaking change to \DeclareNameFormat in biblatex. The older syntax takes hard-wired arguments for each name part, but that obviously can’t be extended. The new format only deals with one argument (the name as a whole), but this requires changes to (non-standard) styles.

At the moment, the change is only true for Biber, which means some conditional code is needed. The best way to do that is to test for the older (BibTeX) back-end. For example, in the latest release of biblatex-chem I have in chem-acs.bbx:

% Modify the name format
\@ifpackageloaded{biblatex_legacy}
  {
    % Original syntax for BibTeX model
    \DeclareNameFormat{default}{%
      \renewcommand*{\multinamedelim}{\addsemicolon\addspace}%
      \usebibmacro{name:last-first}{#1}{#4}{#5}{#7}%
      \usebibmacro{name:andothers}%
    }

    \DeclareNameFormat{editor}{%
      \renewcommand*{\multinamedelim}{\addcomma\addspace}%
      \usebibmacro{name:last-first}{#1}{#4}{#5}{#7}%
      \usebibmacro{name:andothers}%
    }
  }
  {
   % New syntax for flexible back end
    \DeclareNameFormat{default}{%
      \renewcommand*{\multinamedelim}{\addsemicolon\addspace}%
      \nameparts{#1}%
      \usebibmacro{name:family-given}
        {\namepartfamily}
        {\namepartgiveni}
        {\namepartprefix}
        {\namepartsuffix}%
      \usebibmacro{name:andothers}%
    }

    \DeclareNameFormat{editor}{%
      \renewcommand*{\multinamedelim}{\addcomma\addspace}%
      \nameparts{#1}%
      \usebibmacro{name:family-given}
        {\namepartfamily}
        {\namepartgiveni}
        {\namepartprefix}
        {\namepartsuffix}%
      \usebibmacro{name:andothers}%
    }
  }

I’ll deal with the differences in back-ends in another post, but for the present this formulation will keep styles working for everyone.

A new maintainer for etoolbox (and csquotes)

One of the most significant new LaTeX packages of recent years has been biblatex, originally developed by Philipp Lehman and offering an extremely powerful approach to bibliographies. As I’ve covered before, Philipp Lehman vanished from the TeX world a few years ago. To keep biblatex development going, a team was assembled led by Philip Kime. However, Philipp Lehman’s other packages have up to now been left unmaintained.

The LaTeX team are currently working on some LaTeX2e improvements, and they have a knock-on effect on Philipp Lehman’s etoolbox package. To date, it’s automatically loaded etex, but the team are moving that functionality to the LaTeX kernel so it will no longer be needed. Thus we needed to sort out a minor update to etoolbox. As I’m already involved with biblatex, it seemed natural for me to take up this challenge. I’ve therefore forked etoolbox (see The LPPL: ‘maintainer’ or ‘author-maintained’ for why it’s technically a fork), set up a GitHub site and made the changes. Of course, two days after that ‘one off’ fix I got my first bug report!

Philipp Lehman’s other big contribution along with biblatex and etoolbox is csquotes. While I don’t have any immediate need to make a change there, this seems like a good time for someone to pick it up too. So I’ve set up a (technical) fork and GitHub page for that too, and expect to have a few minor changes to make (I’ve had informal discussions about at least one). Should there be a need I’ll also be looking at Philipp’s other packages (he and I had interesting discussions about logreq, for example, and how the ideas might make it into expl3).

Beamer overlays beyond the \visible

I wrote earlier this year about using the beamer overlay concept with relative slide specifications to produce dynamic slide structures. Another question about overlays came up recently on TeX StackExhange, but this time wanting to do something a bit different.

The ‘standard’ beamer overlay system does the same as the \visible command: makes things appear and disappear, but always keeps space for them on the slide. However, beamer also provides \only, which completely omits items not visible on a slide. So the question was how to combine this idea with the general overlay concept.

It turns out that this is all quite straight-forward if you know what to look for. The standard beamer overlay syntax, for example

\item<+->

extends to include an action type to specify what the overlay should do. That is given as a keyword and an @ before the overlay number(s). So for example

\begin{itemize}
  \item First item
  \item<only@1> Second item
  \item<only@2> Replacement second item
...

will show Second item on the first slide then replace it entirely with Replacement second item on the second slide. That approach can be combined with the idea of relative slide specs, as I talked about before, to give something like

\documentclass{beamer}
\begin{document}
   \begin{frame}
   \begin{itemize}[<+->]
      \item item 1
      \item item 2
      \item<only@+-.(2)> item 3
      \item item 4
      \item item 5
   \end{itemize}

   \end{frame}
\end{document}

to have the ‘normal’ items appear one at a time but with item 3 only on slides 3 and 4.

This doesn’t just apply to only: other keywords that work here include visible and alert. The latter tends to be seen with another syntax element: | to separate out appearance from a second action. A classic example of that is

\documentclass{beamer}

\begin{document}
   \begin{frame}
   \begin{itemize}[<+->]
      \item item 1
      \item item 2
      \item<+-|alert@+(1)> item 3
      \item item 4
      \item item 5
   \end{itemize}

   \end{frame}
\end{document}

where item 3 appears on the third slide and is highlighted on the fourth one. (Note that both + substitutions in this line use the same value for the pause counter, hence needing the (1) offset.) That’s useful even without the ‘one at a time’ effect, with for example

\documentclass{beamer}

\begin{document}
   \begin{frame}
   \begin{itemize}
      \item item 1
      \item item 2
      \item<alert@+(1)> item 3
      \item item 4
      \item item 5
   \end{itemize}

   \end{frame}
\end{document}

highlighting the item on the second slide.

A bit of imagination with this syntax can cover almost any appearance/disappearance/highlight requirement. As I said before: the key thing is not to overdo it!

Reworking and exposing siunitx internals

I’ve been talking for a while about working on a new major version of siunitx. I’ve got plans to add some new features which are difficult or impossible to deliver using the v2 set up, but here I want to look at perhaps what’s more important: the back end, programming set up and related matters.

I’ve now made a start on the new code, working first on what I always think of as the core of siunitx: the unit processor. If you take a look at the new material and compare it with the existing release the first thing that should be obvious is that I’ve finally made a start on splitting everything up into different sub-parts. There are at least a couple of reasons for this. First, the monolithic .dtx for v2 is simply too big to work with comfortably. More importantly, though, the package contains a lot of different ideas and some of them are quite useful beyond my own work. To ensure that these are available to other people, it would seem best to make the boundaries clear, and separate sources helps with that.

That leads onto the bigger picture change that I’m aiming for. As regular readers will know, I wrote the first version of siunitx somewhat by accident and in an ad hoc fashion. Working on v2, I decided to make things more organised and also to use expl3, which I’d not really looked at before. So the process of writing the second version was something of a learning experience. At the same time, expl3 itself has firmed up a lot over the time I’ve been working with it. As such, the current release of siunitx has rather a lot of rough edges. In the new code, I’m working from a much firmer foundation in terms of conventions, coding ideas and testing implementations. So for v3 I’m aiming to do several things. A key one for prospective expl3 programmers is the idea of defined interfaces. Rather than making everything internal, this time I’m documenting code-level access to the system. That means doing some work to have clearly defined paths for information to pass between sub-modules, but that’s overall a good thing. I’m also using the LaTeX3 teams new testing suite, l3build, to start setting up proper code tests: these are already proving handy.

The net result of the work should be a better package for end users but also extremely solid code that can be used by other people. I’m also hopeful that the ideas will be usable with little change in a ‘pure’ LaTeX3 context. Documenting how things work might even have a knock-on effect in emulating siunitx in say MathJax. Beyond that, I’ve viewed siunitx as something of a sales pitch for expl3, and providing a really top-class piece of code is an important part of that. If I can get the code level documentation and interfaces up to the standard of the user level ones, and improve the user experience at the same time, I think I’ll be doing my job there.