# Beastie: an experimental BibTeX processor

- Features
- Resources
- Release notes

Beastie is an experiment in parsing and processing BibTeX files.

Beastie is:

- A parser for `.bib` files, amongst others.
- A implementation of the `.bst` BibTeX Style Language.
- That `.bst` implementation is hosted in a dialect of Scheme, extended with suitable parsing and processing primitives.

The goals of this project are:

- to explore ways of processing `.bib` files which aren't exclusively based on `.bst`;
- to provide a second implementation of the BST language; and
- to initiate some sort of conversation about BibTeX standards, broadly construed.

The `.bst` language counts as rather niche. It has substantial amusement value, but expertise could hardly be said to be widespread (of course, one could complain that Scheme has some niche aspects, but it also has its numerous compensating virtues).

The implementation of the `.bst` language is intended to be fairly complete, in the sense that it is nearly a drop-in replacement for the `bibtex` program, and deviations from that program are bugs, where they are not noted as deliberate exceptions.

Beastie also provides a framework for writing alternative bibliography tools. The ‘experimental’ status means that it is not yet fully-featured in that respect. The fact that it implements the `.bst` language confirms its scope, and acts as some indicator of ‘correctness’, and the `examples/` directory in the distribution illustrates how one would go about writing a from-scratch style file, but there isn't (yet) a drop-in Scheme-based alternative to, for example, `plain.bst`.

It is a goal of Beastie to go slightly beyond BibTeX, and to make it a little easier to use `.bib` files to generate HTML or other formats, or for the output of beastie to be ingested into other processing pipelines. This has informed some of the choices made.

## Features

- Programmatic manipulation of BibTeX bibliographies in a high-level language. And using that...
- An implementation of the `.bst` BibTeX style language.
- `.bib`, `.bst`, `.aux`. and Markdown parser/converters.
- A few minor extensions to the `.bst` language.
- Slightly more informative error handling.
- Parsing of `.bib` files promptly expands a selection of `{\"u}`-style escapes into the corresponding characters (this makes it easier to use this tool for generating HTML and other non-LaTeX bibliographies).
- Unicode-aware throughout, in terms of case-folding and word-boundary recognition (with some qualifications).

The project has a few deliberate limitations, discussed, along with tentative future plans, in the main documentation.

Most of this is implemented in an embedded copy of s7, from ccrma.stanford.edu.

## Resources

- The Beastie home page is `https://purl.org/nxg/dist/beastie` – please use this in any long-term references to the software.

- Repository and issues list (currently): `https://codeberg.org/nxg/beastie` (note that this has changed several times in the past, most recently mid-2026, and it's not impossible it will change again in future, so use the above URL as a long-term reference to the program).

- The distribution should soon be available on CTAN.

- The main documentation contains a more detailed description of the program, including the functions exposed to the Scheme programs, and release notes. There is also a collection of examples.

- The manpage.

- There was an article about Beastie v0.12 in TUGBoat, 46:2, pp.220–227 (2025), doi:10.47397/tb/46-2/tb143gray-beastie.

The program is distributed under the terms of the 2-clause BSD licence, and includes some material with different licences. See the file `LICENCE.txt` for details.

## Release notes

v0.14.1, 2026 August 2

- Distribution fixes (problems identified by the CTAN maintainers, to whom, thanks).

v0.14.0, 2026 August 2

- Substantial reworking of the internal representation of parsed data, and its processing, with (intentionally) little visible effect at the level of the interface exposed by the programs in `examples/`. Most processing is now in terms of `ustring?` objects, rather than `string?` ones, supported by functions which are intended to translate back and forth smoothly.

- Renamed `braced?` to `bstring?` (and associated functions). Various associated internal restructurings, which mostly removed this from the public interface. This might evolve further. This was quite a substantial internal change, but the main visible change is the `ustring->ustring` function mentioned below.

- Markdown: the parser now identifies single-tab indents as indents.

- JSON: Reworking the parser, which is now more fully conformant to RFC 8259.

- Unicode: various additions to the support, and to the interface in the `unicode` module.

  - `unicode-decode/utf8` now returns a ustring?, rather than a list.
  - Added `char-wordbreak?` and `char-nbsp?`.
  - Added ustrings `#"..."`.
  - Add `ustring->ustring` for mildly adjusting the formatting of ustrings for output (specifically, controlling whether braces or `~` appear).
  - Various similar changes.

- Completed (ie, made up my mind, and documented) `user-char-command` (allowing commands such as `\LaTeX` to be recognised and expanded in `.bib` files), and `bib-string-table-set!` (with the equivalent effect to `@string` in a `.bib` file or a `MACRO` in a `.bst` file).

- Regularised option handling for consistency (specifically, calling as `repl` no longer ignores all other options, and `-e` is handled more systematically).

- Replaced the clumsy ‘untexify’ functionality with a `untexify-string-or-list`, which wraps the `ustring->ustring` function from the `'subtex` module. This function can remove braces and tilde-quotes in a principled way. This is intended to make it easier to produce non-TeX output. Correspondingly removed the `BEASTIE_UNTEXIFY_FIELDS` environment variable.

- Changed the behaviour of `printf.push$$` and `printf.pop$$`, to include writing to strings.

- Now uses s7 v11.9 (6-Jul-2026).

- `*beastie*`: added keys `'unicode-version`, `'build-platform` and `'run-platform`.

- Refactoring throughout the code, including documentation and tests.

- Added REUSE headers, throughout, to mechanically confirm the licensing.

v0.13.1, 2025 October 26

- CI/release bug: fix the `upload` stage in the CI configuration.

v0.13, 2025 October 26

- When parsing author lists, the string `and` is (now) identified as an author-separator only case-sensitively. Surprisingly, BibTeX matches this case-insensitively, which seems rather unfair on people called ‘And’ (I can find no evidence of that as an in-use surname, but if there exist people surnamed ‘Null’, there's bound to be someone surnamed ‘And’).

- There is now preliminary support for Unicode-sensitive sorting, and locales, with an optional dependency on the ICU library. This is compiled in if the ICU library is available, or suppressed with `./configure --without-icu`.

- The allowed characters for field names now include those in the set `[-_.:]` (a pragmatic rather than a particularly principled extension).

- Documentation: the repository now includes the TUGboat paper: Norman Gray, TUGboat 46:2, 228–234 (2025) `https://doi.org/10.47397/tb/46-2/tb143gray-beastie`.

- Added the `*beastie*` function, to show metadata.

- The REPL now behaves well when printing multiple values.

- The program compiles tidily as C99 with clang 17 (macOS), clang 19 (FreeBSD), and GCC 14 & 15 (Debian and RHEL), at least. This isn't a particular change, but it required some effort when the compilation was switched to mandate C99.

v0.12, 2025 August 3

- Reworked the author-name parsing, attending to some of the edge-cases mentioned at the end of btxhak (including the `~~` thing, and more systematic handling of the non-default name-part separators). I wouldn't claim that the interpretation of the document is bulletproof here, but the copy-editing of names is an area where it is hard to be dogmatic (well, to be honest, it's an area where it's probably quite easy to be ‘dogmatic’, but hard to get anyone to agree with your personal dogma). Fixes issue #7.

- The JSON reader is now more robust, and consequently now documented.

- Markdown: `<hn>` elements now contain a reference anchor formed from the element content.

- Utils: various additions (specifically `circular-list`, `intersperse`, `zip`).

v0.11.2, 2025 July 18

- Fix a minor lexer error reading .bib (in `"{foo...}"`, the opening brace was missed).

- Add `ustring<?`, but this is not a Unicode-sensitive sort order.

- The command `beastie foo.aux` does now write its output to `foo.bbl` (as opposed to stdout). I was sure it did so already, but I'd never actually got around to it!

- Added a JSON reader, but it is NOT ready for general use (it's currently included only to help support self-tests, and it behaves badly with invalid input).

v0.11.1, 2025 July 13

- Ooops: Debian/gcc build glitch

v0.11, 2025 July 13

- Released just before TUG 2025.

- Adding a number of name-parsing test cases, and expanding the parsing in response. I think the parser will now accept any string including 0, 1 or 2 commas.

- Unicode: malformed UTF-8 characters now result in a ‘replacement character’ (U+fffd), rather than being simply skipped.

- Added significantly more Unicode support, in the sense that parsing of BibTeX file strings is Unicode aware, and the expansion of ‘accented character’ commands is more consistent.

- Added `char-alpha?` and friends, which are Unicode-aware.

- Added or adjusted various Unicode manipulations and character class tests. Refactored some case-changing: the `change-case` function has been replaced by separate `{upper,lower,title}case-string/bst` functions, which respect the idiosyncratic definitions of those case-foldings specified by the bst documentation. Added `uchar-{up,down,title}case` and `ustring-{up,down,title}case`, working on characters/integers and strings, respectively, and Unicode aware (in BMP, and evading special cases). Added `uchar-alphabetic?` and `uchar-word-character?`. Et probably cetera.

- Added a `ustring?` type, to manage Unicode strings, and thence support some Unicode-aware string transformations.

- Completely replaced the `parse-subtex-string` implementation. This should now be much more complete, more principled and (soon) more extensible. This is where the resolution of issue #2 (adjusting the format of output) will come from.

- A complete rewrite of the `.bib` parser, in a way which is Unicode-first, rather than Unicode-works-by-accident.

- Exposed the `klipspringer` parser framework (which is used to implement the ‘subtex’ and ‘bibtex’ parsing).

- Improved parsing error messages (though there's probably more to do here).

- Added the `struct` macro.

- Added `delay`/`force` promises; added `every`/`any` from SRFI-1; added utility functions including `current-directory`, `split-path`, `explode-path`; also `setenv`.

- The `resolve-file/plain` function (which is the fallback implementation of `resolve-file` when the `kpselib` library is not present) now respects `TEXINPUTS` and friends, but does not attempt to implement all of the functionality of `kpselib`.

- Refactoring of regexp functions. Adding `regexp-match{,/multi}` and `regexp-split`.

- Assorted bug-fixes.

- Thanks to Karl Berry and Paulo Ney for comments, and for functionality suggestions (some of which are still in the pipeline).

v0.10, 2024 September 6

- Renamed function `resolve-file/kpse` to `resolve-file`, and improve search behaviour (this will usually use the kpse library, but will fall back to a simple-minded version if that's not available at build-time) (fixes issue #6).

- Formatting fixes and better error handling in `examples/plain.scm` (fixes issue #4).

- `regexp-match-positions*` renamed `regexp-match-positions/multi`.

- Bugfixes to `examples/extract-bib.scm` (fixes issue #6; these probably also close issue #2, but that could do with more testing).

- Various improvements parsing TeX-style accents in `.bib` databases (though there is more to do here).

- Substantial rewrite of the way that TeX-like content is parsed, and some of it expanded, from the .bib file (and adding function `parse-subtex-string`, though it should rarely be necessary for user code to call this). As part of this, the `BIBTEX_UNTEXIFY_FIELDS` support, and the wrinkle of optionally doing this only for content at brace-level-1 (like BibTeX does) has been removed (it might come back, if there's a clear need for it; fixes issue #1).

- Beastie now supports a much wider range of characters in BibTeX entry citation keys (issue #3). At present, it allows any valid Unicode (UTF-8) string that doesn't include characters syntactically important in `.bib` files (specifically, it excludes ASCII characters below and including `space` (0x20), plus 0x7f, and the characters `["%,={}]`; if you're using any of these in a citation key, then... I have absolutely no sympathy: stop it!). This currently includes such slightly arcane Unicode codepoints as separators and marks, and Beastie may decide to exclude these in a future version, or do something sophisticated with the various Unicode normal forms.

  Thanks to Karl Berry for discussion around this point, and for noting a number of the other bugs fixed in this release.

- Documentation improvements.

v0.9.1, 2024 July 13

- Changes to distribution bundle -- now installs successfully.

v0.9, 2024 July 5

- `with-fields-from-entry`: support 1-of-n being required.
- examples: adjusted extract-bib.scm to output (rough) crossref and YAML format (thanks to Karl Berry for the suggestion).
- `en-dashify` now handles "1+"; added `string->page-range`.
- Rename `entry-print` to `entry-print!` (since it's called for side-effects), and `entry-local-variable` to `entry-get-local-variable` (for symmetry).
- Added the `citation?` object.
- Adjustments to example programs, to match changes in this version.
- Reworking/expanding `examples/plain.scm`, and new example program `aux-to-refs.scm`.
- Renamed `list/true` and friends to `maybe-list`, and moved them to module `bibtex`. Added `sentence` to the same place.
- Added more SRFI-1 and SRFI-13 functions.
- Added `regexp` function, and friends.
- Changed `parse-aux-file` so that the list of citations it returns is returned as a list of symbols, rather than a list of strings.

v0.8, 2024 May 11

- Adjusted the options in the Bison grammars so Bison 3.x doesn't give deprecation warnings. Still works with Bison 2.x.
- Utils: added `string-index-right`, `split-path`, and `symbol<?`.
- Utils: `string-index` and friends now accept a string? 'delimiter'.
- Added simple regression tests to the `examples/` collection.
- Define the environment variable `POSIXLY_CORRECT`, to prevent GNU getopt permuting options.
- Added `xexpr-write/md!` to write xexprs as Markdown (very preliminary).
- The `(metadata/type metadata 'citation)` metadata from `parse-markdown/metadata` is now keyed by symbol rather than string.
- Enhancements to the `bibliography-in-markdown.scm` example program.
- Added `examples/plain.scm` as a basic general-purpose bibliography formatter program.
- Added `char-alpha?`... `char-xdigit?`, ctype functions.
- xexpr: optionally expand some entities (eg `&reg;`) in place, rather than turning them into entity references in the output.
- Bugfix: Fix initialise-runtime! so that `BEASTIE_LOAD_PATH` works again.
- Bugfix: the built-in field `crossref` must always be defined, even if it is unset in the sense of `missing$` (btxhak Sect.5.3).
- Bugfixes for xexpr output.

v0.7, 2024 March 17:

- First public release.

Norman
2026 August 02