Haiku Package Management

Discussion on Hacker News Discussion on lobste.rs

I’ve long since been a die-hard BeOS fan and have been running the open-source recreation Haiku for many years. I think it’s interesting to explore the “alternative OS” world and consider some great ideas that for whatever reason never caught on elsewhere.

The way Haiku handles package management and its alternative approach to an “immutable system” is one of those ideas I find really cool. Here’s what it looks like from a desktop user’s perspective - there’s all the usual stuff like an “app store”, package updater, repositories of packages and so on:

It’s all there and works well - it’s easily as smooth as any desktop Linux experience. However, it’s the implementation details behind the scenes that make it so interesting to me. Haiku takes a refreshingly new approach to package management: Despite the user experience “feeling” like a traditional package manager - say, something like apt or dnf - it has seamless support for:

• Immutable system directories
• Rollback to previous states
• User-managed packages separated from system packages

And a whole bunch of infrastructure and tooling to support multiple package repositories, building packages from source and more. As a geek, I find it beautifully elegant and an idea that I’d love to see other platforms exploring.

Let’s take a closer look, starting with how Haiku handles the split between “system” and “user” directories.

Filesystem layout

Running a df command from the Haiku shell (BASH is the default, but others like ZSH can be easily installed) shows the following:

~> df -h
 Mount             Type      Total     Free      Flags   Device
----------------- --------- --------- --------- ------- ------------------------
/boot             bfs       232.9 GiB 219.6 GiB QAM-P-W /dev/disk/scsi/0/0/0/0
/boot/system      packagefs   4.0 KiB   4.0 KiB QAM-P--
/boot/home/config packagefs   4.0 KiB   4.0 KiB QAM-P--

In Haiku, the root / is a ram-based virtual filesystem set up by the kernel when it’s booted. All other filesystems and devices are mounted under /, so /boot refers to the entire boot volume - and not a boot partition as is the case with e.g. most Linux installs.

The /boot/system mountpoint is essentially the system directory which makes up the Haiku OS and installed applications. In the root directory, there are a bunch of symlinks that point there for convenience:

~> ls -l /
total 6
lrwxrwxrwx 1 user root   16 Feb 13 14:18 bin -> /boot/system/bin
drwxr-xr-x 1 user root 2048 Mar 31  2021 boot
drwxr-xr-x 1 user root    0 Feb 13 14:18 dev
lrwxrwxrwx 1 user root   25 Feb 13 14:18 etc -> /boot/system/settings/etc
lrwxrwxrwx 1 user root    5 Feb 13 14:18 Haiku -> /boot
lrwxrwxrwx 1 user root   26 Feb 13 14:18 packages -> /boot/system/package-links
lrwxrwxrwx 1 user root   12 Feb 13 14:18 system -> /boot/system
lrwxrwxrwx 1 user root   22 Feb 13 14:18 tmp -> /boot/system/cache/tmp
lrwxrwxrwx 1 user root   16 Feb 13 14:18 var -> /boot/system/var

So we can access e.g. /boot/system as /system and so on. Note that this mount point is backed by packagefs - this is provided by a virtual filesystem that presents a merged view of all the packages installed. It’s sort of like an overlay filesystem (commonly used by container runtimes) in the Linux world.

This mount-point is read only:

~> touch /system/test
touch: cannot touch '/system/test': Read-only file system

But we can however write to anything in our home directory, which ensures a clean separation of system and user data/configuration.

NOTE : Due to its BeOS ancestry, Haiku is not currently a multi-user system so there is only one /boot/home directory and the user is effectively the sole administrator account. As I understand it, the scaffolding is present to support multiple users, but it won’t be a priority until after R1 is released. However, this opens up the later possibility for packages to be installed and configured on a per-user basis.

The Package Daemon and packagefs

There’s a great overview of how all these components fit together in the developer documentation, but here’s my lay-persons understanding of it…

Haiku has many packages available, ranging from system components, development libraries and end-user applications. There’s even recent ports of Wine, LibreOffice and KDE Applications. These are available as .hpkg files which are placed in a special packages directory, and the packagefs service mounts the contents into e.g. the /boot/system mountpoint.

Unlike traditional package management tools, the contents of the package are not unarchived and copied into the filesystem; When you’re looking at /boot/system, you’re essentially looking at a collection of multiple packages and their contents, all dynamically mounted and made available on-the-fly as a read-only, virtual filesystem.

This is why /boot/system is read-only: It’s not a “real” filesystem and only exists as a virtual union of all the different packages that are activated at that point in time.

NOTE : There are some exceptions to this, as some directories such as packages, cache, var etc. are writeable. There are called “shine-through directories” which reside on the underlying BFS volume. You can read more about these in the developer documentation. And while we’re geeking out, BeFS itself is definitely also worth investigating!

When packagefs detects a new .hpkg file has been copied to the packages directory, it performs some checks such as searching for dependencies or conflicts. If everything is OK it “activates” the package, and the contents are then available.

Installing a package

Here’s an example of installing a package using the CLI pkgman tool, showing what happens behind the scenes. First, let’s install an example package, in this case something simple like the awesome CLI Pipe Viewer tool. It’s very much like running apt-get, yum or other similar tools on a Linux system:

~> pkgman install pv
Validating checksum for Haiku...done.
100% repochecksum-1 [64 bytes]
Validating checksum for HaikuPorts...done.
The following changes will be made:
  in system:
    install package pv-1.6.6-2 from repository HaikuPorts
100% pv-1.6.6-2-x86_64.hpkg [40.57 KiB]
Validating checksum for https://eu.hpkg.haiku-os.org/haikuports/master/x86_64/current/packages/pv-1.6.6-2-x86_64.hpkg...done.
[system] Applying changes ...
[system] Changes applied. Old activation state backed up in "state_2023-02-13_14:33:27"
[system] Cleaning up ...
[system] Done.

If we now look at /system/packages we can see the downloaded .hpkg file (which can also be inspected with CLI or Desktop tools):

~> ls -l /system/packages/pv-1.6.6-2-x86_64.hpkg
-rw-r--r-- 1 user root 41543 Feb 13 14:33 /system/packages/pv-1.6.6-2-x86_64.hpkg

Sure enough, the package daemon picked it up and mounted its contents so they are available through the merged packagefs under /boot/system. Here’s where the pv binary now appears installed:

~> ls -l /system/bin/pv
-r-xr-xr-x 1 user root 70136 Aug  6  2018 /system/bin/pv

Uninstalling is as simple as pkgman uninstall pv which removes the .hpkg file, and the contents then “disappear” from /boot/system.

State and Rollback

What’s really nice about this, is that it enables a simple and elegant way of rolling-back your system state to a previous set of packages or even OS releases. If you check the last output from the pkgman install pv command above, you’ll see there’s a message saying that an “old activation state” is being backed up to a newly-created directory. The package manager does this at the start of every transaction, and if we check that directory we’ll see a simple plain-text file called activated-packages:

~> head -n5 /system/packages/administrative/state_2023-02-13_14\:33\:27/activated-packages

netcat-1.10-4-x86_64.hpkg
ncurses6-6.3-2-x86_64.hpkg
mpfr3-3.1.6-6-x86_64.hpkg
mesa_swpipe-22.0.5-2-x86_64.hpkg
mesa_devel-22.0.5-2-x86_64.hpkg

This contains a record of the exact package versions that were installed at that time. If any packages were to be upgraded, then the state directory will also contain a set of packages to facilitate roll-back. Here’s what it looks like from the Haiku Desktop:

Along with a listing of old packages in a state directory, the currently installed packages are shown on the top left. You can see for example, that BASH in the current system is at 5.1 but in an old backup state from 2021 it was 5.0. You can also clean these old state directories up according to your needs - such as only keeping the last 30 days of state to preserve disk space.

Tying this all together, the Haiku Boot Loader can make use of activation lists and saved packages to get back to any particular state very easily - all it has to do is pick a backup package activation file, and only activate the packages found in it when booting. You can select a backup state from the “Select Boot Volume” option in the boot loader and your system will boot back to the previous state using the archived packages and activation list. Your virtual /boot/system directory will then be reverted to the desired state - and this all happens on-the-fly at boot time; there is no long roll-back process or destructive operations and you can reboot into different states at will.

User packages

So far, I’ve just been talking about the packagefs mountpoint at /boot/system, but there’s also another mountpoint shown in my first df -h command which lives under /boot/home/config. Here’s what exists under that directory:

~/config> tree -d /boot/home/config -L 1
/boot/home/config
├── apps
├── cache
├── data
├── non-packaged
├── packages
├── settings
└── var

This is more-or-less a copy of the system directories, but they are specific to my user account. This means I can use this location to install software and test new packages out without “polluting” the system directories. And when Haiku gets full multi-user support this also means each user can have their own distinct set of packages available.

A couple of points of interest as an aside:

• For non-native software packages (e.g. something installed with ./configure && make install) you can use ~/config/non-packaged. This is somewhat similar to /usr/local on Unix systems.

• The settings directory is where Haiku-packaged software keeps local configuration. To use an analogy again, it’s sort of like $XDG_CONFIG_HOME on other Unix-like systems. Haiku software tends to make use of this directory: For example, instead of SSH keeping configuration under ~/.ssh, it’s now stored under a directory in ~/config/settings:

~/config> ls -l settings/ssh/
total 32
-rw------- 1 user root 1238 Mar 31  2021 authorized_keys
-rw------- 1 user root  476 Mar 31  2021 config
-rw------- 1 user root 1679 Mar 31  2021 id_rsa
-rw------- 1 user root  410 Mar 31  2021 id_rsa.pub
-rw------- 1 user root 1790 Dec 18  2021 known_hosts

Building your own packages

I’ll finish off by showing an example of building a user package and installing it using Haikuporter and the community HaikuPorts collection. These tools can be used to build and customise a massive amount of software for Haiku ranging from old BeOS tools to a complete LibreOffice installation.

NOTE : Think of building HaikuPorts from source like the FreeBSD “ports” collection. All the HaikuPorts packages are available through their repository as binaries through pkgman or the HaikuDepot graphical desktop application. As an end-user, you typically do not need to use Haikuporter unless you want to customise a package, create a new one, or submit patches/bug-fixes. I’m just using it as an example and because it’s cool!

After setting up Haiku Ports and Haikuporter by following the instructions, I can now build a package from source. I’ll use the GUI FTP client “FTP Positive” as an example:

~/haikuports/haiku-apps> alias hp="haikuporter -S -j8 --no-source-packages --get-dependencies"
~/haikuports/haiku-apps> hp ftppositive

Checking if any dependency-infos need to be updated ...
Looking for stale dependency-infos ...
----------------------------------------------------------------------
haiku-apps::ftppositive-1.2.2
        /boot/home/haikuports/haiku-apps/ftppositive/ftppositive-1.2.2.recipe
----------------------------------------------------------------------
Skipping download of source for 48a5acdfe0981697018abf151a82802f4f3e500e.tar.gz
Validating checksum of 48a5acdfe0981697018abf151a82802f4f3e500e.tar.gz
Unpacking source of 48a5acdfe0981697018abf151a82802f4f3e500e.tar.gz
...
... Build output truncated
...
mimesetting files for package ftppositive-1.2.2-7-x86_64.hpkg ...
creating package ftppositive-1.2.2-7-x86_64.hpkg ...
----- Package Info ----------------
header size:                     80
heap size:                   211523
TOC size:                      1110
package attributes size:        695
total size:                  211603
-----------------------------------
waiting for build package ftppositive-1.2.2-7 to be deactivated
grabbing ftppositive-1.2.2-7-x86_64.hpkg and moving it to /boot/home/haikuports/packages/ftppositive-1.2.2-7-x86_64.hpkg

The last lines of output show me that a .hpkg file has been created. I can then simply copy the package to my local ~/config/packages directory to “activate” it. Once again, packagefs will check it and then add the contents to the /boot/home/config directory. Here’s what it looks like on the Haiku desktop:

Note the top two windows “stuck” together using the built-in stacking and tiling window manager!

You can see the package has been activated after copying it into the user’s packages directory. It’s installed the application to /boot/home/config/apps/FtpPositive/ instead of the system directories and has added a DeskBar menu entry which has also been picked up and “merged” into the global system Applications menu - where it appears alongside all the system-installed packages.

Conclusion

Like the rest of Haiku, package management is a refreshingly different and well thought-out experience. Given how niche an OS it is, it still surprises me how polished the system is - I have it running natively on an old Lenovo ThinkStation SFF PC and it absolutely flies. Apart from my Amigas it’s easily my favourite “hacking on code in the evening” system and for many tasks is perfectly capable of being a daily-driver.

There’s a fascinating world of alternative OSes out there, many of them following entirely different paradigms than the current mainstream world of Windows/Mac/Linux. I’ve had the good fortune to be exposed to these different ways of thinking all the way back to my school days in the UK, where RISC OS was commonplace in the classroom and Amigas and Ataris ruled the playground.

From this once-rich polyculture of competing processor architectures and platforms, the world seemingly consolidated itself around an x86 or ARM processor running something based on Windows or Unix. Which gets the job done, but it’s y’know… a little boring. There are so many interesting - and some downright crazy - ideas that either failed in the commercial marketplace, were never given a proper chance (yeah, I’m still salty about Commodore killing the Amiga) or only found a hobbyist following.

But here’s the thing: Just because they never got wide adoption doesn’t mean they were wrong. I think it’s great that people are exploring new ideas, continuing the lineage of legacy systems, and simply creating stuff because they damn well feel like it and it’s fun! There’s lots of really interesting code out there and it makes you wonder what an alternate time-line would look like where Apple did use BeOS as the basis for their next operating system. What if VMS had “won” over UNIX? What if Commodore had known what to do with the Amiga ?

Anyway, I hope this has whetted your appetite for all things Alt-OS, and Haiku in particular. If you haven’t yet tried it, there’s detailed instructions on their website, and I highly recommend taking it for a spin.

Happy Hacking!