toolbox

Toolboxes started life often described as disposable containers – and that is still one of their major uses: install stuff, then try it out in the relative safety of a container, and lastly, cleanly dispose of it. Minimal risk, fuss and without pesky residual libraries and applications hanging around on the host long after you have finished.

So — why would you backup a Toolbox? Sometimes, they have more permanent uses, contain complex and lengthy installs, or are being used for critical applications. For example, Toolboxes can be used as a development environment, containing hardware associated drivers and applications. Or they could be used for an application you want to run in a container for which there is no Flatpak, or one that has requirements a Flatpak doesn’t satisfy. While they can be handy to use on Fedora Workstation, toolbox containers are often essential for Silverblue users since they offer an easy solution to installing applications that can’t successfully be installed by rpm-ostree. Or for applications that may not have a Flatpak version readily available. In the above situations a busted Toolbox can be a major headache. But if a backup exists, you can quickly restore a Toolbox or move it to another workstation.

The backup process uses Podman to create an image of an existing toolbox container, and save that image to an archive file. To restore the toolbox container, load the image from the archive file and then create a Toolbox from that image. The new toolbox container will be an identical copy of your backed up toolbox container.

It is important to note this process does not backup data, just what you have installed in the toolbox container. This includes packages installed from repositories or from a local rpm file using dnf. If you need to backup data, Podman’s commit command that will be used to capture an image of the toolbox container, has an option to include volumes attached to the container.

Creating a backup

To backup a toolbox container you will need it’s name and container ID which can be gotten by using toolbox list. For this example I am going to backup my golang development toolbox container, imaginatively named go.

$ toolbox list CONTAINER ID CONTAINER NAME CREATED STATUS IMAGE NAME
00ff783a102f go 5 weeks ago exited registry.fedoraproject.org/f32/fedora-toolbox:32

If the container’s status shows as running , you should stop it using podman container stop container_name. Although the commit command has a -p for pause option, make sure that the Toolbox is not running, which helps it initialize correctly when restored from backup.

$ podman container stop go

To create an image of the toolbox container use

podman container commit -p container_ID backup-image-name

Depending on the complexity of the Toolbox, this can take a little while.

 $ podman container commit -p 00ff783a102f go-backup

Now to confirm the image has been created type…

$ toolbox list

You should get output similar to what is below…

IMAGE ID IMAGE NAME CREATED
cfcb13046db7 localhost/go-backup:latest About a minute ago CONTAINER ID CONTAINER NAME CREATED STATUS IMAGE NAME
00ff783a102f go 5 weeks ago exited registry.fedoraproject.org/f32/fedora-toolbox:32

Now to save the backup image to a tar archive file using podman save -o backup-filename.tar backup-image-name.

$ podman save -o go.tar go-backup

Confirm the archive file, our toolbox container backup, was created.

$ ls go.tar

Do some tidying up, remove the backup image and, if needed, remove the original Toolbox.

$ podman rmi go-backup $ toolbox rm go

Restore a backup

To create an image from the backup file that was made above, you do it with the command podman load -i backup_filename.

$ podman load -i go.tar

Then you can confirm the image was created with…

$ toolbox list IMAGE ID IMAGE NAME CREATED
cfcb13046db7 localhost/go-backup:latest 17 minutes ago

Now create a toolbox container from the restored image, with toolbox create ––container container_name ––image image_name, specifying the full repository and version tag as the image name.

$ toolbox create --container go --image localhost/go-backup:latest

Confirm that the toolbox was created.

$ toolbox list IMAGE ID IMAGE NAME CREATED
cfcb13046db7 localhost/go-backup:latest 20 minutes ago CONTAINER ID CONTAINER NAME CREATED STATUS IMAGE NAME
34cef6b7e28d go 21 seconds ago configured localhost/go-backup:latest

Finally, you can test that the restored Toolbox works…

$ toolbox enter --container go

If you can enter the newly created toolbox container, you will see the toolbox prompt and will have successfully backed up and restored your Pet toolbox container.

Fedora Silverblue provides a useful workstation build on an immutable operating system. In “What is Silverblue?“, you learned about the benefits that an immutable OS provides. But what pieces go into making it? This article examines some of the technology that powers Silverblue.

The filesystem

Fedora Workstation users may find the idea of an immutable OS to be the most brain-melting part of Silverblue. What does that mean? Find some answers by taking a look at the filesystem.

At first glance, the layout looks pretty much the same as a regular Fedora file system. It has some differences, like making /home a symbolic link to /var/home. And you can get more answers by looking at how libostree works. libostree treats the whole tree like it’s an object, checks it into a code repository, and checks out a copy for your machine to use.

libostree

The libostree project supplies the goods for managing Silverblue’s file system. It is an upgrade system that the user can control using rpm-ostree commands.

libostree knows nothing about packages—an upgrade means replacing one complete file system with another complete file system. libostree treats the file system tree as one atomic object (an unbreakable unit). In fact, the forerunner to Silverblue was named Project Atomic.

The libostree project provides a library and set of tools. It’s an upgrade system that carries out these tasks.

Pull in a new file system
Store the new file system
Deploy the new file system

Pull in a new file system

Pulling in a new file system means copying an object (the entire file system) from a remote source to its own store. If you’ve worked with virtual machine image files, you already understand the concept of a file system object that you can copy.

Store the new file system

The libostree store has some source code control qualities—it stores many file system objects, and checks one out to be used as the root file system. libostree’s store has two parts:

a repository database at /sysroot/ostree/repo/
file systems in /sysroot/ostree/deploy/fedora/deploy/

libostree keeps track of what’s been checked in using commit IDs. Each commit ID can be found in a directory name, nested deep inside /sysroot .A libostree commit ID is a long checksum, and looks similar to a git commit ID.

$ ls -d /sysroot/ostree/deploy/fedora/deploy/*/
/sysroot/ostree/deploy/fedora/deploy/c4bf7a6339e6be97d0ca48a117a1a35c9c5e3256ae2db9e706b0147c5845fac4.0/

rpm-ostree status gives a little more information about that commit ID. The output is a little confusing; it can take a while to see this file system is Fedora 31.

$ rpm-ostree status
State: idle
AutomaticUpdates: disabled
Deployments:
● ostree://fedora:fedora/31/x86_64/silverblue Version: 31.1.9 (2019-10-23T21:44:48Z) Commit: c4bf7a6339e6be97d0ca48a117a1a35c9c5e3256ae2db9e706b0147c5845fac4 GPGSignature: Valid signature by 7D22D5867F2A4236474BF7B850CB390B3C3359C4

Deploy the new filesystem

libostree deploys a new file system by checking out the new object from its store. libostree doesn’t check out a file system by copying all the files—it uses hard links instead. If you look inside the commit ID directory, you see something that looks suspiciously like the root directory. That’s because it is the root directory. You can see these two directories are pointing to the same place by checking their inodes.

$ ls -di1 / /sysroot/ostree/deploy/fedora/deploy/*/
260102 /
260102 /sysroot/ostree/deploy/fedora/deploy/c4bf7a6339e6be97d0ca48a117a1a35c9c5e3256ae2db9e706b0147c5845fac4.0/

This is a fresh install, so there’s only one commit ID. After a system update, there will be two. If more copies of the file system are checked into libostree’s repo, more commit IDs appear here.

Upgrade process

Putting the pieces together, the update process looks like this:

libostree checks out a copy of the file system object from the repository
DNF installs packages into the copy
libostree checks in the copy as a new object
libostree checks out the copy to become the new file system
You reboot to pick up the new system files

In addition to more safety, there is more flexibility. You can do new things with libostree’s repo, like store a few different file systems and check out whichever one you feel like using.

Silverblue’s root file system

Fedora keeps its system files in all the usual Linux places, such as /boot for boot files, /etc for configuration files, and /home for user home directories. The root directory in Silverblue looks much like the root directory in traditional Fedora, but there are some differences.

The filesystem has been checked out by libostree
Some directories are now symbolic links to new locations. For example, /home is a symbolic link to /var/home
/usr is a read-only directory
There’s a new directory named /sysroot. This is libostree’s new home

Juggling file systems

You can store many file systems and switch between them. This is called rebasing, and it’s similar to git rebasing. In fact, upgrading Silverblue to the next Fedora version is not a big package install—it’s a pull from a remote repository and a rebase.

You could store three copies with three different desktops: one KDE, one GNOME, and one XFCE. Or three different OS versions: how about keeping the current version, the nightly build, and an old classic? Switching between them is a matter of rebasing to the appropriate file system object.

Rebasing is also how you upgrade from one Fedora release to the next. See “How to rebase to Fedora 32 on Silverblue” for more information.

Flatpak

The Flatpak project provides a way of installing applications like LibreOffice. Applications are pulled from remote repositories like Flathub. It’s a kind of package manager, although you won’t find the word package in the docs. Traditional Fedora variants like Fedora Workstation can also use Flatpak, but the sandboxed nature of flatpaks make it particularly good for Silverblue. This way you do not have to do the entire ostree update process every time you wish to install an application.

Flatpak is well-suited to desktop applications, but also works for command line applications. You can install the vim editor with the command flatpak install flathub org.vim.Vim and run it with flatpak run org.vim.Vim.

The toolbox project provides a traditional operating system inside a container. The idea is that you can mess with the mutable OS inside your toolbox (the Fedora container) as much as you like, and leave the immutable OS outside your toolbox untouched. You pack as many toolboxes as you want on your system, so you can keep work separated. Behind the scenes, the executable /usr/bin/toolbox is a shell script that uses podman.

A fresh install does not include a default toolbox. The toolbox create command checks the OS version (by reading /usr/lib/os-release), looks for a matching version at the Fedora container registry, and downloads the container.

$ toolbox create
Image required to create toolbox container.
Download registry.fedoraproject.org/f31/fedora-toolbox:31 (500MB)? [y/N]: y
Created container: fedora-toolbox-31
Enter with: toolbox enter

Hundreds of packages are installed inside the toolbox. The dnf command and the usual Fedora repos are set up, ready to install more. The ostree and rpm-ostree commands are not included – no immutable OS here.

Each user’s home directory is mounted on their toolbox, for storing content files outside the container.

Put the pieces together

Spend some time exploring Fedora Silverblue and it will become clear how these components fit together. Like other Fedora variants, all these of tools come from open source projects. You can get as up close and personal as you want, from reading their docs to contributing code. Or you can contribute to Silverblue itself.

Join the Fedora Silverblue conversations on discussion.fedoraproject.org or in #silverblue on Freenode IRC.

Tag: toolbox

Backup and restore Toolboxes