Omegacrash

One of my new favorite tools in Debian is schroot. schroot makes it easy to maintain and use chroot environments. If you are not familiar with chroot environments, they are essentially a complete system installed in a directory which you can switch to using the chroot(2) system call. You can think of this as a basic form of operating system virtualization.

schroot allows you enter a chroot without requiring root credentials, automatically mounts important file systems inside the chroot, and allows you to create sessions from a chroot that function like an independent chroot that disappears when you are finished with it, leaving the source chroot in its original condition. You can install software, make configuration changes, create new users, run daemons, etc. and all of that will disappear when you terminate the session.

Creating a chroot With debootstrap

To use schroot, you first need a chroot environment to use it with. The easy way to do this in Debian is with the debootstrap program, available in the package of the same name. debootstrap downloads and installs packages into a new directory to create an initial install that can be used as a chroot environment:

# debootstrap --include=locales,less,sudo,ncurses-term,vim sid sid-chroot

This command installs a Debian Sid system in the directory sid-chroot, and includes a few additional packages. locales and less are probably good things to include in general, the rest are specific to how I work, and you may not need them.

It's possible to switch immediately to this chroot with the chroot command, but that's not as interesting or as useful as what we can do with schroot.

Basic schroot Configuration and Use

To get started, we need an entry for our chroot in the schroot configuration file, /etc/schroot/schroot.conf:

[sid-basic]
description=Sid sample chroot
type=directory
directory=/home/kevin/test/chroots/sid-chroot
union-type=aufs
users=kevin
source-users=kevin

Note that kevin is my username and the directory is set to the full path to the chroot environment created with debootstrap. Replace these with something appropriate.

With this configuration we can use schroot to enter the chroot environment:

$ schroot -c sid-basic

There are a few things to note about this:

1. Users specified in the users field of the configuration file don't need to provide credentials to enter the chroot environment.
2. This type of chroot is "session managed", meaning that entering it creates a new session that exists until you exit the chroot. Changes made to the system within the session disappear when the session ends. More on this later.
3. While some changes in this chroot are transient, schroot mounts parts of the host file system inside the chroot environment. Changes here will affect the host file system, so be careful.

To get a better understanding of what is happening, exit the chroot environment (just use the exit command to end the shell process and return to the previous shell). Next, we'll start a named session. You can enter and leave the session as needed, and the session will persist until it is explicitly ended. The commands to create and enter the session follow:

$ schroot --begin-session --session-name=test -c sid-basic
$ schroot --run-session -c test

Now, from a second terminal, run mount on the host system. In addition to the normal mounts you typically have on your system, you'll see several associated with this schroot session:

/home/kevin/test/chroots/sid-chroot on /var/lib/schroot/union/underlay/test type none (rw,bind)
test on /var/lib/schroot/mount/test type aufs (rw,br:/var/lib/schroot/union/overlay/test:/var/lib/schroot/union/underlay/test=ro)
/proc on /var/lib/schroot/mount/test/proc type none (rw,bind)
/sys on /var/lib/schroot/mount/test/sys type none (rw,bind)
/dev on /var/lib/schroot/mount/test/dev type none (rw,bind)
/home on /var/lib/schroot/mount/test/home type none (rw,bind)
/tmp on /var/lib/schroot/mount/test/tmp type none (rw,bind)

The first two mounts create a union file system for the root file system of the chroot. The "underlay" portion is the base chroot directory, and it is mounted as a read-only layer. The "overlay" on top of that is writable, but is deleted when the session ends.

The remaining mounts add common file systems to the chroot environment—/proc, /sys, /dev, /home, and /tmp. Mounting /home is convenient, but has some undesirable consequences. The first is that it makes experimenting in the chroot environment rather dangerous, since you can destroy your files. The second is that creating new users in the chroot environment will create home directories that persist after the session is ended. I'll discuss the first concern later. To deal with the second concern while still having the convenience of being able to access my files, I like to mount /home/kevin in the chroot instead of mounting all of /home. The next section explains how to further customize a chroot to accomplish things like this.

When you are finished with a named session, you can end it with a command like this:

$ schroot --end-session -c test

Custom schroot Configurations

schroot has a stock set of configuration types located in directories in the /etc/schroot directory: default, desktop, minimal, and sbuild. You can create your own configurations based on these. To start with, copy the default directory to a new directory called custom. Edit the file custom/config and change the paths to refer to the files in the custom directory:

# Filesystems to mount inside the chroot.
FSTAB="/etc/schroot/custom/fstab"

# Files to copy from the host system into the chroot.
COPYFILES="/etc/schroot/custom/copyfiles"

# System NSS databases to copy into the chroot.
NSSDATABASES="/etc/schroot/custom/nssdatabases"

Next, edit the custom/copyfiles file. This is a list of files that are copied from the host system to the chroot system each time a new session is started. By default this includes only /etc/resolv.conf and /etc/gshadow. I like to add /etc/sudoers since I use sudo rather than the root account. I also prefer to remove /etc/resolv.conf, because I think a static file in the chroot is more useful. For example, if I start a session on my laptop while connected to some coffee shop wifi network, the DNS servers provided may not be useful on any other network.

Now edit the fstab file. The main change I typically make here is to replace the /home mount with a /home/kevin mount. The first two fields should both be changed. This allows access to my files inside the chroot while also allowing me to create new users with their own home directories which will not persist beyond the end of the session. This wouldn't work so well if the chroot was configured for use by multiple users.

I'm skipping the nssdatabases file because I don't typically find any reason to make changes there. You may want to, however.

To make use of the custom configuration, make sure you've ended any schroot sessions and edit the entry in /etc/schroot/schroot.conf adding this line:

script-config=custom/config

Managing Source Chroots

Chroots with a union-type setting fall into a category known as source chroots. Source chroots are used as sessions, where changes made within the session only last as long as the session is active. In other words, they have a source which is used to generate the session, but which is not modified by using the session.

Source chroots provide an additional chroot with a name constructed by appending -source to the end of the original name, and which is accessible by users listed in the source-users setting. Changes made in this chroot are persistent, affecting all future sessions.

As a simple example of modifying a source chroot, lets look at a problem you are likely to encounter using chroots. If you have a terminal that uses UTF-8 (as many do today), you might have LANG=en_US.UTF-8 (for example) in your environment. In a new chroot, this will produce some problems. When running man, you may see this error:

man: can't set the locale; make sure $LC_* and $LANG are correct

Many other commands will produce similar errors. This is because the system locales are not configured. This is easy to fix, but if you fix it in a session you'll just need to fix it again in the next session. Instead, go to the chroot's source and make the changes there:

$ schroot -c sid-basic-source

Edit /etc/locale.gen, uncomment the locales you might actually use, save and quit, then run locale-gen. Now you can leave the chroot and the changes will persist into your future sessions.

By using the chroot's source you can install software you always want to have available in your sessions, update installed packages, etc.

Another Custom Configuration

One advantage of working in a chroot environment is that you can protect files on the host system from accidental modification or deletion. However, as mentioned earlier, the configurations we've looked at so far mount users' home directories inside the chroot, negating some of these benefits. This is a matter of convenience versus security, and the right way to go depends on what you are trying to accomplish. By using a second schroot configuration, we can support both arrangements. This new configuration will use a home folder that exists inside the chroot rather than on the host system, and furthermore will allow running X applications using the host's display.

To begin with, in /etc/schroot/ copy the directory desktop to a new directory named nohome. Starting with the desktop configuration makes it easier to get X applications working. The changes that need to be made to the new configuration are similar to what was done previously. Fix up the paths in nohome/config and make any desired changes in nohome/copyfiles and nohome/nssdatabases. In nohome/fstab, delete the /home line. If you use gdm3, uncomment the line to mount /var/run/gdm3. Sid seems to put this in a different location, so I also had to change the mount point to /run/gdm3.

Now in /etc/schroot/schroot.conf add a section to create a new chroot using this configuration:

[sid-nohome]
description=Sid with /home not mounted
type=directory
directory=/home/kevin/test/chroots/sid-chroot
union-type=aufs
users=kevin
source-users=kevin
script-config=nohome/config
preserve-environment=true

This is just like the sid-basic chroot, but using the nohome configuration and including a new setting, preserve-environment=true. This setting imports environment variables from the host to the chroot, including things like DISPLAY and XAUTHORITY that allow X applications to access the display.

If you don't use gdm3, you will probably need an alternative strategy to get your Xauthority file into the chroot environment. One option that seems to work is to add /home/<user>/.Xauthority to /etc/schroot/nohome/copyfiles.

Once this is set up, you will probably want to enter the source chroot and create your home directory there. You may want to add some base files that will be available in any session using the chroot, such as shell initialization files.

I was recently given a form to fill out that was in MS Word format. As a Debian user I of course used OpenOffice.org to open it (Debian's stable release doesn't have LibreOffice).

While I don't expect Word files to import perfectly into OpenOffice, I was flummoxed by the way the form fields were imported. Instead of editable fields, they were grayed-out sequences of space characters. These could be edited, but not very conveniently. I also couldn't find any way to convert them to the normal OpenOffice input fields. There was no indication that the application had any way of identifying or changing the field type.

Since OpenDocument files are just zip files with XML to define the document structure, it wasn't too hard to find the XML markup for the problem fields. The fields were identified with the tags <field:fieldmark-start/> and <field:fieldmark-end/>, and a few minutes on Google suggested that these are an extension specifically for MS Word compatibility.

Fortunately I found a (sort of) convenient way to convert these fields. First, I discovered that OpenOffice can save in a format that I didn't know about before: a flat XML version of OpenDocument. This makes editing or just viewing the XML much more convenient. It even produces nicely indented XML rather than the one huge line of text you get if you unzip an ODF file.

After converting to this format, all I had to do was open it in Vim and search for all the instances of the fieldmark tags, replacing them with the OpenDocument text-input tag, <text:text-input text:description=""/>. The description text corresponds to the field name, and the element text can be used to provide an initial value for the field.

The resulting file can be opened in OpenOffice and saved as a normal ODF. This could be a very handy tool for sticky OpenOffice/LibreOffice formatting issues, something that isn't too far removed from everyone's favorite feature from WordPerfect, Reveal Codes.

My last few posts were about the solutions I've been writing for the problems from Project Euler. There are a lot of problems, and commentary on the individual solutions aren't all that interesting.

Therefore, instead of posting solutions here I've started pushing them to a GitHub repository. At the time of writing, I've solved the first 45 problems.

A few of the problems offered opportunities to use dynamic programming, which is a technique I haven't encountered much since leaving college. I was glad to have a chance to use it again, and refresh my memory. The result tends to be a very fast solution.

Some problems require very large integers, and in some cases I'm using the long long type which isn't officially part of C++. However, even this isn't big enough for some problems. In a few cases I "faked" arbitrary precision integers, then at problem 20 I started to implement a separate BigInt class, but only having the parts I needed for that problem. I quickly decided that it would be more useful to use a common arbitrary precision library, and switched to using GMP for later problems that required very large integers. Reinventing the wheel isn't really the point of the problems, after all.

I eventually added a utility module for things that I've been using over and over again, such as a primality testing function. There are still many copies of this function in the earlier problems because I didn't bother to update them. Once I have the solution, I consider the program complete.

Project Euler, Problems 6 through 10

Project Euler, Problems 1 Through 5

Diagnosable Software Failures

The Trouble With Terminals