LXC for running problematic apps

A friend recommended Dwarf Fortress as a quality real-time strategy game with a good community and active development. Sadly, I found it hard to run the game since it's distributed as 32-bit binaries and like most games has many library dependencies. Gentoo distributes pre-baked 32-bit libs for these cases but they didn't work for me. I then launched a 32-bit Ubuntu VM. The game worked this way, but it's unsatisfying to play through an extra window, with emulation latency, and the audio broke up.

Since brushing up on LXC 1.0, I realized that this could be the perfect solution for Dwarf Fortress. I'm all set up for unprivileged containers, so I pulled up Graber's article on how to run X apps in a container. I was successful after encountering several problems, so here's how I did it.

The Plan

Graber's technique for trasferring host-unprivileged actions (e.g. normal user can run X on host) to a container-unprivileged user can be summarized:

Bind-mount any required special files/devices into the container.
Thoughtfully craft your user ID map so that the container-unprivileged uid/gid map directly to the host-unprivileged uid/gid

So, these went into my container config, taken directly from Graber's article (except /dev/video0 which I don't have and apparently don't need):

lxc.mount.entry = /dev/dri dev/dri none bind,optional,create=dir
lxc.mount.entry = /dev/snd dev/snd none bind,optional,create=dir
lxc.mount.entry = /tmp/.X11-unix tmp/.X11-unix none bind,optional,create=dir

And also this user mapping (my host-unprivileged user is 1000):

lxc.id_map = u 0 100000 1000
lxc.id_map = g 0 100000 1000
lxc.id_map = u 1000 1000 1
lxc.id_map = g 1000 1000 1
lxc.id_map = u 1001 101001 64535
lxc.id_map = g 1001 101001 64535

The effect of this is that uid/gid 1000 on the guest has all the same privileges as uid/gid 1000 on the host in terms of bind-mounted resources. There is an unexpected consequence of this mapping: user 'erik' on the guest will be able to interact with the bind-mounted resources but not user 'root' on the guest. This strange inversion of privileges applies only to those bind mounts (X socket, sound device); guest-root still trumps guest-'erik' when dealing with all other permissions in the container.

Building the system

I can now create the container. It is configured to bridge to br0.

lxc-create -f dwarf.conf.init -t download -n dwarf -- -d gentoo -r current -a i386

I start the container. It's necessary to fix up the networking manually at least the first time since the Gentoo download image doesn't DHCP.

lxc-start -n dwarf -d
lxc-attach -n dwarf ip addr add 172.21.117.80/24 dev eth0
# also add default route and fix /etc/resolv.conf
lxc-attach -n dwarf wget google.com # Prove networking success

I now enter the container through SSH instead of lxc-attach, since some things don't quite work correctly under lxc-attach (no hostname, ~ isn't expanded correctly).

Prove X

I installed xclock, so I could isolate the "use X in a container" problem from the "satisfy binary linker dependencies" problem. I got an error about "no protocol specified", and the search engines told me that I needed the xauth program. I installed xauth and imported the cookie from the host environment. On the host:

msi erik # xauth list $DISPLAY 
msi/unix:0  MIT-MAGIC-COOKIE-1  029beac9a3cecde93ffbf0007ea7a4b6

And in the guest:

erik@dwarf ~ $ export DISPLAY=:0
# Note I omitted the hostname before :0
erik@dwarf ~ $ xauth add :0  MIT-MAGIC-COOKIE-1  029beac9a3cecde93ffbf0007ea7a4b6
erik@dwarf ~ $ xclock

And behold, a big clock appears!

Summoning dwarves

In the guest, I download the binary tarball for the game and extract it. It won't run, of course, until dependencies are satisfied.

dwarf portage # ldd /usr/src/df_linux/libs/Dwarf_Fortress 
        linux-gate.so.1 (0xf7732000)
        libSDL-1.2.so.0 => not found
        libgraphics.so => /usr/src/df_linux/libs/libgraphics.so (0xf7310000)
        libstdc++.so.6 => /usr/src/df_linux/libs/libstdc++.so.6 (0xf7233000)
        libm.so.6 => /lib/libm.so.6 (0xf71ee000)
        libgcc_s.so.1 => /usr/src/df_linux/libs/libgcc_s.so.1 (0xf71d2000)
        libc.so.6 => /lib/libc.so.6 (0xf702c000)
        libpthread.so.0 => /lib/libpthread.so.0 (0xf7011000)
        libgtk-x11-2.0.so.0 => not found
        libgobject-2.0.so.0 => /usr/lib/libgobject-2.0.so.0 (0xf6fbf000)
        /lib/ld-linux.so.2 (0xf7733000)
        libSDL-1.2.so.0 => not found
        libSDL_image-1.2.so.0 => not found
        libGLU.so.1 => not found
        libSDL_ttf-2.0.so.0 => not found
        libglib-2.0.so.0 => /usr/lib/libglib-2.0.so.0 (0xf6e83000)
        libffi.so.6 => /usr/lib/libffi.so.6 (0xf6e7c000)

It should be possible to iteratively identify and install missing packages until "not found" is "not found" in the ldd output. However, I took a shortcut by consulting an ebuild in the sunrise overlay.

less /var/lib/layman/sunrise/games-roguelike/dwarf-fortress/dwarf-fortress-0.34.05.ebuild
...

media-libs/fmod:1
media-libs/freetype
media-libs/libsdl[opengl,video,X]
media-libs/libsndfile[alsa]
media-libs/openal
media-libs/sdl-image[png,tiff,jpeg]
media-libs/sdl-ttf
sys-libs/zlib
x11-libs/cairo[xcb,X]
x11-libs/gtk+:2[xinerama]
x11-libs/libXcomposite
x11-libs/libXcursor
x11-libs/pango[X]

I tweaked some package.use settings until they installed. There was also a dependency failure that required me to manually install dev-perl/XML-Parser. At this point dynamic linking looks clean.

The chicken exit

At some point in here I learned that Dwarf Fortress can run under curses

emacs data/init/init.txt
...
Linux/OS X users may also use PRINT_MODE:TEXT for primitive ncurses output.

[PRINT_MODE:TEXT]

... but the output goes to a very small fixed area, and that's not what we're after here.

Let's try running the game:

su erik
export DISPLAY=:0
./df

...it fails because there a couple of things left to debug. One error was about being unable to load a png file, and a search engine turned up a solution which involved symlinking *.png to *.bmp. It seems that the game needs an older libpng, but it still didn't work when I tried that solution so I just went with the symlink hack.

And it worked!

Extra credit: re-using portage

So, all this took a few attempts, especially since tweaking the user map requires re-creating the container. I ran out of space on my notebook, and realized that it would be nice if the "download" template could re-use the host's portage and distfiles just like the privileged "gentoo" template does. This saves about 850M per container.

I started by deleting the contents of /usr/portage in the container. Then I restarted the container with a new bind-mount:

lxc.mount.entry = /usr/portage usr/portage none bind,optional,create=dir

A little extra user-map magic was required, since the container boosted privileges for it's 'erik' user but now I wanted the guest's 'root' user to also have privileges to write into the new bind-mount. There are probably multiple ways to achieve this, but my little brain only came up with this:

root on the guest is mapped to 100000.
This uid should have rw access to /usr/portage on the host.
The existing 'portage' group (gid 250) has rw access /to /usr/portage on the host
The guest 'portage' group should have the same gid as the host 'portage' group.

So, I created a new user on the host named 'dwarfroot' with the desired uid and added it to the 'portage' group. Also, a hole was punched in the subgid map for the guest 'portage' group, just like we did earlier with the 'erik' user in the subuid map.

/etc/subuid and /etc/subgid:

erik:1000:65536
erik:100000:65536
dwarfroot:165536:65536

And in the container config file, the map has two special-case one-id ranges for the 'portage' and 'erik' uid/gid's:

lxc.id_map = u 0 100000 250
lxc.id_map = g 0 100000 250
lxc.id_map = u 250 250 1
lxc.id_map = g 250 250 1
lxc.id_map = u 251 100251 749
lxc.id_map = g 251 100251 749
lxc.id_map = u 1000 1000 1
lxc.id_map = g 1000 1000 1
lxc.id_map = u 1001 101001 64535
lxc.id_map = g 1001 101001 64535

After restarting the container, the guest 'root' user can write files into the bind-mounted /usr/portage/distfiles as needed.

Audio

Audio isn't working, and I believe that's again a permissions issue. The files in /dev/snd/* are writable by the host's group 'audio' (gid 18) which isn't mapped. By now, this seemed like it should be simple. I punched a new hole in the gid map so that host gid 18 remained guest gid 18:

lxc.id_map = u 0 100000 250
lxc.id_map = g 0 100000 18
lxc.id_map = g 18 18 1
lxc.id_map = g 19 100019 231
lxc.id_map = u 250 250 1
lxc.id_map = g 250 250 1
lxc.id_map = u 251 100251 749
lxc.id_map = g 251 100251 749
lxc.id_map = u 1000 1000 1
lxc.id_map = g 1000 1000 1
lxc.id_map = u 1001 101001 64535
lxc.id_map = g 1001 101001 64535

I repeatedly got "newgidmap: write to gid_map failed: Invalid argument" when starting the container. Debugging, I noticed that commenting out the final lines of the map allowed the container to start. Finally I found this gem in "man user_namespaces":

There is an (arbitrary) limit on the number of lines in the
file.  As at Linux 3.18, the limit is five lines.

Gah! Too many 'g' entries! Let me collapse the system entries, at the risk of giving the guest slightly too many permissions on the files in /dev. I must also adjust /etc/subgid accordingly.

lxc.id_map = u 0 100000 250
lxc.id_map = g 0 100000 18
lxc.id_map = g 18 18 232
lxc.id_map = u 250 250 1
lxc.id_map = u 251 100251 749
lxc.id_map = g 251 100251 749
lxc.id_map = u 1000 1000 1
lxc.id_map = g 1000 1000 1
lxc.id_map = u 1001 101001 64535
lxc.id_map = g 1001 101001 64535

While this succeeded in rendering the files in /dev/snd/* as owned by 'audio' (and presumably writeable), the game still threw the same audio errors:

Picking OpenAL Soft. If your desired device was missing, make sure you have the appropred a different device, configure ~/.openalrc appropriately.
ALSA lib /var/tmp/portage/media-libs/alsa-lib-1.0.28/work/alsa-lib-1.0.28/src/confmisc.
ALSA lib /var/tmp/portage/media-libs/alsa-lib-1.0.28/work/alsa-lib-1.0.28/src/conf.c:42card_driver returned error: No such file or directory
ALSA lib /var/tmp/portage/media-libs/alsa-lib-1.0.28/work/alsa-lib-1.0.28/src/confmisc.ings
ALSA lib /var/tmp/portage/media-libs/alsa-lib-1.0.28/work/alsa-lib-1.0.28/src/conf.c:42concat returned error: No such file or directory
ALSA lib /var/tmp/portage/media-libs/alsa-lib-1.0.28/work/alsa-lib-1.0.28/src/confmisc.e
ALSA lib /var/tmp/portage/media-libs/alsa-lib-1.0.28/work/alsa-lib-1.0.28/src/conf.c:42refer returned error: No such file or directory
ALSA lib /var/tmp/portage/media-libs/alsa-lib-1.0.28/work/alsa-lib-1.0.28/src/conf.c:47ch file or directory
ALSA lib /var/tmp/portage/media-libs/alsa-lib-1.0.28/work/alsa-lib-1.0.28/src/pcm/pcm.cfault
AL lib: (EE) alsa_open_playback: Could not open playback device 'default': No such file
Initializing OpenAL failed, no sound will be played

Graber's example used PulseAudio not ALSA, so there's a good chance that something's missing in my mounted audio devices. I'm no ALSA ace, but let me try bind-mounting all of the 'audio'-owned devices.

# All audio-owned devices
lxc.mount.entry = /dev/snd dev/snd none bind,optional,create=dir
lxc.mount.entry = /dev/adsp dev/adsp none bind,optional,create=file
lxc.mount.entry = /dev/dsp dev/dsp none bind,optional,create=file
lxc.mount.entry = /dev/audio dev/audio none bind,optional,create=file
lxc.mount.entry = /dev/mixer dev/mixer none bind,optional,create=file
lxc.mount.entry = /dev/sequencer dev/sequencer none bind,optional,create=file
lxc.mount.entry = /dev/sequencer2 dev/sequencer2 none bind,optional,create=file

It didn't help. To isolate the issue, like with xclock, I found a .wav file I could test with aplay. Indeed, the error is reproducible with aplay. The venerable strace utility showed a failure to write to /dev/snd/controlC0. I hadn't added 'erik' to 'audio' in the guest also. Problem solved, and audio works.

Now, down to the mines...

2015-04-18 by stasibear Permalink

containers