[Libva] Weird behaviour when dup()'ing the file descriptor for /dev/dri/card0

[Jorge Lucangeli Obes]

Hi all,

My name is Jorge. I'm trying to get libva to work inside the syscall
filtering sandbox (seccomp mode 2 [1], landed on Linux 3.5) in
Chromium/Chromium OS. Syscall filtering in Chromium allows us to deny
access to syscalls deemed "dangerous" (e.g. ptrace()), or syscalls
that could be used by malicious code after compromising a Chromium
process using a vulnerability in the code.

More specifically, we would like to have Chromium's GPU process run
without unlimited access to the open() system call. To do this, we
have to warm up/preload some resources, including the
i965_drv_video.so library and /dev/dri/card0 which are legitimately
opened by libva. The issue I'm hitting is with /dev/dri/card0.

The way we limit access to open() is by preopening the files needed by
libva, and then hooking open() when the sandbox is enabled, and
returning dup()'s of the original fd's opened before enabling the
sandbox. One of the files libva opens is /dev/dri/card0:

http://cgit.freedesktop.org/vaapi/libva/tree/va/x11/dri2_util.c#n198
va/x11/dri2_util.c

Bool
isDRI2Connected(VADriverContextP ctx, char **driver_name)
{
/* snip */
    dri_state->fd = open(device_name, O_RDWR); <--- device_name ==
"/dev/dri/card0"
    assert(dri_state->fd >= 0);

/* snip */

    if (drmGetMagic(dri_state->fd, &magic))
        goto err_out;

    if (!VA_DRI2Authenticate(ctx->native_dpy,
RootWindow(ctx->native_dpy, ctx->x11_screen),
                          magic)) <--- failing
        goto err_out;
/* snip */

Now, the issue that I'm seeing is the following: the call to
open(device_name) happens after our sandbox is on, so the call gets
intercepted. Instead of letting it go through, we dup() a fd for
|device_name| that we opened before enabling the sandbox. drmGetMagic
succeeds when called on that dup()'ed fd (all it does is an ioctl),
but for some reason the call to DRI2Authenticate fails. This makes
va_getDriverName() fail as well, and Chromium falls back to not using
libva. I've tried using the LIBVA_DRIVER_NAME env var to sidestep the
call to va_getDriverName() but since va_getDriverName() does a whole
bunch of things, sidestepping makes an assert trigger in the actual
libva Intel driver code.

Stracing the behaviour with the open() restrictions disabled and
enabled, the difference appears to be in what the driver reads from fd
5, which is a socket. My question would be, then, if anyone knows
whether reopening /dev/dri/card0 clears or resets some state, which
could be causing DRIAuthenticate to fail when called after calling
drmGetMagic on a dup()'ed fd.

*Open hook disabled:
open("/dev/dri/card0", O_RDWR)          = 15
...
dup(15)                                 = 38
ioctl(38, 0x80046402, 0x7fff7ee87360)   = 0
poll([{fd=5, events=POLLIN|POLLOUT}], 1, -1) = 1 ([{fd=5, revents=POLLOUT}])
writev(5, [{"\204\2\3\0\255\0\0\0\n\0\0\0", 12}, {NULL, 0}, {"", 0}], 3) = 12
poll([{fd=5, events=POLLIN}], 1, -1)    = 1 ([{fd=5, revents=POLLIN}])
read(5, "\1\0a\1\0\0\0\0\1\0\0\0\t\0\0\0\0\0\0\0\0\0\0\0\260O\374\364y\177\0\0",
4096) = 32
...
write(2, "libva: ", 7)                  = 7
write(2, "va_getDriverName() returns 0\n", 29) = 29

*Open hook enabled:
open("/dev/dri/card0", O_RDWR)          = 15
...
dup(15)                                 = 32
ioctl(32, 0x80046402, 0x7fff7ee87360)   = 0
poll([{fd=5, events=POLLIN|POLLOUT}], 1, -1) = 1 ([{fd=5, revents=POLLOUT}])
writev(5, [{"\204\2\3\0\255\0\0\0\n\0\0\0", 12}, {NULL, 0}, {"", 0}], 3) = 12
poll([{fd=5, events=POLLIN}], 1, -1)    = 1 ([{fd=5, revents=POLLIN}])
read(5, "\1\0A\4\0\0\0\0\0\0\0\0\t\0\0\0
\340\n\365y\177\0\0\260O\374\364y\177\0\0", 4096) = 32
...
write(2, "DRI2Authenticate failed\n", 24) = 24
close(32)                               = 0
...
write(2, "libva: ", 7)                  = 7
write(2, "va_getDriverName() returns -1\n", 30) = 30

Thanks!
Jorge

[1] http://lwn.net/Articles/475043/