Building for ARM systems

[Dan Nicholson]

On Tue, Mar 14, 2017 at 8:58 AM, Aleix Pol <aleixpol at kde.org> wrote:
> On Tue, Mar 14, 2017 at 11:54 AM, Dan Nicholson <nicholson at endlessm.com> wrote:
>> On Mar 14, 2017 3:08 AM, "Alexander Larsson" <alexl at redhat.com> wrote:
>>
>> On Mon, 2017-03-13 at 22:31 -0500, Dan Nicholson wrote:
>>> On Mar 13, 2017 9:19 PM, "Aleix Pol" <aleixpol at kde.org> wrote:
>>>
>>> $ flatpak build sdk bash
>>> execvp bash: No such file or directory
>>> $ cat /proc/sys/fs/binfmt_misc/status
>>> enabled
>>> $ ls -l /proc/sys/fs/binfmt_misc/qemu-arm
>>> -rw-r--r-- 1 root root 0 mar 13 14:19 /proc/sys/fs/binfmt_misc/qemu-
>>> arm
>>>
>>> Can you show the contents of that qemu-arm file?
>>
>> Yeah, it seems like it is dynamically linked and is not finding the
>> loader.
>>
>>
>> Nah, I just went through this debugging with someone at Endless yesterday.
>> The kernel unfortunately returns ENOENT when it can't handle the binary
>> format. Or possibly the ENOENT is because it can't find the interpreter for
>> the binfmt. Either way, I'm pretty sure it's not a dynamic linking problem
>> unless Aleix just installed the wrong package.
>>
>> What I suspect is actually the problem is that Debian's qemu-user-static
>> package is setting a restrictive mask in the binfmt configuration that
>> doesn't have the new F flag needed so that the interpreter is opened
>> immediately at registration and kept open. I noticed yesterday that the
>> setup on xenial is that the flags are set to OC. I'm guessing this hasn't
>> been updated and can't really since it would make the configurations in the
>> package dependent on a pretty new kernel. I believe the kernel throws EINVAL
>> when an unrecognized option is in the mask.
>>
>> What Fedora does (I'm pretty sure), and what I think Debian should do, is
>> leave the mask blank so the kernel defaults are used. Then you pick up the
>> new F flag automatically when the kernel supports it.
>>
>> So, I'd like to see the actual registered configuration. If it contains OC
>> like I suspect, then it needs to be updated to include F or just have the
>> mask stripped off and use the kernel defaults.
>>
>> --
>> Dan
>
> It does include OC, not that I know what it means ^^'.
>
> $ cat /proc/sys/fs/binfmt_misc/qemu-arm
> enabled
> interpreter /usr/bin/qemu-arm-static
> flags: OC
> offset 0
> magic 7f454c4601010100000000000000000002002800
> mask ffffffffffffff00fffffffffffffffffeffffff

Documentation for the entries is at
https://www.kernel.org/doc/html/latest/admin-guide/binfmt-misc.html.
Here's a temporary workaround you can try where you're clearing the
flags so you use the kernel defaults (which should include F on your
yakkety kernel).

# echo -1 > /proc/sys/fs/binfmt_misc/qemu-arm
# echo ':qemu-arm:M::\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x28\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/bin/qemu-arm-static:'
> /proc/sys/fs/binfmt_misc/register

After this, /proc/sys/fs/binfmt_misc/qemu-arm should show an empty
flags, and I bet things work.

If you want to make this more permanent, remove the binfmt support
file from qemu-user-static and use systemd's binfmt support:

# update-binfmts --package qemu-user-static --remove qemu-arm
/usr/bin/qemu-arm-static
# echo ':qemu-arm:M::\x7fELF\x01\x01\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02\x00\x28\x00:\xff\xff\xff\xff\xff\xff\xff\x00\xff\xff\xff\xff\xff\xff\xff\xff\xfe\xff\xff\xff:/usr/bin/qemu-arm-static:'
> /etc/binfmt.d/qemu-arm.conf

This should really be fixed in debian's qemu-user-static (drop
--credential yes in call to update-binfmts) or binfmt-support (support
F flag with some --option) packages.

--
Dan