[question] How to best package an app that uses Tensorflow?

[Giovanni Campagna]

On Fri, 2019-04-26 at 10:35 +0200, Alexander Larsson wrote:
> On Wed, Apr 24, 2019 at 6:55 AM Giovanni Campagna
> <scampa.giovanni at gmail.com> wrote:
> > So I'm wondering: is it acceptable for Flathub to download the
> > prebuilt
> > Tensorflow binaries for x86_64 and arm?
> > Should I specify them as plain sources or do I need to use "extra-
> > data"?
> 
> In general we are pretty flexible, but there are certain rules:
> 
> If the source is GPL (or similarly licenced), we need to be able to
> provide sources, which we do via the '*.Sources' generated extension.
> So, any such software must be built from sources.
> 
>  Its fine to bundle binaries in the build, but *only* if they are
> actually redistributable. If you don't have explicit
> permissions/license to redistribute the binaries you must use
> extra-data. However, prefer building from source if possible over
> bundling binaries.

As an update on this, I found that the existing binary for ARM is not
compatible, because it's built for Raspbian only, and pip refuses to
install it.

So I started working on getting Tensorflow to build from source, and I
succeeded for x86_64.

If you're interested, you can find the result of that effort here:
https://github.com/flathub/edu.stanford.Almond/tree/wip/update

In particular, the sticking point is that Tensorflow's build system
(bazel) normally makes a number of network calls for dependencies
during the build. There is no way to know these ahead of time, so the
only way is to run the build, watch it fail, collect the failing URL
and add it to the source:
https://github.com/flathub/edu.stanford.Almond/blob/wip/update/build-data/tensorflow-bazel-deps.json


When I then went to build on the other arches though I find that:

- bazel will not build on anything but openjdk 8 (9, 11 or 12 will not
do), and openjdk 8 is not available on arm
- bazel will flat out refuse to run on 32-bit x86
- building tensorflow itself causes an Internal Compiler Error on
aarch64

Which means that, after all the effort, I can still only support
x86_64.

Tensorflow is Apache 2.0 licensed, which means sources are not needed
and binaries should be redistributable, so I think I'll stick to
downloading the prebuilt wheel.

FWIW, building TF from source is about 1h on the x86_64 flathub
machines, but almost 3h on our lab machines (which run the nightly
builds). Building anything on the arm flathub machines is a hit or miss
- it could be as fast as x86_64, or many hours, depending on which
machine you get.

-- 
Cheers,

Giovanni
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <https://lists.freedesktop.org/archives/flatpak/attachments/20190503/01c5d42d/attachment.sig>