Flathub, initial proposal

Thu Sep 1 13:59:07 UTC 2016

Hi,

I'm interested in participating in the design & development of the
proposal.

Some days ago I shared  with this list [1] a design document [2] that
matches quite a lot with the "App Store" side of your proposal.

For the developer's side, I considered this options:
* Develop something like GitHub but using OSTree
* Develop something like Docker Hub
* Develop something like "Google Play Developer console"

In the end I opted for the Google Play model (the developer generates a
bundle "application.flatpack" file and uploads it to the store).

PROS:
* It solves the principal need: provide an easy channel for devs to publish
their apps
* It's a well known process used in other platforms (Android, iOs, ...)
* It can be more agile (no need to wait for a build server)
* This kind of service seems easier to implement than the other options
(GitHub  & Docker Hub)

I personally don't see the need of controlling the build process and keep
the sources. If we trust a developer to control his source code and publish
it in his own git repo, why don't we trust him to build his code and upload
it to the store?

However, there should be some kind of control for the apps before being
published, like controlling that the the "official Firefox" is published by
Mozilla and no 3rd parties.

Best regards,

[1] https://lists.freedesktop.org/archives/xdg-app/2016-July/000340.html
[2]
https://drive.google.com/file/d/0B-021DnJVTcZSXNyQWFZS0JDYmc/view?usp=sharing

On Tue, Aug 30, 2016 at 4:20 PM, Alexander Larsson <alexl at redhat.com> wrote:

> Flatpak and the surrounding software projects
> (ostree/gnome-software/appstream/etc) are now in a pretty good
> state. We need to work on the portal implementation and port
> applications to that to make the sandboxing aspects work, but for the
> pure distribution usecase things mostly work.
>
> However, there are two big missing pieces in the full flatpak story:
>
> The App Store Problem
> =====================
>
> When a user logs in to a newly installed Linux laptop and launches the
> flatpak supporting software installer app (e.g. gnome-software) it
> should be full of all the major FOSS software from the start. Right
> now this only works if the user (or the admin/distro) manually found
> all the various upstream repositories and added them. This is neither
> a good experience, or very scalable.
>
> Instead we want to have a single curated entity where you can get most
> major FOSS software. We also want that entity to have some degree of
> trust, in that it does some kind of verification (manual and
> automatic) of the software in the repositories.
>
> The Build/Distribution Problem
> ==============================
>
> flatpak-builder makes it reasonably easy to package an application as
> a flatpak, including whatever dependencies it needs. However, once you
> have a working build of your application things get more tricky. How
> do you make an official, signed build of it? How do you build on
> multiple arches? How do you set up and maintain a public repository?
> What about bandwidth costs? Mirroring, etc?
>
> All these are solvable. GPG is daunting but tutorials exists,
> scripting the release builds on some machine is doable, and all you
> need for distribution is a dumb web server somewhere. However, this is
> not exactly easy, and its not work that an upstream developer is
> interested in. There should be a simpler way to do this, that everyone
> could reuse.
>
> I imagine a web service that you create an account on, somewhat similar
> to say github or the ubuntu PPA service. Your account gets access to a
> per-user ostree repository, and you can create projects and upload
> builder json manifests for them. These get automatically build on
> several arches and you can pull the builds for testing. Once the build
> is working you can click a button to push them to a stable repository
> that you can tell your users to pull from.
>
> These per-user repos would be similar to github repos. I.e. many repos
> with little guarantees, rather than the vet:ed single access global
> app store. I.e. you can easily fork some existing app and throw a
> patch in it to let someone test your new feature.
>
> The proposal: Flathub
> =====================
>
> Conceptually these are two different problem, with different
> audiences. For instance, one can easily imagine the app store working
> without the shared build system. Or the build system being used without
> a centralized app store.
>
> We discussed this quite a bit at the Flatpak BOF at guadec however,
> and came to the conclusion that an initial attempt at this would best
> handle these as a single thing. Partly this is because it gives nice
> synergy effects, and that many people would probably want to use
> both. But it also lets us give certain guarantees for the app-store,
> which are important to the organization that hosts the service.
>
> For instance, having everything locally build from source, with all
> the sources available, makes it possible to fulfill licensing
> requirements (e.g. the GPL), and makes it possible to verify that the
> software is free software. We hope that the Gnome Foundation (or some
> similar organization) would help us host and maintain the system
> during an incubation phase, and that means we can only support free
> software projects.
>
> So, the proposal is to create a web service that combines these two.
> In many ways it is similar to github, but focusing on ostree instead
> of git.  The minimal viable product for this has the following
> features:
>
>  * Register and authenticate as a user.
>  * Create and edit projects.
>  * Assign members to projects
>  * Create OSTree repositories for a user
>  * Create OSTree repositories for a project
>  * Maintain member permissions for project repos.
>  * Create and/or register GPG keys for users/projects/repos
>    These would be used automatically for signing builds
>    in the repos.
>  * Trigger a build in a repository by uploading a json file
>    or a tarball with json file + dependent files.
>  * Migrate a commit from one repo/branch to another.
>    For instance, you may have a testing repo where you build
>    things. Once tested you can then take the latest build
>    from that and put (+re-sign) in the stable repo that you
>    give to users. (This is the build-commit-from flatpak
>    command.)
>  * A well known Flathub project+repo that contains the curated
>    "Linux upstream app store".
>
> Bring the features!
> ===================
>
> This MVP should be pretty simple to get going. Once that is working we
> can start doing more complex things. Here is a mostly unsorted list of
> things we probably want:
>
> A way to request that a project or build should be available in the
> app store. This should put the build in a review queue that the
> curators of the app store can view and work on.
>
> Automatic scanning/review of builds. For instance, this could ensure
> that apps have valid appdata, desktop files, icons etc. It could also
> do some simple license scanning, CVE scanning, etc. The sky is the
> limit. This is very useful for automatic-review of the app store
> review queue, but is also nice feedback for normal builds.
>
> We want to keep all downloaded source for all builds forever in a
> directory that is shared by every build, both to avoid multiple
> downloads, but also so that we can always rebuild things, and fulfill
> license requirements for GPL:ed software.
>
> Also, we need to keep the json manifest and required files for all
> builds done ever, so that they are reproducible. Maybe these can be
> stored in the ostree repository itself?
>
> Set up mirroring/CDN for the repos that get heavy traffic, to handle
> heavy network loads and to get better download performance.
>
> Collect download statistics and make available to users/projects.
>
> Automatically scan and hardlink common object files across repos to
> keep down disk requirements, while still allowing users full access to
> do whatever they want in their repos.
>
> A REST API and a command line tool to make it easier to submit builds,
> etc.
>
> A web app-store frontend equivalent to gnome-software. This lets you
> browse the appdata files, rate/comment on apps and install them (it
> could generate key-value files that would open in gnome-software for
> installation).
>
> Also, there is the question of what runtimes we should allow on the
> site. They would need to be on the build servers to be able to build
> things. Should we allow people to add runtimes themselves? Should we
> redistribute the runtime on the appstore (probably, to get the
> mirroring, etc).
>
> Have some way to donate/pay for applications. Here be dragons...
>
> Allow pre-built (i.e. not built on Flathub) flatpaks in the
> app-store. Dragons abound here too, but useful for larger projects
> like libreoffice that do their own builds.
>
> Support for user-private repositories. This seems useful, but there is
> no support for authentication in ostree atm, so would have to be added.
>
> Build on weaker architectures, like ARM, using the userspace emulation
> stuff that bastien has been working on.
>
> Going forward
> =============
>
> I don't really have a lot of experience building services like this,
> and I'm pretty busy with the flatpak core stuff. So I would love if
> people who are interested in working on this could chime in.
>
> One very important issue is where to host this initially. I want to
> bring this up with the Gnome Foundation as a start, but we may also
> want to talk to e.g. Digital Ocean for hosting. They seem to be
> sponsoring a lot of free software stuff.
>
> --
> =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> =-=-=-=-=-=-=-=
>  Alexander Larsson                                            Red Hat, Inc
>        alexl at redhat.com            alexander.larsson at gmail.com
> He's a lonely guerilla sorceror with a passion for fast cars. She's a
> ditzy communist bounty hunter living on borrowed time. They fight crime!
>
>
> _______________________________________________
> xdg-app mailing list
> xdg-app at lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/xdg-app
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/flatpak/attachments/20160901/29269084/attachment.html>