Using appstream in xdg-app

[Alexander Larsson]

(CCing matthias and richard as being appstream guys)

Yesterday I started looking at appstream and how it should work when
integrated with xdg-app.  The general idea is that each app ships an
appstream file describing in, then we run an extractor on the repo
when it is updated which creates the combined metadata for all the
apps in the repo. These metadata are then commited on a separate
branch in the repo.

On to the details.

For apps
========

Each app ships an exported appinfo file. It goes into
export/share/appdata, similar to how the desktop files goes in
export/share/applications.

The name of the appinfo file needs to be "$(APP_ID).appdata.xml", in
order to be allowed for export (xdg-app only allows files with the app-
id as prefix to be exported, to avoid conflicts in the exported
directories). The id of the app should be that of the desktop file,
which due to the prefix rule is "$(APP_ID).desktop", and similarly the
icon will have the app-id as the prefix. This is nice because it
forces a shared identifier for things like desktop file, dbus service
name and appstream id, and it means we will not get conflicts in the
appinfo icon cache. 

Note: This is not only about accidental conflicts, we don't want some
app to deliberately cause a conflict as a way to attack the user
somehow.

In the repo
===========

Whenever the repo is updated we run appstream-builder on it,
enumerating all app and runtime branches for each architecture and
looking at the exports in them. All appinfo files (that comply with
the above rules) will be extracted into a directory with this format:

 appdata.xml
 icons/64x646/...
 icons/128x128/...

i.e. a standard (uncompressed) appstream and corresponding icon cache.
This directory is then checked into the repository as a branch called
"appdata/$arch".  We also want to update static delta files for it, one
from-scratch to allow initial fetch (which is otherwise slow for the
many small icon files), and one that is from the last version (as this
allows the deltas to the uncompressed appdata.xml to be encoded as a
diff, which, assuming we sort it, should allow very small delta
files).  Note that the xml will be compressed on the repo/network, but
not on the client.

Additionally, during the extraction of the appinfo we can supply
custom xml that overrides the appdata from the apps, or supply extra
translations.

On the client
=============

Each xdg-app directory (i.e. /var/xdg-app and ~/.local/share/xdg-app)
will maintain a local mirror of the "appdata" branch of each
configured remote. This will typically be a reference called
"€remotename:appdata/$arch" and be stored in
"repo/remotes/$remotename/appdata/$arch".

Additionally, there will be a deployed version of this branch in
"appdata/$remotename". This is where e.g. gnome-software would look
for appdata info and icons.

Issues
======

Localized data is pretty large, and we generally don't need all
locales. For instance, on my fedora 23 install 62% of the uncompressed
repo appdata is made up of localized strings. We could split up the
xml files per-locale and have separate branches (say
"appdata/$arch/$locale" and "appdata/$arch/icons"). Would this be
worth it? Also, does appstream support localized icons?

What do we do about appinfo id conflicts? For instance, a single
repository can have two branches of org.gnome.Gimp, say "master"
(tracking 2.10.*) and "2.8" (tracking 2.8.*). Both of these would have
an id of "org.gimp.Gimp.desktop". In this setup, how can we tell the
difference between the app ids? Also, if they have the same icon name,
but different icon files, which one will be stored in the icon cache?

The way xdg-app handles conflics on the client is that you have to
specify which one you want when you pull, and you can have multiple
installed locally, but only one of the is "current". Only the current
one gets its files (e.g. desktop file) exported to the system, and the
other ones need to be started manually. Should we do something like
that on the repo too? i.e. mark one current and ignore the others in
the UI? Or should we extend the ids and icon names with the branch
name?

The conflicts can also happen between different remotes. I.e. multiple
remotes can contain the same app. gnome-software has to handle that
somehow by showing a list of possible sources when installing the app.

Does this make sense?

-- 
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
 Alexander Larsson                                            Red Hat, Inc 
       alexl at redhat.com            alexander.larsson at gmail.com 
He's an unconventional umbrella-wielding dwarf moving from town to town, 
helping folk in trouble. She's a cynical hip-hop archaeologist looking 
for love in all the wrong places. They fight crime!