Using appstream in xdg-app

[Alexander Larsson]

On fre, 2015-12-18 at 10:27 -0500, Richard Hughes wrote:
> > 
> > The name of the appinfo file needs to be "$(APP_ID).appdata.xml",
> > in
> > order to be allowed for export (xdg-app only allows files with the
> > app-
> > id as prefix to be exported, to avoid conflicts in the exported
> > directories). The id of the app should be that of the desktop file,
> > which due to the prefix rule is "$(APP_ID).desktop", and similarly
> > the
> > icon will have the app-id as the prefix.
> 
> Is this an upstream change required for apps? If so we probably want
> to blog about this a bit more, and if this does stick it makes the
> "find an application icon" code in appstream-glib becomes SO much
> simpler.

I've not really done much public announcement of it, and its only
necessary for upstreams to change if and when they create a xdg-app
bundle. For instance, xdg-app-builder has some features to make the
rename easy. Of course, everything makes most sense when you make the
change upstream.

The reasoning behind this is pretty simple. In a regular distro
environment, filename conflicts in the repos are easily detected and
centrally managed by manual decisions. But in a distributed environment
that is much harder. Additionally, if we are to have some kind of
protection against malicious apps then we must take the case of
filenames much more seriously, because while uncommon by chance, they
are very easy to explicitly create as a form of attack.

For instance, one could imagine an app exporting a dbus service file
with the name "org.freedesktop.secrets.service" and then start
collecting passwords. Clearly there must be some kind of limit on what
files you can export to the host side.

The typical filetypes you export are: desktop file, dbus service file
and icons. Dbus service file filenames must be the same as the well-
known name, and with the new style dbus activation for desktop files
the same is true for them. This makes it natural to make the export
rule to be that it must be prefixed by the dbus name (if any), and the
natural follow-up on that is to make the application id the same as the
dbus id. This means that the only real extra requirement we make of
upstream is that they rename the app icon to the dbus name.

Note that the exact requirment is that the exported filenames must be
*exactly* the app id, or the app id followed by a dot, followed by
anything. That means "org.gnome.Gimp", can additionally export service
files like "org.gnome.Gimp.helper.service", but not
"org.gnome.GimpHelper.service". So, once can still expose sub-service-
names (and in fact the default dbus filtering we do allow you to own
these too). Similarly, the app can contain multiple icons, but they are
all easy to verify as coming from the app.


> > In the repo
> > ===========
> > 
> > Whenever the repo is updated we run appstream-builder on it,
> 
> The binary as a spawned exec or using libappstream-builder? If it's
> the latter we get a lot more control about where logs go and that
> kind of thing.

I'm not against using libappstream-builder from xdg-app, but I would
like it to be in a separate binary from the regular things that a user
would use so that you only need to install it if you're building apps.
Maybe we can do things like git, and have "xdg-app build-finish" spawn
"xdg-app-build-finish" which is in an optional package.

> > Additionally, during the extraction of the appinfo we can supply
> > custom xml that overrides the appdata from the apps, or supply
> > extra
> > translations.
> 
> Any ideas on the format of this?

No, i don't have any idea about how this could work, but I know that at
least endless have interest in injecting app metadata (translations,
etc) from the side. I was hoping you would know how to best do this. :)

> > Additionally, there will be a deployed version of this branch in
> > "appdata/$remotename". This is where e.g. gnome-software would look
> > for appdata info and icons.
> 
> Is there an issue here if more than one remote is shipping the same
> application ID?

I think that depends a bit on how you handle it. You can install
multiple remotes, and those remotes can contain the same id. You can
even pull those ids, and we would know to separate them (because the
remote is part of the reference path, for instance:

 ~/.local/share/xdg-app/repo/refs/remotes/nightly/app/org.gimp.GimpDevel/x86_64/master

Or, as the ref will be known to ostree

  nightly:app/org.gimp.GimpDevel/x86_64/master

However, once the app is installed, which in xdg-app terms is the same
as deployed, it will have an origin file:

  ~/.local/share/xdg-app/app/org.gimp.GimpDevel/x86_64/master/origin

Which means that once installed, the app-id+arch+branch uniquely
identifies which particular remote the app is from.

> I'd also really like to keep the translations in one place if
> possible; 62% sounds a lot, but the total size of the XML file is
> tiny. Having all the locale data there also means you can switch your
> "language" in the control center and do searches in the new language
> *without* waiting for the next idle refresh of all the remotes to get
> your new locale appstream metadata. It also means I don't have to
> write that code to auto-fetch certain appdata locales depending on
> your locale, which would work okay for the --user target but be much
> harder to do system-wide.

True. Lets ship it all then.

> > What do we do about appinfo id conflicts? For instance, a single
> > repository can have two branches of org.gnome.Gimp, say "master"
> > (tracking 2.10.*) and "2.8" (tracking 2.8.*). Both of these would
> > have
> > an id of "org.gimp.Gimp.desktop". In this setup, how can we tell
> > the
> > difference between the app ids? Also, if they have the same icon
> > name,
> > but different icon files, which one will be stored in the icon
> > cache?
> 
> At the moment, the last added "wins". I'm not terribly sure it's a
> huge problem, but you certainly could put the commit id in the <id>
> rather than the application "name" if we needed to support this. As
> long as we had some way of getting the xdg-app app "name" from the
> AppStream file I don't think it matters. I think renaming the file
> like we do the .desktop file would also work.

Things would probably make most sense if we used the full xdg-app
"name" as the id in the appinfo files. I.e:

<id>app/org.gimp.GimpDevel/x86_64/master</id>

However, that would make the appdata id not the same as one if the app
came from a rpm install. Would that be a problem? Its pretty easy to
extract "org.gimp.GimpDevel.desktop" from the above.

> > The way xdg-app handles conflics on the client is that you have to
> > specify which one you want when you pull, and you can have multiple
> > installed locally, but only one of the is "current". Only the
> > current
> > one gets its files (e.g. desktop file) exported to the system, and
> > the
> > other ones need to be started manually. Should we do something like
> > that on the repo too? i.e. mark one current and ignore the others
> > in
> > the UI? Or should we extend the ids and icon names with the branch
> > name?
> 
> I think we have to do the current thing; we don't want to return ~15
> different "GIMP" applications when searching.

How does appdata currently handle multiple arches? Is there a metadata
file per arch? xdg-app can share a single repo for multiple arches,
which is not traditionally done for e.g. rpm repos (although multiarch
does it a bit i guess).

I guess we need some way to mark the "current", or "default" branch in
the repo. Or alternatively, always use "master" for that (its the
default in the tools if you specify no branch, etc).

> > The conflicts can also happen between different remotes. I.e.
> > multiple
> > remotes can contain the same app. gnome-software has to handle that
> > somehow by showing a list of possible sources when installing the
> > app.
> 
> This sounds more like a #gnome-design discusssion, but I'm pretty
> sure Allans response would be "JDI without asking questions" -- i.e.
> we need to 'prefer' one remote, or have some kind of prioritisation
> even if it's as simple as remote-added-first-wins or remote-updated-
> last-wins. People who want more control than that can use the command
> line.

Yeah, I'll add a priority metadata field to the remotes.

-- 
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
 Alexander Larsson                                            Red Hat, Inc 
       alexl at redhat.com            alexander.larsson at gmail.com 
He's a one-legged gay grifter who hides his scarred face behind a mask. 
She's a cold-hearted green-skinned mechanic with the power to bend men's 
minds. They fight crime!