[AppStream] Extensibility in AppStream

[Matthias Klumpp]

2016-07-20 20:01 GMT+02:00 Simon McVittie <simon.mcvittie at collabora.co.uk>:
> Hi,
> I'm currently working on an automotive Linux platform that aims to have
> an "app store". We're trying hard to use standard Linux/freedesktop
> concepts rather than going off in our own weird directions (for example
> our app launcher consumes .desktop files), so I'm evaluating AppStream
> as a format for metadata about installable or installed "apps" - more
> specifically, the things that AppStream calls components, including
> launchable programs but also more general user-installable components
> like themes.
>
> One thing that I notice is missing from the AppStream specification is
> an official way to provide unofficial extensions, similar to the role of
> X-Vendor-MyField in .desktop files. We're keen to have extensibility
> available for a couple of reasons:

Prefixing things with "x-" is the way to add non-standard spec
extensions. The AppStream validator will ignore those when validating.

> * prototyping: when new features that would make sense in the upstream
>   AppStream spec are still under discussion, we'd prefer to be able to
>   experiment with them without patching/forking AppStream libraries -
>   particularly if some implementation experience with a prototype
>   design is necessary to see the flaws in that design and propose
>   something that's better in the long term

As soon as things are in libappstream, I can't remove the things
easily anymore, because there should be ABI and API stability.
You could experiment with things by carrying downstream patches and
upstream them.
Another thing that could be done is exposing the libxml XML tree for
other services to use...
And quite honestly, it's XML or YAML, both formats can be read with any tool :-)

> * specialized requirements: if there's a commercial need for a
>   particular feature in an automotive platform but the AppStream spec
>   authors consider it to be too niche to incorporate, we'll still need
>   to provide it somehow
>
> (I don't have any concrete examples of a need for either at the moment,
> but I'm fairly confident that both will happen at some point.)

Let's solve this problem when it actually appears. I very rarely
reject adding metadata, and if the feature is generally useful, even
if it is a nice, it can be added.
Some company-internal extensions would never be supported in the
upstream spec, the same applies to things which aren't metadata (e.g.
stuff that influences how the application works, or things like build
recipes).

> I've also seen that the appstream-glib library used by GNOME Software
> does offer extensibility, in the form of a <metadata> element. For
> example, <https://github.com/hughsie/appstream-glib/issues/72> mentions:
>
>     <metadata>
>       <value key="X-CacheID">firefox_43.0.1-1_amd64.deb</value>
>     </metadata>
>
> and GNOME Software seems to insert other <value>s as required. This
> seems ideal for our purposes: how feasible is it to get that into the
> AppStream spec?

Never. This is a terrible hack which I hope Richard will kill off at
some time. A specification should really not define some field which
contains random stuff which downstream consumers may or may not
support and which we do not (and can not) validate for correctness at
all. Additionally, "metadata" is a terrible name, because everything
in the files already is metadata - could just be called "stuff"
instead, that would be just as accurate.

I would rather want people to use x-prefixed fields and read the XML
directly for their extensions, than adding hacks like this one to the
AppStream libs, which adds fields which consumers of the API and
writers of metainfo files can not rely on.
Exposing the libxml2 document tree could actually be something useful for you.
For simple key-value things I could even think about collecting all
the x-prefixed things and adding them to a hashtable in AsComponent to
consume for clients if libas is compiled in some kind of experimental
mode (that would be really ugly, so that's something only to consider
if there is an actual strong need for it (so if for whatever reason
you can't consume the XML directly)). Adding some blanko "but anything
you want in here" tags to the spec is something I will avoid though.

It would be helpful to see some concrete issues you might be facing,
and to know about the concrete scenario you want to use AppStream for
(e.g. who will be writing the metadata, how open will the store thing
be, will it be based on some preexisting solution, etc.)

Cheers,
    Matthias