[AppStream] Using AppStream for "Server Applications", Cockpit

Mon Apr 3 11:38:33 UTC 2017

2017-03-30 14:09 GMT+02:00 Marius Vollmer <marius.vollmer at redhat.com>:
> Matthias Klumpp <matthias at tenstral.net> writes:
>
>> I have been thinking about adding a component-type "daemon" or
>> "service" to AppStream for a while, which would cover background
>> services like webservers, question is whether you could use something
>> like that for your use-case, [...]
>
> Yes, this makes much more sense than type="server-application".
>
> Content-type "service" would be for a systemd service?  Cockpit could
> work very directly with that.

Would be a component-type, and yes, it would affect anything that can
be started via systemd (which would also include SysV-init scripts).
The component-type would probably have to be accompanied by a provided
element "service" or similar, so components can tell which services
they contain.
E.g.
```
<component type="service>
  <id>org.samba.Samba</id>
  ...
  <provides>
    <service>nmbd</service>
    <service>samba</service>
    <service>smbd</service>
    ...
  </provides>
</component>
```

This will need more thought and feedback though before I would be
comfortable to add it - especially knowing how exactly it would be
used will be helpful.

>> This will be tricky, I assume (as web server admins will also want to
>> install services that are *not* web-facing, like MySQL or even stuff
>> that would be a generic-type component, suhc as Python).
>
> Cockpit can offer to install console-application components, and then
> explain how to get to a shell (either ssh or using the builtin terminal
> emulator).

Okay, so it would display components of type console-applications from
AppStream then - that's nice!

> What about adding component-type "cockpit-application"?  This would be
> for components that add their UI to Cockpit.  And then we make little
> UIs for setting up MySQL, PostgreSQL, FreeIPA, etc.  I have a plan to do
> this for FreeIPA.
>
> So Cockpit would filter by type and include those that make sense for a
> remote session in a web browser: console-application,
> cockpit-application, and service.

I generally don't want to add too many component types - they should
indicate the basic types of software or data a system consists of,
while a "Cockpit application" would be way too specific, especially
since the application could be used without Cockpit.

But, there are ways AppStream can deal with this. Depending on the
nature of what "Cockpit integration" actually is, a component could:

  * Provide it as a new provides type, in case it is some kind of
interface, e.g. by adding <provides><cockpit>blah</></> to a
component.

  * Be a separate "addon" type component extending the actual
software, in case cockpit integration can be an usually is separated
out from the main product. In that case, the addon would either extend
Cockpit itself or the main software (e.g. MySQL). In the latter case,
some new AppStream tag would be required to split addons into groups
(on my todo list as addon_class tag).

In any case, you would have enough metadata to easily filter out
applications which say that they have explicit Cockpit support.

> We might need to add some categories maybe, but for now I think I can
> ignore categories.  If someone makes a game for Cockpit, why not.  No
> need to filter that out.  Let's see how many console-applications there
> are already... :-)

Right :) We can expand on it as soon as there is a demand for further
filtering. At time, AppStream supports the XDG categories for
categorization, I wonder if that would work for server things as
well... https://standards.freedesktop.org/menu-spec/latest/apa.html
(if not, we can always extend the spec, in case that's a generally
useful thing to do)

>> At Debian, only the data the respective frontend requires and only the
>> data of enabled sources is downloaded and installed.
>
> This is explained in DEP-11, right?  (I noticed the sentence "On servers
> this file is not really required." :-)

Yes :) And it looks like I need to review the spec more often ^^ - on
Debian, servers *do* have the AppStream data depending on what other
stuff is installed. E.g. Debian has a tool which uses AppStream
metadata to install missing firmware packges and other stuff
on-demand, and there is also fwupd which might be potentially useful
on a server and is using AppStream as well.
(by default, no AppStream is available on a minimal Debian installation).

>>>  - In this modern world, we will have to have a container story.  If
>>>    AppStream is the way to go, how do you feel about getting containers
>>>    in it as well?  I know Flatpak uses AppStream, and I'll make sure
>>>    that Server Applications via Flatpak will totally be possible, but
>>>    maybe we also need to include Docker, System Containers, etc to the
>>>    party.  Has anyone thought about this already?
>>
>> I don't really know what you mean by that. AppStream, as metadata
>> format, doesn't care at all which bundling system is used.
>
> AppStream needs to be able to say how to acquire the component, and we
> might need to extend the collection metadata for containers that are
> pulled from registries.

Getting this kind of logic into AppStream is something I was very
careful to avoid so far, although to make Richard happy some stuff to
fetch firmware from web locations is in there.
The thing is, AppStream metadata should really be the bare metadata
describing a component, instructions how to get it and properly
install it are better suited for other systems like package managers,
the Docker hub, Flatpak etc.

What the AppStream collection metradata does, however, is giving
pointers on how to query the install/remove instructions from other
systems. E.g. it contains a package name to allow software centers to
install software via PackageKit, or a Flatpak bundle ID to install it
from a Flatpak repo.

> Also, after we have the component, the client needs to know what happens
> next, no?
>
> For example, a desktop-application component is known to have a Desktop
> file in it that makes it appear in some launcher menu.  A
> cockpit-application component has a Cockpit plugin manifest in it that
> makes it appear in the Cockpit menus.  A service component has a systemd
> unit file in it, so it can be managed with systemctl.  A font install
> something can then be found with fontconfig, so Cockpit ignores that.

Yes, but does the software center need to know about that in advance?
In any case, with the exception of desktop-files, this is what the
"provides" tag is used for:
https://www.freedesktop.org/software/appstream/docs/chap-Metadata.html#tag-provides

> A container image (or Nulecule atomic app, ...) might need a different
> way to manage it, so we might need new component types.
>
> I want to concentrate on "System Containers"[0] first, which present
> themselves as systemd services.  I also want to make it possible for a
> system container to bring a Cockpit plugin, and then they might be
> component-type "cockpit-application".  So nothing new just because they
> are containers.  I think this might be similar to how a Flatpak is a
> desktop-application and not some new component type, right?  Let's see.
>
> [0] http://www.projectatomic.io/blog/2016/09/intro-to-system-containers/

It looks like I need to dive into Cockpit at least shortly to see what
it's all about.
Generally, AppStream doesn't "see" the container or makes it known to
the user - it describes the software inside one in it's metainfo file.
The file gets processed by some system (e.g. a package manager or
containerization service) which produces a collection AppStream file
containing all metadata in it's repository, augmented with information
how to get the software (e.g. "install flatpak bundle XY").

So, what we might need is a new <bundle/> type in the AppStream
collection spec for the specific bundling systems / repositories.
But it could be that I am misunderstanding the problem here.

>> It would be awesome to know what exactly you are planning, so can you
>> maybe give some details on what you want to do with AppStream
>> specifically?
>
> I try to keep this up-to-date:
>
>     https://github.com/cockpit-project/cockpit/wiki/Server-Applications
>
> You might think of this as "GNOME Software running in a browser".  So we
> want to discover available applications via AppStream, and learn enough
> about that application to know how to manage it further inside Cockpit.

Looking at this, it actually appears like we wouldn't need to add much
to AppStream except for the "service" component type and maybe a
couple of new bunding system types, which would be quite neat.
I'll need to try out Cockpit to see how it works and get a clearer
picture, but at time it appears like AppStream - with some minor
extensions - would do exactly what you are looking for.

Cheers,
    Matthias

-- 
Debian Developer | Freedesktop-Developer
I welcome VSRE emails. See http://vsre.info/