[AppStream] Adding QA information to a component

[Richard Hughes]

On Sat, 3 Jul 2021 at 20:08, Matthias Klumpp <matthias at tenstral.net> wrote:
> Maybe you could fire `appstreamcli validate` at the files, just to
> check that?

I'm not sure I can turn off <checksum> and just rely on <artifact>
just yet; at the moment we just do "both" so we support old and new
versions of fwupd. Maybe a few more years!

> I would make the XML look differently though - while freeform
> "anything goes" key value fields are nice, in the past it always paid
> off to have more specific tag names and narrowly defined value spaces.

Yes, you're probably right. <value tag="foo">bar<value> is the a{sv}
of API designs :)

> In AppStream, we usually have one larger tag with children if we
> expect there to be usually more tags of the same type, which I think
> applies here (there are exceptions, like the <url/> tag for backwards
> compatibility).

Yes, agree. At least from a libxmlb point of view, using a parent tag
enclosure like <testing> means we do one integer compare and a jump,
compared to N(integer compare and a jump) so it makes a lot of sense.

>   <testing>
>     <test_result date="2020-07-02">

date! I forgot that bit, 100% agree.

>       <tester>Red Hat QA</tester>

not <name>? It seemed like the "name" of the test_result i..e the bit
you'd show in the UI.

>       <device>Lenovo Thinkpad P1</device>
>       <os version="9">rhel</os>

These make sense.

> Questions for this: Is there ever a case where a test has *failed*
> that should show up here? Or will all listed tests be always
> successful, so actually be some kind of "it works!" certificates?

I did think about only doing successful tests; hopefully any failures
would mean the firmware isn't available.

> Since you went for the key/value approach, do you expect there to be a
> need for freeform key value pairs, like in the custom tag in
> AppStream? If so, what are the applications for that?

The only other thing that I might need is the version the tester
upgraded *from* i.e. 1.2.3 to 1.2.4 is much more likely to succeed
than 0.1.2 to 3.4.5 and also the version of the client that updated
it. e.g. what I've got so far is:

      <testing>
        <test_result id="1" date="2021-07-02">
          <name>LVFS</name>
          <device>LENOVO ThinkPad X1 Carbon 7th</device>
          <os version="34" variant="workstation">fedora</os>
          <custom>
            <value key="RuntimeVersion(org.freedesktop.fwupd)">1.6.2</value>
            <value key="VersionOld">1.2.6</value>
          </custom>
        </test_result>
      </testing>
    </release>

Getting there! Thanks!

Richard.