[AppStream] Adding QA information to a component

[Richard Hughes]

On Sun, 4 Jul 2021 at 23:14, Matthias Klumpp <matthias at tenstral.net> wrote:
> Appstreamcli shouldn't throw an error yet if `artifact` isn't used...
> But it's actually very likely that it will do that in future,
> supporting both syntax versions is a bit of a pain and definitely
> something that I'll want to go away when AppStream 1.0 is released.

Right; I don't plan to be 1.0 compliant for another few years then :)

> > not <name>? It seemed like the "name" of the test_result i..e the bit
> > you'd show in the UI.
> Ah! I thought the string value was for a person / team that did the
> testing, not a name for the specific test that has been performed. For
> the latter, `name` is obviously the best choice.

Nope, I've confused you. I was using <name> as the vendor name of the
entity that did the testing. Maybe <developer_name> is what I want
here? or maybe a new <vendor_name>?

> > I did think about only doing successful tests; hopefully any failures
> > would mean the firmware isn't available.
>
> That makes sense. What will the UI do if a test result isn't available
> for the specific hardware the user is running, but the firmware itself
> is available? Warn about the update? (that would IMHO be a bit
> weird...)

I think the idea here is one of:

 * Only mirror firmware that's had a successful test from a specific QA team
 * Only allow deployment of firmware that's been tested by a specific QA team

> Looks sensible! What is the `id` attribute of test_result for?

Ohh, that was more for me; it points to the specific test result on
the LVFS that allows us to see more details about the test. Maybe that
should just be <url>THE_WHOLE_URL</url> instead?

> Ordering? Or just some unique ID for the specific result?

I didn't even think about ordering; naively I thought in the UI to
just use alphabetical or something to avoid getting political.

> I also have to make one correction: The `platform` attribute that I
> mentioned in my previous mail is obviously a property of an *artifact*
> and not of a *release*.

Very true; good point.

> This means we could either add the testing
> block to individual artifacts

I think this is fine; it's okay to say "if you want to use this super
new feature you can't use the old way of doing release artifacts"

> That "VersionOld" thing could potentially be generalized (I'd call it
> updated_from_version maybe, or previous_version as tag

<previous_version> makes a lot of sense to me.

> runtime version piece is very much implementation specific.

You have any preference between:

<value key="RuntimeVersion(org.freedesktop.fwupd)">1.6.2</value>
<value key="fwupd">1.6.2</value>

Other ideas very welcome.

Richard

---
      <artifacts>
        <artifact type="binary">
          <filename>firmware.bin</filename>
          <checksum
type="sha1">111784ffadfd5dd43f05655b266b5142230195b6</checksum>
          <checksum
type="sha256">fce1847b0599bb19cd913d02268f15107691a79221ce16822b4c931cd1bda2c5</checksum>
          <testing>
            <test_result date="2021-07-02">
              <name>LVFS</name>
              <device>LENOVO ThinkPad X1 Carbon 7th</device>
              <url>https://127.0.0.1:5000/lvfs/reports/1/details</url>
              <os version="34" variant="workstation">fedora</os>
              <previous_version>1.2.6</previous_version>
              <custom>
                <value key="RuntimeVersion(org.freedesktop.fwupd)">1.6.2</value>
              </custom>
            </test_result>
          </testing>
        </artifact>
      </artifacts>