[Wayland-bugs] [Bug 86110] Backwards compatibility testing

[bugzilla-daemon at freedesktop.org]

https://bugs.freedesktop.org/show_bug.cgi?id=86110

            Bug ID: 86110
           Summary: Backwards compatibility testing
           Product: Wayland
           Version: unspecified
          Hardware: All
                OS: All
            Status: NEW
          Severity: enhancement
          Priority: medium
         Component: wayland
          Assignee: wayland-bugs at lists.freedesktop.org
          Reporter: ppaalanen at gmail.com
            Blocks: 83980

To ensure the continued backwards-compatibility of the Wayland protocol and
libwayland-server,client ABI, we should have some automated testing in place. I
very much doubt any developer ever tests mixed versions of different components
in purpose.

The components in play:
- Wayland (libwayland-server, libwayland-client, protocols)
- Weston
- an autonomous test app to be written, that excercises as much as possible
given the versions available

We would like to test building Weston and app against one version of wayland,
and run them against another version. This produces a test matrix on Wayland
build version vs. Wayland runtime version.

Only official release tags are the tested versions. Alpha and beta releases
should be included if resources allow. In addition, the master version of
Wayland and Weston should be included, so we can catch breakage before
releases. The rolling tests could run, say, weekly or more often if resources
allow.

Weston very often requires the contemporary version of wayland, which is fine,
and produces expected build failures. These should be found by testing rather
than pre-encoding the assumption. Once a test is confirmed to legitimately
fail, it can be marked as known-to-not-work.

We need some way to limit the combinatorial explosion, because both Weston and
the app maybe built or ran against different versions of wayland. It is also
useless to re-run tests with the exact same setup as before (unless the test
app is updated).

The essential idea is, that we would get an automated test bot, that ensures
that we do not break the backwards-compatibility of wayland.

There exists
http://upstream.rosalinux.ru/versions/wayland.html
but I think doesn't cover the Wayland-specifics, like different code generation
between versions of wayland and how generated code gets built into compositor
and app binaries.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/wayland-bugs/attachments/20141110/c61b4a57/attachment.html>