[Intel-gfx] [RFC PATCH i-g-t 0/3] Test the plane formats on the Chamelium

[Maxime Ripard]

Hi,

Here is an RFC at starting to test the plane formats using the
Chamelium over the HDMI. This was tested using the vc4 DRM driver
found on the RaspberryPi.

This is still pretty rough around the edges at this point, but I'd
like to get feedback on a few issues before getting any further.

  * I've used pixman for now to convert back and forth the pattern and
    the captured frame. While this worked quite well for the RGB
    formats since pixman supports most (but not all) of them. However,
    the long term plan is to also test YUV and more exotic (like
    vendor specific) formats that pixman has 0 support for. So I
    really wonder whether this is the right approach compared to:
    - Using something else (but what?)?
    - Rolling our own format conversion library?

  * I've so far had a single big test that will test all the formats
    exposed by the planes that have a pixman representation. I wonder
    whether this is preferrable, or if we want to have a subtest per
    format. I guess the latter will be slightly better since we would
    be able to catch regressions in the number of formats exposed that
    we wouldn't be able to with the former.

  * Kind of related, I'm not sure what is the policy when it comes to
    tests, and whether I should merge this tests with kms_chamelium or
    leave it as a separate file.

  * One of the biggest challenge of the serie is to support formats
    that have less bits than the reference frame. Indeed, the flow of
    patterns is this one: the pattern will first be generated in
    ARGB8888. It will then be converted to whatever format we want to
    test, be fed into the display engine, that will output it, and the
    Chamelium will capture it in ARGB8888.
    However, when the plane format has less than 8 bits per color,
    some upsampling will happen, where the less significant bits will
    be filled with values that probably depend on the display
    engine. Another side effect is that the CRC used in the Chamelium
    tests cannot be used anymore.
    The way I'm testing currently is that I'm retrieving the frame,
    and then compare each pixels on their most significant bits. This
    sounds inefficient, and it is, especially on the RPi that doesn't
    have the best networking throughput out there.
    I guess we could also generate a CRC for both an upsampling with
    the lowest bits set to 1, and one for the lowest bits set to 0,
    and try to see if one of them match. I guess this should cover
    most of the situation.

Let me know what you think,
Thanks!
Maxime

Maxime Ripard (3):
  tests/chamelium: Move some functions and structures to a common place
  tests/chamelium: Add test case for plane formats
  tests: Add vc4 test suite

 tests/Makefile.am                        |   1 +
 tests/Makefile.sources                   |  10 +
 tests/helpers_chamelium.c                | 165 +++++++++++++++++
 tests/helpers_chamelium.h                |  65 +++++++
 tests/kms_chamelium.c                    | 170 +----------------
 tests/kms_chamelium_formats.c            | 305 +++++++++++++++++++++++++++++++
 tests/meson.build                        |   1 +
 tests/vc4_ci/vc4-chamelium-fast.testlist |   4 +
 tests/vc4_ci/vc4-chamelium.testlist      |   9 +
 tests/vc4_ci/vc4.testlist                |  35 ++++
 10 files changed, 596 insertions(+), 169 deletions(-)
 create mode 100644 tests/helpers_chamelium.c
 create mode 100644 tests/helpers_chamelium.h
 create mode 100644 tests/kms_chamelium_formats.c
 create mode 100644 tests/vc4_ci/vc4-chamelium-fast.testlist
 create mode 100644 tests/vc4_ci/vc4-chamelium.testlist
 create mode 100644 tests/vc4_ci/vc4.testlist

-- 
2.14.3