[Mesa-dev] Driver testing system for Steam games

[Karen Ghavam]

Hi Mesa Developers,

Since going public with the Mesa Graphics Testing Services system last
week, I have received some questions and comments that made me think it
would be good to articulate the objectives of the Mesa Graphics Testing
Services:


   1. Detection of regressions (both image rendering and performance)
   2. Provide a service to Mesa Developers to test out their development
   branches for regressions before merging.
   3. Provide ongoing testing on Mesa Releases and maintain a history of
   results that is helpful in continued monitoring of Mesa driver release
   quality as it relates to Steam games.
   4. Provide a maintained baseline of known good images and performance
   for various Steam game frames.
   5. NOT Objective: Performance bench-marking test suite. The tests in the
   Mesa testing system are not a measure of overall game performance since
   game logic will be removed at test time. The tests are focused on Mesa
   driver functionality and performance through the GPU.

Karen Ghavam
CEO and Engineering Director
LunarG, Inc.


On Wed, Nov 15, 2017 at 1:32 PM, Pierre-Loup A. Griffais <
pgriffais at valvesoftware.com> wrote:

> Hi Mesa-Devel,
>
> Today we're happy to announce that we're making the LunarG driver
> testing system available to the Mesa development community. We've been
> using this system internally for quite some time, but we've recently
> done some more work so the results could also be publicly available for
> the community to track regressions, and for developers to be able to
> submit their branches for pre-submit testing.
>
> The LunarG test system is a service that provides regular regression
> testing on Mesa releases for Intel and AMD OpenGL graphics drivers. It
> also enables users to test their own Mesa builds and compare results to
> LunarG baselines, LunarG test runs, or user test runs. The objective of
> the LunarG test system service for Mesa is to identify regressions in
> performance or rendering for Mesa releases and developer's branches that
> would negatively impact the gaming experience.
>
> The test suite is a collection of trace files created from hundreds of
> Linux OpenGL games from Steam. These trace files are used to automate
> testing of rendering correctness as well as game performance. There are
> currently 6 graphics platforms available for test (3 configurations each
> of Intel and AMD graphics). The full suite of games typically runs
> overnight with results posted the next day.
>
> To access the test system and browse results: https://share.lunarg.com
> (click on "Test System") on the left.
>
> Here's an example of a test report that a Mesa bug could possibly be filed
> from after inspection: https://imgur.com/a/cBonz
>
> To learn more about how the test system works, read the short user
> manual: https://share.lunarg.com/test/manual
>
> Let us know if you have any feedback or questions. Vulkan support is
> planned for the future, but not ready quite yet.
>
> Thanks,
>   - Pierre-Loup
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://lists.freedesktop.org/archives/mesa-dev/attachments/20171120/e80665d5/attachment-0001.html>