[Piglit] [RFC] Piglit tags/releases

[Ken Phillis Jr]

On Tue, Sep 30, 2014 at 12:05 PM, Emil Velikov <emil.l.velikov at gmail.com>
wrote:

> So in a nutshell what I've gathered so far:
>
>  * Concerns that it will take too much effort.
> Dare I say it - it will not. A single build + sanity run takes a few
> minutes.
>
>  * There is no point if tagging/shipping (distro or otherwise) piglit
> for people that do not know/have the time/etc to build it.
> With all respect - even if all of us have plenty of trained members in
> their QA team, I believe that we all can welcome the little extra
> assistance this can bring us. Imho AMD and Nvidia hardware tend to have
> more (versatile) products, so additional testing there would always be
> appreciated.
>
>
The tagging/version information can also be used as a reference to improve
the reliability of the test framework by giving reference points to work
with. Just as a note, Since July 1st more than 440 commits have been made
to the piglit tree. Also, I believe that these tags can be used by any
vendor implementing opengl to improve overall driver quality while also
improving the interoperability of user space code with mesa.


>  * But to what end? Why is packaging snapshots of an always changing
> developer test suite useful?
> - Does it hurt ? I believe not.
> - Is there a possibility of being useful ? Not amazingly large, but yes.
> - Would it help the versatility "problem" mentioned above ? Possibly.
> - Will it align with i-g-t requirements, without the hassle of splitting
> the tests from the testrunner ? Yes.
>
>
 * Split the runner from the tests, so that it can be used by others -
> i-g-t.
> As already pointed this alone will be more than enough hassle for devs
> and QA alike. Perhaps we can hold off the idea for now ?
>
>
> I believe this covers most/all topics. Are there any actual
> concerns/issues that someone can foresee with this idea ? I feel that
> someone may envision/feels that this may cause headaches but is keeping
> low :)
>
>
> Thanks
> Emil
>

I agree with the idea of having tags/versions. I believe piglit is a large
enough project now that it seriously is needing this to help improve the
ability to find major bugs. I believe it is not exactly a major issue to
give a driver two quick test runs.  The first test run would be to run both
drivers against the last tag of piglit, and then the next test run would be
to run the tests against the revision of piglit that will be flagged as the
tag for this driver. It would be expected that the old tag would possibly
see some regressions, but if there is a major problem that appears it would
be easily seen, however, the final say would be against the most current
release of piglit.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.freedesktop.org/archives/piglit/attachments/20140930/0e695375/attachment.html>