[Piglit] [RFC] Piglit tags/releases

[Dylan Baker]

On Tuesday, September 30, 2014 02:12:48 PM Ken Phillis Jr wrote:
> On Tue, Sep 30, 2014 at 9:19 AM, Dylan Baker <baker.dylan.c at gmail.com>
> wrote:
> 
> > Although it hasn't been nearly as bad recently as it was before we
> > started writing unit tests for the framework we still do have
> > regressions, especially when refactoring touches the API's. I know that
> > IGT integration was broken until recently, and I suspect that if we
> > looked several other suites are broken.
> >
> > While I think this is a good idea, it's a lot of work and it will
> > probably be disruptive for a lot of developers, and a lot of work. So
> > it's not very high on my list of priorities.
> >
> > However, I'm not convinced that having versions of the tests themselves
> > is all that useful.
> >
> > Dylan
> >
> 
> I agree that applying version information to the tests is also not very
> useful. However, I do think that it is extremely useful to apply version
> information to the results files themselves. This can be useful in the fact
> that it will allow an easier method to detect outdated test results. I
> believe it is not a good idea to compare results from a test run from about
> 5 years ago to a test run that was made just last week. There is too many
> changes in the results file themselves over the past couple of years that
> it is nearly impractical to keep a test result map from that time period.
> At least with quarterly releases we can add a flag ( version number ) in
> the file that will indicate that the test results are not compatible
> between the two files.

Our JSON output is numbered, we use these version number to track
changes to the format of the file, and we transparently updated old
versions to new versions.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.freedesktop.org/archives/piglit/attachments/20140930/487877af/attachment-0001.sig>