[Mesa-dev] A proposal for new testing requirements for stable releases

[Michel Dänzer]

On 09.07.2014 08:10, Carl Worth wrote:
> 
>   3. I'd like to receive a testing report from each driver team
> 
> 	This is the meat of my proposal. I'm requesting that each driver
> 	team designate one (or more) people that will be responsible for
> 	running (or collecting) tests for each release-candidate and
> 	reporting the results to me.
> 
> 	With a new release-candidate pushed by the end of the day on
> 	Monday, and me starting the actual release work on Friday, that
> 	gives 72 hours for each team to perform testing.
> 
> 	I'm happy to let each driver team decide its own testing
> 	strategy. I would hope it would be based on running piglit
> 	across a set of "interesting" hardware configurations, (and
> 	augmenting piglit as necessary as bug fixes are performed). But
> 	I do not plan to be involved in the specification of what those
> 	configurations are. All I need is something along the lines of:
> 
> 		"The radeon team is OK with releasing commit <foo>"
> 
> 	sent to me before the scheduled day of the release.
> 
> 	Obviously, any negative report from any team can trigger changes
> 	to the patches to be included in the release.

This sounds good, but...

> 	And in order to put some teeth into this scheme:
> 
> 	I propose that on the day of the release, the release manager
> 	drop all driver-specific patches for any driver for which the
> 	driver team has not provided an affirmative testing report.

... this seems a bit harsh.

A lot of backported fixes are specific to one driver and have zero
impact on anything else. Surely it should be up to the discretion of the
driver maintainers whether or not such changes should be backported.

OTOH, who's supposed to give that sort of OK for changes to say st/mesa
or Mesa core? The former may affect all Gallium based drivers, the
latter basically anything. It's unlikely that any individual or
organization can test all affected setups in advance.


Given those issues, and that as you say, the current process seems to
have been working out pretty well overall, is introducing that kind of
bureaucracy really justified?


> [...] who from each driver team is willing to volunteer to send me
> testing reports?

We at AMD are working towards having such regular testing, but we're not
quite there yet, nor do we have an exact schedule yet for when we will be.


-- 
Earthling Michel Dänzer            |                  http://www.amd.com
Libre software enthusiast          |                Mesa and X developer