[Piglit] Atomic Writes for piglit backends

[Dylan Baker]

I'm getting a little antsy to land this. It's a pretty invasive series
and I'd like to land it sooner rather than later to save myself rebasing
pains.

On Friday, September 26, 2014 09:28:17 PM Dylan Baker wrote:
> bump.
> 
> On Tuesday, September 23, 2014 05:55:47 PM Dylan Baker wrote:
> > For a long time piglit has had a considerable problem with resuming
> > interrupted runs. This problem comes from the recovery tool, a very
> > fragile piece of code that relies on the formatting of the json to walk
> > backwards throwing away code. The problem with this approach are two
> > fold, first if the indentation changes the tool breaks. Second, it can
> > only handle corruption at the end of the file.
> > 
> > There is a second problem with our resume method, and that is that the
> > json file is read, truncated, and completely re-written on every resume,
> > this can be quite expensive.
> > 
> > Initially I began writing a complicated tool that actually could parse
> > json and clean it, but this quickly became a significant amount of code,
> > and the complexity was overwhelming, it was a complete project in
> > itself.
> > 
> > What I have implemented is a much simpler idea, that I borrowed from
> > databases, Atomic writes. By atomic we mean that a transaction should
> > either completely succeed or completely fail. Using a file backend there
> > are two basic ways to go about doing this. One method Copy the file to a
> > temporary location, write to the temporary file, verify the results, and
> > copy it back over the original. This has a couple of distinct drawbacks.
> > First it requires a lot of copying, second it means that the file needs
> > to be locked for long periods, and finally it requires reading the file.
> > 
> > What I implemented instead is a system that writes each test to it's own
> > file using built-in methods for both json and xml. At the end the runner
> > reads in each of these files, and creates a combined file from the
> > results. This has advantages and drawbacks. Because each test gets it's
> > own file there is no locking required at all, and flushing and syncing
> > is cheaper. It also means that there is no need to verify writes, when
> > the final file is being assembled json.load and xml.ElementTree.parse
> > are called on each file, and if they fail the result is considered
> > failed and ignored.
> > 
> > This series also takes a few more steps toward abstracting backends, by
> > creating a separate package for backends, and including a small subset
> > of the classes and functions from the package in the __init__ module,
> > allowing them to be accessed as if backends itself was a module. This is
> > sufficient for most applications, with mainly unit tests needing to
> > bypass this and access the modules directly.
> > 
> > Patch 1 removes a broken executable script that has been broken for some
> > 		time
> > Patch 2 does most of the abstraction
> > Patches 3-6 do some code cleanup
> > Patches 7 and 8 implement atomic writes in the two backends
> > Patch 9 does some additional refactoring that is possible becasue of
> > 		patches 7 and 8
> > 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: This is a digitally signed message part.
URL: <http://lists.freedesktop.org/archives/piglit/attachments/20140930/8e50cadd/attachment.sig>