[Piglit] [PATCH 4/4] framework: Repair result file if test run is interrupted

[Chad Versace]

On 07/23/2011 01:30 AM, Ian Romanick wrote:
> On 07/19/2011 12:36 PM, Chad Versace wrote:
>> If the JSON result file was not closed properly, perhaps due a system
>> crash during a test run, then TestrunResult.parseFile will attempt to
>> repair the file before parsing it.
> 
> What happens if I run piglit-summary-html.py on a set of results while
> piglit-run.py is generating those results?  Will it attempt to "repair"
> the output of the in-progress test results?  Will this make everything
> explode?

I did not consider this, since I only run piglit-summary-html.py after the test run has finished. Your fear is accurate. It will attempt to "repair" the in-progress result file and hence destroy it.

I see a simple way to fix this. Currently, TestrunResult.__repairFile 1) copies the result file into a buffer, 2) repairs the buffer's contents, and 3) then overwrites the result file with the buffer. To fix this problem, we can replace step 3 with: return the buffer as the repaired file. This way, __repairFile() never overwrites the result file.

>> Signed-off-by: Chad Versace <chad at chad-versace.us>
>> ---
>>  framework/core.py |   68 +++++++++++++++++++++++++++++++++++++++++++++++++++-
>>  1 files changed, 66 insertions(+), 2 deletions(-)
> 
>> diff --git a/framework/core.py b/framework/core.py
>> index afc5a28..cbd3ef3 100644
>> --- a/framework/core.py
>> +++ b/framework/core.py
>> @@ -291,6 +291,68 @@ class TestrunResult:
>>  			raise ResultFileInOldFormatError(file.name)
>>  		file.seek(saved_position)
> 
>> +	def __repairFile(self, file):
>> +		'''
>> +		Reapair JSON file if necessary
>> +
>> +		If the JSON file is not closed properly, perhaps due a system
>> +		crash during a test run, then the JSON is repaired by
>> +		discarding the trailing, incomplete item and appending braces
>> +		to the file to close the JSON object.
>> +
>> +		Warning: This sets the file's position to 0.
>> +		'''
>> +
>> +		file.seek(0)
>> +		lines = file.readlines()
>> +
>> +		if lines[-1] == '}':
>> +			# JSON object was closed properly. No repair is
>> +			# necessary.
>> +			file.seek(0)
>> +			return
>> +
>> +		# JSON object was not closed properly.
>> +		#
>> +		# To repair the file, we execute these steps:
>> +		#   1. Find the closing brace of the last, properly written
>> +		#      test result.
>> +		#   2. Discard all subsequent lines.
>> +		#   3. Remove the trailing comma of that test result.
>> +		#   4. Append enough closing braces to close the json object.
>> +		#   5. Replace file's contents with repaired JSON.
>> +
>> +		# Each non-terminal test result ends with this line:
>> +		safe_line =  3 * JSONWriter.INDENT * ' ' + '},\n'
>> +
>> +		# Search for the last occurence of safe_line.
>> +		safe_line_num = None
>> +		for i in range(-1, - len(lines), -1):
>> +			if lines[i] == safe_line:
>> +				safe_line_num = i
>> +				break
>> +
>> +		if safe_line_num is None:
>> +			raise Exception('failed to repair corrupt result file: ' + file.name)
>> +
>> +		# Remove corrupt lines.
>> +		lines = lines[0:(safe_line_num + 1)]
>> +
>> +		# Remove trailing comma.
>> +		lines[-1] = 3 * JSONWriter.INDENT * ' ' + '}\n'
>> +
>> +		# Close json object.
>> +		lines.append(JSONWriter.INDENT * ' ' + '}\n')
>> +		lines.append('}')
>> +
>> +		# Replace contents of corrupt file.
>> +		file.truncate(0)
>> +		file.seek(0)
>> +		file.writelines(lines)
>> +		file.flush()
>> +		os.fsync(file.fileno())
>> +		file.seek(0)
>> +
>>  	def write(self, file):
>>  		# Serialize only the keys in serialized_keys.
>>  		keys = set(self.__dict__.keys()).intersection(self.serialized_keys)
>> @@ -299,7 +361,7 @@ class TestrunResult:
> 
>>  	def parseFile(self, file):
>>  		self.__checkFileIsNotInOldFormat(file)
>> -
>> +		self.__repairFile(file)
>>  		raw_dict = json.load(file)
> 
>>  		# Check that only expected keys were unserialized.
>> @@ -510,7 +572,9 @@ def loadTestResults(path):
> 
>>  	testrun = TestrunResult()
>>  	try:
>> -		with open(filepath, 'r') as file:
>> +		# Open the file in r+ mode so that it can be repaired if
>> +		# necessary.
>> +		with open(filepath, 'r+') as file:
>>  			testrun.parseFile(file)
>>  	except OSError:
>>  		traceback.print_exc()
> 
> _______________________________________________
> Piglit mailing list
> Piglit at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/piglit

-- 
Chad Versace
chad at chad-versace.us

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 900 bytes
Desc: OpenPGP digital signature
URL: <http://lists.freedesktop.org/archives/piglit/attachments/20110725/47ddcee3/attachment.pgp>