[Ezbench-dev] ezBench errors & error handling

[Eero Tamminen]

Hi,

Run-time errors...

First got following:
----------------------------------------------
  ./core.sh -P mesa -r 1 -b gputest:fur:1080p:fullscreen -b 
gputest:pixmark_volplosion:1080p:fullscreen -b 
gputest:pixmark_julia_fp32:1080p:fullscreen -b 
gputest:pixmark_piano:1080p:fullscreen -b gputest:tess_x64:576p:window 
-b unigine:heaven:1080p:fullscreen HEAD
Configuration scripts used:
Repo type = git, directory = work/upstream/mesa, version = 4e1f3af, 
deployed version =
Tests that will be run: gputest:fur:1080p:fullscreen 
gputest:pixmark_volplosion:1080p:fullscreen 
gputest:pixmark_julia_fp32:1080p:fullscreen 
gputest:pixmark_piano:1080p:fullscreen gputest:tess_x64:576p:window 
unigine:heaven:1080p:fullscreen
Testing 1 versions: 4e1f3af
Estimated finish date: 17-04-27 - 18:14:32 (00h:09m:15s)

WARNING: automatic sudo rights are missing for function 
'xserver_setup_start'
WARNING: automatic sudo rights are missing for function 
'cpu_reclocking_disable_start'
WARNING: automatic sudo rights are missing for function 
'cpu_reclocking_disable_stop'
rm: cannot remove 'logs/2017-04-27-18:05:17/lock': No such file or directory
Exiting with error code 6
----------------------------------------------

This was with just cloned ezBench repository i.e. empty logs directory 
before calling core.sh.  The script created 
"logs/2017-04-27-18\:05\:17/" and "results" file under it before breaking.

If WARNING issues prevent it from running, they should be reported as 
ERRORs.


Accidentally giving bad repository to ezbench results in nested errors. 
Maybe error checking for that could also be improved:
----------------------------------------------
$ ./ezbench -p ~/work/upstream/mesa/ -r 3 -b 
gputest:fur:1080p:fullscreen -b 
gputest:pixmark_volplosion:1080p:fullscreen -b 
gputest:pixmark_julia_fp32:1080p:fullscreen -b 
gputest:pixmark_piano:1080p:fullscreen -b gputest:tess_x64:576p:window 
-b unigine:heaven:1080p:fullscreen -c "911391b 637e571 bf3cdf0 c942faf 
2f8d6bd 8307124 f7b1371 39644fa ec30149" mesa-regressions
2017-04-27 17:56:19: (II) Created report 'mesa-regressions'
Traceback (most recent call last):
   File "ezbench/python-modules/ezbench/smartezbench.py", line 420, in 
set_profile
     runner = self.__create_ezbench(profile=profile)
   File "ezbench/python-modules/ezbench/smartezbench.py", line 341, in 
__create_ezbench
     runner.set_profile(profile)
   File "ezbench/python-modules/ezbench/runner.py", line 442, in set_profile
     self.__send_command__("profile,{}".format(profile))
   File "ezbench/python-modules/ezbench/runner.py", line 390, in 
__send_command__
     "err_code":err_code, "err_str":errstr}))
ezbench.runner.RunnerError: {'err_str': 'The profile does not exist', 
'msg': "The runner returned the error code 12 (The profile does not 
exist) for the command 'profile,~/work/upstream/mesa/'", 'err_code': 
<RunnerErrorCode.CMD_PROFILE_INVALID: 12>}

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
   File 
"/home/testrunner/work/ezbench/python-modules/ezbench/smartezbench.py", 
line 426, in set_profile
     if e.args['err_code'] == RunnerErrorCode.ARG_PROFILE_INVALID:
TypeError: tuple indices must be integers or slices, not str

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
   File "./ezbench", line 146, in <module>
     sbench.set_profile(args.profile)
   File 
"/home/testrunner/work/ezbench/python-modules/ezbench/smartezbench.py", 
line 434, in set_profile
     del runner
UnboundLocalError: local variable 'runner' referenced before assignment
-------------------------------------------2504---
?


This should probably be fatal error:
----------------------------------------------
Traceback (most recent call last):
   File "./ezbenchd.py", line 357, in <module>
     sbench.run()
   File "ezbench/python-modules/ezbench/smartezbench.py", line 806, in run
     runner.start_testing()
   File "ezbench/python-modules/ezbench/runner.py", line 476, in 
start_testing
     self.__send_command__("start_testing")
   File "ezbench/python-modules/ezbench/runner.py", line 399, in 
__send_command__
     "err_str":"Stream ended before we got '<--'"}))
ezbench.runner.RunnerError: {'err_str': "Stream ended before we got 
'<--'", 'msg': 'Incomplete command:\n["WARNING: automatic sudo rights 
are missing for function \'xserver_setup_start\'", "WARNING: automatic 
sudo rights are missing for function 
\'cpu_reclocking_disable_start\'"]', 'err_code': 
<RunnerErrorCode.UNKNOWN: -1>}

Exception ignored in: <bound method Runner.__del__ of 
<ezbench.runner.Runner object at 0x7fca98d8da58>>
Traceback (most recent call last):
   File "ezbench/python-modules/ezbench/runner.py", line 356, in __del__
     self.done()
   File "ezbench/python-modules/ezbench/runner.py", line 418, in done
     self.__send_command__("done")
   File "ezbench/python-modules/ezbench/runner.py", line 374, in 
__send_command__
     self.runner.stdin.flush()
----------------------------------------------

It's also pretty nasty that the script requires sudo rights for 
everything.  Things that are done with sudo should be separate, so that 
just them need to be granted sudo rights.


And at end of ezbenchd.py build run there were more exceptions:
----------------------------------------------
2017-04-27 18:34:32: (II) Ezbench running m2504ode set to 'ERROR'
Traceback (most recent call last):
   File "./ezbenchd.py", line 357, in <module>
     sbench.run()
   File "ezbench/python-modules/ezbench/smartezbench.py", line 843, in run
     time, cmd_output = runner.run(e.commit, e.test, False)
   File "ezbench/python-modules/ezbench/runner.py", line 472, in run
     r = self.__send_command__("run,{},{},{}".format(commit, test, verbose))
   File "ezbench/python-modules/ezbench/runner.py", line 374, in 
__send_command__
     self.runner.stdin.flush()
BrokenPipeError: [Errno 32] Broken pipe

Exception ignored in: <bound method Runner.__del__ of 
<ezbench.runner.Runner object at 0x7f8baa57a668>>
Traceback (most recent call last):
   File "ezbench/python-modules/ezbench/runner.py", line 356, in __del__
     self.done()
   File "ezbench/python-modules/ezbench/runner.py", line 418, in done
     self.__send_command__("done")
   File "ezbench/python-modules/ezbench/runner.py", line 374, in 
__send_command__
     self.runner.stdin.flush()
BrokenPipeError: [Errno 32] Broken pipe
----------------------------------------------


Build errors...

ezBench should have a configuration option for where suitable version of 
libdrm is located, or it should build one itself.


	- Eero