[Piglit] [PATCH] add new --ignoreStderr option to piglit-run.py
Tom Gall
tom.gall at linaro.org
Fri Apr 19 07:29:12 PDT 2013
On Fri, Apr 19, 2013 at 9:22 AM, Brian Paul <brianp at vmware.com> wrote:
> On 04/19/2013 04:33 AM, Jose Fonseca wrote:
>>
>> Yes, assuming stderr messages are driver warnings seems a bit pedantic.
>>
>> We could do the other way around -- have a whitelist of warning regular
>> expressions that we do known that are worrisome.
>
>
> Yeah, that's an idea. But I think the only ones that comes to mind are
> "Mesa implementation error" from _mesa_problem() and "Mesa warning" from
> _mesa_warning().
FWIW, there are some binary GLES drivers which emit info/warnings
which would be nice to ignore via some kind of mechanism.
> "Mesa: User error" from _mesa_error() should be ignored since quite a few
> piglit tests probe for GL errors and the message is always printed for debug
> builds.
>
> I guess I'd like to get some more opinions on this patch vs. just removing
> the "stderr=warn" detection code entirely.
I'm not keen on removing it either, but with the idea of a list of
messages that can be ignored, I'd say that's moving in the right
direction.
> -Brian
>
>
>
>> ----- Original Message -----
>>>
>>> Normally, if anything unexpected is printed to stderr we report 'warn'
>>> for the test result. The list of expected/allowed stderr message is
>>> found at the end of core.py. It's a PITA to update this list and if
>>> you've temporarily inserted extra debug code in Mesa/gallium it causes
>>> piglit to just report 'warn' instead of the more useful 'pass/fail'.
>>>
>>> I wonder if anybody depends on stderr output for generating 'warn'
>>> results. If not, maybe we should just remove that "feature" entirely.
>>> ---
>>> framework/core.py | 5 +++--
>>> framework/exectest.py | 5 +++--
>>> piglit-run.py | 7 ++++++-
>>> 3 files changed, 12 insertions(+), 5 deletions(-)
>>>
>>> diff --git a/framework/core.py b/framework/core.py
>>> index 08f442f..8d0d0a6 100644
>>> --- a/framework/core.py
>>> +++ b/framework/core.py
>>> @@ -372,13 +372,14 @@ class TestrunResult:
>>>
>>> class Environment:
>>> def __init__(self, concurrent=True, execute=True,
>>> include_filter=[],
>>> - exclude_filter=[], valgrind=False):
>>> + exclude_filter=[], valgrind=False,
>>> ignoreStderr=False):
>>> self.concurrent = concurrent
>>> self.execute = execute
>>> self.filter = []
>>> self.exclude_filter = []
>>> self.exclude_tests = set()
>>> self.valgrind = valgrind
>>> + self.ignoreStderr = ignoreStderr
>>>
>>> """
>>> The filter lists that are read in should be a list of
>>> string objects,
>>> @@ -455,7 +456,7 @@ class Test:
>>> try:
>>> status("running")
>>> time_start = time.time()
>>> - result = self.run(env.valgrind)
>>> + result = self.run(env.valgrind,
>>> env.ignoreStderr)
>>> time_end = time.time()
>>> if 'time' not in result:
>>> result['time'] = time_end -
>>> time_start
>>> diff --git a/framework/exectest.py b/framework/exectest.py
>>> index b8d97b8..007cf49 100644
>>> --- a/framework/exectest.py
>>> +++ b/framework/exectest.py
>>> @@ -55,7 +55,7 @@ class ExecTest(Test):
>>> raise NotImplementedError
>>> return out
>>>
>>> - def run(self, valgrind):
>>> + def run(self, valgrind, ignoreStderr):
>>> """
>>> Run a test. The return value will be a dictionary with
>>> keys
>>> including 'result', 'info', 'returncode' and 'command'.
>>> @@ -161,7 +161,8 @@ class ExecTest(Test):
>>> results['returncode'] = returncode
>>> results['command'] = ' '.join(self.command)
>>>
>>> - self.handleErr(results, err)
>>> + if not ignoreStderr:
>>> + self.handleErr(results, err)
>>>
>>> else:
>>> results = TestResult()
>>> diff --git a/piglit-run.py b/piglit-run.py
>>> index 6d6ec77..d6aadf0 100755
>>> --- a/piglit-run.py
>>> +++ b/piglit-run.py
>>> @@ -94,6 +94,10 @@ def main():
>>> parser.add_argument("--valgrind",
>>> action = "store_true",
>>> help = "Run tests in valgrind's memcheck")
>>> + parser.add_argument("--ignoreStderr",
>>> + action = "store_true",
>>> + help = "Ignore messages printed to stderr
>>> (don't report 'warn'"
>>> + " for unexpected stderr
>>> messages)")
>>> parser.add_argument("testProfile",
>>> metavar = "<Path to test profile>",
>>> help = "Path to testfile to run")
>>> @@ -155,7 +159,8 @@ def main():
>>> exclude_filter=args.exclude_tests,
>>> include_filter=args.include_tests,
>>> execute=args.execute,
>>> - valgrind=args.valgrind)
>>> + valgrind=args.valgrind,
>>> + ignoreStderr=args.ignoreStderr)
>>>
>>> # Change working directory to the root of the piglit directory
>>> piglit_dir = path.dirname(path.realpath(sys.argv[0]))
>>> --
>>> 1.7.3.4
>>>
>>> _______________________________________________
>>> Piglit mailing list
>>> Piglit at lists.freedesktop.org
>>> http://lists.freedesktop.org/mailman/listinfo/piglit
>>>
>
> _______________________________________________
> Piglit mailing list
> Piglit at lists.freedesktop.org
> http://lists.freedesktop.org/mailman/listinfo/piglit
--
Regards,
Tom
"Where's the kaboom!? There was supposed to be an earth-shattering
kaboom!" Marvin Martian
Tech Lead, Graphics Working Group | Linaro.org │ Open source software
for ARM SoCs
w) tom.gall att linaro.org
h) tom_gall att mac.com
More information about the Piglit
mailing list