<p dir="ltr">Not off the top of my head. But I do have a patch Jose reviewed that would probably cover the problem, (falling back to not fast skipping). What test is it you're running?</p>
<p dir="ltr">Sent from my Nexus 6.<br>
I'm on mobile, please excuse autocorrect fail.</p>
<div class="gmail_quote">On Jan 1, 2016 16:59, "Marek Olšák" <<a href="mailto:maraeo@gmail.com">maraeo@gmail.com</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Dylan,<br>
<br>
-p gbm seems to be broken:<br>
<br>
Traceback (most recent call last):ail: 62 /<br>
File "/home/marek/dev/piglit/framework/test/base.py", line 181, in execute<br>
self.run()<br>
File "/home/marek/dev/piglit/framework/test/base.py", line 235, in run<br>
self.is_skip()<br>
File "/home/marek/dev/piglit/framework/test/opengl.py", line 322, in is_skip<br>
if (self.__info.glsl_version is not None<br>
File "/home/marek/dev/piglit/framework/core.py", line 204, in __get__<br>
value = self.__func(instance)<br>
File "/home/marek/dev/piglit/framework/test/opengl.py", line 226, in<br>
glsl_version<br>
raw.split('\n'), 'OpenGL shading language').split()[-1])<br>
File "/home/marek/dev/piglit/framework/test/opengl.py", line 106, in __getline<br>
raise Exception('Unreachable')<br>
Exception: Unreachable<br>
<br>
Any idea what's wrong?<br>
<br>
Marek<br>
<br>
On Thu, Nov 5, 2015 at 11:16 PM, <<a href="mailto:baker.dylan.c@gmail.com">baker.dylan.c@gmail.com</a>> wrote:<br>
> From: Dylan Baker <<a href="mailto:baker.dylan.c@gmail.com">baker.dylan.c@gmail.com</a>><br>
><br>
> This Mixin provides a way for OpenGL tests to skip very fast. Currently<br>
> it only applies to GL extensions, but will be extended to cover GLSL<br>
> version requirements and GL version requirements (and ES)><br>
><br>
> This is split into a separate module because it's going to grow into a<br>
> fairly large amount of code (mostly around querying wflinfo).<br>
><br>
> Signed-off-by: Dylan Baker <<a href="mailto:dylanx.c.baker@intel.com">dylanx.c.baker@intel.com</a>><br>
> ---<br>
> framework/test/opengl.py | 206 ++++++++++++++++++++++++++++++++++++++++<br>
> framework/tests/base_tests.py | 5 +-<br>
> framework/tests/opengl_tests.py | 188 ++++++++++++++++++++++++++++++++++++<br>
> 3 files changed, 398 insertions(+), 1 deletion(-)<br>
> create mode 100644 framework/test/opengl.py<br>
> create mode 100644 framework/tests/opengl_tests.py<br>
><br>
> diff --git a/framework/test/opengl.py b/framework/test/opengl.py<br>
> new file mode 100644<br>
> index 0000000..3485d3a<br>
> --- /dev/null<br>
> +++ b/framework/test/opengl.py<br>
> @@ -0,0 +1,206 @@<br>
> +# Copyright (c) 2015 Intel Corporation<br>
> +<br>
> +# Permission is hereby granted, free of charge, to any person obtaining a copy<br>
> +# of this software and associated documentation files (the "Software"), to deal<br>
> +# in the Software without restriction, including without limitation the rights<br>
> +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell<br>
> +# copies of the Software, and to permit persons to whom the Software is<br>
> +# furnished to do so, subject to the following conditions:<br>
> +<br>
> +# The above copyright notice and this permission notice shall be included in<br>
> +# all copies or substantial portions of the Software.<br>
> +<br>
> +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR<br>
> +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,<br>
> +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE<br>
> +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER<br>
> +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,<br>
> +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE<br>
> +# SOFTWARE.<br>
> +<br>
> +"""Mixins for OpenGL derived tests."""<br>
> +<br>
> +from __future__ import absolute_import, division, print_function<br>
> +import errno<br>
> +import os<br>
> +import subprocess<br>
> +<br>
> +from framework import exceptions, core<br>
> +from framework.options import OPTIONS<br>
> +from .base import TestIsSkip<br>
> +<br>
> +# pylint: disable=too-few-public-methods<br>
> +<br>
> +__all__ = [<br>
> + 'FastSkipMixin',<br>
> +]<br>
> +<br>
> +<br>
> +class StopWflinfo(exceptions.PiglitException):<br>
> + """Exception called when wlfinfo getter should stop."""<br>
> + def __init__(self, reason):<br>
> + super(StopWflinfo, self).__init__()<br>
> + self.reason = reason<br>
> +<br>
> +<br>
> +class WflInfo(object):<br>
> + """Class representing platform information as provided by wflinfo.<br>
> +<br>
> + The design of this is odd to say the least, it's basically a bag with some<br>
> + lazy property evaluators in it, used to avoid calculating the values<br>
> + provided by wflinfo more than once.<br>
> +<br>
> + The problems:<br>
> + - Needs to be shared with all subclasses<br>
> + - Needs to evaluate only once<br>
> + - cannot evaluate until user sets OPTIONS.env['PIGLIT_PLATFORM']<br>
> +<br>
> + This solves all of that, and is<br>
> +<br>
> + """<br>
> + __shared_state = {}<br>
> + def __new__(cls, *args, **kwargs):<br>
> + # Implement the borg pattern:<br>
> + # <a href="https://code.activestate.com/recipes/66531-singleton-we-dont-need-no-stinkin-singleton-the-bo/" rel="noreferrer" target="_blank">https://code.activestate.com/recipes/66531-singleton-we-dont-need-no-stinkin-singleton-the-bo/</a><br>
> + #<br>
> + # This is something like a singleton, but much easier to implement<br>
> + self = super(WflInfo, cls).__new__(cls, *args, **kwargs)<br>
> + self.__dict__ = cls.__shared_state<br>
> + return self<br>
> +<br>
> + @staticmethod<br>
> + def __call_wflinfo(opts):<br>
> + """Helper to call wflinfo and reduce code duplication.<br>
> +<br>
> + This catches and handles CalledProcessError and OSError.ernno == 2<br>
> + gracefully: it passes them to allow platforms without a particular<br>
> + gl/gles version or wflinfo (resepctively) to work.<br>
> +<br>
> + Arguments:<br>
> + opts -- arguments to pass to wflinfo other than verbose and platform<br>
> +<br>
> + """<br>
> + with open(os.devnull, 'w') as d:<br>
> + try:<br>
> + raw = subprocess.check_output(<br>
> + ['wflinfo',<br>
> + '--platform', OPTIONS.env['PIGLIT_PLATFORM']] + opts,<br>
> + stderr=d)<br>
> + except subprocess.CalledProcessError:<br>
> + # When we hit this error it usually going to be because we have<br>
> + # an incompatible platform/profile combination<br>
> + raise StopWflinfo('Called')<br>
> + except OSError as e:<br>
> + # If we get a 'no wflinfo' warning then just return<br>
> + if e.errno == errno.ENOENT:<br>
> + raise StopWflinfo('OSError')<br>
> + raise<br>
> + return raw<br>
> +<br>
> + @staticmethod<br>
> + def __getline(lines, name):<br>
> + """Find a line in a list return it."""<br>
> + for line in lines:<br>
> + if line.startswith(name):<br>
> + return line<br>
> + raise Exception('Unreachable')<br>
> +<br>
> + @core.lazy_property<br>
> + def gl_extensions(self):<br>
> + """Call wflinfo to get opengl extensions.<br>
> +<br>
> + This provides a very conservative set of extensions, it provides every<br>
> + extension from gles1, 2 and 3 and from GL both core and compat profile<br>
> + as a single set. This may let a few tests execute that will still skip<br>
> + manually, but it helps to ensure that this method never skips when it<br>
> + shouldn't.<br>
> +<br>
> + """<br>
> + _trim = len('OpenGL extensions: ')<br>
> + all_ = set()<br>
> +<br>
> + def helper(const, vars_):<br>
> + """Helper function to reduce code duplication."""<br>
> + # This is a pretty fragile function but it really does help with<br>
> + # duplication<br>
> + for var in vars_:<br>
> + try:<br>
> + ret = self.__call_wflinfo(const + [var])<br>
> + except StopWflinfo as e:<br>
> + # This means tat the particular api or profile is<br>
> + # unsupported<br>
> + if e.reason == 'Called':<br>
> + continue<br>
> + else:<br>
> + raise<br>
> + all_.update(set(self.__getline(<br>
> + ret.split('\n'), 'OpenGL extensions')[_trim:].split()))<br>
> +<br>
> + try:<br>
> + helper(['--verbose', '--api'], ['gles1', 'gles2', 'gles3'])<br>
> + helper(['--verbose', '--api', 'gl', '--profile'],<br>
> + ['core', 'compat', 'none'])<br>
> + except StopWflinfo as e:<br>
> + # Handle wflinfo not being installed by returning an empty set. This<br>
> + # will essentially make FastSkipMixin a no-op.<br>
> + if e.reason == 'OSError':<br>
> + return set()<br>
> + raise<br>
> +<br>
> + return {e.strip() for e in all_}<br>
> +<br>
> +<br>
> +class FastSkipMixin(object):<br>
> + """Fast test skipping for OpenGL based suites.<br>
> +<br>
> + This provides an is_skip() method which will skip the test if an of it's<br>
> + requirements are not met.<br>
> +<br>
> + It also provides new attributes:<br>
> + gl_reqruied -- This is a set of extensions that are required for running<br>
> + the extension.<br>
> + gl_version -- A float that is the required version number for an OpenGL<br>
> + test.<br>
> + gles_version -- A float that is the required version number for an OpenGL<br>
> + ES test<br>
> + glsl_version -- A float that is the required version number of OpenGL<br>
> + Shader Language for a test<br>
> + glsl_ES_version -- A float that is the required version number of OpenGL ES<br>
> + Shader Language for a test<br>
> +<br>
> + This requires wflinfo to be installed and accessible to provide it's<br>
> + functionality, however, it will no-op if wflinfo is not accessible.<br>
> +<br>
> + The design of this function is conservative. The design goal is that it<br>
> + it is better to run a few tests that could have been skipped, than to skip<br>
> + all the tests that could have, but also a few that should have run.<br>
> +<br>
> + """<br>
> + # XXX: This still gets called once for each thread. (4 times with 4<br>
> + # threads), this is a synchronization issue and I don't know how to stop it<br>
> + # other than querying each value before starting the thread pool.<br>
> + __info = WflInfo()<br>
> +<br>
> + def __init__(self, *args, **kwargs):<br>
> + super(FastSkipMixin, self).__init__(*args, **kwargs)<br>
> + self.gl_required = set()<br>
> + self.gl_version = None<br>
> + self.gles_version = None<br>
> + self.glsl_version = None<br>
> + self.glsl_es_version = None<br>
> +<br>
> + def is_skip(self):<br>
> + """Skip this test if any of it's feature requirements are unmet.<br>
> +<br>
> + If no extensions were calculated (if wflinfo isn't installed) then run<br>
> + all tests.<br>
> +<br>
> + """<br>
> + if self.__info.gl_extensions:<br>
> + for extension in self.gl_required:<br>
> + if extension not in self.__info.gl_extensions:<br>
> + raise TestIsSkip(<br>
> + 'Test requires extension {} '<br>
> + 'which is not available'.format(extension))<br>
> +<br>
> + super(FastSkipMixin, self).is_skip()<br>
> diff --git a/framework/tests/base_tests.py b/framework/tests/base_tests.py<br>
> index a7afd25..c005273 100644<br>
> --- a/framework/tests/base_tests.py<br>
> +++ b/framework/tests/base_tests.py<br>
> @@ -28,7 +28,10 @@ from nose.plugins.attrib import attr<br>
><br>
> import framework.tests.utils as utils<br>
> from framework.test.base import (<br>
> - Test, WindowResizeMixin, ValgrindMixin, TestRunError<br>
> + Test,<br>
> + TestRunError,<br>
> + ValgrindMixin,<br>
> + WindowResizeMixin,<br>
> )<br>
> from framework.tests.status_tests import PROBLEMS, STATUSES<br>
> from framework.options import _Options as Options<br>
> diff --git a/framework/tests/opengl_tests.py b/framework/tests/opengl_tests.py<br>
> new file mode 100644<br>
> index 0000000..aa42738<br>
> --- /dev/null<br>
> +++ b/framework/tests/opengl_tests.py<br>
> @@ -0,0 +1,188 @@<br>
> +# Copyright (c) 2015 Intel Corporation<br>
> +<br>
> +# Permission is hereby granted, free of charge, to any person obtaining a copy<br>
> +# of this software and associated documentation files (the "Software"), to deal<br>
> +# in the Software without restriction, including without limitation the rights<br>
> +# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell<br>
> +# copies of the Software, and to permit persons to whom the Software is<br>
> +# furnished to do so, subject to the following conditions:<br>
> +<br>
> +# The above copyright notice and this permission notice shall be included in<br>
> +# all copies or substantial portions of the Software.<br>
> +<br>
> +# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR<br>
> +# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,<br>
> +# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE<br>
> +# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER<br>
> +# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,<br>
> +# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE<br>
> +# SOFTWARE.<br>
> +<br>
> +"""Test the opengl module."""<br>
> +<br>
> +from __future__ import absolute_import, division, print_function<br>
> +import subprocess<br>
> +<br>
> +import mock<br>
> +import nose.tools as nt<br>
> +<br>
> +import framework.tests.utils as utils<br>
> +from framework.test import opengl<br>
> +from framework.test.base import TestIsSkip<br>
> +<br>
> +# pylint: disable=invalid-name,protected-access,line-too-long,pointless-statement,attribute-defined-outside-init<br>
> +<br>
> +<br>
> +class TestWflInfo(object):<br>
> + """Tests for the WflInfo class."""<br>
> + __patchers = []<br>
> +<br>
> + def setup(self):<br>
> + """Setup each instance, patching necissary bits."""<br>
> + self._test = opengl.WflInfo()<br>
> + self.__patchers.append(mock.patch.dict(<br>
> + 'framework.test.opengl.OPTIONS.env',<br>
> + {'PIGLIT_PLATFORM': 'foo'}))<br>
> + self.__patchers.append(mock.patch(<br>
> + 'framework.test.opengl.WflInfo._WflInfo__shared_state', {}))<br>
> +<br>
> + for f in self.__patchers:<br>
> + f.start()<br>
> +<br>
> + def teardown(self):<br>
> + for f in self.__patchers:<br>
> + f.stop()<br>
> +<br>
> + def test_gl_extension(self):<br>
> + """test.opengl.WflInfo.gl_extensions: Provides list of gl extensions"""<br>
> + rv = 'foo\nbar\nboink\nOpenGL extensions: GL_foobar GL_ham_sandwhich\n'<br>
> + expected = set(['GL_foobar', 'GL_ham_sandwhich'])<br>
> +<br>
> + with mock.patch('framework.test.opengl.subprocess.check_output',<br>
> + mock.Mock(return_value=rv)):<br>
> + nt.eq_(expected, self._test.gl_extensions)<br>
> +<br>
> +<br>
> +class TestWflInfoSError(object):<br>
> + """Tests for the Wflinfo functions to handle OSErrors."""<br>
> + __patchers = []<br>
> +<br>
> + @classmethod<br>
> + def setup_class(cls):<br>
> + """Setup the class, patching as necissary."""<br>
> + cls.__patchers.append(mock.patch.dict(<br>
> + 'framework.test.opengl.OPTIONS.env',<br>
> + {'PIGLIT_PLATFORM': 'foo'}))<br>
> + cls.__patchers.append(mock.patch(<br>
> + 'framework.test.opengl.subprocess.check_output',<br>
> + mock.Mock(side_effect=OSError(2, 'foo'))))<br>
> + cls.__patchers.append(mock.patch(<br>
> + 'framework.test.opengl.WflInfo._WflInfo__shared_state', {}))<br>
> +<br>
> + for f in cls.__patchers:<br>
> + f.start()<br>
> +<br>
> + def setup(self):<br>
> + self.inst = opengl.WflInfo()<br>
> +<br>
> + @classmethod<br>
> + def teardown_class(cls):<br>
> + for f in cls.__patchers:<br>
> + f.stop()<br>
> +<br>
> + @utils.not_raises(OSError)<br>
> + def test_gl_extensions(self):<br>
> + """test.opengl.WflInfo.gl_extensions: Handles OSError "no file" gracefully"""<br>
> + self.inst.gl_extensions<br>
> +<br>
> +<br>
> +class TestWflInfoCalledProcessError(object):<br>
> + """Tests for the WflInfo functions to handle OSErrors."""<br>
> + __patchers = []<br>
> +<br>
> + @classmethod<br>
> + def setup_class(cls):<br>
> + """Setup the class, patching as necissary."""<br>
> + cls.__patchers.append(mock.patch.dict(<br>
> + 'framework.test.opengl.OPTIONS.env',<br>
> + {'PIGLIT_PLATFORM': 'foo'}))<br>
> + cls.__patchers.append(mock.patch(<br>
> + 'framework.test.opengl.subprocess.check_output',<br>
> + mock.Mock(side_effect=subprocess.CalledProcessError(1, 'foo'))))<br>
> + cls.__patchers.append(mock.patch(<br>
> + 'framework.test.opengl.WflInfo._WflInfo__shared_state', {}))<br>
> +<br>
> + for f in cls.__patchers:<br>
> + f.start()<br>
> +<br>
> + @classmethod<br>
> + def teardown_class(cls):<br>
> + for f in cls.__patchers:<br>
> + f.stop()<br>
> +<br>
> + def setup(self):<br>
> + self.inst = opengl.WflInfo()<br>
> +<br>
> + @utils.not_raises(subprocess.CalledProcessError)<br>
> + def test_gl_extensions(self):<br>
> + """test.opengl.WflInfo.gl_extensions: Handles CalledProcessError gracefully"""<br>
> + self.inst.gl_extensions<br>
> +<br>
> +<br>
> +class TestFastSkipMixin(object):<br>
> + """Tests for the FastSkipMixin class."""<br>
> + __patchers = []<br>
> +<br>
> + @classmethod<br>
> + def setup_class(cls):<br>
> + """Create a Class with FastSkipMixin, but patch various bits."""<br>
> + class _Test(opengl.FastSkipMixin, utils.Test):<br>
> + pass<br>
> +<br>
> + cls._class = _Test<br>
> +<br>
> + _mock_wflinfo = mock.Mock(spec=opengl.WflInfo)<br>
> + _mock_wflinfo.gl_version = 3.3<br>
> + _mock_wflinfo.gles_version = 3.0<br>
> + _mock_wflinfo.glsl_version = 3.3<br>
> + _mock_wflinfo.glsl_es_version = 2.0<br>
> + _mock_wflinfo.gl_extensions = set(['bar'])<br>
> +<br>
> + cls.__patchers.append(mock.patch.object(<br>
> + _Test, '_FastSkipMixin__info', _mock_wflinfo))<br>
> +<br>
> + for patcher in cls.__patchers:<br>
> + patcher.start()<br>
> +<br>
> + @classmethod<br>
> + def teardown_class(cls):<br>
> + for patcher in cls.__patchers:<br>
> + patcher.stop()<br>
> +<br>
> + def setup(self):<br>
> + self.test = self._class(['foo'])<br>
> +<br>
> + @nt.raises(TestIsSkip)<br>
> + def test_should_skip(self):<br>
> + """test.opengl.FastSkipMixin.is_skip: Skips when requires is missing from extensions"""<br>
> + self.test.gl_required.add('foobar')<br>
> + self.test.is_skip()<br>
> +<br>
> + @utils.not_raises(TestIsSkip)<br>
> + def test_should_not_skip(self):<br>
> + """test.opengl.FastSkipMixin.is_skip: runs when requires is in extensions"""<br>
> + self.test.gl_required.add('bar')<br>
> + self.test.is_skip()<br>
> +<br>
> + @utils.not_raises(TestIsSkip)<br>
> + def test_extension_empty(self):<br>
> + """test.opengl.FastSkipMixin.is_skip: if extensions are empty test runs"""<br>
> + self.test.gl_required.add('foobar')<br>
> + with mock.patch.object(self.test._FastSkipMixin__info, 'gl_extensions', # pylint: disable=no-member<br>
> + None):<br>
> + self.test.is_skip()<br>
> +<br>
> + @utils.not_raises(TestIsSkip)<br>
> + def test_requires_empty(self):<br>
> + """test.opengl.FastSkipMixin.is_skip: if gl_requires is empty test runs"""<br>
> + self.test.is_skip()<br>
> --<br>
> 2.6.2<br>
><br>
> _______________________________________________<br>
> Piglit mailing list<br>
> <a href="mailto:Piglit@lists.freedesktop.org">Piglit@lists.freedesktop.org</a><br>
> <a href="http://lists.freedesktop.org/mailman/listinfo/piglit" rel="noreferrer" target="_blank">http://lists.freedesktop.org/mailman/listinfo/piglit</a><br>
</blockquote></div>