[Piglit] [PATCH 00/15] [RFC] Replace glew with a custom Piglit dispatch mechanism

[Paul Berry]

Piglit currently uses the GLEW library (http://glew.sourceforge.net/)
as a dispatch layer to invoke OpenGL functions.  Though GLEW is a
reasonable choice for normal desktop OpenGL applications, it has a
number of disadvantages that make it suboptimal for use in Piglit.
Notably, it doesn't support GLES, Wayland, or forward compatible
contexts, and those are all things we're going to need to start
testing soon*.  I believe it would be fairly easy to replace GLEW with
a new dispatch mechanism that's more suited to the needs of Piglit.

(*Note: Piglit does contain some GLES tests already, but it's my hope
that a new dispatch mechanism will make it much easier to add more,
and to adapt the existing GL tests to also test GLES).

This patch series contains my preliminary efforts, which for the
moment I'm calling "piglit-dispatch".  I'm very interested in hearing
people's thoughts about the approach I'm taking.  In an effort to get
early feedback, I'm publishing these patches in unfinished form--in
particular, support for Mac, Windows, Wayland, and GLES is currently
missing.  I plan to add Mac and Windows support before landing the
patches, so that no one's work is interrupted.  Note that the patch
series is complete as far as Linux and desktop GL are concerned--I
have tested against Intel Sandy Bridge and found no regressions.

Two of the patches in the series are tens of thousands of lines long
(since one of them removes glew.h/c, and the other adds a bunch of
generated code).  So I'm omitting them from the e-mail list.  If you
want to try out this code, please feel free to pull from branch
"dispatch" of git://github.com/stereotype441/piglit.git.

What follows is a brief description of how the proposed dispatch
mechanism works, its advantages, and some questions I'm particularly
interested in hearing feedback on.


How it works:
------------

As with GLEW, piglit-dispatch maps each GL function to a global
function pointer using preprocessor macros (for example, glBeginQuery
is mapped to a function pointer called
__piglit_dispatch_glBeginQuery).  However, unlike in GLEW, function
names that are aliases of each other map to the same function pointer
(so for example, glBeginQueryARB also maps to
__piglit_dispatch_glBeginQuery).

Initially, each function pointer points to a stub function (in the
case of glBeginQuery, it's called stub_glBeginQuery).  The first time
the stub funciton is called, it inspects the implementation's version
and extension strings to determine which alias the GL implementation
supports, and then uses the appropriate GetProcAddress() function to
get a pointer to the GL function.  Then it calls that function.
Additionally, it updates the global function pointer, so that future
calls will go directly to the GL function, bypassing the stub.

If the implementation doesn't support any alias for a given GL
function, piglit-dispatch automatically terminates the test,
generating a Piglit "SKIP" condition.

As in GLEW, the dispatch code is largely autogenerated.  However,
unlike GLEW, which generates dispatch code based on heuristic parsing
of the text extension descriptions on www.opengl.org, piglit-dispatch
uses the same XML files used by Mesa to build its dispatch tables.

Here is an example of the autogenerated header code for the
glBeginQuery function:

/* glBeginQuery (GL 1.5) */
/* glBeginQueryARB (GL_ARB_occlusion_query) */
extern PFNGLBEGINQUERYPROC __piglit_dispatch_glBeginQuery;
#define glBeginQuery __piglit_dispatch_glBeginQuery
#define glBeginQueryARB __piglit_dispatch_glBeginQuery

And here is the autogenerated C code:

/* glBeginQuery (GL 1.5) */
/* glBeginQueryARB (GL_ARB_occlusion_query) */
static void stub_glBeginQuery(GLenum target, GLuint id)
{
	__check_initialized();
	if (__check_version(15))
		__piglit_dispatch_glBeginQuery = (PFNGLBEGINQUERYPROC) __get_core_proc("glBeginQuery");
	else if (__check_extension("GL_ARB_occlusion_query"))
		__piglit_dispatch_glBeginQuery = (PFNGLBEGINQUERYARBPROC) __get_ext_proc("glBeginQueryARB");
	else
		__unsupported("BeginQuery");
	__piglit_dispatch_glBeginQuery(target, id);
}
PFNGLBEGINQUERYPROC __piglit_dispatch_glBeginQuery = stub_glBeginQuery;

The functions __get_core_proc() and __get_ext_proc() wrap around
function pointers that are set up at initialization time.  This will
allow piglit-dispatch to make a run-time decision of how to resolve
function pointers (e.g. whether to use glXGetProcAddressARB() or
eglGetProcAddress()).


Advantages:
----------

Piglit-dispatch has the following advantages over our existing use of
GLEW:

1. Since __get_core_proc() and __get_ext_proc() wrap around function
pointers that are provided at initialization time, each Piglit
executable will be able to bind to either desktop GL or GLES based on
a run-time decision (a command-line parameter or an environment
variable, for example).  This should allow us to dramatically simplify
Piglit's build scripts, which currently do separate builds for desktop
GL, GLES 1, and GLES 2.  Similarly, we'll be able to make a run-time
decision of whether to test using X windows (which traditionally uses
glXGetProcAddressARB()) or Wayland (which uses eglGetProcAddress()).

2. Since __get_core_proc() and __get_ext_proc() are separate
functions, piglit-dispatch will be able to deal with GLES's
restriction that eglGetProcAddress() is only guaranteed to work for
extension functions.  (When testing GLES, __get_core_proc() will
determine the function pointer using some other OS-specific mechanism,
such as dlsym() on Linux).

3. Since __get_core_proc() and __get_ext_proc() are invoked lazily,
piglit-dispatch will work properly with forward compatible contexts
(since it won't try to get the address of a deprecated function unless
the test requires it).  As a side benefit, this should make test
startup faster, since we won't waste time querying the address of
functions the test doesn't need.

4. Since piglit-dispatch understands function aliases, we won't have
to go to extra work in Piglit to figure out which version of a
function to call based on which extensions are supported.  This should
allow us to eliminate the utility files piglit-shader.h and
piglit-transform-feedback.h, as well as a lot of ad-hoc code in
individual tests.

5. Since piglit-dispatch automatically reports a Piglit "SKIP" if a
test tries to access an unsupported function, we won't have to
remember to explicitly check that the necessary extensions are
supported before beginning a test.

6. Piglit-dispatch addresses a limitation of GLEW that we've had to
work around several times: it correctly handles the situation where a
function with the same name appears both in a GL version and in an
extension that is not required by that version (for example,
glGetIntegeri_v is defined by both GL 3.0 and
ARB_uniform_buffer_object, but ARB_uniform_buffer_object is not a
required part of glGetIntegeri_v.  With an unpatched version of GLEW,
glGetIntegeri_v() segfaults on an implementation that supports GL 3.0
but not ARB_uniform_buffer_object).

7. Since piglit-dispatch is autogenerated from Mesa's XML files, we
avoid having to write a bunch of tricky text parsing code, and it
should be easy to add new extensions and fix bugs in the future.  As a
side benefit, running Piglit on platforms other than Mesa will help
validate that Mesa's XML files are correct.


Questions:
---------

1. Does this mechanism seem like it would suit your needs?  If not,
what is missing?

2. How would you prefer to share the XML files between Piglit and
Mesa?  In my preliminary effort, I've moved the XML files to their own
project (temporarily hosted at
git://github.com/stereotype441/glapi.git).  However, an alternative
would be to leave the XML files in Mesa, and require Mesa to be
present on the machine when generating the piglit-dispatch code.  What
would you prefer?

3. Should code generation be done on every build, or should the
generated code be checked into source control, and rebuilt manually
when the XML files change?  At the moment I've adopted the latter
approach, since it makes the CMake integration easier.

4. Do you forsee any difficulties in adapting this approach to
Windows, MacOS, Android, Wayland, GLES, or forward compatible
contexts?  I don't think I've painted myself into any corners, but I'd
appreciate others' insights.

5. Is anyone interested in collaborating on the Windows portion of
this effort?  I'm sure I can do it, but there's probably someone out
there who can do the work faster than I can (Jose, Vinson, or Brian,
perhaps?)


Thanks in advance for your review,

Paul Berry


[PATCH 01/15] Stop using GLEW_VERSION_* macros
[PATCH 02/15] Stop using GLEW extension macros
[PATCH 03/15] Stop using glewIsSupported
[PATCH 04/15] Stop using GLEW_GET_FUN
[PATCH 05/15] Stop referring to nonstandard enums GL_CLIP_PLANE{6,7}.
[PATCH 06/15] Convert more older tests to use piglit-framework.c.
[PATCH 07/15] Add a call to glewInit() to windowoverlap.c
[PATCH 08/15] glean: call glewInit()
[PATCH 09/15] Add glewInit() calls to glx tests.
[PATCH 10/15] piglit-dispatch: Code generation script
[PATCH 11/15] piglit-dispatch: CMake code to build generated files
[PATCH 12/15] piglit-dispatch: Add generated files
[PATCH 13/15] piglit-dispatch: Remaining infrastructure
[PATCH 14/15] Switch to using piglit-dispatch instead of GLEW.
[PATCH 15/15] Remove GLEW.