[Piglit] Replacing GLUT -- Testing ES, EGL, core profiles, Wayland, etc

[Chad Versace]

I propose that we replace GLUT with something that meets Piglit's needs.
I want to hear your opinion on this.

Problems with Piglit
--------------------

Piglit has several inadequacies that need to be addressed. Some inadequacies,
such as poor support for ES tests, have been an inconvenience that the Mesa
devs have succeeded in overlooking so far (and it's embarrassing how long
we've been content to overlook it). Others, such as no support for Wayland, or
core profiles, have not become a problem yet, but will become pain points in
the near future if not addressed.

Here are the major inadequacies I see that need to be fixed soon:
    - Piglit does not support core profiles.

      This will prevent us from testing core 3.x profiles in Mesa when we
      eventually begin implementing them.  And today, this prevents us from
      running GL 3 tests on Apple.

    - Piglit doesn't really support ES tests.
    
      We have a few tests, specific to GLES1 and GLES2, scattered about, but
      we have no coherent way to write ES tests and integrate them into the
      framework.

      Many of Piglit's GL tests could just as well run on GLES2 with a little
      tweaking. But to take advantage of that, Piglit needs a method to run
      a single test under different API's. (Paul's work on piglit-dispatch was
      a prerequisite for this).

    - Piglit does not support X11/EGL.

      Mesa has an implementation of libEGL, but we don't really test it. There
      is not way to run the GL tests with X11/EGL; they run only on GLX.
      Piglit does have a few scattered tests for EGL extentions, but that's
      not the same as supporting EGL as a first-class platform.

    - Piglit does not support Wayland or Android.


What a solution might look like
-------------------------------

I think the first step to fix these problems is to replace GLUT with something
more flexible. I mean *really* flexible, something that will allow individual
tests to select a GL flavor (api/version/profile) and a window system (GLX,
X11/EGL, Wayland, etc) at runtime.  Now that piglit-dispatch has eliminated
the need for tests to link to libGL and include GL headers, there is a real
possibility that we can make this happen.

This will eventually allow us to write *one* test, annotate which GL flavors
it is compatible with in a comment block (like the glslparser tests), and run
it with different API's and winsys. For example, if a test was properly
annotated, libpiglitutil would allow running it once like this
  ./test --gl-api=gl --gl-profile=core --gl-version=3.2 --winsys=glx
and again like this
  ./test --gl-api=gles2 --winsys=x11_egl
(Those options are from a brainstorm session Ken and I had. There's no working
code that does that). 

I originally considered that Piglit should keep GLUT, and that I should just
contribute the necessary features to GLUT upstream. But I feel that
a better choice is to abandon GLUT altogether because
    1) Piglit's needs, such as dynamic choice of GL flavor and winsys, are
       outside of GLUT's goals. And those needs may need to evolve at a pace
       that upstream GLUT is uncomfortable with.
    2) GLUT has a lot of feature cruft that Piglit doesn't care about. I don't
       want to implement and maintain that cruft for Wayland and possibly
       other future platforms.


Waffle as a replacement for GLUT
--------------------------------

My desired goal is that we replace GLUT with a lightweight library tailored to
meet Piglit's needs, not with another GLUT clone.  I started a new project,
Waffle, to be that replacment.

homepage:       http://people.freedesktop.org/~chadversary/waffle/index.txt
source:         git://people.freedesktop.org/~chadversary/waffle.git
gitweb:         http://cgit.freedesktop.org/~chadversary/waffle

I want to hear people's concerns and opinions about 1) replacing GLUT with
a library that enables the "write one test, run many API's" goal discussed
above, about 2) how suitable Waffle is as a replacment, and about 3) bad
design choices I may have made in the Waffle API. If you like the idea of
Waffle, please voice your opinions on it. I want its API and feature-set to
meet everyone's Piglit needs, not just my short-sighted needs.

Waffle isn't ready yet to replace GLUT (though I do have a working
proof-of-concept branch where it does). It still needs a lot of work. It only
supports Linux and X11/EGL at the moment. It doesn't support any input. It has
a weird bug when creating windows with alpha. But I'm actively working on it,
so hopefully it will be complete in the next month or so.

Below is what basic usage of the Waffle API looks like. What enables the
dynamic choice of GL flavor and window system is the combination of
waffle_init(), waffle_dlsym_gl(), and waffle_get_proc_address().  Those
functions should integrate well with piglit_dispatch_init(). [Paul
(stereotype441) and I had previously discussed how piglit-dispatch would
integrate with Waffle. So, waffle_dlsym_gl() and waffle_get_proc_address()
were written with the expectation that they would be given as the
`get_core_proc` and `get_ext_proc` parameters to piglit_dispatch_init()].

    #include <waffle/waffle.h>

    // Declare function pointers and enums for glClear, glClearColor.

    int
    main()
    {
        const int32_t init_attrs[] = {
            WAFFLE_PLATFORM,            WAFFLE_PLATFORM_X11_EGL,
            WAFFLE_OPENGL_API,          WAFFLE_OPENGL_ES2,
            0,
        };

        const int32_t config_attrs[] = {
            WAFFLE_RED_SIZE,            8,
            WAFFLE_BLUE_SIZE,           8,
            WAFFLE_GREEN_SIZE,          8,
            0,
        };

        // Setup waffle objects.
        waffle_init(init_attrs);
        struct waffle_display *dpy = waffle_display_connect(NULL);
        struct waffle_config *config = waffle_config_choose(dpy, config_attrs);
        struct waffle_window *window = waffle_window_create(dpy, width, height);
        struct waffle_context *ctx = waffle_context_creat(config, NULL);

        glClearColor = waffle_dlsym_gl("glClearColor");
        glClear = waffle_dlsym_gl("glClear");
        // Use waffle_get_proc_address() for extension functions.

        waffle_make_current(dpy, window, ctx);
        glClearColor(1.0, 0.0, 0.0, 1.0);
        glClear(GL_COLOR_BUFFER_BIT);
        waffle_window_swap_buffers(window);

        // Teardown waffle.
        waffle_make_current(dpy, NULL, NULL);
        waffle_window_destroy(window);
        waffle_context_destroy(ctx);
        waffle_config_destroy(config);
        waffle_display_disconnect(dpy);
        waffle_finish();

        return 0;
    }

As a proof-of-concept, I have an experimental Piglit branch [20] that uses
Waffle. It runs all GL, GLES1, and GLES2 tests with X11/EGL.  The small subset
of the tests that I have tried running (the fbo and hiz subdirectories) pass
as expected.

The branch works only on Linux.  There is no command-line switch for choosing
GLX; X11/EGL is hard-coded into Piglit's initialization routine.

[20] git://people.freedesktop.org/~chadversary/piglit.git ; waffle-wip

- Chad