[Xorg] Damage/Composite + direct rendering clients

[Andy Ritger]

I've given some thought to how best to integrate direct rendering
clients with Damage/Composite.  For the below discussion, I'll focus
on GLX as the direct rendering client but the same concepts should
apply to XvMC or any other direct rendering client.

For anyone not already familiar with the Damage and Composite
extensions, please see Keith Packard's description of how compositing
works in the modular X server:

    http://www.freedesktop.org/Software/TranslucentWindows

The relevant extensions are Damage and Composite (explanations
of both, as well as links to the specs are available in Keith's
above description).  Damage has already been integrated back into
the X.org X server; integration of Composite is in progress.


How should a direct rendering client interact with Damage/Composite?
There seem to be two pieces to this: damage notification, and
synchronization.


Damage Notification

Obviously, when rendering is performed by the X server, the X server
knows what regions of the X screen have been damaged so that it can
send notification to all Damage clients.  When a direct rendering
client performs the rendering, the X server does not know when the
rendering has been performed.  When a direct rendering client damages
the X screen, it needs to communicate that information to the X
server so that the X server can notify Damage clients of the damage.

Presumably, any direct rendering client today requires support from a
vendor-specific X module to negotiate things like window moves, etc.
Given that, it seems best to leave the details up to each vendor.
As long as the core X server exposes to X drivers some function
such that an X driver can inform the X server of damage, then each
vendor can solve the client -> server communication with whatever
mechanism works best with their architecture.


Synchronization

There are atleast two possible race conditions:

   1) client kicks off rendering, notifies X server of damage,
      X server sends Damage event to composite manager, composite
      manager sends compositing request back to server, server
      performs composite.  There needs to be some synchronization to
      guarantee that the composite is not performed until the client
      rendering is completed by the hardware.

   2) some damage occurs, composite manager sends composite request,
      additional rendering is performed, part of which the composite
      operation picks up, but the rest of the rendering is not
      composited until the next "frame" of the composite manager,
      and we see visible tearing.

      Consider this example: a translucent xterm partially overlaps
      glxgears.  If the xterm is damaged, and the composite manager
      requests a composite, and then glxgears is updated (between
      when the composite request is sent, and when the composite
      operation is performed), then the part of the glxgears beneath
      the xterm will be composited this frame of compositing.  Later,
      the composite manager will receive a damage event for glxgears,
      and will composite, causing the visible screen to be brought
      up to date.  But in the period of time between the first and
      second composites, glxgears will tear.

      The above xterm+glxgears scenario is not limited to direct
      rendering clients.  The same should be reproducible with any
      regular X rendering -- there is a race between when the
      composite manager retrieves the damage region(s), when it
      sends the composite requests, and any rendering protocol
      (or direct rendering) that is processed in between.

      It seems that the complete solution would be for the composite
      manager to perform an XGrabServer(3X11) before retrieving the
      damage regions, then send the compositing requests, and then
      XUngrabServer(3X11).  Unfortunately, that seems very heavy
      weight.  On the other hand, it may ensure faster compositing
      by effectively raising the priority of the composite manager's
      protocol while all other X clients are locked out.

      Some may be inclined to accept the tearing rather than pay
      the heavy weight operation of grabbing/ungrabbing around every
      compositing frame.  For X clients, that may be OK, but I expect
      the tearing will be much more pronounced with OpenGL clients,
      because by nature they are more often animating.


Perhaps the best solution is to introduce two new requests to the
Composite extension: a "BeginComposite" and an "EndComposite" that
composite managers would call, bracketing their compositing requests.
The X server would dispatch these requests into the X driver.
This would give vendors the flexibility to perform any necessary
synchronization to protect against the above race conditions.


In short, some sort of bracketing around the compositing is the
only thing I see lacking from the current specs that may be necessary
to adequately support direct rendering clients with Damage/Composite.



Below are 2 additional ideas that we may want to consider:

   1) Truly double buffer the compositing system.  Keith's sample
      xcompmgr double buffers the compositing by creating a pixmap the
      size of the root window, compositing into that, and then after
      each frame of compositing is complete, copying from the pixmap
      to the visible X screen (is that accurate, Keith?)

      I can't make a strong argument for it, but if instead a back
      buffer for the root window were automatically allocated when a
      composite manager started redirecting windows, and compositing
      was done into that buffer, then this might allow for various
      minor optimizations:

        - the backbuffer could be allocated, by the X driver,
          with a different surface format optimal for compositing,
          but not necessarily optimal for normal pixmaps.

        - possibly swap by flip rather than blit (though obviously
          there would be other complications: the back buffer is
          expected to accumulate, rather than get completely redrawn
          each composite frame).

        - swaps could be done by an X driver at vblank


    2) An actual fullscreen mode.  This is admittedly orthogonal
       to compositing, but the overhead of compositing suggests that
       we should have a mode of operation that clients can request
       where they are given exclusive access to the hardware,
       bypassing the compositing system.

       Today, applications use the XF86VidMode extension (or possibly
       RandR?) to set the mode that they want, and then place their
       window such that it is the only thing that is visible.

       It seems desirable to provide a way for an application to
       bypass the compositing system.  The vidmem for the X root
       window could be freed (along with everything else in video
       memory?)  A surface just the necessary size would be
       allocated, leaving all extra vidmem available for pbuffers,
       multisample buffers, textures, etc.


Do either of the above ideas sound interesting?


Here are a few other important implementation details:

- It is important that X.org maintain a binary compatible driver
  interface, so that vendors are not required to provide multiple
  driver binaries (how to determine which binary to install? etc...)

- An X driver should be able to wrap the redirection of windows to
  offscreen storage:
    - so that the window can be allocated with optimal surface
      format by the X driver
    - The X driver needs to be able to tell its direct rendering
      clients where in video memory the buffer is located

- An X driver should be able to call into the core X server to
  notify X of damage done by direct rendering clients.

- A Video Overlay Xv Adaptor is obviously fundamentally incompatible
  with Damage/Composite.  Should X drivers no longer advertise
  Video Overlay Xv adaptors if they are running in an X server that
  includes Composite support?

- As window managers and desktop environments start folding composite
  manager functionality into their projects, it would be nice
  for them to provide a way to dynamically disable/enable
  compositing... I expect some users will need to disable
  compositing while using film tools, CAD applications, etc, where
  OpenGL performance is more important to them than window system
  compositing.


Feedback very welcome.

Thanks,
- Andy