Nesting sub-surfaces?

[Pekka Paalanen]

Hi,

I have been pondering about sub-surface nesting, i.e. allowing
sub-sub-surfaces and so on, and I have hard time figuring out how
commits should work.

The v2 sub-surface protocol proposal is here:
http://cgit.collabora.com/git/user/pq/weston.git/tree/protocol/subsurface.xml?h=subsurface-v2

Adding support for nesting does not require any API changes to the
protocol. We just need to define, that it is ok to have a sub-surface
as a parent for another sub-surface, and how commits work. Commit
behaviour is again the hard part.

Suppose we have a window consisting of the following:

main surface, wl_surface A
	↳ sub-surface, wl_surface B

Every surface has current state (how it is currently drawn on screen),
and pending state (to accumulate pieces of state, that will then be
made current atomically on wl_surface.commit).

Additionally, sub-surfaces have a state cache. When a sub-surface
itself is explicitly committed, the pending state either becomes
current, or goes to the cache, depending on the commit mode of the
sub-surface. The pending state can only be used when a commit request
was received for the surface itself, never via parent commit.

If the commit mode for the sub-surface B is parent-cached, then
B.commit() will push the pending state into the cache. The cached state
is made current on A.commit(), so that all surfaces get updated
atomically.

If the commit mode for the sub-surface B is independent, then
B.commit() will make the pending state current. A.commit() does not
affect B's current state, except for the wl_subsurface.set_position.

The position set by a wl_subsurface.set_position request is always
stored, and applied only on parent surface commit (A.commit()). This
way the contents of wl_surface B can be updated independently, but
positioning it is still tied to the wl_surface A updates.

(I'm not considering the corner cases arising from transitions from one
commit mode to another at arbitrary times here, but they are solved
for non-nested sub-surfaces.)


Now, add a sub-sub-surface. Suppose we have a window consisting of the
following:

main surface, wl_surface A
	↳ sub-surface, wl_surface B, parent-cached
		↳ sub-sub-surface, wl_surface C, parent-cached

Assume the commit mode of all sub-surfaces is parent-cached, as
written above. C.commit() will push C's pending state into C's cache.
B.commit() will push B's pending state into B's cache. But does
B.commit() do anything to C?

As B is a sub-surface itself, needing a parent commit to change its
current state, B.commit() certainly cannot make C's cached or pending
state current.

Suppose we have this sequence in protocol:

C.commit(C1)
B.commit(B1)
C.commit(C2)
A.commit(A1)

What state should be current for each surface?
For A, it is A1, of course.
For B, it is B1, trivially.
For C, should it be C1 which has been "blessed" by B, or C2 which is
the latest but not "blessed" by B?

If we say C1, then we will need a multi-level cache for each
sub-surface, with a level of cache for each level of nesting.
Sub-surface C would need a 2-level cache. C.commit() pushes pending
state to C.cache-1. B.commit() pushes C.cache-1 to C.cache-2.
A.commit() makes C.cache-2 as current state of C.

Furthermore, each level of cache may hold a reference to a wl_buffer.
This means, that the component driving wl_surface C will actually need
a pool of 4 buffers:
- a buffer that is currently on screen
- a buffer that is in cache-2
- a buffer that is in cache-1
- a buffer that it is rendering to

I do not think this is feasible. So, the answer must be that C2 is the
current state for C. In other words, whether B.commit is called or not,
it has no effect on what happens with C. (But does it make sense for
applications?)


Next a case with mixed commit modes:

main surface, wl_surface A
	↳ sub-surface, wl_surface B, independent
		↳ sub-sub-surface, wl_surface C, parent-cached

C.commit() pushes C's pending state to C's cache.
B.commit() make B's pending state current, and make C's cached state current, too.
A.commit() make A's pending state current, and... does nothing to B,
since B is independent. Also, does not even look past B, so nothing for C.

Ok, I guess that works. And the case where B is parent-cached and C is
independent should be obvious, too.



This leads to the following commit algorithm:

- Main surface commit, and sub-surface commit in independent mode, will
  make own pending state current, and will call recursive commit on all
  its immediate sub-surfaces.

- Sub-surface commit in parent-cached mode will push the pending state
  into the sub-surface's cache. No recursion.

- A recursive commit on a sub-surface with independent commit mode does
  nothing. No recursion.

- A recursive commit on a sub-surface with parent-cached commit mode
  will make the surface's cached state current, and call recursive
  commit on all its immediate sub-surfaces.

And then add all the corner cases when commit modes are changing, and
e.g. a sub-surface is committed in independent commit mode with some
state in its cache.

Oh, and I forgot to specify what happens with
wl_subsurface.set_position. D'oh.


So, what are the benefits from nesting sub-surfaces?
- Each sub-surface's position is relative to its parent, so you don't
  have to add up the offsets in the client.
- An independent sub-surface can have its own sub-surfaces, and update
  the whole wl_surface sub-tree in sync.
- ...?


Is this all really worth it?
Does anyone have a good use case for nested sub-surfaces, which would
be inconvenient to implement with just sibling sub-surfaces without
nesting?

Unless there is a good use case I can design for, I'm tempted to just
not bother with this, this time. If we later see, that nesting
would actually have value, we can add the support by just bumping the
wl_subcompositor interface version, and without any changes to the
protocol signature.


Thanks,
pq