[PATCH wayland v3] protocol: Add minimize/maximize protocol

[Scott Moreau]

Hi Jason,

On Sat, Mar 23, 2013 at 2:58 PM, Jason Ekstrand <jason at jlekstrand.net> wrote:
> Scott,
> I have a few technical comments to make down below.  Before I go onto
> those, I want to make sure we are perfectly clear about the purpose of
> this discussion.  So please read the following through (multiple times
> if needed) before going on to the technical bits.

Yes, I'm glad you brought up the purpose of this discussion.
Ultimately, I don't think we all share the same general purpose.

>
> The Wayland project is primarily about PROTOCOL.  Specifically, it is
> about the Wayland core protocol: that which is described in
> wayland/protocol/wayland.xml.

I am curious, is the primary purpose or ultimate goals of wayland
documented somewhere? Kristian?

> What we (myself, Bill, and Pekka) are
> trying to discuss with you is a proposed min/max PROTOCOL that will,
> eventually, get added to wayland/protocol/wayland.xml.  This has a
> number of implications.

I've had these patches on the list open for discussion for five months
now, give or take. During this time, the window list was briefly
reviewed by Kristian yet, none of the core protocol commented on. This
was surprising to me but I assumed since the window list patch was
reviewed, silence on the protocol implied acceptance. Unfortunately,
during this time some things have also changed and I have come to many
realizations.

>
> First, we are talking about what the protocol OUGHT to be, not what it
> is right now.  Ideas that get thrown around in the mailing list need
> to be evaluated and discussed based on their technical merits and how
> clean they make the client/compositor interactions.  We need to be
> looking for the best possible solution to the problem as a whole.  In
> particular this means that "I have it working" is not a valid
> justification in the face of technical issues or potentially better
> solutions.

Yes, you are right that the protocol should serve the greater good as
a whole. "I have it working" means the same thing from any weston
component. Sure, things work but this does not mean that the
implementations upstream are perfect. There are plenty of bugs in
weston as well as my implementations. The point is, that once you see
the code in a 'working' state, this provides more insight as to what
might be good things to look out for, for future cases. Anyone can
talk and have ideas but if the implementation cannot be worked out as
discussed in the code for technical reasons, naturally adjustments
need to be made. "I have working" does not mean "My implementation is
the only correct one and only one that should exist" which you seem to
be misunderstanding.

>
> Second, none of this is yet in wayland master.  Until the min/max
> protocol makes its way into a Wayland release, nothing is final and
> everything is flexible.  This means that the entire min/max protocol
> is up for revision.  Just because you "have it working" in your gh
> next branch doesn't mean that it can't be thrown out and re-worked.
> Also,  It is not practical for us to hold a discussion based on the
> incremental changes that you commit to your gh next branch.  As the
> protocol gets re-worked, please send new versions to the list, rebased
> against wayland master, that incorporate those changes.  This will
> make it much easier to see how everything fits into the Wayland
> protocol as a whole.  (I already asked you to do this a few e-mails
> ago.)

Alright, now you're moving into an assumption that wayland is and will
be the only protocol to use the new driver infrastructure and it will
be absolute. This would be nice but it's not likely going to be the
case as time progresses. See the ever-so-(un)popular Mir
implementation. People are humans that have desires and wants, not a
plugin to the wayland protocol. In order to attempt considering all
scenarios, you might expand your thoughts outside of the wayland
realm. For the purposes of this discussion, you can stay isolated to
your wayland box ideals. I for myself, will continue to strive for
things that makes sense to myself and others.

>
> Third, this discussion is NOT about implementation.  While Weston and
> your gh next Weston may be valuable, they are largely irrelevant to
> the current discussion.  What we need to focus on is trying to make
> the client/compositor interactions as clean as possible.  This has
> nothing to do with the internal implementation details of Weston or
> any other compositor.  Those implementation details are useful only in
> so far as they help us understand the client/compositor interactions
> and the potential pitfalls of a given protocol.

I have given my representations of what I think is a clean solution.
The implementation must follow protocol but beyond that isn't
extremely important. However, it has been the policy in wayland/weston
to only push protocol with working implementations in the official
reference compositor, weston. You are right that a wayland compositor
implementation isn't specifically important to the wayland protocol
but specifically, the implementation in weston is in fact important.
Again, not saying my solution is correct or absolute in any way, just
that we can't say 'let us make the protocol first, push it and then
figure out how or if it works, later'

>
> Finally, this is about the Wayland protocol, not the Weston protocol.
> This means that we don't just throw a bunch of undocumented
> events/requests into the protocol file, start implementing it in
> Weston, and document how we ended up implementing it.  That is exactly
> backwards of how the protocol should be developed.  I'm not saying
> that implementing it in Weston isn't useful for getting the kinks out.
>  What I'm saying is that the protocol comes first and then we
> implement it to make sure it works.

I think that you are completely wrong here. We *definitely* want to
get new protocol in place and try it out. The policy of wayland/weston
has been, that when new protocol is added, the example implementations
are added to weston and other relevant components. As I explained, the
min/max descriptions were intentionally and thoughtfully left blank
because there is a whole lot involved for the expected desktop cases
and it's a touchy subject too. However, you don't have to overthink
this. The best solution is to add protocol for everyone, not just
isolating to the immediate use cases that you see and disregard
everyone elses ideas as unneeded/crazy.

>
> One more thing before I go onto the technical details: Bill Spitzak.
> Just because he gets on your nerves doesn't mean you should just
> ignore him.  You are very much into the implementation details whereas
> Bill tends to come at things from a perspective that is more
> high-level and theoretical.  This is a very useful perspective when
> discussing a protocol because the protocol should transcend the
> details as much as possible.  Bill does read the e-mails and he
> frequently points out potential pitfalls that other people miss.  He
> is also usually very clear in his examples (in his first e-mail in
> this series he drew you an ASCII-art picture to demonstrate what he
> meant).  You need to stop ignoring him and writing off everything he
> says as worthless.  At the very least you need to act towards him in a
> more professional manner.

Don't care about Bill. He doesn't contribute, he wastes time. You are
beginning to do the same thing by trolling me here. I really don't
appreciate it.

>
> Technical comments follow.
>
> On Fri, Mar 22, 2013 at 6:29 PM, Scott Moreau <oreaus at gmail.com> wrote:
>> Hi Pekka, thanks for your comments here.
>>
>>>
>>> Scott,
>>>
>>> do you mean that these unminimize, unmaximize, etc. requests would
>>> actually work like undo? Unmaximize would undo what the last
>>> maximization did, as opposed to just set_normal which might do
>>> something slightly different since its aim is to make the window
>>> normal, not undo something?
>>
>> Right, I am asking if there is a case for such functionality. The more
>> I think about it, it seems like it might seem like a bug in normal
>> desktop usage. But if you happen to have another external program that
>> you might want to use to manipulate window states, maybe you'd want a
>> finer grain control there. I'm not sure of all the possible cases,
>> which is why I'm asking for input.
>>
>>>
>>> If so, does it make sense to have state specific undo requests?
>>>
>>> Consider this sequence, driven by the client:
>>> 1. the window is normal
>>> 2. maximize the window
>>> 3. minimize the window
>>> 4. unmaximize the window
>>>
>>> What happens at step 4? Is it possible to define the outcome of
>>> such cases in an intuitive way?
>>
>> This illustrates the point perfectly. This is definitely all about
>> semantics and expected behavior. This must be clearly documented in
>> the protocol description. I had it working so you could do this. You
>> could minimize, toggle maximized state and unmaximize with the correct
>> state. This works fine when doing
>> maximize->minimize->unmaximize->unminimize but causes a frame glitch
>> when doing unmaximize->minimize->maximize->unminimize. The frame
>> glitch is probably easily worked around. However, it is noteworthy
>> that if you have this behavior, then you can't assume that maximize
>> and toplevel will be counterparts, but instead, that you'd need
>> explicit unmaximize notification.
>
> The last thing we want is a protocol full of "semantics and expected
> behaviour".  If implementing this correctly requires a lot of expected
> behaviour, we need to re-think the protocol.
>
> To solve this problem I suggested above that we simply replace all of
> the set_X and unset_X requests with set_X requests and one
> "set_normal" or whatever that returns to a default window state.  This
> way, instead of having to worry about what to set/unset to get to a
> desired state, the client simply tells the compositor what the next
> state should be.  In order to do things correctly, the client is going
> to have to track state anyway, so it might as well explicitly tell the
> compositor what state it wants next.  (This suggestion has gone
> completely unanswered.)
>
>>> Does it ever make sense to send a unBAR when the previous operation
>>> was FOO, not BAR? Could you do with just one undo request for all
>>> the un* cases? (If not, why?)
>>>
>>> What if a client undoes twice? N times?
>>>
>>> Add this to the above sequence:
>>> 5. unminimize the window
>>>
>>> Is the window now in normal state or maximized state?
>>
>> Since this email, I have made it so you cannot toggle maximize while
>> in a minimized state. This is the way xfce works and it's reasonably
>> sane behavior.
>>
>>>
>>>
>>> I'm thinking this purely from the compositor point of view, and I
>>> don't have any tangible suggestions here, but the above just
>>> seems to generate more illegal than legal sequences, which also
>>> means the code in the compositor must be checking for all the
>>> illegal cases, and emit errors. The illegal sequences might not
>>> make any sense to use, but the compositor (and the protocol spec)
>>> must be aware of what happens when they are received. It might be
>>> worth to actually draw (as on paper) the state machine with all the
>>> requests you might get in each state.
>>>
>>> Is there any way this protocol could be designed in a way, that
>>> illegal sequences would not exist, or at least would be in the
>>> minority?
>>
>> I would like to dismiss the term 'illegal' here in it's entirety
>> because it is biased by definition. I prefer 'possibly problematic'.
>
> When talking about a protocol "illegal" is a perfectly valid word.
> Until we define how everything should work there's a significant
> possibility for "illegal" interactions.
>
>>> As such, having only the set-requests without corresponding
>>> un-requests would cut down the number of illegal (or just
>>> unintuitive) sequences a lot. Adding a single undo request won't make
>>> it much worse, the only suspicious case is undoing multiple times
>>> in a row, I think. Adding corresponding un-request for each state
>>> request leads to a minor combinatorial explosion of possible
>>> sequences for which there is no obvious idea on what should happen.
>>
>> Yes, this is an orchestration between the clients and shell. It
>> doesn't sound too complicated until you take a look at what's actually
>> going on. I did not have time to draw a picture but mainly, you have
>> to keep everyone in sync. And by everyone I mean:
>>
>> 1) The shell plugin
>> 2) The wl_shell_surface clients
>> 3) The xwayland clients
>> 4) The desktop-shell client
>>
>> These all must be 'on board', for everything to go as intended.
>
> This is entirely an implementation detail of weston.  It is completely
> irrelevant to the current discussion.
>
>>> Btw. how do you intend to restore the stacking order on undo, in
>>> practice, in the Weston implementation? It is possible other
>>> windows have been deleted, created, and shuffled between set and
>>> undo, so what will you use as the anchor to go back to?
>>
>> I am using gh next as a testbed to work out many of the details.
>>
>>>
>>>
>>> As for the whole idea of undoing stacking order changes; you seem
>>> to assume that set_<state> requests will change the stacking order.
>>> Is that right?
>>
>> The stacking order is (optionally) only changed when a state is restored.
>>
>>>
>>> Or is that just a convenient workaround for the fact, that we do
>>> not have protocol for explicitly controlling stacking order? So you
>>> just add implicit stacking side-effects to unrelated requests?
>>
>> There's a lot of missing protocol, again, gh next is the testbed for
>> the current implementation I have.
>>
>>>
>>> If we had orthogonal requests for controlling stacking order, then
>>> set_<state> requests would not need to touch stacking order at all.
>>> Excluding stacking order, is there something else you would want to
>>> undo, or would the whole undo thing become unneeded?
>>
>> I would like to think, that there are far and few between cases where
>> we'd want such a behavior. Weston is reportedly a toy, not a real DE
>> and I'm playing with it a bit. I'm not trying to enforce policy, I'm
>> trying to open up the possibility of doing more interesting things.
>>
>>>
>>> I would like to strongly suggest to consider splitting the protocol
>>> into orthogonal concepts. That is what I did when I separated
>>> clipping and scaling from the sub-surfaces protocol. It may seem
>>> like more work to start with, but the end result will be cleaner,
>>> more intuitive, and more versatile. It will also allow you to
>>> reduce the interactions and implications, making designing the
>>> protocol easier, and leading to a better end result. The short-term
>>> downside is that you cannot take shortcuts in the design to have
>>> a certain use case running sooner; you will have all use cases
>>> running correctly later. As a compromise to allow development and
>>> testing, your implementation can violate your own spec while
>>> unreleased.
>>
>> I'm not sure what you mean by 'splitting the protocol into orthogonal
>> concepts' here.
>
> By "splitting into orthogonal concepts" he means developing two
> completely separate protocols: one for specifying stacking order, the
> other for minimizing/maximizing.  At the protocol level, they would
> have basically no overlap.  In particular, the min/max protocol should
> specify NOTHING about stacking order other than, perhaps, fullscreen
> is on top.
>
>>> Window state and stacking order often change hand-to-hand, but I
>>> see no reason to tie them together on the protocol level. That is
>>> why I would suggest to handle window state in one set of requests,
>>> and stacking order in a another disjoint set of requests. Moving a
>>> fullscreen window to the top could still be implemented by moving
>>> it to the fullscreen layer in Weston, but that really is just an
>>> implementation detail. From the client's point of view the
>>> fullscreen window is simply "on top" at that time.
>>
>> The bottom line is, the four components I mentioned here, must work
>> together and be 'on the same page' regarding semantics. Each component
>> is responsible for it's own state tracking. I have found it is easier
>> to track state when the calls are in consistent pairs. This also
>> yields potentially more flexible control for the window manager (shell
>> plugin).
>
> I don't see how this comment has anything to do with the interaction
> between stacking and min/max.  Also, it's mostly about implementation
> details again.
>
> On Fri, Mar 22, 2013 at 7:09 PM, Bill Spitzak <spitzak at gmail.com> wrote:
>> The underlying problem is that if a window is full-screen or maximized, and
>> you minimize it, then un-minimize should put it back to full-screen or
>> maximized. Thus un-minimize cannot be the "normal" state.
>>
>> The compositor could track the previous state and set that but then the
>> client can't change the previous state while minimized.
>>
>> The proposed "un-minimize request" means the compositor does not know what
>> state will result after the un-minimize.
>>
>> I think this can be solved by merging all these states into a single integer
>> as bits:
>>
>>         MAXIMIZED = 1;
>>         FULL_SCREEN = 2;
>>         MINIMIZED = 4;
>>         ...
>>
>> The rules are that if MINIMIZED is on then it is minimized, and the other
>> bits are ignored. If not then FULL_SCREEN is on it is fullscreen and
>> MAXIMIZED is ignored. Etc.
>
> Yes, If we're going to handle min/max in terms of setting/unsetting
> flags, we definitely need a precedence order.  There are other ways
> that you could probably describe what to do with each given
> combination.  However, I think simply setting a precedence order is
> the cleanest and easiest to get consistent.
>
>> The compositor can send an event to a surface saying that the state should
>> change to a new set of values. Un-minimize just means it turns off the
>> minimize bit and sends the new value.
>>
>> The client can send commands changing the state of surfaces. If it wants to
>> turn off maximize while minimized, it just turns off the bit.
>>
>> This has a number of advantages:
>>
>>  1. greatly reduces the api count
>>
>>  2. Easy to add some new "state" to existing shell api
>>
>>  3. All plausable implementations of clients and shells I can think of would
>> end up storing a set of flags just like this anyway and tracking the state.
>
> Thanks for your comments Bill!  I have thought about this solution and
> I think it would work fairly well if we were to have the client send
> an explicit next state such as minimized, maximized, or normal to the
> compositor.  However, I'm not sure I like it for the flags.  For one
> thing, it implies a precedence order which is good until we add a flag
> whose precedence is somewhere in the middle.  Second, it allows the
> client to flip any number of flags before sending them to the
> compositor.  At that point, it might as well just send the final state
> and forget the flags.
>
> One more thing: Could you please send out a version 2 with whatever
> changes have been made since the original version you sent to the
> list.  This will greatly help with the discussion.
>
> Thanks,
> --Jason Ekstrand

I don't have time for this, you are trolling by now. You guys can take
and make the protocol however you see best fit for your wayland
project.


Thanks,

Scott